The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2016-03-17

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 bpmedley pink_mist : May I ask where simcop2387 mentioned what the issue was?  I'm not seeing anything in my history.
00:01 bpmedley Oh!  I see, there was a side conversation in #perl on freenode..
00:01 bpmedley Cool.  I may want to have better examples in that module.
00:01 pink_mist his issue was that his tx was being freed too early
00:02 pink_mist which is unrelated to ProcBackground
00:02 pink_mist ... Grinnz_ did convince him to switch to ForkCall though
00:02 pink_mist unsure whether he did or not
00:02 heytrav left #mojo
00:03 bpmedley Hrmm.  I wonder what would make ForkCall better..
00:04 pink_mist s/did or not/switched already or not/
00:04 Grinnz I don't know anything about Proc::Background, but I have used ForkCall a lot. as I mentioned.
00:04 bpmedley That makes sense.
01:12 jb360 joined #mojo
01:35 Grinnz_ this seems a lot faster than before http://paste.fedoraproject.org/341335/17845714/
01:36 Grinnz_ but the only relevant change really was using JSON1 for the json columns, which isn't really an optimization
01:36 Grinnz_ (Minion::Backend::SQLite)
01:38 mattp_ Grinnz_: what per/second does the PG backend hit roughyl/
01:38 mattp_ roughly?*
01:38 Grinnz_ a lot faster than that, lol
01:38 Grinnz_ don't remember
01:38 mattp_ Grinnz_: is ::sqlite on github?
01:38 mattp_ yes it is
01:39 Grinnz_ i'm testing it on the commit before the JSON1 change and it's taking forever
01:39 Grinnz_ i wonder what's so different
01:39 Grinnz_ Minion doesn't really use any database json features, and the json is still encoded and decoded by Mojo::JSON either way
01:40 Grinnz_ http://paste.fedoraproject.org/341340/14581787/
01:41 mattp_ were you using varchar before?
01:42 Grinnz_ it was in a blob, now it's in a text, but SQLite types are ... weird
01:42 mattp_ id imagine a json specific type would only slow things down, not speed up
01:43 Grinnz_ I wonder if it was changing that index on (state) to (state, priority desc, created)
01:45 Grinnz_ sri does say it made dequeue two orders of magnitude faster for Pg https://github.com/kraih/minion/commit/70c09d4d66de7ef344ce76dc25f710175b2ec2cf
01:46 Grinnz_ well, it definitely helped, same as the last one with that one index changed http://paste.fedoraproject.org/341344/81791031/
01:46 Grinnz_ these times vary wildly on my machine because of the disk access unfortunately
01:47 Grinnz_ http://paste.fedoraproject.org/341347/58179217/ this one looks closer to the original time
01:47 Grinnz_ so, yay indexes
01:50 ivi joined #mojo
01:54 cpan_mojo Minion-Backend-SQLite-0.004 by DBOOK https://metacpan.org/release/DBOOK/Minion-Backend-SQLite-0.004
01:58 mattp_ Grinnz_: sqlite_busy_timeout = 0 or N seconds might yield a decent boost
02:01 mattp_ rather than sleeping for a full second. im not sure how often that dequeue with SQLITE_BUSY
02:06 Grinnz that just seems to cause it to error out when trying to access in parallel
02:07 Grinnz there isn't a lot of info about sqlite_busy_timeout especially in relation to wal mode
02:11 mattp_ Grinnz: what error is thrown?
02:12 Grinnz "database is locked"
02:12 Grinnz oh: "specifying zero milliseconds for the timeout restores the default behavior." wonder which default it's referring to
02:15 Grinnz seems to be same result with any timeout value
02:15 mattp_ what happens if you set to 10000?
02:16 Grinnz that worked
02:16 mattp_ any faster? if not screw it I guess :)
02:17 Grinnz not that i can tell, with the variance in my tests
02:17 asarch joined #mojo
02:20 Grinnz I really want to figure out why the parallel testing throws errors when I don't disconnect the parent process, but debugging DBD::SQLite is almost as painful as debugging DBD::mysql
02:21 Grinnz and it's probably something in the sqlite library itself
02:23 mattp_ lolya nevermind re; sqlite_busy, looks like the internal implementation is just usleep'ing the same way you are
02:24 Grinnz well the usleep in the minion backend is waiting for more jobs, not for the database
02:25 mattp_ ah
02:25 Grinnz there's not really any protection against queries returning busy, I guess I should consider adding that
02:26 Grinnz the trouble is nothing that talks about SQLITE_BUSY says what it does by default, or in wal mode
02:27 sri mattp_: on a macbook air i can dequeue 4500 jobs/s
02:28 sri with 4 workers
02:29 Grinnz and DBD::SQLite doesn't discuss it whatsoever
02:30 sri these days i tend to benchmark roundtrips though
02:30 mattp_ Grinnz: sqlite3.c isnt too bad. just ctrl-f  for busy_timeout
02:30 Grinnz maybe when I get really bored again :)
02:30 sri benchmark i use currently https://gist.github.com/anonymous/8cb045a84edb3fb74aa8
02:31 Grinnz cool
02:32 mattp_ it looks like default behavior is immediate SQLITE_BUSY if lock exists, if you set timeout to N seconds it will usleep / retry until timeout
02:32 disputin joined #mojo
02:32 noganex joined #mojo
02:34 disputin1 joined #mojo
02:34 disputin joined #mojo
02:35 Grinnz something must be setting it to not default behavior, then
02:35 Grinnz because it's not throwing any errors by default
02:35 sri the repetitions really help with getting a feel for performance degradation when the queue gets more diverse, that's how i got to the new index :)
02:36 disputin joined #mojo
02:37 Grinnz_ http://paste.fedoraproject.org/341366/81822491/
02:40 sri usually when i have repetitions getting faster it's still warming up caches
02:41 sri not sure how that works with sqlite
02:42 Grinnz I think the disk access on this machine is just really terrible and sporadic
02:46 sri for a file backed database 100/s is really really good though
02:46 Grinnz I'm still running the one without the magic index :P
02:46 sri DBM::Deep was incapable of finishing these benchmarks
02:46 sri it just locked upand refused to do anything if i remember correctly
02:46 Grinnz going to try them on my other machine, which should have less flaky disk access
02:46 Grinnz_ http://paste.fedoraproject.org/341367/82637145/ from the first machine, minus the new index
02:46 Grinnz_ big difference
02:47 Grinnz_ http://paste.fedoraproject.org/341368/82797145/ from my other server with the index, LOL
02:47 Grinnz_ ok not running benchmarks on this server anymore
02:54 mcsnolte joined #mojo
02:58 sri lol
02:59 sri those are amazing results
03:00 Grinnz I think it's creating the temp database in /tmp, which is actually a tmpfs (in-memory) on that server, lol
03:00 Grinnz so that would explain it
03:00 sri ah
03:01 Grinnz well, i can test that easily enough by using a non temp file
03:02 Grinnz_ http://paste.fedoraproject.org/341374/14581837/
03:02 Grinnz_ only a bit slower on dequeueing, but the enqueueing is way slower lol
03:03 Grinnz_ lots of writing i guess
03:04 disputin1 joined #mojo
03:09 mattp_ Grinnz_: does mojo::db cache prepared sth?
03:09 Grinnz you mean Mojo::SQLite?
03:09 Grinnz yes
03:10 Grinnz_ https://metacpan.org/source/DBOOK/Mojo-SQLite-0.021/lib/Mojo/SQLite/Database.pm#L109
03:11 Grinnz Mojo::Pg also does, but Mojo::mysql doesn't (mysql cached handles are weird)
03:26 noganex_ joined #mojo
03:29 simcop2387 pink_mist: got pulled off it for a bit for something higher priority but it waas started to switch to ForkCall.  Has a nicer API
03:32 McA joined #mojo
03:35 zivester joined #mojo
04:43 inokenty-w joined #mojo
04:54 melo joined #mojo
04:57 aborazmeh joined #mojo
05:14 melo1 joined #mojo
05:19 melo joined #mojo
05:20 melo1 joined #mojo
05:31 melo joined #mojo
05:39 z\ joined #mojo
05:50 irqq joined #mojo
06:08 PryMar56 joined #mojo
06:46 ichi joined #mojo
06:59 dod joined #mojo
07:04 dod joined #mojo
07:28 Guest-quest joined #mojo
07:28 lsm joined #mojo
07:38 salva joined #mojo
07:38 Vandal joined #mojo
07:43 kes joined #mojo
07:50 kes It is not critical, because problem resolved in client code without problem, but for code: $v->input( $block->{ $id } ); and $v->optional( "name" ) will result: Can't use an undefined value as a HASH reference at lib/lib/perl5/Mojolicious/Validator/Validation.pm line 69
07:51 kes if $block->{ $id } not exists.
08:04 cpan_mojo MojoX-Mysql-0.20 by KOSTYA https://metacpan.org/release/KOSTYA/MojoX-Mysql-0.20
08:20 noganex joined #mojo
08:28 trone joined #mojo
08:34 trone joined #mojo
08:49 melo joined #mojo
08:50 melo1 joined #mojo
08:52 melo joined #mojo
09:15 nicomen joined #mojo
09:21 melo1 joined #mojo
09:25 cpan_mojo XML-Compile-SOAP-Mojolicious-0.02 by MARKOV https://metacpan.org/release/MARKOV/XML-Compile-SOAP-Mojolicious-0.02
09:30 cpan_mojo MojoX-Mysql-0.21 by KOSTYA https://metacpan.org/release/KOSTYA/MojoX-Mysql-0.21
09:33 melo joined #mojo
09:59 trone joined #mojo
10:02 absolut_todd joined #mojo
10:05 dhg joined #mojo
10:20 dh7320 joined #mojo
10:38 anparker joined #mojo
10:50 janus joined #mojo
11:25 dod1 joined #mojo
11:31 martin joined #mojo
11:36 kaare joined #mojo
11:55 dhg joined #mojo
12:04 dotan joined #mojo
12:09 dh7320 joined #mojo
12:16 dhg joined #mojo
12:39 dh7320 joined #mojo
12:40 mdom Okay, probably a stupid question but i can't wrap my head around. I have an application with Mojolicious::Lite and Minion. I have a helper add_user ... i call it in a route with $controller->adduser(...), but now i also need to call it in a task, with $job->app->adduser
12:40 mdom wait a sec
12:55 disputin joined #mojo
12:57 mdom never mind... :)
12:59 neilhwatson joined #mojo
13:11 neilhwat1on joined #mojo
13:12 neilhwat1on joined #mojo
13:15 asarch joined #mojo
13:23 disputin joined #mojo
13:35 punter joined #mojo
13:36 zivester joined #mojo
13:46 dhg joined #mojo
13:52 disputin1 joined #mojo
13:55 cpan_mojo Mojo-Weixin-1.0.7 by SJDY https://metacpan.org/release/SJDY/Mojo-Weixin-1.0.7
14:08 mcsnolte joined #mojo
14:16 arthas joined #mojo
14:29 lluad joined #mojo
15:11 PryMar56 joined #mojo
15:12 irqq joined #mojo
15:14 punter joined #mojo
15:21 zivester joined #mojo
15:57 anparker joined #mojo
16:53 dotan joined #mojo
17:09 melo joined #mojo
17:42 Mikey joined #mojo
17:43 claudio joined #mojo
17:56 dod joined #mojo
18:05 trone joined #mojo
18:06 sri django joins the real-time web party too https://blog.heroku.com/archives/2016/3/17/in_deep_with_django_channels_the_future_of_real_time_apps_in_django
18:07 jberger good luck with that
18:08 sri looks like they are using https://github.com/joewalnes/reconnecting-websocket
18:08 jberger lots of things use tat
18:08 jberger that
18:11 sri and require redis
18:12 jberger sri: any opinion on this: https://github.com/kraih/mojo/pull/937
18:12 jberger I seem to recall that line being discussed at length
18:13 sri meh
18:13 jberger but again, I haven't use the validation framework so I'm always at a loss
18:14 Mikey hi that was me.  it broke between 6.40 and 6.41.
18:14 sri stopped looking when i saw the 100+ char lines... don't want to be the bad guy criticizing style again ;p
18:14 Mikey i'll re-do it however you want it if you say you'll accept it
18:14 sri oh, now i am the bad guy again :(
18:14 * sri vanishes
18:14 Mikey whatever i don't have hurt feelings.
18:15 Mikey code style is important and i kinda knew i blew it... but figured no matter what i did first pass it wouldn't be up to snuff
18:15 jberger Mikey: there is an included .perltidy
18:16 Mikey oh thats handy.  i'll run it next time, my bad.
18:16 jberger so write it however you like and then perltidy it
18:16 jberger Mikey: can you do me a favor and find the commit that changed the behavior?
18:16 Mikey sure.
18:16 jberger I'd like to see why the change was made
18:17 Mikey im guessing it had to do with adding filters
18:17 Mikey and perhaps a filter made more values than the input
18:17 Mikey but i'll double check one sec
18:20 Mikey https://github.com/kraih/mojo/commit/1a3ccef03e3e03aa316e74d6cad5ce68ee3ac729 broke it
18:21 Mikey instead of assigning input to output unless.. it does a thing where it counts the values of @input and if it's only a single value array it treats it as a scalar otherwise it returns a reference to @input
18:22 Mikey i was using the validator stuff to validate JSON sent over the websocket.. so my array refs can't just become strings
18:22 anparker joined #mojo
18:22 Mikey i will monkey patch if need be but thought i'd try this route first
18:23 sri is "disarrayed" a real word?
18:24 Mikey no i dont think so.
18:24 * Mikey puts on the dunce cap.
18:31 sri wonder how long it will take for other languages to copy this badly https://channels.readthedocs.org/en/latest/asgi.html
18:35 Grinnz_ "disarrayed" is a word, but it doesn't have anything to do with arrays :P
18:35 Mikey well the pun was intended
18:35 Mikey if you catch my drift
18:38 Mikey but again i will delete the request and shape it however you want if you'll agree that the substance is something you agree with.  'not disarrayed' can become 'right result' the same as the others.
18:45 mitya joined #mojo
18:51 sri now i know what django channels reminded me of, it works like mongrel2
18:51 sri they want to turn all protocols into pub/sub channels
18:57 sri it's a cute idea
19:00 sri much better idea than rails actioncable
19:20 Rubes joined #mojo
19:36 cfedde If I invert control far enough then I wont need any of my own.
19:36 jberger Mikey: I usually dislike magical changes of type (array to scalar) but we need to consider the primary use as multiple values of the same key to a form
19:38 Mikey so the way it worked before was just accidental
19:38 Mikey or coincidental rather
19:38 Mikey i understand.  thanks for your consideration.
19:42 jberger I'm not even discouraging if we/you can show that it doesn't break that case
19:42 Mikey all the other tests passed?
19:42 Mikey if the ample test coverage there doesn't cover the "primary use" then, i'm at a loss for how to show that it doesn't break that case
19:44 jberger Mikey: the problem is that I'm least familiar with the validation stuff
19:44 jberger it happens that I don't use it in anything I do
19:44 jberger maybe if you review it with someone else (batman, Grinnz_?) just as a sanity check
19:44 Mikey at this point i'm wishing i'd have taken the same approach as you haha
19:45 Mikey nah it's moot, monkey patch done.  issue resolved.
19:45 Mikey thanks again
20:04 kaare joined #mojo
20:25 sri fwiw, i might have given it a +1 if the code wasn;t ugly
20:35 Grinnz_ this looks neat, i wonder if anything uses it yet https://tools.ietf.org/html/rfc7239
20:35 sri think that came up like a year ago
20:36 sri nobody every used it as far as i'm aware
20:36 Mikey the de-facto standard for that is haproxy's proxy protocol which does that at a lower level than http
20:37 Mikey http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt
20:39 sri Mikey: umm, your patch doesn't have to be ugly
20:39 Mikey what patch?
20:39 sri your pull request earlier
20:40 Mikey ahh i said i'd make it pretty right off the bat.
20:41 Mikey ok, for you sri, i'll take another swing at it.
20:47 sri you can even make it more robust
20:48 Mikey ok.. let me run this by you to save time
20:48 Mikey i'd like to just check ref $input eq 'ARRAY' one time
20:49 Mikey and set it as a variable, and then use if/else instead of the tertiary
20:50 sri this is what i had in mind https://gist.github.com/anonymous/c06089084c9707036c9d
20:50 sri fixes a few edge cases
20:51 sri foo => [] is especially interesting, since it ends up with {foo => undef} in output
20:51 sri which happens to pass the ->is_valid check
20:51 Mikey ahh yeah i hadn't thought of that one
20:51 Mikey yeah much better
20:51 Mikey just do yours.  i'm not after the glory, just the working code :)
20:52 sri performance of the ref check seems irrelevant
20:52 Mikey *nod* i agree
20:52 Mikey probably just as expensive as checking a boolean
20:52 Mikey maybe a little more ram
20:54 Mikey i am not able to write code as succinct as yours, though I can read it well enough.
20:54 Mikey anyway this is much closer to the old $output = $input unless ...
20:54 Mikey which is what im after
20:55 sri just take this patch and make a pull request for it :) https://gist.github.com/anonymous/73851aad09f016b92f36
20:55 Mikey a gift!
20:55 sri still your patch, i only cleaned it up
20:55 Mikey thanks, sri :)
20:57 sri wonder if there's a nicer way to write "&& !grep { !length } @input"
20:58 Grinnz_ if it's just a binary, List::Util::any (is mojo requiring a new enough List::Util?)
20:58 sri we have no List::util dep yet
20:58 Grinnz_ ah
20:59 sri 5.20 is very new
20:59 pink_mist List::Util just got ::uniq! looking forward to it being in a perl near me :P
21:00 Grinnz_ won't be in perl till 5.26, but i might be bumping a cpan dep or two :P
21:00 sri so new it's not on cpan yet?
21:00 pink_mist oh not in 5.24?
21:00 Grinnz_ 5.24 is in code freeze
21:01 Grinnz_ https://metacpan.org/release/PEVANS/Scalar-List-Utils-1.43_005
21:01 pink_mist https://metacpan.org/pod/release/PEVANS/Scalar-List-Utils-1.43_005/lib/List/Util.pm :P
21:02 sri oh, having an xs version for that is nice
21:04 Rubes joined #mojo
21:05 good_news_everyon joined #mojo
21:05 good_news_everyon [mojo] kraih pushed 2 new commits to master: https://git.io/var6h
21:05 good_news_everyon mojo/master 2465c38 Michael Gregorowicz: fix a bug where arrayrefs were being changed to scalars
21:05 good_news_everyon mojo/master 15c421d Sebastian Riedel: Merge pull request #938 from mgregoro/master...
21:05 good_news_everyon left #mojo
21:06 Mikey thanks!!
21:06 ichi joined #mojo
21:09 good_news_everyon joined #mojo
21:09 good_news_everyon [mojo] kraih pushed 1 new commit to master: https://git.io/var1Z
21:09 good_news_everyon mojo/master 2857428 Sebastian Riedel: test errors too
21:09 good_news_everyon left #mojo
21:10 sri thanks for bringing it up
21:15 good_news_everyon joined #mojo
21:15 good_news_everyon [mojo] kraih pushed 1 new commit to master: https://git.io/varDq
21:15 good_news_everyon mojo/master 89c78b2 Sebastian Riedel: test a few more special cases
21:15 good_news_everyon left #mojo
21:16 punter joined #mojo
21:20 good_news_everyon joined #mojo
21:20 good_news_everyon [mojo] kraih pushed 1 new commit to master: https://git.io/varyI
21:20 good_news_everyon mojo/master cf9d51c Sebastian Riedel: handle undefined values more gracefully
21:20 good_news_everyon left #mojo
21:41 sri i wasn't sure if undef support was really necessary
21:41 sri but, at the end of the day it's a validator, it should be able to handle whatever data you throw at it
22:14 Rubes joined #mojo
22:16 Rubes joined #mojo
22:21 sri MAKE PERL GREAT AGAIN
22:22 bpmedley I believe perl is regaining mindshare again.
22:23 sri </trump meme>
22:24 Grinnz_ build a wall to keep the PHP out, and bomb all the javascript?
22:24 sri and make the PHP pay for it!
22:43 Rubes joined #mojo
22:52 marty joined #mojo
23:36 chorny joined #mojo
23:55 jberger Mikey++ sri++
23:57 * jberger keeps out the Ruby refugees

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary