The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2014-11-17

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:37 * sri releases minion with advisory locks and notifications o/
00:52 laouji joined #mojo
00:58 rem_lex|pivo joined #mojo
01:21 atom joined #mojo
01:21 atom if this is my helper: helper redis => sub { state $r = Mojo::Redis2->new };
01:22 atom this means Mojo will be using the same Mojo::Redis2 object every time?
01:23 sri yes
01:24 atom cool, thank you
01:28 zivester joined #mojo
01:36 davido_laptop joined #mojo
01:42 sri damn, the advisory lock falls apart with an order by and lots of jobs in the queue :o
02:01 bpmedley sri: Back to the old SQL for _try or something else?
02:01 sri i'm not giving up yet
02:04 sri the old sql sucked pretty bad, it's going to be something better one way or the other
02:23 klapperl joined #mojo
02:32 atom when a websocket closes and triggers fisnish, I see "$code" 1001 ( going away ) with "$reason" being ''
02:32 atom what is the $reason for, what does it show?
02:33 atom $ws->finish(1003 => 'Cannot accept data!');  ahh so it's when you call finish you can put a reason in there?
02:49 mgrimes joined #mojo
02:52 sri the quick fix https://github.com/kraih/minion/commit/b1497a65db7e139e5cb405c3367d929fe4addaba
02:53 sri really too bad the order by kills advisory locks
02:54 sri i think there might be a possibility to make it work with recursion, but i'm not good enough at postgres yet
03:04 sri that version is still at least twice as fast as the old (before advisory locks)
03:15 woz joined #mojo
03:23 crab what should i read to understand the problem you're discussing?
03:28 * crab reads commits
03:31 sri i was trying to make this more efficient https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L200-L211
03:32 sri the method takes a job from the queue, and needs to avoid race conditions (other workers trying to do the same)
03:33 sri i tried adding an advisory lock, so other workers wouldn't have to wait for the exclusive lock on the row
03:35 sri i had a test queue with about 100k jobs, it immediately ran out of shared memory with order by clase
03:35 sri s/clase/clause/
03:35 sri without order by it actually works
03:38 sri i'd be happy about any suggestions for performance improvements though
03:50 crab what error did you get?
03:50 crab (ran out of shared memory)
03:57 crab there's a patch that adds SKIP LOCKED to postgres that would help this case
03:57 crab (just saying)
04:04 sri haha, yea i linked to an article about that earlier ;p
04:05 sri but it's not happening before 9.5
04:05 basic6 joined #mojo
04:28 laouji joined #mojo
05:07 Gedge joined #mojo
05:13 sri this is supposed to actually work, with recursion https://github.com/chanks/que/blob/master/lib/que/sql.rb#L5-L36
05:13 sri but i don't grok it yet
05:14 sri it uses pg_try_advisory_lock() though, which seems sketchy
05:15 sri wrong scope as far as i understand it
05:17 crab nope. that's not sketchy.
05:18 sri why would you lock it for the whole session?
05:19 sri instead of using pg_try_advisory_xact_lock()?
05:21 crab oh, sorry, i misunderstood where you thought the sketchiness was. sure, you can use pg_try_advisory_xact_lock if your code suits. the main disadvantage in that case is that you can't explicitly release the lock. but in this case that doesn't matter.
05:26 crab when i wrote those advisory lock functions, there wasn't any mechanism to release the locks at transaction end.
05:27 rem_lex|pivo joined #mojo
05:58 marcus joined #mojo
06:03 sujithm joined #mojo
06:14 marcus joined #mojo
06:15 woz joined #mojo
06:17 sujithm joined #mojo
06:23 dotandimet joined #mojo
07:03 dod joined #mojo
07:08 sugar joined #mojo
07:14 sugar_ joined #mojo
07:17 marmez joined #mojo
07:24 trone joined #mojo
07:34 basiliscos joined #mojo
07:59 dotandimet joined #mojo
08:03 woz joined #mojo
08:11 Vandal joined #mojo
08:46 trone joined #mojo
08:46 odc joined #mojo
08:47 woz joined #mojo
08:49 denis_boyun joined #mojo
09:04 KCL_ joined #mojo
09:16 basiliscos joined #mojo
09:17 vytas joined #mojo
09:26 basiliscos joined #mojo
09:36 fhelmber_ joined #mojo
09:37 basiliscos joined #mojo
09:39 Shaeto joined #mojo
09:42 dotandimet joined #mojo
09:43 basiliscos joined #mojo
09:46 basiliscos joined #mojo
09:51 neyasov______ joined #mojo
09:54 dp_ joined #mojo
09:58 ua joined #mojo
10:08 basiliscos joined #mojo
10:11 basiliscos joined #mojo
10:13 basiliscos1 joined #mojo
10:16 basiliscos joined #mojo
10:19 basiliscos1 joined #mojo
10:22 basiliscos joined #mojo
10:24 basiliscos1 joined #mojo
10:27 basiliscos joined #mojo
10:29 basiliscos1 joined #mojo
10:32 basiliscos joined #mojo
10:35 basiliscos joined #mojo
10:38 basiliscos joined #mojo
10:40 basiliscos1 joined #mojo
10:42 dotandimet joined #mojo
10:43 basiliscos joined #mojo
10:45 basiliscos1 joined #mojo
10:51 basiliscos joined #mojo
11:01 aleksey joined #mojo
11:10 basiliscos joined #mojo
11:12 basiliscos joined #mojo
11:14 basiliscos1 joined #mojo
11:16 basiliscos joined #mojo
11:18 basiliscos1 joined #mojo
11:21 basiliscos joined #mojo
11:25 basiliscos joined #mojo
11:28 basiliscos1 joined #mojo
11:31 basiliscos joined #mojo
11:34 basiliscos1 joined #mojo
11:36 basiliscos joined #mojo
11:38 sugar__ joined #mojo
11:40 basiliscos joined #mojo
11:43 basiliscos joined #mojo
11:47 basiliscos joined #mojo
11:49 basiliscos1 joined #mojo
11:52 basiliscos joined #mojo
11:55 basiliscos1 joined #mojo
11:56 basiliscos joined #mojo
11:59 basiliscos joined #mojo
12:02 basiliscos1 joined #mojo
12:06 basiliscos joined #mojo
12:07 basiliscos1 joined #mojo
12:11 basiliscos joined #mojo
12:13 basiliscos joined #mojo
12:16 basiliscos1 joined #mojo
12:19 basiliscos joined #mojo
12:22 basiliscos joined #mojo
12:22 janus joined #mojo
12:25 basiliscos joined #mojo
12:28 basiliscos1 joined #mojo
12:31 basiliscos joined #mojo
12:32 aleksey joined #mojo
12:33 basiliscos joined #mojo
12:36 basiliscos joined #mojo
12:40 basiliscos joined #mojo
12:42 basiliscos1 joined #mojo
12:43 Shaeto left #mojo
12:44 basiliscos joined #mojo
12:50 basiliscos joined #mojo
12:53 basiliscos joined #mojo
12:56 basiliscos1 joined #mojo
12:56 ignacio_ joined #mojo
12:58 neyasov______ joined #mojo
13:02 basiliscos joined #mojo
13:07 basiliscos1 joined #mojo
13:09 basiliscos joined #mojo
13:12 basiliscos1 joined #mojo
13:14 basiliscos joined #mojo
13:15 Kripton joined #mojo
13:18 basiliscos joined #mojo
13:22 basiliscos joined #mojo
13:24 basiliscos joined #mojo
13:28 denis_boyun joined #mojo
13:32 basiliscos joined #mojo
13:36 basiliscos joined #mojo
13:39 basiliscos joined #mojo
13:43 basiliscos joined #mojo
13:43 neilhwatson joined #mojo
13:46 basiliscos joined #mojo
13:49 basiliscos joined #mojo
13:53 basiliscos joined #mojo
13:55 basiliscos1 joined #mojo
13:58 basiliscos joined #mojo
14:01 sugar joined #mojo
14:01 basiliscos joined #mojo
14:05 basiliscos joined #mojo
14:08 basiliscos1 joined #mojo
14:11 basiliscos joined #mojo
14:14 basiliscos1 joined #mojo
14:18 basiliscos joined #mojo
14:18 sujithm joined #mojo
14:21 basiliscos joined #mojo
14:24 basiliscos joined #mojo
14:27 basiliscos joined #mojo
14:29 basiliscos1 joined #mojo
14:31 basiliscos joined #mojo
14:32 sri basiliscos: you might want to fix that
14:33 inbioz joined #mojo
14:39 punter joined #mojo
14:50 zivester joined #mojo
14:57 meredith Tweeted by @perlbuzz: Minion is a job queue for Mojolicious http://blogs.perl.org/users/brian_medley/2014/11/using-minion-with-a-rest-api.html
14:58 hasan joined #mojo
14:58 hasan hi guyes
14:59 hasan can someone have a look at http://nopaste.info/26930909b4.html please. I get an inactivity timeout in my foreach loop (line 130).
14:59 hasan my goal is to SELECT ... (line 137) foreach id I have from the first Mojo delay
15:00 hasan aggregate them to my hashref requests. and finally render my template with requests in my stash.
15:03 hasan of course it shall be the non blocking way.
15:04 sri bpmedley++ # that's a nice little example app
15:05 sugar_ joined #mojo
15:11 hasan I guess I'm doing it the wrong way. I think I don't need a second Mojo::IOLoop->delay in my second sub of the first delay, I just can call $delay->begin again and catch my results in a third sub in that first delay, right?
15:11 hasan so the whole second IOLoop is useless I guess.
15:14 sh4 joined #mojo
15:15 sh4 joined #mojo
15:16 tgh joined #mojo
15:22 marty joined #mojo
15:36 sujithm_ joined #mojo
15:56 Ptolemarch joined #mojo
15:57 risugg joined #mojo
16:06 disputin joined #mojo
16:06 marty_ joined #mojo
16:16 Ptolemarch joined #mojo
16:18 disputin joined #mojo
16:22 powerman joined #mojo
16:23 powerman hi. how to get ioloop used with some $c?
16:23 s1037989 joined #mojo
16:24 powerman I need to change default timeout, and this doesn't work when non-default ioloop is used: Mojo::IOLoop->stream($c->tx->connection)->timeout(...)
16:24 powerman i.e. when while testing blocking I/O is used instead of non-blocking
16:36 sri i guess with Minion now using notifications we should be getting a little more real world testing for that part of Mojo::Pg
16:37 sri just wish this eval wasn't necessary https://github.com/kraih/mojo-pg/blob/master/lib/Mojo/Pg/Database.pm#L150
16:56 inbioz joined #mojo
17:01 inbioz joined #mojo
17:05 inbioz joined #mojo
17:46 sugar__ joined #mojo
17:54 sugar joined #mojo
18:08 sugar_ joined #mojo
18:11 Ptolemarch joined #mojo
18:15 gtodd strange how sometimes people devote resources to redevelop a "legacy" web site/web application in Java or a new enterprise supported thingy but leave one crucial perl script untouched ....
18:15 ua left #mojo
18:18 s1037989 joined #mojo
18:22 neilhwatson Lack of perl talent?
18:24 stephanj the app is just the frontend the real buisnesslogic is done by that one script which nobody dares to touch
18:27 neilhwatson A CGI script, running perl 4.
18:30 sugar__ joined #mojo
18:30 disputin joined #mojo
18:40 Ptolemarch joined #mojo
18:40 sugar joined #mojo
18:47 gtodd neil: yeah that is probably the reason
18:47 * sri wonders why no postgres job queue seems to create indexes by default
18:48 gtodd perhaps that one left over "CGI script" that everyone was afraid to touch can be a job for ... Mojo...licious
18:49 denis_boyun joined #mojo
18:50 gtodd sri: I thought newer Pg did something tricky with indexes
18:50 gtodd sri: multi-indexing ?
18:51 gtodd sri: I was told that was good but I don't know what it is :-)
18:52 mst sri: because 90% of everything is shit?
18:52 gtodd that :-D
18:58 powerman sri: sorry, I've already asked a couple of times, but is it ever possible to change stream's default timeout on server-side after receiving incoming request, i.e. do this using Mojolicious::Controller object $c? Mojo::IOLoop->stream($c->tx->connection)->timeout(...) works for normal requests but fails for internal blocking requests executed while running tests.
18:59 powerman I don't see a way to get correct ioloop object used for given $c.
19:06 neyasov______ joined #mojo
19:07 gtodd sri: there is a ruby<->postgresql job queue application called errm "Queue" (?)  I recall reading somewhere that it uses features specific to Pg to avoid a number of issues with indexing ... but I don't know enough about RDBMS or database theory to know if that is relevant to your Mojo-Pg efforts  ... I guess Mojo needs to attract some deep Postgresql hackers to help
19:07 Ptolemarch joined #mojo
19:10 gtodd sri: oops that ruby project is actually called "Que" https://github.com/chanks/que
19:20 amon joined #mojo
19:23 mst gtodd: #postgresql on freenode tends to be awesome.
19:24 gtodd :-)
19:24 irq joined #mojo
19:26 neyasov joined #mojo
19:27 sri gtodd: i linked to that project a few times yesterday ;p
19:27 gtodd doh!
19:27 sri even the magical recursion code it uses
19:28 sri https://github.com/chanks/que/blob/master/lib/que/sql.rb#L5-L36
19:28 sri apparently that's the way to handle massive concurrency
19:29 gtodd :-D ...
19:30 sugar_ joined #mojo
19:31 good_news_everyon joined #mojo
19:31 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/FOhKAA
19:31 good_news_everyon mojo/master 36f64ab Sebastian Riedel: fixed bug in Mojolicious::Routes::Pattern where optional placeholders in nested routes would sometimes not work correctly (closes #709)
19:31 good_news_everyon left #mojo
19:32 neilhwatson joined #mojo
19:34 gtodd sri: http://www.postgresql.org/about/newsarchive/ has mentioned POWA/Dalibo a few times - it uses a Mojo UI for stats on postgres server workload etc. Pg hackers with Mojo are out there I guess
19:35 disputin joined #mojo
19:38 sri we actually have someone on the core team who also hacks on postgres, bu he's not around much these days *hint hint*
19:43 sri crab is literally the guy that added advisory locks to postgres :)
19:44 sri the open source world is small
19:44 gtodd wha?!?! that is good
20:11 dod joined #mojo
20:30 gtodd1 joined #mojo
20:31 Ptolemarch joined #mojo
20:35 rawler joined #mojo
20:52 marty_ joined #mojo
20:59 neyasov_ joined #mojo
21:01 denis_boyun joined #mojo
21:06 neyasov_ joined #mojo
21:08 neyasov__ joined #mojo
21:14 spikkie joined #mojo
21:17 bobkare joined #mojo
21:18 sri so... the postgresql questions is still have are 1) how to make this update better for high concurrency https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L200-L211 2) what indexes to create to make it faster in general, and 3) if DBD::Pg has a better way to handle pg_notifies failures than an eval https://github.com/kraih/mojo-pg/blob/master/lib/Mojo/Pg/Database.pm#L150
21:19 sri s/is/i/
21:22 neyasov__ joined #mojo
21:32 basiliscos joined #mojo
21:42 disputin joined #mojo
22:02 disputin joined #mojo
22:23 Averna joined #mojo
22:23 marty joined #mojo
22:32 Grinnz_ joined #mojo
22:37 Grinnz_ I have a controller package that is already in camelcase and Mojo is trying to use it as an application name even though it's specified as 'controller'. How do I get mojo to stop changing the case of my controller names?
22:39 bobkare joined #mojo
22:54 bobkare joined #mojo
23:02 Grinnz_ I found the problem. Seems mojo really doesn't like when my handler functions are named "handler"
23:03 Grinnz_ bug, or undocumented feature?
23:04 sri what does your log say?
23:05 firnsy Grinnz_: it's documented ... http://mojolicio.us/perldoc/Mojolicious/Controller#stash
23:05 Grinnz_ when I name it "handler", it says 'Routing to application "My::Application::SomeController"'
23:06 Grinnz_ whereas when i changed it it properly says Routing to controller ... and action ...
23:06 sri mojolicious version?
23:07 Grinnz_ 5.60
23:07 Grinnz_ firnsy: i don't see what the reserved keys have to do with a value of "handler"
23:09 firnsy Grinnz_: hmm, i thought you were using functions with a name handler ... i just avoid any of those reserved keywords when i'm in a controller
23:10 Grinnz_ If I change the controller back to a camelcase name, it routes it as an application again
23:10 Grinnz_ even though the handler is named 'dock'
23:12 tianon sri: slightly old news, but thanks for cpanm -M :D <3
23:18 Grinnz_ is there any way I can get mojolicious to use a camelcase controller name?
23:32 firnsy Grinnz_: for Camel::Case::Controller you can do $r->get('/foo')->to('camel_case_controller#foo');
23:33 good_news_everyon joined #mojo
23:33 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/cf9HDg
23:33 good_news_everyon mojo/master 8a40c41 Sebastian Riedel: fixed bug where "handler" was not an allowed name for controller methods
23:33 good_news_everyon left #mojo
23:34 sri should be fixed now
23:34 sri bit of an odd case
23:34 sri the documented feature is http://mojolicio.us/perldoc/Mojolicious/Guides/Routing#Embedding-applications
23:35 bpmedley sri: Thanks for the ++
23:35 Grinnz_ firnsy: it is CamelCase not Camel::Case
23:36 Grinnz_ sri: ah gotcha
23:36 firnsy my bad ... my example was wrong
23:36 firnsy for CamelCase::Controller you can do $r->get('/foo')->to('camel_case-controller#foo');
23:37 sri the camelization process is of course, also documented http://mojolicio.us/perldoc/Mojolicious/Guides/Routing#Special-stash-values
23:38 sri (hint: it's easier than you may think)
23:38 Grinnz_ yes, i should have followed that link to Mojo::Util... didn't occur to me
23:38 sri :)
23:39 Grinnz_ still routing it as an application.
23:39 Grinnz_ when it's camelcase, but it's fine when it's not
23:40 sri that does not make any sense
23:42 Grinnz_ yes it's strange. I copied the module so I have a CamelCase version and a Noncamelcase version, and using the controller "camel_case" leads to Routing to application...
23:42 Grinnz_ wait. think its my fault
23:42 bobkare joined #mojo
23:43 Grinnz_ yeah it was the same problem. I wasn't looking at the original version of the module and it still had a handler() function
23:44 * sri wonders if 10000 jobs are not enough for a minion benchmark... because indexes don't seem to make any difference whatsoever
23:44 Grinnz_ thanks for the help :)
23:45 marmez joined #mojo
23:46 Grinnz_ my next question would be "what if I have modules with underscores in them?" but none of those modules will be used for my application, lol
23:47 sri you'd get tarred and feathered
23:52 sri hmm, from what i've been readin so far an "order by priority desc, created" is pretty shitty to optimize :S
23:59 marty joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary