The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2014-11-18

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 sri a multicolumn index of (created, priority DESC) makes a little bit of a difference, hmm
00:08 sri i guess the interest for job queues and postgres is not very big here atm. :o
00:09 * sri shuts up
00:59 bpmedley sri: http://explain.depesz.com/s/VqRk
01:00 bpmedley Have you tried: CREATE INDEX an_index ON minion_jobs (priority DESC, created ASC);
01:04 bpmedley As an aside, that’s a weird cloud as the favorite icon for that site.. ;)
01:08 sri wtf?
01:08 sri does order actually matter?
01:08 bpmedley Yes
01:08 sri OMG
01:08 bpmedley http://www.postgresql.org/docs/9.3/static/indexes-ordering.html
01:09 bpmedley An important special case is ORDER BY in combination with LIMIT n: ...
01:10 sri omg... this boost my benchmark from 5 dequeues per second to 1000
01:10 sri at 100k jobs
01:10 bpmedley Not to shabby, eh?
01:11 sri how did i not know the order in multi column indexes matters :(
01:13 sri and when i say order... i mean (created ASC, priority DESC) vs (priority DESC, created ASC)
01:14 bpmedley I understand.  The LIMIT throws things for a loop and sometimes more eyeballs are needed.. :)
01:16 sri and the index is only used after around 50k jobs
01:19 bpmedley What other points to consider are there for Minion before 1.0?
01:23 sri these are just optimizations, no blockers
01:23 sri it depends more on Mojo::Pg becoming stable
01:26 sri indexes on delayed, state and task don't seem to make much of a difference
01:27 sri explain analyze uses them a few times, and then gives up falling back to plain filtering because it's faster, even at 100k jobs
01:28 sri hmm, and now it jumps back
01:41 sri i've been using a rather naive benchmark https://gist.github.com/anonymous/d98d2cdef86b650c5ae1
01:41 sri but it did show the massive gain with the right index for order by limit
01:42 sri even if 50k jobs have the wrong task, it doesn't use an index on the task field
01:43 bpmedley I’m not sure what the magic numbers are with Postgres for when an index is used, or no.
01:47 sri even if i get it to use the index, results are pretty much the same
01:47 sri anyway, order by limit index is not in master
01:48 sri s/not/now/
01:56 sri bpmedley++
02:23 klapperl_ joined #mojo
02:43 Grinnz joined #mojo
03:12 davido_laptop joined #mojo
03:46 damaya joined #mojo
04:05 basic6_ joined #mojo
04:08 tbushell joined #mojo
04:08 gtodd joined #mojo
04:24 absolut_todd joined #mojo
04:26 alnewkirk joined #mojo
04:45 sujithm joined #mojo
04:45 sujithm_ joined #mojo
04:46 irq joined #mojo
04:52 jberger bpmedley++ cool post
04:52 bpmedley Thanks.  I think I need to use your handy gist viewer.
05:12 zivester joined #mojo
05:18 inbioz joined #mojo
05:20 sri i guess 260 jobs per second on my little macbook air is not too bad
05:20 sri (dequeue and finish)
05:20 bpmedley Does the job do anything or just return?
05:21 sri $worker->dequeue(0)->finish
05:21 bpmedley That’s quite a lot of jobs per second.. :)
05:21 sujithm joined #mojo
05:22 sri enqueue seems to scale pretty well, 1800 enqueue per second with one worker, 3150 per second with two workers
05:23 sri dequeue does not scale so well, the row locking is noticeable
05:25 sri funny thing, $worker->dequeue(0)->finish while 1 is faster than $worker->dequeue(0) while 1
05:26 sri less congestion
05:28 sri reconsidering advisory locks might be worth it for setups with more 100 workers, but otherwise i think we're fine now
05:28 sri *+than
06:01 memowe joined #mojo
06:04 * sri would have liked more postgresql talk around here, too bad there's not more participants
06:12 bpmedley http://pastie.org/9726913 <— What am I doing wrong?  The command “joy” doesn’t show up in the “COMMANDS” list.
06:17 sri the command system only finds classes in actual .pm files
06:18 bpmedley Thx, glad I asked before spending much more time on that.. :)
06:23 sujithm joined #mojo
06:28 Shaeto joined #mojo
06:47 zivester joined #mojo
07:07 dod joined #mojo
07:11 dod joined #mojo
07:11 Vandal joined #mojo
07:15 dod joined #mojo
07:19 inbioz joined #mojo
07:22 rem_lex|pivo joined #mojo
07:28 Lee joined #mojo
07:49 sugar joined #mojo
07:51 rawler joined #mojo
07:56 dod joined #mojo
08:02 neyasov__ joined #mojo
08:06 Lee joined #mojo
08:08 trone joined #mojo
08:13 inbioz joined #mojo
08:14 basiliscos joined #mojo
08:19 woz joined #mojo
08:23 irq joined #mojo
08:28 denis_boyun joined #mojo
08:41 spikkie joined #mojo
08:45 dp_ joined #mojo
08:47 marmez joined #mojo
08:55 irq_ joined #mojo
09:01 amon joined #mojo
09:01 moritz joined #mojo
09:01 augensalat joined #mojo
09:01 omega joined #mojo
09:01 kablamo joined #mojo
09:01 panzana` joined #mojo
09:01 yuni joined #mojo
09:01 adhoc joined #mojo
09:01 rwp joined #mojo
09:01 zivester joined #mojo
09:01 yusukebe joined #mojo
09:01 tardisx joined #mojo
09:01 HtbaaPi joined #mojo
09:01 salparadise joined #mojo
09:01 basic6_ joined #mojo
09:01 stokachu joined #mojo
09:01 Vandal joined #mojo
09:01 oalders joined #mojo
09:01 ua joined #mojo
09:01 xtetsuji joined #mojo
09:01 dod joined #mojo
09:01 charsbar joined #mojo
09:01 Dandre joined #mojo
09:01 jwang joined #mojo
09:01 marmez joined #mojo
09:01 sugar joined #mojo
09:01 throughnothing joined #mojo
09:01 rawler joined #mojo
09:02 DaTa joined #mojo
09:02 gtodd joined #mojo
09:02 sweet_kid joined #mojo
09:02 robinsmidsrod joined #mojo
09:02 bjoernfan joined #mojo
09:02 phillipadsmith joined #mojo
09:02 chansen joined #mojo
09:02 jnbek joined #mojo
09:02 bowtie joined #mojo
09:03 Lee joined #mojo
09:06 saki joined #mojo
09:06 hahainternet joined #mojo
09:11 romel joined #mojo
09:11 rblackwe joined #mojo
09:11 cfedde joined #mojo
09:11 maze joined #mojo
09:12 mtj- joined #mojo
09:16 aleksey joined #mojo
09:16 sugar_ joined #mojo
09:18 neyasov joined #mojo
09:22 fhelmber_ joined #mojo
09:50 dp_ joined #mojo
10:06 inbioz joined #mojo
10:07 dod joined #mojo
10:44 basic6__ joined #mojo
10:46 basic6__ is "my $json_bytes = j($hash); render(data => $json_bytes, format => 'json')" (assuming use Mojo::JSON qw(j)) the same as render(json => $hash) ?
10:49 basic6__ oh never mind i've overlooked this line in the docs, which says $c->render(data => encode_json({test => 'I ♥ Mojolicious!'}));
10:50 Snelius render(json => { key => 'value'});
10:51 da5id joined #mojo
10:54 da5id joined #mojo
11:02 absolut_todd joined #mojo
11:08 dod joined #mojo
11:13 Dandre joined #mojo
11:18 denis_boyun joined #mojo
11:52 inbioz joined #mojo
12:28 ashimema joined #mojo
12:42 jberger sri: the new job currently uses mysql, but given what I've been reading and the general sentiment here, if they can switch to pg, I'm going to recommend it
12:43 jberger the fact that mojo::pg exists well gradually increase its usage around here, as mango did for mongo
12:44 Kripton joined #mojo
12:54 bzero Hello. How to use I18N plugin from another script? I want to use Mojolicious helpers for Mojo::Template from separated script.
13:00 neilhwatson joined #mojo
13:05 bzero $app->l() does not work properly. I18N plugin does not load App::I18N::en
13:28 human39 joined #mojo
13:32 Zx3 joined #mojo
13:39 neyasov joined #mojo
13:39 Grinnz joined #mojo
13:47 doby joined #mojo
13:47 aleksey left #mojo
14:06 Dandre joined #mojo
14:06 neyasov_ joined #mojo
14:06 basiliscos joined #mojo
14:17 Grinnz_ joined #mojo
14:39 sugar__ joined #mojo
14:43 Claudio joined #mojo
14:45 basic6 joined #mojo
14:47 Mikey joined #mojo
14:58 purl joined #mojo
15:08 sri think my postgresql questions are down to only one... if there's a better way to handle pg_notifies errors than an eval https://github.com/kraih/mojo-pg/blob/master/lib/Mojo/Pg/Database.pm#L150
15:08 sri (or rather, a way to handle async connection errors)
15:12 inbioz joined #mojo
15:12 sh4 joined #mojo
15:13 sujithm joined #mojo
15:15 sri review of the minion schema would also be appreciated, regarding performance and future prooofness https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L411
15:15 * sri would really like to push both to 1.0
15:20 KCL_ joined #mojo
15:30 Ralesk joined #mojo
15:37 arthas joined #mojo
15:43 davido_ joined #mojo
15:50 sujithm joined #mojo
16:02 Ptolemarch joined #mojo
16:03 Ptolemarch joined #mojo
16:19 disputin joined #mojo
16:30 neyasov_ joined #mojo
16:38 bzero https://gist.github.com/su-bzero/e7169825bbc4bd287c74 That's what I mean... Now it works. :)
16:39 Ptolemarch joined #mojo
16:43 sugar joined #mojo
16:57 rawler joined #mojo
17:06 ua left #mojo
17:16 niczero joined #mojo
17:26 denis_boyun joined #mojo
17:51 Ptolemarch joined #mojo
18:02 batman anyone want to give me feedback on the new assetpack extension? https://github.com/jhthorsen/mojolicious-plugin-browserify
18:03 batman it enables you to do var Mod = require("./my-module.js"); in your javascript
18:03 batman it's cheating though, by using the node app "browserify"...
18:04 batman no idea how to avoid that, without doing some crazy hacking myself :/
18:04 disputin joined #mojo
18:14 sugar_ joined #mojo
18:21 good_news_everyon joined #mojo
18:21 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/ck1gDg
18:21 good_news_everyon mojo/master 0bff738 Sebastian Riedel: a few more routing tests
18:21 good_news_everyon left #mojo
18:23 good_news_everyon joined #mojo
18:23 good_news_everyon [mojo] kraih tagged v5.62 at adc29e3: http://git.io/JZGQhw
18:23 good_news_everyon left #mojo
18:24 good_news_everyon joined #mojo
18:24 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/-tqGXw
18:24 good_news_everyon mojo/master 53060ab Sebastian Riedel: bump version
18:24 good_news_everyon left #mojo
18:27 inbioz joined #mojo
18:31 neyasov_ joined #mojo
18:33 fhelmber_ joined #mojo
18:38 Ptolemarch joined #mojo
18:38 rawler joined #mojo
19:01 sugar__ joined #mojo
19:06 denis_boyun joined #mojo
19:15 dod joined #mojo
19:19 dotandimet joined #mojo
19:23 meshl joined #mojo
19:36 meshl I have an encoding issue... In an AJAX call rendering JSON via render(json => $stringarray) I have some strings of the array with the Perl UTF-8 flag set (Encode::is_utf8 returning true) and others not. The data is consumed by the JS select2 plugin. Accentuated chars are not properly encoded by the browser. Any idea of what I could try/do to debug/resolve this issue?
19:37 rawler joined #mojo
19:38 meshl Sorry I should have said "decoded" or rather "rendered" by the browser...
19:46 hernan604 whats the metacharset of your page
19:48 meshl utf8
19:48 tbushell joined #mojo
19:48 amon joined #mojo
19:48 priodev joined #mojo
19:49 meshl I also use utf8 in my modules and if I hardcode the data (instead of doing a DB query) the problem vanishes
19:49 disputin joined #mojo
19:50 hernan604 when you inspect the regex,
19:50 hernan604 are the characters garbled:
19:50 hernan604 ?
19:51 meshl I do not use any regex
19:52 hernan604 the request*
19:52 hernan604 typo sorry
19:54 meshl how would you inspect the request? I am using DBIx::NoSQLite BTW
19:56 rawler joined #mojo
20:02 hernan604 well you said its a ajax request
20:03 hernan604 you could use firebug and check if characters are garbled
20:04 hernan604 or you already know they are garbled on the terminal
20:05 irq joined #mojo
20:07 chansen meshl: usage of the utf8.pm pragma only tells the interpreter that your literal strings is in UTF-8
20:08 rawler joined #mojo
20:09 meshl I knew this chansen. This only had an effect when I was hardcoding my strings for testing. I just can't figure out where the problem lies.
20:10 meshl hernan604: when I call my ajax URL directly I already see garbled chars in the JSON rendered by the browser
20:10 hernan604 cool
20:10 meshl so basically this tells me it is not a select2 / JS issue
20:10 hernan604 yeah
20:11 hernan604 maybe you are missing some:  pg_enable_utf8
20:11 hernan604 or something equivalent for DBIx::NoSQLite
20:12 hernan604 meshl: try opening that page
20:12 hernan604 with garbled response
20:12 batman Make sure you have the correct collate on the table (at least with MySQL)
20:12 hernan604 and manually change the browser charset, try force it to iso
20:12 batman Or else, you have to decode to avoid duble encoding... Or something... Haven't figured that part out yet :/
20:12 hernan604 see if it renders correctly.. if it does, its responding in that charset
20:13 basiliscos joined #mojo
20:13 hernan604 yeah could be doubleencoding also
20:13 meshl under DBIx::NoSQLite lies SQLite - someone knows about encoding config params of that one?
20:13 chansen meshl: then you probably have a issue where one of your points asume ISO8859-X
20:14 hernan604 sqlite_unicode
20:14 batman Or you get ISO8859 with UTF8 flag set.
20:14 batman Encoding <3
20:14 meshl sure chansen, but how could the string be flagged UTF8 correctly and not be considered as-is?
20:14 batman J/k
20:15 hernan604 http://search.cpan.org/~ishigaki/DBD-SQLite-1.44/lib/DBD/SQLite.pm#DRIVER_PRIVATE_ATTRIBUTES
20:15 batman meshl: it's just bytes with a flag...
20:15 hernan604 browser trusts and renders
20:18 chansen meshl: by decoding it using a appropriate encoding, UTF-8?
20:19 abhishekisnot joined #mojo
20:24 chansen meshl: Unlike many other languages, perl only have one type to represent characters
20:24 Ptolemarch joined #mojo
20:27 chansen batman: are you trying to confuse developer's/users that are using Perl, or you don't understand perls model for strings?
20:31 meshl hmmmm, setting the sqlite_unicode does not seem to make a difference
20:33 chansen Probably the latter, there is a lot of people that claim to understand how Perl models Unicode
20:35 * moritz once thought he understood, and now isn't so sure anymore
20:37 meshl Alright, the right setting was $NOSQLDB = DBIx::NoSQLite->connect($dbfilename); and then $NOSQLDB->dbh->{sqlite_unicode} = 1;
20:37 meshl Everything's fine now
20:38 hernan604 meshl: are you sure the data is in utf8 ?
20:38 hernan604 check the data you are saving
20:38 hernan604 then check whats inside that sqlite
20:38 hernan joined #mojo
20:38 hernan back
20:38 hernan604 meshl: are you sure the data is in utf8 ?
20:38 purl i already had it that way, hernan604.
20:38 neyasov_ joined #mojo
20:38 hernan604 check the data you are saving
20:38 hernan604 (before save)
20:38 hernan604 then check whats inside that sqlite
20:39 chansen moritz: how can blame you, even the most seasoned p5p developer gave no clue/understanding of UTF-8
20:40 * chansen s/gave/have/
20:41 dotandimet joined #mojo
20:42 meshl sorry everyone for such a mess on #mojo... but yeah, encoding has always been a big mess since every element in the SW dev stack works differently and may corrupt the data
20:44 mst meshl: ->connect($dbfilename, undef, undef, { sqlite_unicode => 1 })
20:44 * sri never encountered encoding issues with postgresql and mongodb
20:47 * sri really hopes this will end up killing all other CAs https://letsencrypt.org
20:47 hernan604 hopefully
20:47 purl hopefully is lightweight.
20:48 sri bit odd google is not participating
20:48 meshl mst: nope, but yeah I trust there should be a quicker way ;)
20:48 hernan604 should work at least in mozilla browsers
20:48 mst meshl: that would work with DBI->connect and also works with e.g. DBIx::Class' ->connect
20:49 mst meshl: if it doesn't work for DBIx::NoSQLite then I would suggest filing a feature request
20:49 sri mozilla marketshare is shrinking last i heard
20:49 mst meshl: since poking the dbh directly is kinda fuckin' ugly
20:50 hernan604 sri: i dont like how chrome handles the url box.. everything you type there is sent for google autocomplete
20:50 hernan604 plus other stuff im not aware
20:50 hernan604 they have all my email.. i think its enough :P
20:51 hernan604 but yeah its a great browser
20:51 hernan604 and i dont search for weird stuff, usually
20:56 sugar joined #mojo
20:59 hernan604 ha ! that guy is brasilian
20:59 hernan604 lol
20:59 hernan604 didnt know
20:59 hernan604 im br too
20:59 Grinnz_ question: if one does my $parent = $r->get(...); $parent->any(...); will that work, and restrict the latter route to GET?
21:01 sri perl -Mojo -E 'app->routes->get()->any({text => "works!"}); app->start' get -M POST /
21:02 Grinnz_ seems it does :)
21:26 tbushell joined #mojo
21:26 priodev joined #mojo
21:31 basiliscos joined #mojo
21:37 Ptolemarch joined #mojo
21:57 powerman left #mojo
22:13 disputin joined #mojo
22:22 jnbek joined #mojo
22:58 D4RK-PH0ENiX joined #mojo
23:09 denis_boyun joined #mojo
23:23 Averna joined #mojo
23:29 d4rkie joined #mojo
23:31 * sri thinks he might just bump Mojo::Pg and Minion to 1.0
23:31 disputin joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary