The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2016-02-05

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:43 sri that reminds me... i still need to try chicago style pizza
00:44 Grinnz_ I prefer it to new york style, though i'm right near new york :P Deep dish ftw
00:46 sri new york style is mostly italian style
00:47 sri (which is great)
01:00 jontaylor joined #mojo
01:00 vicash so Minion requires the Mojo::Pg to support json type so really Mojo::Pg needs Postgres > 9.4 ?
01:01 vicash I use Postgres 9.1 and I am getting "DBD::Pg::st execute failed: ERROR: type 'json' does not exist" from Mojo::Pg::Database.pm line 78
01:03 jberger new york style pizza isn't really like italian, though it is closer to the right shape
01:03 jberger sri: you should come to Chicago sometime
01:03 sri need to get some body armor first ;p
01:04 jberger I won't take you to the south side
01:04 vicash If Minion::Backend::Pg requires the json type to exist shouldn't that be tested by checking if the Postgres version is 9.4 and above  before even loading ?
01:05 sri minion will soon require 9.5
01:05 sri basically just waiting for travis
01:06 vicash well then you should really fail Minion when a lower Postgres is detected
01:06 sri it is at least in the docs already
01:08 sri personally i don't care about older versions of postgres at all
01:09 disputin joined #mojo
02:00 jontaylor joined #mojo
02:05 CandyAngel Are there any terminals that don't do any special handling of '&'.. pasting urls into bash is a pain -.-
02:06 jberger yeah, I always do that once first too
02:20 vicash sri: Minion::Backend::Pg91 uploaded to CPAN
02:21 vicash it uses Mojo::JSON's functions to encode/decode json into a text column as done by bpmedly in his Minion::Backend::mysql
02:21 cpan_mojo Minion-Backend-Pg91-4.04 by VIKAS https://metacpan.org/release/VIKAS/Minion-Backend-Pg91-4.04
02:21 vicash *bpmedley
02:22 meredith i've had good experience pasting into zsh but on the flipside sometimes it's annoying when i intend to use like, { } as shell metacharacters and it enters \{ for me
02:27 jberger vicash: packaging error?
02:28 vicash jberger: what do you mean ?
02:29 jberger I only see the META files
02:29 jberger I'm assuming you forgot to make manifest
02:29 * vicash checks
02:29 jberger https://metacpan.org/source/VIKAS/Minion-Backend-Pg91-4.04
02:30 vicash shit yea
02:30 jberger I think a 4.0401 is in our future soon :-P
02:38 vicash thanks jberger. i fixed it should be in soon.
02:39 cpan_mojo Minion-Backend-Pg91-4.0401 by VIKAS https://metacpan.org/release/VIKAS/Minion-Backend-Pg91-4.0401
02:42 jberger can't say I'm a big fan of the Changes file format, but otherwise, seems like a reasonable plan
02:46 jberger I know sri is going to add SKIP LOCKED once 9.5 comes out but that's just a performance improvement
02:46 jberger well, once travis supports 9l5
02:46 jberger (I can't even blame that on autocorrect, I'm on the laptop now)
02:46 jberger 9.5
02:46 vicash i did a quick Changes using git log --pretty=oneline for now since otherwise I will just get emails from PAUSE saying stuff is missing. i can fix that when I do the next update.
02:46 jberger k
02:46 jberger vicash++
02:46 jberger always nice to see a solution to a problem rather than complaining about the problem
03:30 noganex joined #mojo
03:44 zivester joined #mojo
04:37 irqq joined #mojo
04:45 jontaylor joined #mojo
04:49 c--__ joined #mojo
05:27 jacoby_ joined #mojo
05:39 inokenty-w joined #mojo
06:18 cstamas joined #mojo
06:26 jontaylor_ joined #mojo
06:31 abra joined #mojo
06:37 dod joined #mojo
06:42 dod joined #mojo
06:48 cstamas joined #mojo
07:09 dod joined #mojo
07:17 marcus jberger: lies. We're just here to plunder and pillage
08:13 Vandal joined #mojo
08:14 sugar joined #mojo
08:20 AndrewIsh joined #mojo
08:23 batman i wonder if this is useful for anyone else: https://gist.github.com/jhthorsen/f7bf9cfde19d268800f6#file-podify-pl (generate pod for a new module - mojo style)
08:23 trone joined #mojo
08:23 batman at least i think it's mojo style :)
08:27 cstamas joined #mojo
08:31 mitya joined #mojo
08:32 osfabibisi joined #mojo
08:47 CandyAngel batman: <3 that is amazing
08:48 * CandyAngel immediately runs it for ~50 modules
08:52 CandyAngel I wonder if I can convince Edward to add POD handling to Leo..
08:53 CandyAngel Also, I will probably modify that to put the POD in another file, if you don't
08:53 CandyAngel I prefer to separate them
08:53 CandyAngel (code/documentation)
09:09 batman CandyAngel: cool :) did the 50 modules already have pod?
09:10 batman CandyAngel: i wonder if i should make a dist of it. let me know in private if you want to contribute.
09:13 gregf joined #mojo
09:29 Averna joined #mojo
09:58 CandyAngel They had minimal POD, just notes what they need to do when I finish the code in them
10:00 batman ok :)
10:03 osfabibisi joined #mojo
10:08 pink_mist batman: please do make a dist of it :D
10:08 irqq joined #mojo
10:08 pink_mist batman++ awesome tool
10:21 batman :)
10:33 batman pink_mist: pink_mist: https://github.com/jhthorsen/app-podify <-- watch that :)
10:49 nic jberger: You were almost half right about ojo: yes, I'm putting strict into @ISA (why does that matter for oneliners?); no, ojo does not turn on strict
10:55 nic batman: I'm pondering whether app-podify could use ep templates for per-site customisation
10:55 batman that sounds like POD::Weaver
10:55 batman anyway... join #podify for more comments
10:56 nic /join #batman-has-too-many-channels      :)
10:56 batman haha
10:56 nic Can we just have #batman to avoid death-by-tabs?
10:57 batman sure.
10:58 nic sorry, I should really have taken that discussion offline
10:58 nic batman, please join #discuss-batmans-channels
10:58 CandyAngel I'm.. actually surprised IRC servers don't do something like that.. reserve user_* and have channels for each user (and you are autojoined to yours?)
10:59 CandyAngel Easy to spam though, I guess
11:00 nic I've only met one person who needed one, and tbh a spotlight aimed at a nearby cloud usually does a better job
11:00 CandyAngel Nanananananananananana-#batman
11:04 batman nic: just made #darkcode ;)
11:06 osfabibisi joined #mojo
11:11 marcus for batman's darkpan?
11:11 batman no. this is brand new code, hehe
11:31 nic :D
11:31 kaare joined #mojo
11:37 osfabibisi joined #mojo
11:39 kes joined #mojo
11:44 jontaylor joined #mojo
11:47 abra joined #mojo
11:52 cpan_mojo Mojo-Webqq-1.7.1 by SJDY https://metacpan.org/release/SJDY/Mojo-Webqq-1.7.1
11:57 osfabibisi joined #mojo
12:21 osfabibisi joined #mojo
12:29 dvinci joined #mojo
13:05 jberger nic: I checked $^H and it seemed to be the same, but maybe I did it wrong
13:05 jberger Anyway adding strict to @ISA does not enable it
13:06 nic easy check: perl -Mojo -E'say ($x =1)'
13:06 jberger True
13:06 jberger I'm on my phone so I'll trust that you're right
13:07 nic now try the above with -MMojo::Base=strict and you'll see it works
13:07 jberger All I was saying is you want -MMojo::Base=-strict
13:08 nic and I'm saying I do not
13:08 jberger Why?
13:08 jberger What benefit do you get from inheriting from strict?
13:09 nic You're wanting to correct my usage without understanding what's going on (and giving wrong information in the process)
13:11 kes joined #mojo
13:13 batman nic: -MMojo::Base=strict != -MMojo::Base=-strict
13:13 nic correct
13:14 batman (ignore me)
13:14 nic :)
13:15 jberger nic: I've conceded on the point that strict and friends need to be enabled separately
13:16 jberger And it probably does no harm to inherit from strict
13:17 jberger But I wonder what understanding I could be missing
13:17 jberger strict is not a class, is it?
13:18 batman jberger: strict->new() ftw!!!! ;)
13:18 jberger Putting the package into your @ISA seems to serve no purpose
13:18 jberger Unless I'm wrong, in which case I'd be interested in the explanation
13:19 ribasushi jberger: strict.pm is very much "a class", `perldoc -m strict`
13:20 ribasushi shoving it in @ISA will gain you a *method* of bits() which is neither here nor there
13:20 ribasushi it won't cause direct harm
13:20 ribasushi but makes no sense either
13:21 ribasushi the "I will just inherit the import()" dance has been done many times, it has very well known failure modes, and everything we collectively known about it is encapsulated in Import::Into
13:21 ribasushi look at source for edge cases / inspiration
13:22 ribasushi </PSA> ;)
13:23 nic Four suggestions for what you're missing: 1. Why a benign char omission is useful when doing oneliners, 2. Why it's harmful to the channel to have people being too quick to tell others they're doing it wrong, 3. Why it's better to show people how to do the quick test rather than tell them the wrong answer to that test, 4. Why it's unhelpful to tell people what they want when maybe they...
13:23 nic ...already know what they want
13:27 jberger nic: well clearly you took offense to what I said, so for that I'm sorry
13:28 jberger I'll leave it be
13:28 nic cheers
13:29 sri vicash: looks like your minion backend doesn't perform a version check either!
13:43 zivester joined #mojo
13:49 neilhwatson joined #mojo
13:53 sri hmm, looks like postgres 9.5 will be on travis very soon https://github.com/travis-ci/travis-cookbooks/pull/651
14:00 wingfold joined #mojo
14:52 asarch joined #mojo
15:08 osfabibisi joined #mojo
15:12 ramortegui joined #mojo
15:14 jberger This is cute: http://blogs.perl.org/users/tom_wyant/2016/02/stupid-state-trick.html
15:16 sri yea
15:18 cpan_mojo Minion-Backend-Pg91-4.0402 by VIKAS https://metacpan.org/release/VIKAS/Minion-Backend-Pg91-4.0402
15:20 gtodd joined #mojo
15:20 vicash sri: added the check now. it explicitly checks for Pg 9.1 otherwise croaks.
15:21 gtodd so Minions-1.00000 has nothing to do with Minion-4.0.4 :)
15:22 gtodd https://metacpan.org/pod/Minions  "This is an early release available for testing and feedback and as such is subject to change."  ....
15:22 gtodd I'd suggest a name change :-D
15:24 pink_mist maybe tell the author instead of #mojo :P
15:25 sri vicash: interesting, looks like you found a way to break migrations :)
15:27 sri ah, you make two instances of Mojo::Pg
15:28 sri i guess that's ok
15:28 vicash sri: yes i struggled to figure out why migrations kept breaking. then in the unit test I tried using a new Mojo::Pg object and that worked. so I did that
15:29 vicash sri: if you could explain why migrations breaks that will help my curiosity
15:29 sri *+not
15:30 sri because they run when a connection gets established
15:30 sri usually for the first connection
15:31 gtodd pink_mist: was thinking the authors of Minion could suggest the change to the author of Minions :-D  (I think the plural and singulars are correct?)
15:32 vicash sri: ok. that makes sense. i was still surprised that a SELECT/SHOW query would break migrations. i couldn't figure it out by browsing the code.
15:32 ribasushi gtodd: I think the point is that one of the minions of the author of minion can approach the minion of minions about the name change
15:33 gtodd ribasushi: heheh
15:33 berov left #mojo
15:34 ribasushi pink_minion: ^^ did I get it right? :D
15:34 gtodd Minons seems inspired by Moose Moo Mo but also by a smalltalky messaging approach to OOP, Alan Kay etc. ... there has to be a better more historically resonant name  :-D
15:38 jberger first releases in 2014, think that ship has sailed
15:38 jberger though, I can't say that the semver worked very well if Minions 1.0 still includes "This is an early release available for testing and feedback and as such is subject to change."
15:40 meredith i was just gonna guess they forgot to remove it when cutting 1.0
15:40 dvinci joined #mojo
15:40 meredith still there's the forever-beta approach, too ;)
15:43 cpan_mojo Minion-Backend-Pg91-4.0403 by VIKAS https://metacpan.org/release/VIKAS/Minion-Backend-Pg91-4.0403
15:44 gtodd haha
15:44 sri vicash++ # leading by example, i like that
15:44 sri vicash: https://github.com/kraih/minion/commit/0ff7be0538defecc3914542a000ad0c3f2fcefec
15:45 vicash sri: that one liner is nice
15:46 sri vicash: one small thing though, it is customary to ask before forking a module, in this case it's fine though, the backport backend gets my blessing
15:47 vicash sri: sorry about that. i only needed to fork the single module file and git doesn't allow that so had to create a new module.
15:48 sri that's fine, it's just organizational, difference between hostile and friendly fork
15:50 vicash sure, i am willing to support it since I am stuck with postgres 9.1
15:53 sri these days my main concern with forks is that they might take resources away from the main project, but that's clearly not the case here, so i'm fine with it
15:53 sri the name is also good
15:54 sri maybe you'd like to keep an eye on https://github.com/kraih/minion/issues to chime in on new feature discussions
15:56 vicash sure will do. thanks.
15:57 gtodd jberger: well nothing uses it yet ... anyway if it's truly smalltalky messaging approach to OOP then a nod to that might be a better name ...  maybe Miniscule.pm or OKay.pm  :-D
15:57 sri as a backend maintainer, i imagine the unique job thing might be of interest to you
15:58 cpan_mojo Minion-4.05 by SRI https://metacpan.org/release/SRI/Minion-4.05
15:58 sri personally, i'm interested in index optimizations atm., so if you find any please poke me ;)
15:58 jberger gtodd: if its been on CPAN for years there's no way to know that nothing is using it
15:58 jberger nothing **on cpan** maybe, but off CPAN usage isn't tracked
15:59 jberger I have lots of modules that nothing else on CPAN depends on that I know is used
16:02 vicash sri: ok i will look into it.
16:05 gtodd jberger: heh well I guess it doesn't matter .... unless someone uses Minions to write a job queue :)  it would be unlikely to use Minion; and use Minions; in the same application.
16:08 Grinnz gtodd, and if they do, i'm sure it will be a joy to read
16:10 jberger yo dawg
16:11 gtodd 8-\
16:12 genio o.O There's a Minions too?
16:14 vicash sri: how does one define a unique job for any generic context ? shouldn't that be context or application specific ?
16:15 jberger vicash: that has been a major point of contention
16:16 vicash jberger, sri: for example, let's say you're generating a random 2048-bit prime number as a job. this job is not unique in function but may have unique results. so is a unique job defined based on the task to be performed or the result to be achieved.
16:16 jberger vicash: I'd assume that its usually composed of a hash or the like of a (possibly subset) of job inputs
16:17 jberger but there's no way to know which inputs
16:17 jberger for example, an input might include the username of the job creator
16:17 jberger some applications might only allow one job per user, others might not want user included in the unique hashing
16:18 vicash having worked with third-party schedulers a lot, i feel that unique jobs is really job specific and the job itself should determine whether it should run only once or not. on the other hand to prevent a user from DoSing a scheduler, rate-limiting users is more useful
16:20 vicash maybe i should update the github issue with my thoughts
16:21 mullagain is there a timeline for Mojo to be supported in perl6?
16:22 jberger mullagain: it has been discussed and at the moment there are no official plans for a Perl6 port
16:22 mullagain ok
16:23 jberger a direct port isn't useful since the language provides non-blocking features internally
16:34 PryMar56 joined #mojo
16:38 bradjm joined #mojo
16:38 sri vicash: this was the original unique job idea https://github.com/kraih/minion/issues/7#issuecomment-65731228
16:41 vicash sri: right. i am reading that now.
16:41 vicash sri: also your comment earlier in this chat about index optimizations. is that in an issue as well ?
16:42 sri nope, dunno where to start on that one yet
16:44 sri we actually had an index on the state column, but recent changes for named queues made the query planner not use it anymore
16:45 sri so far my experiments were mostly just adding indexes on individual columns and then run a bunch of stress tests to see which indexes would get used
16:45 sri without success
16:45 vicash is the aim to achieve query performance improvement or scheduling performance improvment ?
16:46 sri i think dequeue performance is most important
16:46 sri but it might be worth looking into stuff like ->repair too
16:48 sri we also kinda don't have a baseline for optimizations... like how many jobs do we expect to be in the queue on average, or how many active workers...
16:48 sri i've been testing with 100k jobs and 5 workers mostly
16:48 sri which is prolly a bit low for indexes to become significant
16:49 vicash ok. and is the expected minimum time required to complete all these jobs is too high ?
16:49 vicash sri: why is the state column text ?
16:49 sri there is no actual flaw that needs to be fixed
16:50 sri just overall performance improvements
16:50 vicash you can use ENUM type
16:50 vicash and set those fixed string values, Pg will optimize it faster than text
16:51 sri see, that's the kind of optimizations i'm interested in :)
16:51 sri when the schema was first created we didn't really know what we wanted yet
16:52 Grinnz enum is always annoying to add values to later, but maybe its stable enough now to be worth using
16:52 sri before postgres, mongodb was actually the main backend :o
16:52 vicash yes that's understandable. but i recommend Enum because I use it and it saves a lot of time since Pg internally optimizes it.
16:52 vicash Grinnz: Pg has ALTER TYPE. not sure about other backends
16:53 Grinnz right, i mean that you have to alter the schema to add values, at a conceptual level
16:53 vicash it is still going to be faster than a text column with arbitrary values and re-indexing requirements
16:53 jberger Grinnz: at least migrations makes that easy
16:53 Grinnz indeed
16:54 sri vicash: for quick benchmarks i've been using this https://gist.github.com/anonymous/6750bea1e2bce74180d7
16:54 sri like, for testing SKIP LOCKED
16:56 vicash sri: i have another question. the "host" field in minion_workers. Is that updated by the worker
16:56 sri which turned out to be 30% faster in the best of cases
16:56 sri see ->register_worker
16:58 vicash yea it is not being used except in a simple select query. otherwise you could use the inet type in Pg to optimize that as well.
16:59 sri it's not an address, but a hostname
17:00 sri it's only meant for admins to identify workers
17:01 vicash right. i think state as an enum is a good bet to try. it will remove the not null check that needs to be done as well
17:13 sri hmm, enum might be a little faster
17:14 sri patch https://gist.github.com/anonymous/d3a2d21b71f2da6cd759
17:17 sri yea, with minion_bench.pl it seems to be 970 vs 990 dequeues per second
17:24 mcsnolte joined #mojo
17:32 sri might be worth converting the json columns to jsonb if we add a new migration anyway... for future proofing
17:34 vicash cool
17:37 sri https://github.com/kraih/minion/commit/e32240d657f5c6faecd74f4ef233aa8910ffd4b5
17:37 dod joined #mojo
17:41 sri enum makes sense in this case
17:41 sri or does anyone know a good reason not to use enum here?
17:46 orev1 joined #mojo
18:06 disputin joined #mojo
18:08 orev joined #mojo
18:11 sri this is where we stand right now with dequeue https://gist.github.com/anonymous/6d791da40fe25e096d89
18:11 sri running that after minion_bench.pl
18:12 sri i've been unable to get the query planner to use an index for the Filter part even with 500k jobs
18:26 irqq joined #mojo
18:38 vicash sri: that's because Pg has reasoned that using the priority_created_idx is a faster approach.
18:40 vicash sri: one way to have a better idea of what the query is causing is to do: BEGIN; DROP INDEX myindex; EXPLAIN ANALYZE myquery; ROLLBACK; This will not force rebuilding of the index on rollback either but will give a better idea of why the query picks an index vs filter
18:44 vicash also i see that the queue is a text column with mostly limited values so indexing will never get used on it. indexing is not needed for enums unless you probably hit some 10s of millions of rows either... indexing is mostly useful when you want to sort arbitrarily.
18:54 irqq_ joined #mojo
20:32 sri hmm, i guess migrations 3 and 6 don't really serve a purpose anymore
20:35 sri looks funny when they are gone, but is totally fine https://github.com/kraih/minion/commit/6612ae9829d509b983cb93811b3861c582a87c6d
20:48 trone joined #mojo
21:09 sri i find Mojo::Pg with delays much nicer looking than pg-promise https://blog.andyet.com/2016/02/04/postgres-9.5-document-store-hybrid/
21:36 sri looking through job queue uis... i guess we might need ->stats({queue => 'foo'}) at some point https://cloud.githubusercontent.com/assets/1458008/5229932/76dd0e70-7716-11e4-9551-e87ce799d8dc.png
21:55 perlpilot joined #mojo
22:06 disputin joined #mojo
22:39 cpan_mojo Minion-4.06 by SRI https://metacpan.org/release/SRI/Minion-4.06
22:41 good_news_everyon joined #mojo
22:41 good_news_everyon [mojo] kraih tagged v6.44 at bc44f88: https://git.io/vgCNX
22:41 good_news_everyon left #mojo
22:42 good_news_everyon joined #mojo
22:42 good_news_everyon [mojo] kraih pushed 1 new commit to master: https://git.io/vgCAe
22:42 good_news_everyon mojo/master f7a807a Sebastian Riedel: bump version
22:42 good_news_everyon left #mojo
23:12 cpan_mojo Minion-Backend-Pg91-4.06 by VIKAS https://metacpan.org/release/VIKAS/Minion-Backend-Pg91-4.06
23:19 disputin joined #mojo
23:51 asarch joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary