The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2014-06-01

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:05 sri the more i think about a file backend the more i dislike it ;p
00:05 sri it's just not something i would ever use
00:07 sri you know...it seems nice in theory...but adding a backend we don't actually recommend using in production, just for the sake of having it in core... it's not the mojo way
00:10 tempire It's something I would use all the time
00:10 tempire I think a lot of people would use it
00:10 tempire Sometimes I need a long running process, but I don't have a backend available.
00:10 tempire Nor do I care to set one up
00:10 tempire I just want_it_to_work_damnit
00:10 tempire that sort of thing
00:10 purl hmmm... that sort of thing is an interesting idea
00:11 tempire This is where people start using fork instead of going through the effort of setting up a backend
00:11 sri with one server and bad performance?
00:11 sri hmm
00:11 tempire performance isn't the issue here - it's low bar of entry to get it working for whatever is needed right now
00:12 tempire then, it's just a matter of switching out a uri to make it work better
00:12 tempire file:///asdf > mongodb://asdf
00:12 tempire I think that's worthwhile
00:14 sri i'll try to keep an open mind
00:17 nicomen https://www.varnish-cache.org/trac/wiki/VCLExampleRemovingSomeCookies - 502 Bad Gateway
00:17 nicomen that's funny
00:17 nicomen tempire++
00:17 nicomen but I still don't see why memory can't cut it. I basically just want an additional process forking and what not doing the heavy lifting
00:18 nicomen and I want to realise eventually that setting up an external backend is worthwhile, but not after I have nb + jobs working
00:26 neyasov__ joined #mojo
00:27 sri tempire: are you working on it? anything on github?
00:44 rem_lex| joined #mojo
00:50 marty joined #mojo
01:05 neyasov__ joined #mojo
01:13 riche` joined #mojo
01:14 ryozi_ joined #mojo
01:15 tempire sri: I haven't made any code yet
01:18 maze joined #mojo
01:23 Lee_ joined #mojo
01:24 kbenson joined #mojo
01:26 asarch joined #mojo
01:38 ryozi joined #mojo
01:49 perl2012 joined #mojo
01:50 Lee joined #mojo
01:51 marty joined #mojo
01:52 klapperl joined #mojo
01:53 ivan joined #mojo
01:59 shmuel joined #mojo
02:06 neyasov__ joined #mojo
02:16 Akron joined #mojo
02:28 mad_hatter joined #mojo
02:28 shmuel joined #mojo
02:37 mad_hatter hello good people...I'd like to get my hands dirty with a little web development. id like to create a dashboard type web app using mojolicious but i seriously have no experience with perl web development. and only a little with php. so what would be a good starting point to go about doing this?
02:41 jberger mad_hatter: have you read the Mojolicious::Lite documentation and the Guides?
02:42 jberger http://mojolicio.us/perldoc
02:42 purl somebody said http://mojolicio.us/perldoc was lovely
02:42 * jberger pats purl
02:42 * purl pats jberger back, harder
02:44 jberger mad_hatter: also there is mojocasts.com
02:44 jberger start here: http://mojocasts.com/e1
02:44 mad_hatter thanks guys
02:45 jberger mad_hatter: those mojocasts are great, but there are a few things that are out of date, see here for the errata: https://github.com/kraih/mojo/wiki
02:46 jberger have fun!
02:51 marty joined #mojo
03:06 neyasov__ joined #mojo
03:48 basic6 joined #mojo
03:52 marty joined #mojo
04:06 dotandimet joined #mojo
04:07 neyasov__ joined #mojo
04:39 Rallias joined #mojo
04:44 jberger sri: here is a possible start for a DBM::Deep backend
04:45 jberger https://github.com/jberger/minion/compare/dbmdeep?expand=1
04:45 jberger meantime, nn all
04:46 jberger oh dotandimet: https://gist.github.com/jberger/5153008
04:53 marty joined #mojo
05:07 mad_hatter joined #mojo
05:09 neyasov__ joined #mojo
05:10 irq joined #mojo
05:17 avenj joined #mojo
05:19 irq_ joined #mojo
05:26 shmuel joined #mojo
05:30 dotandimet jberger: thanks, saw it. Still reading backlog ;)
05:37 davido_ joined #mojo
05:42 davido_ joined #mojo
05:46 dotandimet joined #mojo
05:54 marty joined #mojo
06:10 neyasov__ joined #mojo
06:17 tfiala joined #mojo
06:34 zivester joined #mojo
06:54 marty joined #mojo
06:55 eugen joined #mojo
06:59 Vandal joined #mojo
07:10 neyasov__ joined #mojo
07:40 dotan Wow, 117 comments on the slashdot thread? yesterday I saw only one.
07:55 marty joined #mojo
07:58 shmuel joined #mojo
08:00 tfiala joined #mojo
08:03 dod joined #mojo
08:11 neyasov__ joined #mojo
08:54 dotan although, slashdot comments. blah.
08:56 marty joined #mojo
08:57 dotan joined #mojo
09:04 neyasov__ joined #mojo
09:21 voegelas joined #mojo
09:45 mire_ joined #mojo
09:57 marty joined #mojo
09:59 basiliscos joined #mojo
10:02 denis_boyun joined #mojo
10:04 rem_lex|pivo joined #mojo
10:07 rem_lex joined #mojo
10:10 voegelas joined #mojo
10:14 sugar joined #mojo
10:14 voegelas joined #mojo
10:35 alnewkirk joined #mojo
10:48 neyasov__ joined #mojo
10:58 marty joined #mojo
11:40 nicomen dotan: I wanted to write "oh please shut up. it's 2014. perl is now beatiful, and mojolicious is the future"
11:45 jberger nicomen: I'll admit I wrote over of the anonymous defenses
11:46 jberger though I was only anonymous because I didn't want to set up a user
11:48 neyasov__ joined #mojo
11:58 marty joined #mojo
12:35 d4rkie joined #mojo
12:42 dotan I think my slashdot user is tied to an email from 3 jobs ago, back in the 20th century.
12:47 denis_boyun_ joined #mojo
12:49 d4rkie joined #mojo
12:51 neyasov__ joined #mojo
12:57 janus heh, not sure if anybody has an ikea naming scheme yet for maschines - here's a mojo snippet to collect ~1700 of them:  https://gist.github.com/sbertrang/e0f39bc6a74bd4c3db92 :)
12:59 marty joined #mojo
13:05 neyasov__ joined #mojo
13:12 basiliscos joined #mojo
13:29 janus btw, anyone working on a UA cache plugin already?
13:30 asarch joined #mojo
13:30 janus since I keep (re)writing this almost on a weekly basis I wonder what others think
13:41 jberger janus: cache the result of the request?
13:51 bayashi joined #mojo
13:51 cpan_mojo Mojolicious-Plugin-Notifications 0.2 by Nils Diewald - http://metacpan.org/release/AKRON/Mojolicious-Plugin-Notifications-0.2
13:53 janus well, theoretically anything using Mojo::UserAgent
13:56 hkclark joined #mojo
13:57 janus good example would be rss-aggregation, if-modified-since/etag and such
14:00 marty joined #mojo
14:06 neyasov__ joined #mojo
14:06 jberger so Akron made his own Humane.js plugin eh?
14:06 jberger guess that was coming :-)
14:08 gryphon joined #mojo
14:09 dotan_dimet joined #mojo
14:09 jberger janus: seen https://metacpan.org/pod/Mojolicious::Plugin::CHI?
14:09 purl I haven't seen 'https://metacpan.org/pod/Mojolicious::Plugin::CHI', jberger
14:09 jberger or just CHI if you don't need the plugin form
14:12 sh4 joined #mojo
14:15 dotan janus: I handled if-modified and etag headers in my RSS reader pet project, and I'm pretty sure servers LIE.
14:22 irq joined #mojo
14:33 nicomen janus: I have a Cached Mojo UserAgent I want to publish soon
14:33 nicomen it uses CHI
14:33 nicomen and also allows reading local files
14:35 nicomen it's kind of aggressive by default though, I have used it as a catch-all for stupid code that re-fetches URLs on every request
14:35 nicomen and simply doesn't adhere to If-Modified/Etag (unless Mojo::UserAgent does that underneath)
14:57 Sound joined #mojo
15:00 Sound joined #mojo
15:00 jberger http://blogs.perl.org/users/joel_berger/2014/06/mojoconf-2014-recap.html
15:01 marty joined #mojo
15:01 jberger https://twitter.com/joelaberger/status/473117140816375808
15:02 Sound Hello! First project with Mojo. I need to accept a file upload, then store the file and process it asynchronously (slow, blocking, system() call), then return response as soon as the new processed file is available for download.  How'd you suggest to do?  Minion and WebSocket?
15:03 nicomen jberger++
15:07 jberger Sound: I'm not sure how the notification system upon completion work with Minion (admittedly I haven't spent much time looking at it). For a simple fix, I can recommend Mojo::IOLoop::ForkCall which was made for that specific type of task
15:08 * sri thinks he might give abstracting out the storage layer from minion a try if nobody else is working on it yet
15:08 Sound @jberger: hmm, I'd actually like to queue jobs and use a separate worker process (or pool of processes)
15:09 Sound @jberger: well, I can handle the queue myself, so the question is more likely how to check for completed tasks in order to send a WebSocket message to the client? Should I use a recurring timer?
15:09 nicomen That's pretty similar to what I was advocating yesterday, but I haven't had any time to look at it
15:10 disputin joined #mojo
15:11 jberger Sound: it seems that the Minion::Job object does emit a finish event: https://metacpan.org/pod/Minion::Job#finished
15:13 Sound @jberger: interesting, thank you.  How to pair the finished job to the client connection I want to send the notification to?
15:13 jberger sri: give it a go, last night I did start a fork that was mostly by-hand overloading of most of the methods: https://github.com/jberger/minion/compare/dbmdeep?expand=1
15:14 sri the loss of efficiency for making the api simpler is a bit sad though
15:15 sri gotta dumb it down for fewer different queries
15:15 Sound @jberger: does it automagically remember the client connection if I just include a $self->render in the finish event attached to the Minion job?
15:15 sri do not use the finish event
15:15 sri it doesn't work the way you think it does
15:15 Sound @sri: oh
15:16 zivester joined #mojo
15:16 sri a job may run a completely different server, there is no direct connection to the web server at all
15:16 sri *+on
15:16 sri if you want a result, put it in a database
15:17 Sound @sri: doesn't Minion store things in a MongoDB instance?
15:17 jberger sri: right, I always forget the "a job may run on a completely different server" part
15:17 sri and check that database from your web server
15:18 sri Sound: implementation detail
15:18 purl implementation detail is that an fping child gets the whole list on its input and causes the "node" event in the client session to be called.
15:18 dotan sound: mongodb is also a database, but minion doesn't use it for storing results of the task executed - you can do that yourself in your task code.
15:18 purl okay, dotan.
15:19 Sound dotan: got it, thanks for explaining that
15:20 Sound so I guess I need to implement the queue myself (using, say, a database or a message queue) and poll it using a recurring timer
15:20 dotan I agree with tempire that a file/directory based backend for minion will be used by lots of people
15:20 dotan Sound: yep.
15:21 Sound dotan: okay, thank you very much
15:22 jberger Sound: you don't necessarily need to poll, Mongo/Mango support listening to "capped collections" which can be used to notify
15:24 Sound @jberger: interesting, I'm trying to find where's that documented but I'm not finding it. Where can I learn about capped collections?
15:25 jberger Sound: http://docs.mongodb.org/manual/core/capped-collections/
15:26 Sound oh, I was looking in the Mango docs.  I'm reading that page now
15:26 crab sri: abstracting the storage layer == making it possible to use listen/notify and postgres?
15:26 sri crab: dunno
15:26 sri more like using tables
15:26 sri minion keeps a history
15:27 Sound @jberger: sorry if I look a bit dumb, but how to use those capped collections from a Mojo application without polling?
15:27 sri Sound: tailable cursors
15:27 jberger Sound: http://docs.mongodb.org/manual/tutorial/create-tailable-cursor/
15:27 sri which is a different kind of polling
15:29 sri crab: once i've abstracted out the storage layer you can bikeshed about the underlying data structures ;p
15:31 sri i don't care very much about history, but i do want stats
15:31 dotan jberger: last paragraph of the blogpost, I think you mean "peaking", not peeking.
15:31 jberger dotan: right you are
15:32 Sound @sri: so in the Mojo application I still need to create a recurring timer for each client connection/job and perform a query() call to get such tailable cursor?
15:32 sri honestly, don't use minion
15:32 sri it's experimental and odds are it will break completely next release
15:32 Sound @sri: was that for me?  I already decided not to use Minion :)
15:33 sri jberger, tempire, dotan....and many others are making me redesign it
15:33 jberger I wouldn't say "making"!
15:34 jberger Sound: meantime, would ForkCall do the trick for you?
15:34 crab i like history
15:34 crab i don't quite understand minion because i don't know anything about mongodb
15:34 Sound @jberger: actually nope, it doesn't scale. I can process a limited number of files at the same time, so a queue is needed
15:34 crab i just said listen/notify because that's how my current queue thingy works. i stick data into a table and notify something that's listening to wake up and look at the table
15:35 sri i hate pure pub/sub solutions since i don't want jobs to get lost
15:35 crab i don't know what is a "pure" pub/sub solution, i suppose the lost happens if there's a pub and no sub?
15:37 jberger Sound: what kind of scale are you expecting?
15:37 crab in my (very limited) use case that doesn't happen because i currently treat any notify as a "sendmail -q" style kick in the pants
15:38 crab wake up and smell the^W^Wscan the table
15:38 sri crab: how do you avoid the thundering herd?
15:38 Sound @jberger: I'd say between 10 and 50 files in queue
15:39 Sound @jberger: well, the easy way would be to have the JS client poll for job completion with AJAX requests, but I thought WebSocket would be more Mojo-ish
15:39 neyasov__ joined #mojo
15:39 sri crab: say i have 1000 idle workers all getting notified at the same time
15:40 crab sri: this is the "very limited" part, i just have one worker. (this is also why i was interested in minion in the first place.)
15:40 sri that's the thing that annoys me about hybrid pub/sub (with table)
15:41 sri in minion i decided against pub/sub because of it, and instead went with a sleep a few seconds if you didn't get a job solution
15:42 sri most redis job queues solve it with blpop
15:42 sri 1000 workers can blpop on their connection, but only one gets the next job
15:44 crab so the minion workers are polling the table at random intervals?
15:44 sri yes
15:44 Sound Train wifi sucks a bit, I'll disconnect.  Thank you for your advice, guys
15:46 sri it's actually super simple https://github.com/kraih/minion/blob/master/lib/Minion/Command/minion/worker.pm#L13
15:46 jberger sri: learning what I am now,  I would trade a pluggable back end for a tailable cursor than can notify the originating process of a job's completion
15:47 sri and it's only in the command, so anyone could build a different kind of worker with the primitives
15:47 sri jberger: i believe that would be wrong by design
15:48 crab it was too late for 9.4, but for 9.5 we should have "skip locked", which will help for that kind of wake-up-and-consume-task model
15:48 sri i'm a little scared you seem to view job queues as a silver bullet
15:48 sri there are generally no notifications
15:49 crab personally i never fight a werewolf without a job queue
15:49 sri most jobs are fire and forget maintenance tasks in the background
15:49 crab yes, for me too
15:50 sri job queues always add latency, there may be a huge backlog and you have to wait an hour
15:50 sri it's not a replacement for that forkcall stuff you use for immediate results
15:52 crab http://thumbs.media.smithsonianmag.com//filer/9e/68/9e681b75-e1e5-452e-a261-cb554f871599/50221_2_243512.jpg__1072x720_q85_crop.jpg <- does unicode have a "hawk face"?
15:54 sri if you look at github, a typical job for a job queue would be to prepare your daily contribution graph
15:55 sri or how they handle forking, a job is dispatched and the client side pulls until that job is finished
15:55 sri (not as typical)
15:56 jberger sri: I see.  sorry I never have had that kind of pattern in an app of my own.  I keep losing the forest for the trees
15:56 sri it will never work for the "i just want to perform this blocking operation and continue rendering a response" stuff
15:56 nicomen true, but in a non-blocking world, a job is everything that doesn't have proper non-blocking support
15:57 jberger I like the github graph example, that really helps me
15:57 sri gonna have to make clear that "job queue == latency"
15:58 crab if all the workers do "sleep 5", won't they all wake up together and poll?
15:58 sri jberger: i've actually experimented with the kind of notifications you were thinking about
15:58 crab oh, i guess they're started separately so they might not
15:59 sri crab: times also change whenever one gets a job
16:00 crab right
16:00 crab makes sense
16:00 jberger I'm just thinking about the wrong type of jobs
16:00 * jberger trusts sri
16:01 sri jberger: don't trust me, i'm still learning about job queues as well
16:01 jberger job queues are hard, let's go get pancakes
16:01 crab queue up here for pancakes
16:01 * jberger goes to get pancakes
16:01 marty joined #mojo
16:02 sri jberger: https://github.com/kraih/minion/commit/94288b8f9c528786576492137c39734c664dbb74
16:02 sri that's where i removed support for task results again
16:02 sri which had actual pub/sub notifications
16:02 howitdo left #mojo
16:06 marty joined #mojo
16:07 marty_ joined #mojo
16:07 marty joined #mojo
16:09 marty_ joined #mojo
16:11 * sri is now known as buzzkill
16:13 Akron joined #mojo
16:14 Akron jberger: Jepp, sorry. Hope you like it nonetheless - at least I recommend your plugin for Humane-only usage!
16:15 Akron jberger++ for the recap
16:26 beyondcreed joined #mojo
16:27 neyasov__ joined #mojo
16:43 * sri wonders if $maxTimeMS works on tailable cursors
16:47 dgikiller joined #mojo
16:49 neyasov__ joined #mojo
16:52 mire_ joined #mojo
16:58 neyasov__ joined #mojo
17:05 Akron joined #mojo
17:11 disputin joined #mojo
17:13 meshl joined #mojo
17:20 onur joined #mojo
17:21 onur Hi guys, I have a newbie question. How can I read flash message in my template?
17:21 onur flash () didn't work
17:21 onur I am setting my flash message with $mycontroller->flash ('My flash message');
17:25 Akron onur: You have to set it flash(message => 'My flash message')
17:26 onur Ahh I get it
17:37 howitdo joined #mojo
17:57 neyasov__ joined #mojo
18:13 Akron joined #mojo
18:17 disputin joined #mojo
18:47 Adura joined #mojo
18:47 dod joined #mojo
19:00 neyasov__ joined #mojo
19:07 denis_boyun joined #mojo
19:11 disputin joined #mojo
20:00 punter joined #mojo
20:02 lsm joined #mojo
20:09 sri ok, i have a version of Mango with pluggable backend
20:10 sri backend currently has 13 methods though
20:13 mire__ joined #mojo
20:18 Claudio joined #mojo
20:24 sh4|2 joined #mojo
20:27 tempire mango?
20:27 purl You cannot have mango! or a new grant approved by TPF @ http://news.perlfoundation.org/2006/08/grant_approved_mango.html or claco making a fool out of himself or an ecom solution built upon Catalyst + Handel or a fruit or http://www.newgrounds.com/portal/view/366593 or http://github.com/claco/mango/tree
20:27 tempire you mean minion?
20:27 sri indeed :)
20:32 cpan_mojo Mojolicious-Plugin-Notifications 0.3 by Nils Diewald - http://metacpan.org/release/AKRON/Mojolicious-Plugin-Notifications-0.3
20:35 irq joined #mojo
20:44 basiliscos joined #mojo
20:52 sri tempire, jberger: a first version https://github.com/kraih/minion/commit/32099f5407fc84f31a16a217dd334f3924d68be3
20:54 sri think i'll make a base class so it's easier to see which methods are necessary
20:56 sri class name is lowercase since i wanted to play with the URI idea... Minion->new('some_file_backend:///foo.db')
20:59 punter joined #mojo
20:59 sri basically what you have to implement for any new backend https://github.com/kraih/minion/blob/master/lib/Minion/Backend/mongodb.pm
21:01 sri now that i've started i'm going to revisit a few decisions
21:14 sri and i'm going to need one more method, for listing all existing jobs... not sure how to do that yet
21:15 sri for this https://github.com/kraih/minion/blob/master/lib/Minion/Command/minion/job.pm#L70
21:21 basiliscos joined #mojo
21:28 sugar joined #mojo
22:10 sri hmm
22:10 sri think i'll go with "plugin Minion => {Mango => 'mongodb://127.0.0.1:2717'};" instead
22:44 basiliscos joined #mojo
22:47 hesperaux joined #mojo
22:50 sri ok, this is a lot cleaner now https://github.com/kraih/minion/compare/95acdf417a7d884179a449cc171fe3aae8134953...4d63c72a9a6b4b84810958e6fe61818aa9007b66
22:59 d4rkie joined #mojo
23:10 maze joined #mojo
23:38 sri neyasov: you're welcome to ask here
23:41 neyasov__ mojo get http://www.stylepark.com/ru/flos/romeo-moon-s2-suspension-lamp?ref=show_manufacturer_products body
23:41 neyasov__ Complex regular subexpression recursion limit (32766) exceeded at /Users/neyasov/perl5/perlbrew/perls/perl-5.20.0/lib/site_perl/5.20.0/Mojo/DOM/HTML.pm line 119.
23:41 neyasov__ This is a bug?
23:42 sri likely
23:45 marty joined #mojo
23:54 neyasov__ joined #mojo
23:57 asarch joined #mojo
23:58 sri http://pastie.org/9248520 # i actually have a test case, no clue how to fix it though
23:59 punter joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary