The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2016-10-06

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:05 disputin joined #mojo
00:14 lluad joined #mojo
01:32 lluad joined #mojo
02:05 Janos joined #mojo
02:07 jberger Ummm so then this happened:  https://rethinkdb.com/blog/rethinkdb-shutdown/
02:12 genio :/
02:34 Grinnz open sourcing it at least is good
02:34 Grinnz i mean i guess it was already open source
02:51 lluad joined #mojo
02:55 noganex_ joined #mojo
02:58 lluad joined #mojo
03:11 Janos joined #mojo
03:15 bd joined #mojo
03:26 jberger Indeed. As a user this is the exact reason to demand source
03:41 lluad joined #mojo
03:56 itaipu joined #mojo
04:24 itaipu joined #mojo
04:30 CW joined #mojo
05:09 inokenty-w joined #mojo
06:08 laidback_01 joined #mojo
06:19 dod joined #mojo
06:24 dod joined #mojo
06:42 mbudde joined #mojo
06:47 disputin joined #mojo
07:13 che-quest sri: none use imported monkey_patch https://metacpan.org/source/SRI/Mojolicious-7.08/lib/Mojo/Log.pm#L6
07:14 trone joined #mojo
07:17 AndrewIsh joined #mojo
07:20 Janos joined #mojo
07:27 janl joined #mojo
08:15 rshadow joined #mojo
08:22 mpapec https://hackernoon.com/how-it-feels-to-learn-javascript-in-2016-d3a717dd577f
08:29 geira joined #mojo
08:34 Janos joined #mojo
08:38 coolo joined #mojo
09:20 dod joined #mojo
09:43 Janos joined #mojo
09:59 kaare joined #mojo
10:13 HtbaaPi joined #mojo
10:31 che-quest joined #mojo
10:35 itaipu joined #mojo
10:50 janl mojo apps have insanely deep call stakcs
10:53 vytas joined #mojo
10:54 Janos joined #mojo
11:02 tchaves joined #mojo
11:08 tchaves joined #mojo
11:10 tchaves joined #mojo
11:45 Janos joined #mojo
12:36 itaipu joined #mojo
12:53 gizmomathboy joined #mojo
12:55 che-quest joined #mojo
13:05 ashimema joined #mojo
13:09 Janos joined #mojo
13:32 marty joined #mojo
13:44 mcsnolte joined #mojo
13:51 lluad joined #mojo
14:00 ivi joined #mojo
14:13 Janos joined #mojo
14:17 gregf_ joined #mojo
14:46 disputin joined #mojo
14:58 demian joined #mojo
15:00 demian I would like to event-stream the progress of an intensive task
15:01 demian I am studying http://blog.kraih.com/post/43197537940/mojolicious-hack-of-the-day-html5-eventsource
15:03 demian here is a gist that has a long monte carlo pi calculation.  I would like to stream the value of pi (instead of the dice roll) every second
15:03 demian https://gist.github.com/dmr3/f435e84fb26f4de2e0c8e1f2d2e74f9b
15:14 jberger demian: first of all, is websocket an option for you?
15:14 jberger eventsource is less and less necessary as websockets are more and more prevelent in browsers
15:17 demian it could be an option.
15:17 demian I was viewing the event source as a step in that direction
15:17 jberger I highly suggest just going for that
15:17 demian I am studying your blog entries
15:18 demian on websockets
15:18 jberger cool
15:18 demian is there a simple way to do it with websockets or eventsource for this problem?
15:19 jberger the hardest thing about this is that calculating pi is a cpu bound thing
15:19 jberger and you don't want to block the event loop
15:21 demian I have a reasonably cpu-heavy task with a handful of users
15:22 demian I would like to figure out the best way to carry out the process while providing some progress info
15:23 da5id joined #mojo
15:24 demian each user would be carrying out their own cpu-heavy task
15:25 jberger if I was building something at an "enterprise" scale I would have the tasks be in a job queue and write the progress to a message broker that the clients would be subscribed to
15:25 jberger I don't know if that is overkill for you though
15:27 vicash demian: use Minion. Have the minion task update a cache/database/websocket with progress and have your Mojo controller read that cache/database/websocket and display progress.. that's how i have implemented a viewer for our long running jobs
15:30 jberger vicash: that is basically what I said :P
15:30 Eke joined #mojo
15:32 chansen_ joined #mojo
15:33 demian I would like to keep it as simple as possible at this point.  (Minion is in my learning queue)  Is there a way to send the event-stream periodically from within the costly loop?
15:33 vicash jberger: yes i agree. i actually did exactly that for one of our tasks which is 100% CPU bound for hours sometimes. that's the only place we use Minion. i am adding to your comment with an example that will work
15:33 bjakubski joined #mojo
15:33 demian "hey client, you mc pi=3.14154322"
15:34 demian your mc pi
15:34 vicash demian: for montecarlo pi rolls have you thought of using a GPU ?
15:35 demian I'm not really calculating pi with monte carlo, it's a toy problem
15:35 demian but using a GPU to so would be hot stuff
15:36 vicash demian: Minion is a recommended way. It is really really easy to use. just use Mojolicious::Plugin::Minion. move your task into a function and initiate it with the Mojolicious Plugin wrapper calls.. start a worker that will do it and it will be done
15:36 vicash demian: another way is to use Mojolicious::Plugin::ForkCall or ReadWriteFork but that can slow down your controller...
15:39 jberger RWF is probably the most directly tuned for this problem, in fact I almost mentioned it
15:40 demian in order to keep this example simple, is there a way to do it with minion using backend other than postgresql (another entry in my learning queue!)
15:40 jberger I don't know what you mean by slowing down the controller
15:40 vicash jberger: apologies. i need to elaborate. if the user forks and is waiting for the work to complete the controller hangs around, doesn't it ?
15:41 vicash if the user forks and forgets then that's ok
15:42 vicash demian: maybe you should try ReadWriteFork if you want to minimize work. that is also really easy to use. very good api
15:42 demian I am looking now.  It is looking good
15:42 vicash demian: Minion has a Minion::Backend::Sqlite i believe.. pretty sure it exists
15:46 demian in addition to sqlite like there is Minion::Backend::Storable.  This has been really helpful.
15:47 seatek joined #mojo
15:48 jberger has the Storable one kept up?
15:48 jberger Grinnz maintains the SQLite one
15:49 asarch joined #mojo
15:49 dod joined #mojo
16:20 ptolemarch joined #mojo
16:34 tyldis :)
17:03 ptolemarch joined #mojo
17:16 rshadow joined #mojo
18:13 jamesaxl joined #mojo
18:17 jamesaxl hi mojo
18:19 disputin joined #mojo
18:22 itaipu joined #mojo
18:30 PryMar56 joined #mojo
19:13 jberger jamesaxl: o/
19:25 * sri will be using Mojo::SQLite for a new app at suse
19:26 sri still don't like the pubsub emulation
19:26 sri the concept, not the implementation
19:29 good_news_everyon joined #mojo
19:29 good_news_everyon [mojo] kraih pushed 1 new commit to master: https://git.io/vPWqp
19:29 good_news_everyon mojo/master 684b567 Sebastian Riedel: monkey_patch is not actually used
19:29 good_news_everyon left #mojo
19:29 sri also, everybody sign my gpg key :)
19:31 sri fingerprint 6BA8BB3D9A3D17A2DD2BADDAA4F229961D4827F5
20:07 lluad joined #mojo
20:11 bpmedley sri : May I ask if my gist was helpful for issue #1004?
20:11 sri have not had time to look
20:12 bpmedley That's understandable.  Must be a whirlwind of fun right now.. :)
20:12 bpmedley Can you talk about your new app?
20:42 itaipu joined #mojo
20:44 batman joined #mojo
20:49 batman joined #mojo
20:55 PopeFelix joined #mojo
21:22 stryx` joined #mojo
21:25 itaipu joined #mojo
21:39 demian joined #mojo
21:39 demian I have a websocket version of the mcpi calculator that is helping me explore this stuff:
21:39 demian https://gist.github.com/dmr3/f435e84fb26f4de2e0c8e1f2d2e74f9b
21:40 jberger ummm, so, pg upsert burns a unique id each time you do it :s
21:42 demian (sorry for interrupting) when I run with an expensive enough set of runs with repeats, I see "Rendering cached template", but the client side table doesn't update
21:45 demian it updates after the calculation.  Is there a way to get it update the client side table with each iteration?  I'm not ignoring earlier discussions of minion/readwritefork etc
21:45 bpmedley demian : What are some sample inputs that denote the issue?
21:45 demian just trying to learn
21:46 bpmedley demian : Although, I see an expensive CPU loop that will block the controller.
21:47 demian that must be the trouble.  if you submit with runs: 10000000 with repeats 4, you should get an expensive enough calculation to see the blockage.
21:48 bpmedley I think that shows what jberger mentioned earlier about CPU usage.
21:49 pink_mist earlier when?
21:49 Grinnz jberger: yes I discovered this happened in mysql a few weeks ago and then saw it also happens in pg
21:49 Grinnz i discovered it because one of our tables has IDs in the millions now :P
21:50 Grinnz with nowhere near that many rows
21:50 demian bpmedley: for sure, is there a way to get the update client side, by incrementally carrying out the work?
21:50 jberger Grinnz: yeah, we have 30 items doing regular upserts, and we are suddenly in the 5000s range
21:50 jberger :o
21:50 demian i.e. update the page without reloading for each little chunk of work
21:50 bpmedley demian : I believe that would be a lot more work then just using one of the fork modules.
21:50 bpmedley Or, just minion..
21:54 demian ok, I think I get it.  With each button click in the above example, a new websocket opens and the page is updated without reloading.  You can update arbitrarily from within the websocket with multiple sends.  Is this correct?
21:55 demian I see the benefits of sending a heavy job into the background, I'm using this toy problem to better understand how the websockets work
21:56 demian You can NOT update arbitrarily  from within... (error sorry)
21:57 jberger yeah, you are basically just using a websocket as if it were an http req/res ajax request
21:57 jberger websockets are cooler when you actually use server push
22:00 demian is there a way to restructure the example to do so?
22:00 demian I'm googling "websocket server push"
22:00 bpmedley demian : My understanding is that $c->send is a non-blocking operation that will queue up the data for sending at a later time.  Later, in this example, would be after your CPU intensive task.  Also, you could use callbacks and separate the work in chunks I guess; however, that seems fraught with frailty.
22:02 bpmedley demian : Basically, you must give the event loop time for its operations: http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#REAL-TIME-WEB
22:04 Grinnz when you want to wait for each step, you might want to consider delays
22:04 Grinnz or more precisely, when you want to do something once each step completes
22:05 bpmedley Grinnz : I guess the foreach (1 .. $itr) loop could be broken up in the delay steps?
22:05 Grinnz something like that
22:05 purl something like that is totally possible
22:06 demian that was my hope in using the 1 .. $itr loop.  Break up the problem, and stream back after each itr
22:06 bpmedley demian : http://mojolicious.org/perldoc/Mojolicious/Plugin/DefaultHelpers#delay
22:06 demian I will try with the delay
22:06 demian thanks!
23:02 lluad joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary