The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2016-04-05

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:38 woz joined #mojo
00:44 vicash hello. Suppose a system that runs a Minion::Worker is overloaded (using close to 100% CPU) due to some non-Minion task and a CPU-intensive Minion task gets enqueued on the same system, how do I go about making the worker execute the enqueued task when say the average CPU load is < 70% ?
01:13 tchaves joined #mojo
01:24 asarch joined #mojo
01:38 cpan_mojo Mojolicious-Plugin-SetUserGroup-0.004 by DBOOK https://metacpan.org/release/DBOOK/Mojolicious-Plugin-SetUserGroup-0.004
01:42 bpmedley https://github.com/brianmed/ForkAndGo <-- Here is the latest version of my plugin that runs perl code at startup in a child process.  The children should auto-cleanup when the parent dies.  Also, when restarting via hypnotoad the children cleanup and restart.
01:46 bpmedley Seems to work for starting a minion worker along side a server.
01:46 bpmedley I would be eternally grateful if anyone would code review and/or test.
01:47 ilbot2 joined #mojo
01:47 Topic for #mojo is now ???? cheers | http://mojolicious.org | http://irclog.mojolicious.org | http://code-of-conduct.mojolicious.org
01:48 thowe What would be the typical use case?
01:49 bpmedley thowe : I'd like to make an app that need support for batch jobs and will need no install and only require one command to launch everything.
01:49 bpmedley Possibly a video encoder wrapper.
01:50 thowe huh...  I was thiking of something, but I'm not sure I fully understand the idea...  Let me run it past you.
01:51 thowe Every so often, call it quarterly, we refresh our phone number prefix database and generate new routing tables for our phone switch...
01:53 thowe I want to periodically generate these for download from a static location.  If they don't exist (or the latest doesn't exist), then they should be generated.  This may as well happen on startup as well.
01:53 bpmedley I'd probably create a command for that and use cron.
01:54 bpmedley http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Adding-commands-to-Mojolicious
01:54 thowe yeah, but I want people to be able to trigger that command from my app.
01:55 bpmedley Sounds like an ideal minion use case.
01:55 thowe I actually already have some commands that I've added.  I have one that generates the new style user passwords and another that generates some of the regulatory reports, but I want those available inthe app too.
01:56 bpmedley Perhaps use libraries with shared code?
01:56 thowe Probably a minion use case, but the thing that really makes me want to use minion is invoice generation...
01:56 thowe Oh, I already do that, the business logic is largely in the DBIC lib
01:57 bpmedley If I was adding interactivity to system commands I'd either use minion or one of the Fork modules.
01:58 thowe I want a user to be able to trigger the invoice generation from my app.  It will need to generate a reusable chunk of jason, which goes in the database, but also generate the PDF and stick it in a dir for download.
01:58 thowe That's my big "I want to use minion" thing ^^
01:58 bpmedley Sounds fun.
01:58 thowe but I should have said json
01:59 pink_mist JSON and the Golden Fleece
01:59 thowe You were telling me before that minion might also be good for dealing with long running processes.  Like my queries that take a minute.
02:00 bpmedley thowe : These days you have a variety of ways to deal with that issue.
02:00 thowe I'll get to it soon, right now I need to get back to my non-blocking Pg thing...  I didn't get to it today.
02:01 thowe bpmedley, such as?  I'm all ears.
02:02 bpmedley Sure, Minion; ReadWriteFork; ForkCall; Mojo::Pg in async mode; etc.
02:03 thowe Oh, yeah, Mojo::Pg is what I am going to use, but I wonder what happens if my browser times out waiting for results?
02:03 bpmedley Understood, timeouts are important.  Also, Minion::Notifier might be handy.
02:03 thowe the name alone has my attention.
02:03 bpmedley The issue is not simple, yet we have lots of really sweet tools at present.
02:07 bpmedley s1037989 : Are you around?  Do you have time for testing my ForkAndGo plugin?
02:09 s1037989 bpmedley: Hey!
02:09 bpmedley vicash : Can you use this: https://metacpan.org/pod/Minion::Job#retry?  Something like $job->retry(delay => 15) if $self->overloaded?
02:10 s1037989 I was just going to ping you about that...  Is FAG (sorry)  your new way about doing the Minion workers?  Have you given up on something that could be built-in to Mojo?  Or is this a stepping stone?
02:10 s1037989 bpmedley: Let me know what I can do!
02:11 bpmedley s1037989 : I believe the new approach is much more flexible if I got the API and code correct.  The current setup requires 0 Mojolicious changes.
02:11 bpmedley s1037989 : https://github.com/brianmed/ForkAndGo <-- Would you clone this and run the example?
02:13 Grinnz thowe, if you're worried about the browser timing out you are definitely in Minion territory
02:13 Grinnz You should never run things for a request, even forked, that take longer than a minute to complete
02:14 s1037989 bpmedley: Cloned.  Running.  Browsed to.
02:14 Grinnz 30 seconds is pushing it really
02:14 Grinnz so at that point, you have a minion do it, and some javascript to check for the results in the database
02:14 bpmedley s1037989 : $ echo joy | nc 127.0.0.1 4000
02:15 s1037989 bpmedley: I got joy in my log.
02:15 tchaves joined #mojo
02:15 thowe Hmmm...  Then I need a controller for checking results, maybe.  And some code that keeps track of the fact that things need to be checked.
02:15 bpmedley s1037989 : Cool.  Do you have one from the GET / request and another from the nc?
02:16 s1037989 When I execute joy, I get this in my log: [Mon Apr  4 21:16:11 2016] [debug] 15896: ForkAndGo server: 4000: accept [Mon Apr  4 21:16:11 2016] [debug] 15896: read: joy
02:16 bpmedley Awesome!  And if you do "$ curl http://127.0.0.1:3000" ?
02:18 s1037989 bpmedley: When I start the daemon I get the following immediately: https://gist.github.com/s1037989/dd82688569198a4a9f364170d76025f4
02:19 s1037989 $ curl returns Hello:6
02:19 s1037989 Log shows GET "/"
02:20 bpmedley s1037989 : "$ export PERL5LIB=lib"  <-- Please try this
02:20 bpmedley Run that, and then run the daemon.
02:21 s1037989 See the second file in the gist
02:22 bpmedley Hrmm.
02:23 bpmedley Would you try the curl once more pls?
02:23 s1037989 Got back Hello:7 and this in my log: [Mon Apr  4 21:23:28 2016] [debug] Performing job "7" with task "echo" in process 16032 [Mon Apr  4 21:23:28 2016] [debug] 15987: ForkAndGo server: 4000: accept [Mon Apr  4 21:23:28 2016] [debug] 15987: read: Joy: 15985
02:23 bpmedley I think the issue is that the TCP server on port 4000 was not accepting connections yet.  I didn't account for that.
02:25 bpmedley s1037989 : What just happened is that w/ one command you started a minion worker; tcp server; and web server.
02:25 bpmedley Is there a clean shutdown with CTRL-C?
02:25 s1037989 Great!!
02:26 s1037989 Well, considering I'm a monster with kill -9 a'plenty, yeah, it's clean.  :D  But this is what happened: ^C[Mon Apr  4 21:25:41 2016] [info] 15987: Child exiting: close: 15987 [Mon Apr  4 21:25:41 2016] [info] 15986: Child exiting: close: 15986
02:26 bpmedley Thank you.  What do you think of the ForkAndGo.pm code?
02:28 s1037989 bpmedley: So far I've only looked at the eg.  I'm bothered by what looks like a plethora of boilerplate.  Are the blocks on 11 and 48 boilerplate, or something else?
02:28 s1037989 i.e.  I'm thinking just app->plugin('ForkAndGo') should be all that's needed for an app developer...  Yes?
02:29 s1037989 TBH, I don't know what the blocks on 11 and 48 are doing.  :)
02:29 bpmedley s1037989 : I'm not sure if I would call that boilerplate.  However, the plugin requires that the user setup the code for execution.
02:29 bpmedley With just the plugin there is no code executed.
02:30 bpmedley The minion worker example is simply a perl code sub that happens to launch minion.
02:30 s1037989 Would an app dev have to provide the blocks from 11 and 48 in every app they create?
02:30 bpmedley Yes, because I can't guess what the user will want to execute.
02:31 s1037989 So...  perhaps ForkAndGo is more generic than just Minion: Great!  Could there be a plugin to support launching minion specifically so that there's nothing for a user to setup?
02:31 bpmedley Hrmm, not more generic.  Just simply generic is the goal.
02:31 bpmedley s1037989 : That makes sense.
02:31 s1037989 If you could have built-in support for Minion out of the box, then (me personally) I think you've got a winner!
02:33 s1037989 I could understand the app dev having to config Minion, but ForkAndGo should just know that Minion is plugged into the app and just do it.
02:33 bpmedley s1037989 : What you say makes sense; however, I don't want to second guess the user.  I want them to have control over what is executed.
02:34 s1037989 Yes, yes!  But defaults are nice.  :)
02:34 s1037989 Mojolicious loads up Mojo::HelloWorld if you don't tell it anything else.
02:34 bpmedley I'm thinking of something like plugin(ForkAndGo => "minion")
02:35 s1037989 Ok!  That's solid!
02:35 s1037989 I'm just thinking that the use case of Minion is strong / frequent enough that to have that built-in would be a good game plan.
02:35 bpmedley Thanks for the suggestion, one sec.
02:36 s1037989 To that end, would it even make sense to support the notion of plugins for your plugin?  That is allow an app dev to have reusable code for all these fork calls?  If I have 20 apps that each need Minion, I'd hate to CnP blocks 11 and 48 into each app.  amirite??
02:37 bpmedley If the plugin gets popular, then yes.
02:38 s1037989 :D  But to start, make a special case for Minion?
02:38 s1037989 While also fully supporting the generic fork calls, of course!
02:39 noganex_ joined #mojo
02:40 bpmedley s1037989 : I'm working on the minion special case now.
02:41 s1037989 bpmedley++
02:41 s1037989 I trust you agree...  I don't want to tell you how to do your thing!  :D  Just feedback as a user.
02:42 genio jberger++
02:42 genio preaction++
02:42 jberger ?
02:42 genio Alien::Base makes some things very simple
02:42 s1037989 genio++ You get one, too.
02:43 jberger Ah
02:43 jberger genio: are you on #native?
02:43 s1037989 Oh I thought we were just dishing out ++ all around.  :D
02:43 genio or was that plicease?  oops.
02:43 genio plicease++
02:44 jberger I wrote the original
02:44 jberger plicease has taken the reins
02:44 genio jberger: Nah. I've got too many channels open to pay attention to at the moment.
02:44 jberger k
02:44 jberger It's there if you need it
02:45 jberger I'm very glad you like it
02:46 s1037989 bpmedley: Kill two birds...  https://ide.c9.io/s1037989/junk-trunk
02:46 bpmedley Sweet, one sec
02:48 s1037989 bpmedley: You up for a while?  Maybe it's afternoon where you are...  I'll BB in 30min.
02:48 bpmedley s1037989 : Understood, thx
03:11 s1037989 bpmedley: I'm back.
03:19 bpmedley s1037989 : https://github.com/brianmed/ForkAndGo/blob/master/eg/minion.pl
03:20 s1037989 bpmedley++  Nice!!  Looks good here.  I like the code from an app dev POV.
03:21 bpmedley Thanks!
03:22 bpmedley https://github.com/brianmed/ForkAndGo/blob/master/lib/Mojolicious/Plugin/ForkAndGo.pm <-- jberger , do you happen to have time for perusal again?
03:22 s1037989 Man...  yeah, this is really good.  A simple plugin to allow someone to build up an app and have a minion worker doing its thing.  If I shut down my app it shuts down my Minion.  If I start my app it starts my Minion!
03:23 s1037989 Looking at my process table, I see my perl minion.pl process and a sub process for it!
03:24 s1037989 Is it possible that I could control the number of Minion processes?  Or is ForkAndGo designed to handle just a single process?
03:24 s1037989 e.g.  plugin ForkAndGo => { process => ["minion" => {workers => 4}] };
03:24 bpmedley s1037989 : Via the programmatic API you can launch several.
03:24 s1037989 bpmedley: Great!
03:25 bpmedley Hrmm, would you try this: plugin ForkAndGo => { process => ["minion", "minion"] };
03:26 s1037989 Looked like it started up an extra process, but ps axf shows perl [defunct] and also a running perl sub process
03:26 s1037989 I like the idea behind how that would work!
03:26 bpmedley The minion, minion was untested.
03:27 s1037989 :D  Seems like a solid idea, tho!
03:28 s1037989 bpmedley: There's no reason to consider that this should be used for dev only, right?  That is, this should be perfectly viable for a production deployment?
03:29 bpmedley s1037989 : Hrmm, depends on how bold you are.. :)
03:31 s1037989 bpmedley: Well... I get it's beta code.  It needs some deeper testing and all.  But, theoretically, the *approach* is solid for production?  Put another way: long term, your goal would be for this to be appropriate for production deployment?
03:31 bpmedley s1037989 : Yup, production deployments are the goal.
03:32 s1037989 Great!
03:37 bpmedley s1037989 : Please pull and try the "minion", "minion" again, if you have time.
03:38 s1037989 bpmedley: Success!
03:38 bpmedley s1037989++
03:38 bpmedley Try 10.. ;)
03:38 s1037989 bpmedley: Solid!
03:39 s1037989 That's what I like to see.  That is a really great packaged app -- all in one!
03:40 s1037989 And running minion.pl with morbo, any change that Morbo picks up, it would restart the perl process and all the sub processes as well, yes?
03:41 bpmedley morbo is probably not going to work with this.. just guessing..
03:41 s1037989 Is it correct that the sub process is labelled as: "perl -Ilib eg/minion.pl daemon -l http://*:8080"
03:41 bpmedley Sounds accurate
03:41 s1037989 In other words...  It looks like it's just preforking.  But all the subprocesses are the Minion workers, yes?
03:42 s1037989 None of the subprocesses are actually listening on 8080 as specified are they?  Sorry, I don't know much about forking and that.
03:42 bpmedley I understand your concern.  Perhaps you could try running a job via "$ curl http://127.0.0.1"
03:42 bpmedley $ curl http://127.0.0.1:3000
03:43 s1037989 8080 since I'm listining on 8080, right?
03:43 bpmedley Sorry, yes, 8080
03:43 bpmedley Did I remember correctly that you're on BSD?
03:43 s1037989 Yeah everything worked as expected...  I'm just wondering if the process table is correctly reflecting what the process is actually doing?
03:43 s1037989 No, I'm on Linux.  Ubuntu.
03:44 bpmedley Hrmm.  I might could change $0 dynamically for more straight forward visual inspection.
03:44 s1037989 That's kinda what I was thinking...
03:45 s1037989 Cuz if I ran minion straight up (as per Minion docs) my process table would show (perl minion.pl minion worker)  Correct?
03:47 bpmedley s1037989 : If you'd like, you can pull and test the $0 change
03:47 s1037989 bpmedley: Still shows as daemon -l
03:48 bpmedley Odd, do you see the $0 change in the code?
03:48 s1037989 line?
03:48 bpmedley https://github.com/brianmed/ForkAndGo/blob/master/lib/Mojolicious/Plugin/ForkAndGo.pm#L97
03:49 s1037989 Yep!
03:49 bpmedley Dunno.  Not sure about $0 semantics.
03:50 s1037989 $ perl -E '$0 = "abc"; sleep 20'
03:50 s1037989 That worked.
03:50 bpmedley That appears to be the same thing I'm dong.. hrmm
03:50 s1037989 Yep!
03:52 s1037989 bpmedley: Success after all!
03:52 s1037989 perl -Ilib is no good...  env PERL5LIB=lib:$PERL5LIB did the trick.
03:52 bpmedley Fascinating
03:53 s1037989 Actually, it's super weird.  I'll gist you the process table.  Stand by.
03:54 s1037989 https://gist.github.com/s1037989/dd82688569198a4a9f364170d76025f4
03:55 bpmedley zomg ponies.. seems to be working
03:55 bpmedley /usr/bin/perl eg/minion.pl minion worker <-- Isn't one of these working?
03:56 s1037989 Yeah, but look at the depth of the branching.  Before there were just 10 processes under the parent.  Now each child is a grandparent!
03:57 s1037989 TBH, perhaps the reason I showed only 10 children with no deeper depth was the issue with using -Ilib
03:57 bpmedley I believe that's because we're forking into ForkCall which then runs minion.  That may not be the optimal setup.
03:57 s1037989 When I was doing -Ilib I got this: Plugin "ForkAndGo" missing, maybe you need to install it?
03:57 s1037989 But didn't notice because it scrolled out of view.
03:59 s1037989 I don't know that it's a big deal so much...  But perhaps it looks a bit funny that each pf my 10 minion is a grandparent.  Is that excessive in someway?  Will some users not like that and decide to not use ForkAndGo?  I have no idea.  Just something that's catching my attention.
04:00 bpmedley s1037989 : What you say makes sense.  I might want to optimize the builtin minion process setup.
04:01 s1037989 Optimizing: That's very Mojo!  :D
04:10 bpmedley s1037989 : I remember, I wanted to be able to have an IOLoop present in the forked process' parent, so I used ForkCall
04:10 s1037989 bpmedley: Under hypnotoad I'm getting this: {"message":"Job ID not found","error":1}
04:10 bpmedley s1037989 : URL?
04:11 s1037989 bpmedley: Nevermind.  :)
04:12 s1037989 Under hypnotoad, my 10 minions are all under 1 worker.  The other 3 workers have no sub processes.  That sounds logical.  But is it right?  I have no idea, just asking.  I'll update my gist...
04:12 s1037989 https://gist.github.com/s1037989/dd82688569198a4a9f364170d76025f4
04:13 bpmedley s1037989 : https://github.com/brianmed/ForkAndGo/blob/master/lib/Mojolicious/Plugin/ForkAndGo.pm#L42
04:13 bpmedley See how the code logically groups the forked children?
04:14 s1037989 What happens if that one worker goes away?  Hypnotoad doesn't care, it'll spawn a new worker, right?  But what about ForkAndGo?  What if that one worker dies, what will ForkAndGo do?
04:14 bpmedley Support for monitoring and relaunching of children is not supported, at present.  Probably a nice feature to add.
04:14 s1037989 Yep, I see that at L42!
04:15 s1037989 Ok, good to know.  Indeed, Rome wasn't built in a day!  As you said, it's probably not production ready yet.  :D
04:17 s1037989 I killed that one worker and all ForkAndGo's died with it.  They did not get gobbled up by init.  (I don't know a ton about OS's, but I thought when a parent dies, the children become orphans and adopted by init?)
04:18 bpmedley s1037989 : You killed a minion worker?
04:18 s1037989 No, I killed a hypontoad worker
04:18 s1037989 The one worker that was the parent to all the minions.
04:19 bpmedley That's what is supposed to happen.  When the children's parent dies, then they go away.
04:19 s1037989 Gotcha!
04:20 bpmedley https://github.com/brianmed/ForkAndGo/blob/master/lib/Mojolicious/Plugin/ForkAndGo.pm#L134 <-- This code monitor's for a closed event on the pipe between each child and parent.  When the pipe breaks, the child dies.
04:20 s1037989 Great -- so hot deployment of hypnotoad server kills all the app processes and minion processes, too, but then starts the app processes and the minion processes, too!
04:21 bpmedley s1037989 : Is that the behaviour you would expect?
04:21 s1037989 bpmedley: Yep!
04:22 s1037989 I guess the only open item in that regard is if one process dies (the process with the minions) then the Minions die and have no rebirth.  IIRC, hypnotoad can kill a worker at any time, it might want to do some cleanup or who knows what.
04:26 bpmedley Eep, I forgot about that.
04:27 bpmedley I've got to goto bed, tho.  TTYL
04:27 s1037989 np great work!!  Until next time...
04:43 inokenty-w joined #mojo
05:18 melo1 joined #mojo
05:45 dod joined #mojo
05:50 dod joined #mojo
06:05 woz joined #mojo
06:05 woz joined #mojo
06:08 dod joined #mojo
06:12 cpan_mojo Mojolicious-Plugin-RoutesAuthDBI-0.140 by MCHE https://metacpan.org/release/MCHE/Mojolicious-Plugin-RoutesAuthDBI-0.140
06:12 Bender DAAAHUUUUUT!!!!
06:18 Vandal joined #mojo
06:24 woz joined #mojo
06:27 woz joined #mojo
06:30 woz_ joined #mojo
06:39 woz joined #mojo
06:43 Guest91481 joined #mojo
06:44 Guest91481 hi all
06:47 Guest91481 joined #mojo
07:02 AndrewIsh joined #mojo
07:18 csroli joined #mojo
07:25 trone joined #mojo
07:30 woz joined #mojo
07:33 jicege joined #mojo
07:33 woz joined #mojo
07:38 jicege Hello ?
07:43 jicege I wonder if Mojolicious fits for strong rest server application and if it is, is there any document about this ?
07:44 nicomen What's the difference between strong and regular rest server application?
07:45 jicege good question, just forgot the word "strong"
07:46 nicomen well then it for sure is fit, it's a fast framework, the rest part can be accomplished with regular code, or with specialized plugins
07:47 nicomen to find them search for mojolicious rest in a search engine
07:48 jicege Thx. Is there a a sample of code/doc  that could inspired me ?
07:49 nicomen this looked simple to set up some crud at leat: http://search.cpan.org/~abhiisnot/Mojolicious-Plugin-REST-0.006/lib/Mojolicious/Plugin/REST.pm
07:49 jicege Thx ! I take a look.
07:51 dod joined #mojo
07:59 melo joined #mojo
08:00 osfabibisi joined #mojo
08:07 woz joined #mojo
08:28 dotan_ joined #mojo
08:29 vytas joined #mojo
08:43 bpmedley jicege : http://blogs.perl.org/users/brian_medley/2014/11/using-minion-with-a-rest-api.html <-- This is what I do.   See how the under acts as a REST api authentication mechanism?
08:44 batman jicege: there's also a swagger plugin if you want to write a shareable rest spec. http://thorsen.pm/perl/programming/2015/07/05/mojolicious-swagger2.html
08:44 bpmedley jicege : Swagger2 is awesome in Mojo with batman's code
08:49 batman :)
09:16 jicege bpmedley & batman : Thank you 2 :)
09:17 sebbe left #mojo
09:17 sebbe joined #mojo
09:31 eseyman joined #mojo
09:59 woz joined #mojo
10:15 sri bpmedley: hmm, afraid i see a lot of problems
10:16 bpmedley sri : Awesome, glad you took time and looked over the code.. Would you detail the issues?
10:17 sri starts with the globals, which seem to prevent the use of the plugin in multiple mounted apps
10:18 bpmedley Ok, cool.  I didn't even know about that..
10:18 sri apps are not unique
10:19 bpmedley What do you mean by not unique?
10:19 sri $ENV{HYPNOTOAD_EXE} and $ENV{HYPNOTOAD_REV} are implementation details, and not part of the public api
10:19 sri i mean you can have an arbitrary number of apps in a process
10:20 bpmedley By app do you mean a mounted app?
10:20 sri you filter @ARGV for daemon/prefork, that's very fragile, what if someone subclasses those modules to rename the command?
10:20 bpmedley I agree @ARGV filtering is a hack.
10:22 sri actually, all the HYPNOTOAD_* and MOJO_* env variables you're using are private
10:22 bpmedley Understood, hrrm.
10:25 sri how do you notify the child process that the parent died?
10:26 bpmedley The pipe closes and this code is executed: https://github.com/brianmed/ForkAndGo/blob/master/lib/Mojolicious/Plugin/ForkAndGo.pm#L144
10:26 bpmedley kill('-KILL', $$);  <-- We kill the entire process group
10:27 sri ok, so you require that the forked process calls Mojo::IOLoop->start
10:27 bpmedley Yeah, I'm not sure if that should be a requirement.
10:29 sri ok, the fundamental idea is to use an intermediary process? so, when your app starts you fork off a watchdog process that keeps an eye on the parent, and uses ForkCall to start the actual child
10:30 sri seems overcomplicated
10:31 bpmedley My fundamental approach is off, I think.  I'm attaching the new children to one of the workers and that won't work..
10:31 sri i think what you really want is a way to manually check if the parent died in the child process
10:32 bpmedley Understood; however, how can I handle hypnotoad hot restarts?
10:32 sri without any involvement of an ioloop
10:32 bpmedley Should I only stop/start on server stop and start and ignore the hot restarts?
10:32 sri while (1) { if ($fork_and_go->is_parent_dead) { ... } else { ... } }
10:33 bpmedley Hrmm.  Let me think about that.
10:33 sri especially with minion you want graceful shutdown
10:33 bpmedley That makes sense!
10:34 sri and i think restarting the child was already mentioned, that's actually much harder
10:35 sri for that an intermediary process makes much sense actually
10:35 sri so parent -> watchdog -> child might be what you have to do in the end
10:37 sri you just don't know what the parent will run, it could be a Mojo::Server::Daemon running an ioloop, or a Mojo::Server::Hypnotoad running some custom while (1) loop
10:37 bpmedley By "parent' do you mean the Manager for Prefork?
10:37 sri yes
10:38 sri or just a daemon process, you don't know
10:38 bpmedley How can I attach to the parent in hypnotoad?
10:38 sri and you shouldn't have to know
10:38 sri startup runs in the parent
10:39 sri you'd use Mojo::IOLoop->next_tick from startup to run something in the workers
10:39 bpmedley Hrmm, let me tinker some more.
10:39 sri the more i think about it the more i like the watchdog intermediary process
10:39 sri you can handle most issues gracefully with that
10:40 bpmedley So, create a watchdog in startup and then attach all my children to the watchdog?
10:40 sri on hot restarts i would expect your processes to stop gracefully too
10:41 bpmedley Grr.. :-0
10:41 sri think about what happens during a hot restart
10:41 sri the old manager and workers die
10:41 sri you have a completely new manager
10:41 sri with all new code
10:42 sri possibly updated perl binary
10:42 sri it seems easier to me actually to just let it die with the old manager
10:43 bpmedley Hrmm.  Let me tinker with the watchdog approach..
10:43 sri i can see why you'd want to use $ENV{HYPNOTOAD_REV}, that's the only private variable i think you have to use
10:44 sri if it's done right
10:44 sri note that i'm just ignoring the system() stuff
10:45 sri that's just bad, and i'm sure you know :)
10:46 sri the whole POSIX::setsid stuff just shouldn't be there
10:46 sri imo
10:46 bpmedley That will probably go away with a watchdog.
10:47 bpmedley Attaching trinkets to workers is dumb, in retrospect.
10:51 sri just to be clear, in my mind the plugin always worked with blocks of code, not system('...')
10:52 sri so, i imagined you'd pass a block of code with a while (1) or Mojo::IOLoop->start
10:52 sri and that loop could occasiaonally call $fork_and_go->is_parent_dead to see if it should stop gracefully
10:53 bpmedley Hrmm, that may be helpful!
10:54 sri once you start with system('myapp.pl minion worker') or even exec, the whole plugin seems pointless
10:54 sri imo
10:56 sri as far as core features go, i think we have found a need for one new feature so far
10:56 bpmedley What would that be?
10:57 sri the $ENV{HYPNOTOAD_REV} thing
10:57 bpmedley Would you just need some documentation for that?  Does any code need changing?
10:57 sri even when you do "hypnotoad -s myapp.pl" it starts the app normally, and you would start your watchdog process and so on
10:58 bpmedley Yeah, that bit me a few times..
10:58 dvinciguerra_ joined #mojo
10:58 sri basically what you want is to know if the server is actually going to start up, or if the app is just loaded for introspection
10:59 sri my $app = Mojo::Server->new->load_app('./myapp.pl');
10:59 sri that will have the same problem
11:00 sri at the time the app is loaded it doesn't know its context
11:00 sri http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Application-embedding
11:00 tchaves joined #mojo
11:00 sri for that example you wouldn't want to start a minion when loading the app
11:01 sri which is why i imagine you do the whole @ARGV dance too
11:02 sri it's an interesting problem
11:03 sri there are multiple solutions i can see
11:03 ashimema anyone got any good examples of using Net::LDAP non blocking with Mojo?
11:03 ashimema I'm currently wrapping my head around ldap flows, the async flag and how it's callbacks stuff works
11:04 sri a) expose the information by all servers, maybe as a public env var, b) have a callback or hook in Mojolicious that gets called on actual server start
11:05 sri (and maybe b with a dependency on a)
11:06 bpmedley Hrmm.  How much should the app know about its server?
11:06 sri that is the big question ;)
11:07 sri i don't want to think about how this plugin would work with psgi servers ;p
11:07 bpmedley Maybe start small with knowing the state: "starting", "stopping", "running", "hot-restart" ?
11:07 sri or even the cgi backend, what should happen there?
11:07 bpmedley No clue.. gah
11:08 sri naah, it's more fundamental than knowing states
11:08 sri you basically want to know if the process will be persistent
11:08 sri that covers cgi
11:09 sri yea, that's the lowest common denominator
11:09 sri app->is_persistent or so
11:09 bpmedley app->persists?
11:09 sri lets not bikeshed
11:09 nic app->intends_to_hang_around
11:09 bpmedley Gotcha
11:10 nic app->has_potential_for_bff
11:10 * sri is not going to rush into this, you can use $ENV{HYPNOTOAD_REV} for prototyping, and we'll think some more about it in the meantime
11:11 bpmedley So, the issue is whether to have just a flag for knowing persistence vs having a callback run once persisted?
11:11 bpmedley Cool.
11:11 sri callback is not really important
11:12 sri when i talked about that i still wasn't sure what the root problem really was
11:13 bpmedley Totally, this is complicated with all the scenarios mojo works with and the tightly coupled interactions of ioloop.
11:13 sri but it's the question if the process will be persistent or short-lived
11:14 bpmedley Somewhat embarrassing - I've probably spent 20 - 30 hours on this..
11:14 sri not surprising, this is really hard
11:15 sri will prolly take a lot more time to get bullet proof
11:15 bpmedley Thanks for the validation.. took me 8 or so hours to figure out the ioloop wasn't being processed in the hypnotoad manager process.
11:16 sri with morbo it will look the same at first
11:16 sri but there it's much easier actually
11:16 sri you can treat morbo just like the daemon command
11:17 bpmedley Hrmm, that would be sweet.  Auto launching daemons in dev mode!!
11:17 sri once you get rid of all the @ARGV hacks it should just work
11:18 sri i bet if you're reading the source this line would trip you up https://github.com/kraih/mojo/blob/master/lib/Mojo/Server/Morbo.pm#L49
11:18 sri since it runs in the manager process
11:19 sri it doesn't load the app though, just initializes the socket for later
11:19 sri the app is only loaded later on in the morbo worker process
11:19 sri making it really easy for you, it looks and behaves just like the daemon command would
11:19 bpmedley Wowzers, I haven't focused on morbo at all; I'm going to have to limit my time and focus on daemon, then prefork, and then hypnotoad..
11:20 sri it's a tough subject, took me years to get right ;p
11:24 bpmedley Slightly different topic.. should the ioloop singleton have some type of stash type storage?
11:24 sri what for?
11:26 bpmedley Let me create an example script.. May be a bit later, sorry, I should of had code 1st..
11:30 kaare joined #mojo
11:47 sri the answer is most definitely no anyway ;p
12:02 woz joined #mojo
12:02 Kripton joined #mojo
12:03 woz_ joined #mojo
12:31 arthas joined #mojo
12:52 asarch joined #mojo
13:14 bwf joined #mojo
13:21 Kripton joined #mojo
13:51 mcsnolte joined #mojo
14:05 woz joined #mojo
14:20 disputin joined #mojo
14:26 disputin joined #mojo
14:29 disputin1 joined #mojo
14:30 disputin joined #mojo
14:33 disputin1 joined #mojo
14:37 disputin joined #mojo
14:40 disputin1 joined #mojo
14:52 PopeF joined #mojo
14:55 cpan_mojo Qiniu-0.07 by FUKAI https://metacpan.org/release/FUKAI/Qiniu-0.07
14:56 woz joined #mojo
14:59 lluad joined #mojo
15:11 woz joined #mojo
15:13 disputin joined #mojo
15:13 disputin1 joined #mojo
15:14 disputin2 joined #mojo
15:15 tinita joined #mojo
15:17 disputin joined #mojo
15:20 Atog joined #mojo
15:30 woz joined #mojo
15:44 woz joined #mojo
15:48 zivester joined #mojo
15:50 disputin joined #mojo
15:58 orev joined #mojo
16:02 orev what is the best way to handle performing maintenance tasks with a mojo app?  I have to do stuff like expire records in a database.  cron job would work, but I'm wondering if one could/should do it in an ioloop timer instead
16:08 jberger orev: you certainly can make a app command and have cron fire it regularly
16:09 jberger minion doesn't (yet) have a periodic job capability
16:09 jberger but it has been discussed several times
16:09 jberger in the end though, cron is pretty good at what it does
16:10 preaction you could make a Mojolicious::Plugin::Schedule, which reads a schedule from the config and sets up the IOLoop tasks. i'd search metacpan maybe someone's already done it
16:10 denny joined #mojo
16:10 orev I was looking at ioloop recurring, but cron probably works just as well
16:25 geheimnis` joined #mojo
16:25 stryx` joined #mojo
16:48 woz joined #mojo
17:01 woz joined #mojo
17:07 dod joined #mojo
17:09 trone joined #mojo
17:14 sri hmm, that new tls alternative looks interesting http://noiseprotocol.org/noise.html
17:16 disputin joined #mojo
17:20 woz joined #mojo
17:22 sri used by a billion people already :o
17:28 PryMar56 joined #mojo
17:31 woz joined #mojo
17:33 marty joined #mojo
17:42 pink_mist so uh, 1 out of every 7 people in the world? 0_o I find that hard to believe
17:42 pink_mist (or is it more than 7 nowadays?)
17:58 woz joined #mojo
17:58 disputin joined #mojo
18:17 Gedge joined #mojo
18:43 woz joined #mojo
19:00 thowe_work joined #mojo
19:01 thowe_work It seems odd to me, because I never noticed this before...  I'm using Morbo, and I see all the exact same log messages twice when loading any given route.
19:02 woz joined #mojo
19:02 thowe_work because I'm using under?
19:02 thowe_work maybe
19:02 thowe_work ?
19:04 thowe_work I see 4 repeats if there are two unders in the route...  Is this normal, or have I done something quite wrong?
19:31 woz joined #mojo
19:37 woz joined #mojo
19:51 trone_ joined #mojo
19:54 bwf joined #mojo
20:02 woz joined #mojo
20:15 lluad joined #mojo
20:18 woz joined #mojo
20:36 pink_mist that sounds wrong
20:37 thowe_work yeah, it's happening even for my /login route, which isn't under anything...
20:38 bwf joined #mojo
20:38 thowe_work I would assume I have something screwed up with routing, but I can't seem to find anything wrong..  yet, anyway
20:46 hummeleBop joined #mojo
20:50 thowe_work yeah, I bet I have something wrong with my under
20:51 thowe_work I have an / under / and I wonder if that is sort of looping around on itself?
20:51 jberger thowe_work: they are a tree
20:51 thowe_work http://paste.scsys.co.uk/509625
20:52 thowe_work I wonder if line 10 and 12 there aren't right...  or if that is even the right syntax to use with under
20:52 melo joined #mojo
20:53 thowe_work just going to /login  goes to /login twice in my log.
20:53 thowe_work going to bendor_franchise_fee loops a lot more times and then fails to load the jQuery or somesuch...
20:53 jberger can you have a space between qw and (
20:54 pink_mist seems like it
20:54 thowe_work hm, hadn't noticed that.
20:55 thowe_work doesn't effect my problem..
20:56 thowe_work is line 10 valid?
20:57 jberger thowe_work: which line is line 10
20:57 thowe_work hrm, I thought when I turned on line numbers it would be for everyone...
20:57 pink_mist thowe_work: if you use this linke, it will be: http://paste.scsys.co.uk/509625?ln=on&amp;submit=Format+it%21
20:57 pink_mist *link
20:58 thowe_work Oh, I see.
20:58 thowe_work http://paste.scsys.co.uk/509625?ln=on&amp;hl=on&amp;submit=Format+it!
20:58 thowe_work maybe that helps
20:58 thowe_work line 10 my $root = $r->under('/')->to('user#verify_authenticated');
20:59 jberger I usually use hybrid routes for my unders, but yes I think that should be correct
20:59 pink_mist eek, not highlighting :( that messes up the route list
20:59 thowe_work I'm wondering if line 10 and 12 are confusing things...
20:59 thowe_work http://paste.scsys.co.uk/509625?ln=on&amp;submit=Format+it!  sorry
21:00 thowe_work the routes listed in the route list on lines 56 and 57 make me wonder
21:01 jberger no I'm pretty sure that's fine
21:02 thowe_work hrm...  I get stuff like this with every load...  http://paste.scsys.co.uk/509626
21:03 jberger what client are you using?
21:04 thowe_work Mozilla..  Iceweasel I guess
21:04 thowe_work I could try another
21:04 jberger try with something dumber like curl or mojo get
21:08 thowe_work mojo get /  just seems to go to verify_authenticated and stop
21:08 preaction are you authenticated?
21:09 thowe_work I didn't authenticate with mojo get, so I assume not (?)
21:10 thowe_work get /logout goes to logout, but then not to /login like I would expect...  I'm not used to using that so I don't know what to expect.
21:11 thowe_work If I go to /  in browser it goes to verif_authenticated once and then logs two loads of /login
21:13 thowe_work if I rename the / to /index on line 12 it doesn't change anything.
21:13 hummeleBop joined #mojo
21:14 jberger browsers sometimes make extra requests
21:14 jberger in the background
21:14 jberger watch in the network tab (or whatever firefox's equivalent is)
21:16 thowe_work hm
21:16 lluad And/or check with curl, so you can control exactly what's requested.
21:22 thowe_work that's weird.
21:22 thowe_work why would such a thing be happening?
21:24 thowe_work meh, don't bother answering that, the things my browser decided to do aren't your problem...
21:35 woz joined #mojo
21:48 HtbaaPi joined #mojo
21:49 thowe_work Something with the way angular is loading is freaking it out.  I'm finding references to it online.  Supposedly certain order of loading jQuery and angular or certain include syntax is evil...
21:50 thowe_work not even using angular on this page.
21:54 melo joined #mojo
22:04 dvinciguerra__ joined #mojo
23:02 woz joined #mojo
23:11 bwf joined #mojo
23:33 Lee joined #mojo
23:35 thowe_work I finaly figured out that if I remove this hunk of html, angular stops reloading over and over...  I can find no explanation why.  http://paste.scsys.co.uk/509633

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary