The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2015-06-28

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:21 lluad joined #mojo
00:25 klapperl joined #mojo
00:40 jberger joined #mojo
00:55 buu Ok, where should I look for documentation about writing plugins that hook into the rendering process?
00:56 bpmedley May I ask what you want to do?
00:59 buu Generally speaking I want to insert elements into my responses based on certain request criteria
00:59 bpmedley http://mojolicio.us/perldoc/Mojolicious#HOOKS <-- Perhaps a hook?
01:00 buu oh
01:00 buu I thought those were events
01:00 Grinnz they are?
01:01 buu I thought events used ->on
01:01 buu Anyway, hook->after_render is what I wanted, thanks
01:04 bpmedley I believe events with "on" are handled by Mojo::EventEmitter and hooks are handled by the emit_hook plugin.
01:04 buu Perfect.
01:04 purl La perfection est atteinte non quand il ne reste rien à ajouter, mais quand il ne reste rien à enlever
01:08 buu Just curiously, is there anything like.. route->after that's the inverse of route->under? As in some code that executes *after* a bunch of route handlers?
01:09 Grinnz buu, that would be kind weird cause route handlers generally render something, eventually
01:09 buu I get to a similar point via the after_render hook
01:09 Grinnz if you just want to do something that doesn't affect the rendering, then yeah, after_render
01:09 buu Well, I want to affect the *output* but not the rendering of the specific route
01:10 buu For example, wrapping the output of a route in a wrapper template
01:10 Grinnz but when a route is done, their output is done... otherwise, have them call a common method to do the rendering how you want
01:10 bpmedley http://mojolicio.us/perldoc/Mojolicious#after_render ?
01:10 Grinnz after_render lets you postprocess
01:10 buu I think I mentioned that one
01:12 sri interesting to see a web framework survey (django) https://docs.google.com/forms/d/1Owv-Y_beohyCm9o2xPamdBnvjreNYoWai3rDloKZxWw/viewanalytics
01:14 buu sri: People just want to turn database tables into forms
01:14 sri yea
01:15 klapperl_ joined #mojo
01:17 buu It's not exactly a slot in their survey but I think my answer would be something like "the absolute minimal amount of code required to generate a response to a request combined with the knowledge of near infinite customizability"
01:18 buu I like to think of it as kind of the perl attitude, simple things should be easy
01:23 bpmedley https://bitbucket.org/snippets/bpmedley/Bjz5r <-- Why can't you do something like this to wrap the output in some template?
01:23 bpmedley http://mojolicio.us/perldoc/Mojolicious/Guides/Rendering#Layouts <-- Also, have you seen layouts?
01:24 sri there's also template inheritance and the content_* helpers
01:25 buu bpmedley: what do you mean why can't I?
01:25 sri you can make content blocks in a template and wrap them individually in the next template
01:25 sri there's a lot of very powerful stuff you should check out
01:26 sri http://mojolicio.us/perldoc/Mojolicious/Guides/Rendering#Template-inheritance
01:26 sri http://mojolicio.us/perldoc/Mojolicious/Plugin/DefaultHelpers#content_with
01:26 buu sri: That's EP specific right?
01:26 sri kinda
01:26 buu It's cool though
01:27 sri oh, you're not using ep?
01:27 buu Nope
01:27 sri ok, then ignore me
01:27 buu Never!
01:27 bpmedley Why can't => Can you do
01:27 bpmedley Sorry
01:27 buu sri: I'm playing around with some code that lets me render the template serverside *or* clientside
01:28 buu So I can pre-render the app on the pageload then have subsequent requests be inline
01:28 buu Without requiring the entire thing to be compiled and rendered on first page load
01:28 sri pjax?
01:28 purl it has been said that pjax is really trivial in general, just disable layouts if it's a pjax call, done
01:28 buu bpmedley: And yes, yes I can
01:28 * sri pats purl
01:28 * purl stabs
01:28 buu sri: Uh, I think so, yes
01:29 sri what purl said ;p
01:29 sri with ep i would just make the layout optional for pjax
01:30 buu Yeah, that's what I'm working on right now
01:30 sri % layout 'whatever' unless $c->req->is_xhr;
01:31 buu Oh, er, what does is_xhr look for?
01:31 sri http://mojolicio.us/perldoc/Mojo/Message/Request#is_xhr
01:32 buu was that really easier than saying x-requested-with =[
01:32 sri yes
01:32 sri click, click, ctrl+c, ctrl+v
01:35 buu ok
01:35 bpmedley buu: There are cool things like that all over the framework that makes things easier, consistent.  The code is pretty.. :)
01:48 klapperl joined #mojo
02:02 bpmedley https://bitbucket.org/snippets/bpmedley/xyqx5 <-- buu, not sure how well this achieves the goal; however, it may be a start.
02:02 bpmedley The above is for the cache question.
02:07 noganex_ joined #mojo
02:09 bpmedley buu: Does it make sense?
02:14 neilhwatson joined #mojo
02:43 Adura joined #mojo
03:01 bin_005 joined #mojo
03:03 klapperl joined #mojo
03:11 mattastrophe joined #mojo
03:45 klapperl_ joined #mojo
03:50 davido__ joined #mojo
03:57 klapperl joined #mojo
04:05 irq joined #mojo
04:06 klapperl_ joined #mojo
04:07 mattastrophe1 joined #mojo
04:09 jnbek joined #mojo
04:11 klapperl joined #mojo
04:20 klapperl_ joined #mojo
04:46 kaare joined #mojo
04:53 dotandimet joined #mojo
05:08 klapperl joined #mojo
05:20 klapperl_ joined #mojo
05:25 klapperl joined #mojo
05:31 klapperl_ joined #mojo
05:37 klapperl joined #mojo
05:43 klapperl_ joined #mojo
05:59 klapperl joined #mojo
06:04 klapperl_ joined #mojo
06:27 klapperl joined #mojo
06:34 jontaylor joined #mojo
06:39 batman bpmedley: templates are already cached, so i'm not sure how much gain it is from caching the output from the template.
06:39 batman i would 1) add a reverse proxy with caching (nginx, varnish) 2) use https://metacpan.org/pod/Mojolicious::Plugin::Memorize
06:40 batman (so that's either 1 or 2, i don't think i would do both)
06:48 berov joined #mojo
06:49 jontaylor I was having a think earlier about dropping in some SQL monitoring into a mojo app for a client, which is fairly straightforward to do if its a synchronous app, but seems a lot harder for an async app
06:52 batman jontaylor: what do you mean about "monitoring" ?
06:52 doby joined #mojo
06:52 batman like how long a SELECT takes..?
06:52 jontaylor batman: Correct, the number of and duration of SQL queries for a particular web request
06:53 batman jontaylor: can you post the blocking code to a pastebin?
06:53 batman it should be fairly simple to do the same non-blocking
06:54 jontaylor batman: I’ve not started it yet, i’m just trying to consider the approach so I don’t code myself into a corner
06:54 batman ok. let me give you an idea... *brb*
06:59 batman jontaylor: https://ssl.thorsen.pm/paste/0a033fcca956
06:59 * jontaylor clicks
07:00 batman oh. some copy/paste stuff that wasn't required... like importing qw( gettimeofday tv_interval )
07:01 batman haven't actually run the code, so there might be more weird stuff
07:01 jontaylor ahh yes, I understand what you mean
07:01 jontaylor thank you for your time doing that
07:01 batman you're welcome :)
07:02 batman this is pretty much what Mojolicious and Mojolicious::Controller does to log how long a request takes
07:02 batman jontaylor: https://metacpan.org/source/DBOOK/Mojolicious-6.12/lib/Mojolicious/Controller.pm#L205
07:03 jontaylor One thing that would be neat to acheive is to be able to bolt this onto a legacy application, without having to modify the code for it
07:04 jontaylor for that approach I was thinking of hooking into DBI itself, which would get support for DBI, Dbix::class and rose::db
07:04 batman jontaylor: are you sure that logging how long a complete request takes is not enough?
07:04 jontaylor batman: doing that does acheive 80% of the value yeah
07:04 batman ok
07:04 jontaylor One thing I always like about ruby on rails, was tools like new relic
07:05 jontaylor you could get live performance stats for your apps, see how long it spends in the DB, how long it spends in webservice requests etc
07:05 batman ok.
07:05 jontaylor There is no reason we couldn’t have something similar in perl
07:06 batman sure. have you tried to search cpan for something like that?
07:06 batman maybe someone has made something you could extend...
07:06 doby https://metacpan.org/pod/NewRelic::Agent
07:07 batman :)
07:07 jontaylor That new relic agent is good, and I will most likely extend that with this
07:07 jontaylor thats a very low level utility that works great for sending requests over to new relic
07:08 jontaylor i’m thinking something like Mojolicious::Plugins::NewRelic
07:08 jontaylor that makes use of that
07:08 jontaylor but also hooks into Mojo with some hooks
07:08 batman that shouldn't be too hard
07:08 irq_ joined #mojo
07:09 jontaylor its quite easy to do the entire lot automatically for sync apps, DB and webservice tracing too
07:09 batman magically hooking into async db requests might be harder though... (probably not impossible)
07:09 jontaylor indeed
07:09 jontaylor since requests can overlap can’t they, while also using the same DB handle
07:10 jontaylor one idea I had was to substitute in a “proxy” DB handle
07:10 jontaylor but this would require code changes in the project to support that I guess
07:12 jontaylor an interesting problem though
07:19 batman hm... i thought i was on to something regarding the validation/filter/coerce thing, until i added documentation to the guides :( https://ssl.thorsen.pm/paste/aa4a89986d50
07:19 doby joined #mojo
07:19 zivester joined #mojo
07:24 klapperl_ joined #mojo
07:28 Adurah joined #mojo
07:43 Vandal joined #mojo
07:44 D4RK-PH0ENiX joined #mojo
07:46 klapperl joined #mojo
07:48 mgrimes joined #mojo
07:49 d4rkie joined #mojo
07:51 zivester joined #mojo
08:03 melo joined #mojo
08:40 dave joined #mojo
08:51 trone joined #mojo
09:00 amon joined #mojo
09:00 klapperl_ joined #mojo
09:12 jontaylor batman: Had a quick little play with a toy mojo::lite app to test tracing SQL queries
09:12 jontaylor https://ssl.thorsen.pm/paste/f1c2a8486ffa
09:13 jontaylor as expected the approach i’ve taken works fine for synchronous actions, and fails miserably for async ones
09:14 jontaylor and you get output like this
09:14 jontaylor https://ssl.thorsen.pm/paste/8e27d2966abb
09:15 jontaylor I imagine it works because the first step takes place immediately, whereas the second step is a sub ref executed by the event loop
09:15 jontaylor so my setting of a package variable with local is already undone at that point
09:22 _dave_ joined #mojo
09:32 bpmedley jontaylor: Can you use this: https://metacpan.org/pod/DBI#Tracing-to-Layered-Filehandles
09:32 jontaylor bpmedley: thats what i’m using :)
09:33 bpmedley Doh; so early, I missed the method call.
09:33 mudler joined #mojo
09:33 jontaylor its my bad, I didn’t include the two supporting files
09:33 batman jontaylor: sorry. i'm bailing out on this, since i'm not that interested.
09:33 batman sorry :(
09:34 jontaylor thats ok, I was trying my luck :p
09:34 jontaylor other supporting files btw: https://ssl.thorsen.pm/paste/d4f51cbfefb1
09:34 bpmedley batman: The Memorize plugin looks like the way to go for that issue.
09:34 batman i also try to avoid raw DBI as much as possible. at least using Mojo::mysql or Mojo::Pg
09:34 batman bpmedley: cool
09:35 jontaylor Oh yeah, I never use raw DBI, the only reason i’m doing it for this is if I go to that level, then I get DBIx::Class and Rose::DB support for free
09:35 batman aha. yeah, that makes sense
09:37 jontaylor and as it turns out from a quick google, Mojo::mysql support too :)
09:38 bpmedley jontaylor: Just so I understand, you don't have an issue, we're just talking about ways to monitor an app, right?
09:39 jontaylor bpmedley: I have an issue in that I can’t get it to work for asynchronous apps with my current approach
09:39 jontaylor I just can’t track at this level, which action is doing which SQL query
09:40 bpmedley Are you using Mojo::Pg?
09:41 jontaylor bpmedley: no just raw DBI at the moment
09:42 jontaylor so those DB calls themselves block, but the issue comes in with the second subref in the $c->delay() call
09:43 jontaylor as that subref when its executed, has a different parent in the stack, probably the IO Loop itself
09:44 jontaylor I’m starting to think its not possible to do in app SQL tracing with an async app, without actually modifying the app itself
09:49 jontaylor Yup, verified by printing out a stacktrace :(
10:19 klapperl joined #mojo
10:33 sh4 joined #mojo
11:37 klapperl_ joined #mojo
11:40 dvinciguerra joined #mojo
11:47 bpmedley jontaylor: https://bitbucket.org/snippets/bpmedley/ok7qq <-- Thoughts?  I had to run with DBI_TRACE=1 .. dunno why.
11:51 jontaylor bpmedley: looking now :)
12:01 jontaylor so this is pretty good, if I run the async one, it tells me all the queries done during that call
12:01 jontaylor and it even seems to work if I overlap a second one a few moments later
12:03 jontaylor does this work because the log is now in the scope of the controller?
12:03 bpmedley Let me compare to your original.
12:04 jontaylor i am amazed I can run two of them at the same time, and they don’t get confused with each other
12:05 bpmedley That's one reason.  Also, I believe an issue was that your $fh would go out of scope and close.
12:05 bpmedley What you have here is a fantastic way to add DBI tracing at the route level.  Very sweet.
12:06 bpmedley I'm not 100% sure how to do request_count, tho.
12:10 jontaylor yeah this is looking like it could be really handy
12:11 jontaylor Running locally I get a small issue, if I run an async one, and while thats running then hit the sync one
12:11 jontaylor it prevents the last query from running that inserts last_one
12:12 jontaylor ahh, the query itself does run
12:12 jontaylor but doesn’t get logged
12:12 neilhwatson joined #mojo
12:14 bpmedley https://bitbucket.org/snippets/bpmedley/ok7qq <-- This should do a request count
12:14 bpmedley Sorry, running locally?
12:15 jontaylor I just meant running it myself, I am going to take a look at that now :)
12:15 bpmedley How are you running?  What command(s)?
12:16 jontaylor DBI_TRACE=1 perl dbLog.pl daemon
12:16 bpmedley Odd.  That's how I'm running it and it seems to work.
12:19 jontaylor I’m getting a Can't locate object method "req_count" via package "Mojolicious::Controller" at dbLog.pl line 15.
12:20 bpmedley Sorry, I updated the snippet.
12:20 jontaylor okies
12:21 bpmedley I'm guessing here.. that DBI trace is global scope and only one request per process is sane thing to do.
12:23 bpmedley https://bitbucket.org/snippets/bpmedley/ok7q5 <-- I get this weirdness when I run the /delay request and then /.
12:23 jontaylor yep thats what I see too
12:23 jontaylor but this is a lot closer now
12:24 bpmedley That's a great idea BTW, glad you posted code.
12:28 jontaylor I think this proves the concept nicely, and a lot of people could get some benefit from it
12:28 jontaylor from my testing on this, it works if you just do sync, or just do async
12:28 jontaylor which is probably 99% of mojo installations out there
12:29 sri that sounds odd
12:30 sri mixing sync and async in an app should be no problem whatsoever
12:30 jontaylor oh its not a mojo problem whatsover sri
12:30 sri at least with modules like Mojo::Pg
12:30 jontaylor we are just trying something a bit hacky
12:31 jontaylor per route SQL logging
12:31 klapperl joined #mojo
12:31 jontaylor i should say per request SQL logging really
12:32 sri oh
12:33 sri that will never work async
12:34 sri even with sync you'll get in trouble
12:34 sri all it takes is a sync sql query in a timer
12:34 jontaylor with sync it should be ok as the underlying DB handle is only used by one request at a time
12:35 sri only if your app doesn't use anything non-blocking ever
12:35 jontaylor indeed
12:35 sri no timers, no events, no websockets, no callbacks at all basically
12:36 jontaylor There are a few ways to hack it to work, access to the controller object from within a step would work
12:36 jontaylor or an event before a step is fired by the event loop, to set up some values
12:37 jontaylor but I don’t think these are reasonable use cases
12:38 klapperl_ joined #mojo
12:38 jontaylor The core of the problem is being able to link something from the event loop, back to which request originally triggered it
12:40 sri even the controller object may change per request if you use multiple controllers
12:41 sri like an auth controller followed by a normal controller
12:41 sri you would have to store a uuid in the stash to identify it reliably
12:42 jontaylor that would be ok really
12:43 klapperl joined #mojo
12:43 jontaylor the idea is to be able to report on things like “This route, executed 6 SQL queries that took 23ms”
12:43 jontaylor and then eventually extend it with things like keep tracking of Mojo::UserAgent, LWP etc
12:43 jontaylor to be able to say how long a request spent doing external requests
12:44 sri a route can lead to multiple controllers
12:44 sri with ->under
12:45 sri ->under('/foo')->to('auth#check')->get('/bar')->to('somethingelse#do_stuff')
12:45 sri that's two separate controllers
12:46 jontaylor I’ll have to check the code to see if the controllers are instantiated with something that could be used to dedupe that
12:46 sri not to my knowledge
12:48 jontaylor so I guess a Mojolicious::Routes::Pattern object is the unique point to work from
12:49 sri no
12:49 sri shared between all requests
12:49 sri when i said you shold store a uuid in the stash, i meant it
12:50 ToApolytoXaos joined #mojo
12:50 jontaylor understood
12:52 jontaylor It should be possible to get software analytics working on Mojolicious the same way NewRelic did it for Rails
12:53 jontaylor its the last thing I need to sell the idea of a mojo app to a client
12:53 sri rails sucks at async though, so it's much easier to do there
12:53 sri better inspiration might be node.js analytics solutions
12:55 jontaylor Rails also has the benefit/downside, that you can fairly safely assume ActiveRecord is being used
12:56 klapperl joined #mojo
12:56 sri the mojolicious "default" is Mojo::Pg
12:59 jontaylor The saving grace is that nearly everyone is using DBI under the hood
13:00 jontaylor so if I can support that, i’ve got most things covered
13:04 jontaylor so how about, I create a UUID, as a kind of request id. I can stash that in any and all controllers invoked.
13:04 jontaylor and io loops created via $c->delay I could also stash the UUID
13:06 jontaylor sorry I shouldn’t say IO loop created, as that already exists, I mean when steps are queued
13:06 klapperl joined #mojo
13:07 sri ioloops are not created by ->delay
13:07 jontaylor ^^ :)
13:08 kyshtynbai joined #mojo
13:12 klapperl_ joined #mojo
13:17 klapperl joined #mojo
13:17 sri with Mojo::Pg it actually shouldn't be too hard to get right
13:17 sri hook before_request => sub { shift->stash->{uuid} = ... };
13:19 sri app->pg->options(Callbacks => {...})
13:20 bpmedley https://bitbucket.org/snippets/bpmedley/jkqq7 <-- sri, if you have a sec, how is this incorrect?  The DBI trace happens at line 71.
13:20 sri bpmedley: tldr
13:22 sri bpmedley: anyway, you're only covering sync, so it's uninteresting to me
13:23 dod joined #mojo
13:25 sri bpmedley: there is actually a lot wrong with that example
13:25 bpmedley I'd love to listen, if you have time.
13:26 sri "$clients->{$id} = $self->tx;" should never ever ever ever be done
13:26 sri you're not even reusing connections
13:27 sri your req count is totally arbitrary and changes for every new query, not for requests
13:28 bpmedley https://github.com/kraih/mojo/wiki/Writing-websocket-chat-using-Mojolicious-Lite <-- Might want to update the wiki for the tx bit.
13:28 dod joined #mojo
13:29 sri bpmedley: that is not my article
13:29 bpmedley My bad, I saw your name on it under the "edited" piece and made an improper assumption.
13:30 sri my chat app comes bundled with mojolicious and Mojo::Pg
13:30 bpmedley Moment
13:30 sri https://github.com/kraih/mojo/blob/master/examples/chat.pl
13:31 sri https://github.com/kraih/mojo-pg/blob/master/examples/chat.pl
13:39 sri DBI trace into stash value is not a bad idea, but you need to maintain a request uuid
13:40 bpmedley Thx, one sec.  Given the differences between your code and the wiki I should have known it wasn't yours...
13:40 sri i generally do not write stuff for the wiki
13:42 sri you're lucky actually, because DBI connections even with async queries, can only handle one query at a time
13:43 sri so if you were to trace everything from the moment a connection is dequeued, until it is requeued, odds are everything was for the same request
13:43 sri of course you can't know for sure
13:44 sri it's an event loop, a query may run for a pool of thousands of websocket connections
13:45 sri request scope is a very arbitrary concept for an event loop based framework
13:47 sri you might even make your app worse trying to fit everything into a request scope for profiling
13:48 sri say you have some data from a backend service that can be cached for 5 minutes
13:48 sri you wouldn't want to refresh that in a request scope
13:48 sri you'd want a global recurring timer
13:49 jontaylor i’m still trying to work out what the compromise is, what you are saying is entirely true, especially about 1 querys results being used for many consumers, however i’ve seen how valuable live software analytics can be
13:50 sri i don't think it can be unobtrusive
13:50 bpmedley Thx for all the feedback.  It will take time to soak in.
13:51 sri hooking into DBI and stuff will always have these kinds of problems
13:51 sri think i would go with a special api, that the app developer has to use
13:53 jontaylor thats the only wait it would be reliable isn’t it, which is a shame, as a plugin you can just retrofit trivially into any old app would be really cool
13:53 jontaylor s/wait/way
13:53 sri there is data you can automatically collect, like the time from before_dispatch to after_dispatch, or around_action
13:54 jontaylor real time, but probably not cpu time
13:54 sri but there is no clear request scope, you need the app developer to tell you what he wants to include in the scope
13:57 jontaylor we should do a poll one day to find out what people are using in the wild
13:58 jontaylor my last place had 6 mojo apps
13:58 jontaylor of which 5 only did synchronous things
13:58 jontaylor whereas the 6th made very heavy use of it (it was essentially a peice of middleware for another service)
13:59 sri as soon as you use websockets, you're doing async things
13:59 sri you might not even know it
14:00 sri but actually, even synchronous things can end up being handled async behind the scenes
14:01 sri say you have a route that only renders a big template, that's a very sync thing
14:01 sri but, once the template is rendered it ends up in a Mojo::Asset::Memory buffer
14:02 sri now that buffer is passed to the web server, which streams the data async to the browser
14:02 sri if the connection to the browser happens to be slow, you end up with concurrent requests behind the scenes
14:02 bpmedley Do you happen to have an async insert transaction example handy for Mojo::Pg?
14:04 sri async insert makes no sense
14:05 sri you don't perform fast operations async with anything DBI based
14:05 jontaylor I’m going to have a read through of this https://github.com/newrelic/node-newrelic to see if I can glean anything from it
14:05 sri since all i/o is still blocking
14:06 sri jontaylor: that seems like a good idea
14:08 sri bpmedley: async transactions are no different that this though https://github.com/kraih/mojo-pg/blob/master/lib/Mojo/Pg.pm#L116
14:09 sri you hold on to the $tx scope guard
14:09 sri if your transaction failed just let it go out of scope instead of calling ->commit on it
14:10 sri $err ? return : $tx->commit;
14:22 ajr_ joined #mojo
14:24 Kogurr joined #mojo
14:33 jontaylor so far i’m thinking we can handle the boring sync cases automatically, and provide a simple API for the async calls
14:34 jontaylor maybe something like with_tracing_do $trace_id { code_here  }
14:50 melo1 joined #mojo
14:51 kaare_ joined #mojo
14:51 CandyAngel Hm.. to request a new job state or to just immediately fail a job as soon as it is added to the queue... decisions :P
14:52 melo joined #mojo
14:54 klapperl_ joined #mojo
15:02 sri very convenient http://michael.otacoo.com/postgresql-2/postgres-9-5-feature-highlight-new-jsonb-functions/
15:07 buu So. Um.
15:08 buu If I do: $controller->render(text => "stuff"); why does my after_render hook get $format eq 'hmtl' ?
15:08 buu or html
15:11 pink_mist https://metacpan.org/source/DBOOK/Mojolicious-6.12/lib/Mojolicious/Renderer.pm#L128 <-- I guess this should add ", 'text'" or something
15:11 pink_mist like the json does above it
15:13 pink_mist or maybe it's a bit more involved than that :>
15:15 buu ?
15:15 buu oh
15:15 sri text is the handler, not the format
15:17 buu sri: Why does json set the format but text doesn't then?
15:17 sri because json is pretty obvious
15:17 buu ?
15:18 sri if you render json, what other format could it be than json?
15:18 sri when you render text, how are we supposed to know it is txt and not html?
15:18 mst parse it with regexps
15:18 buu I think I see what you mean, but it's certainly a little .. confusing
15:19 mst if freenode #perl turns up at your house and sets you on fire
15:19 mst it was html
15:19 buu I guess what I actually wanted to know was the handler name
15:19 pink_mist mst: lol
15:20 buu sri: Do you have a recommendation for detecting if I'm getting the output of a template handler or similar in my after_render hook?
15:20 sri no
15:21 sri you seem to be conflating text with txt
15:21 sri http://mojolicio.us/perldoc/Mojolicious/Guides/Rendering#Rendering-text
15:21 sri http://mojolicio.us/perldoc/Mojolicious/Guides/Rendering#Rendering-data
15:22 sri but that is all there is to it
15:22 buu Huh
15:22 sri whole point is to handle unicode properly
15:23 buu ok
15:25 sri btw. http://mojolicio.us/perldoc/Mojolicious/Guides/Rendering#Post-processing-dynamic-content
15:26 sri that's how i imagined after_render would be used
15:27 buu sri: Well, I'm doing something similar, except html mangling
15:27 buu So I need a little more insight into the output
15:27 buu Anyway, I can just set format => txt in my render call
15:28 sri that also sets the Content-Type header properly
15:28 buu sounds useful
15:29 buu sri: While I'm pestering you, is there a clever mojo way to get a list of all the available templates in my templates dir?
15:30 sri Mojolicious::Renderer::_warmup makes a list for detection purposes, but i don't think there's a public api yet
15:32 * sri vaguely remembers proposing an api for it once... but i don't think it went anywhere
15:33 buu Mojo::Home->new->list_files?
15:38 buu self->app->home->list_files('templates') is pretty close
15:51 buu sri: is $options->{template_name} a standard option for renderers?
15:53 sri no
15:55 buu Also what is the 'variant' option?
15:55 buu for template names
15:55 sri http://mojolicio.us/perldoc/Mojolicious/Guides/Rendering#Template-variants
15:56 buu oh
15:56 buu It's in guides =/
15:56 sri it's all in the guides
15:58 sri another variant example is in the hooks section http://mojolicio.us/perldoc/Mojolicious/Guides/Routing#Hooks
16:03 bpmedley https://bitbucket.org/snippets/bpmedley/A4zkb <-- sri, jontaylor this is the best I could do in a few hours..  Of note is how package Trace overrides _enqueue and _dequeue of Mojo::Pg.
16:07 sri bpmedley: why use actual files instead of just tracing into a scalar?
16:08 bpmedley For posterity.  It really depends on what wants to be done with the data.  No significant reason, at present.
16:08 sri anyway, yes, that's how i imagine it would work with Mojo::Pg
16:09 sri it could just have enqueue/dequeue events
16:09 bpmedley That would be useful.
16:10 sri normally i expect 2-3 good use cases for an event
16:10 sri so, if you can come up with some, i'll add it
16:11 sri tracing is a reasonable first
16:12 bpmedley Doesn't dequeue already have the connection event?  So, a "disconnect" event just needs a couple more use cases?
16:16 bpmedley Perhaps a global error message capturing interface?  Perhaps, by using $h->errstr?
16:17 sri no, connection is for new connections only
16:18 sri it's a connection pool
16:18 sri with dbh reuse
16:19 sri (and sth reuse too...everything that can be cached gets cached)
16:19 bpmedley Would the events both have Active handles passed to them?
16:20 buu So "    All templates should be in the "templates" directories of the application, which can be customized with "paths" in Mojolicious::Renderer, or one of" is in the guide for rendering, but where is the actual 'templates' directory added? Or is it just assumed?
16:20 bpmedley http://mojolicio.us/perldoc/Mojolicious/Renderer#paths <-- buu, I think
16:21 buu bpmedley: Yes.. that's where they're stored, but it's empty in my app
16:22 sri bpmedley: not necessarily
16:22 bpmedley buu, sorry I didn't read your msg clearly.
16:23 sri both events would simply get *all* handles
16:23 kaare_ joined #mojo
16:26 bpmedley https://github.com/kraih/mojo/blob/master/lib/Mojolicious/Renderer.pm#L30 <-- Is this what you're looking for?
16:27 bpmedley Nevermind.  I'm being dumb.
16:30 bpmedley buu: /opt/perl script.pl eval -V 'app->renderer->paths'   <-- what directory shows here?
16:31 bpmedley Third times the charm?    https://github.com/kraih/mojo/blob/master/lib/Mojolicious.pm#L148  <-- Seems to be created when making a new app object.
16:33 buu bpmedley: ok, the app->renderer.. has templates in it..
16:35 buu oh.
16:35 buu I'm an idiot.
16:35 bpmedley sri: I'll give the use case question some more thought.  I'll try to remember to give you feedback tomorrow or tuesday.
16:35 buu Basic perl is hard.
16:35 bpmedley What happened?
16:35 purl We don't know what happened, so tell everyone nothing happened.
16:36 buu bpmedley: I forgot that 'warn' doesn't default to $_
16:36 buu I blame perl frankly.
16:36 bpmedley There's so much perl to remember.. ;)
16:36 kaare_ joined #mojo
16:37 buu Dunno why it doesn't really.
16:37 buu hrm
16:38 kaare__ joined #mojo
16:45 CandyAngel Is there a preference between doing things in templates rather than passing things in?
16:46 kaare joined #mojo
16:53 buu what kind of things?
16:54 buu I can ssay that at no point in your career will you ever stop and say "Gee, I'm really glad I put all my database calls inside the template"
16:55 CandyAngel :P
16:55 CandyAngel Like passing in the minion backend for a page the displays the jobs, rather than passing in a hashref of the info
16:55 CandyAngel the jobs info
16:56 ivan_ joined #mojo
16:57 buu As a general rule I prefer the hashref, but it's nice to have a templating language that abstracts hash and method calls
16:57 buu So you can switch between them seamlessly
17:05 CandyAngel Yup, just wasn't sure if it was intended to like that or just an effect of making the templater (infinitely?) flexible
17:24 jontaylor bpmedley: just saw your comment now, that looks really good. I’m already foreseeing having a bunch of adapters working ;)
17:33 berov joined #mojo
17:40 PryMar56 joined #mojo
17:45 ajr_ joined #mojo
18:35 buu I wrote a thing https://metacpan.org/release/RMZG/MojoX-Renderer-Handlebars-0.02
18:36 buu And by 'wrote' I mean copy and pasted
18:36 buu Oh. Should MojoX:: modules declare that they depend on Mojolicious?
18:36 Grinnz do you use mojolicious modules?
18:37 Grinnz use parent qw(Mojolicious::Plugin); -> yes
18:37 buu good point
18:37 purl nice and sharp
18:37 buu Why do I have meta.yml and meta.json
18:37 buu This seems unncessary.
18:38 Grinnz everything does now
18:38 Grinnz because META.json is the new spec, and you need META.yml in case of installers that don't recognize META.json
18:38 buu Oh. Those are autogenerated files.
18:38 buu doh.
18:40 CandyAngel Aw, that's annoying, I can't create a job and immediately fail it :(
18:41 cpan_mojo MojoX-Renderer-Handlebars-0.03 by RMZG https://metacpan.org/release/RMZG/MojoX-Renderer-Handlebars-0.03
18:42 buu =O
18:42 buu Might even work!
18:43 Grinnz you forgot to update Changes :P
18:43 buu lol
18:43 Grinnz i added a dzil plugin to my pluginbundle specifically for that
18:43 Grinnz after seeing Dist::Milla use it
18:44 klapperl joined #mojo
18:44 buu So. Um.
18:54 sri the MojoX namespace is not really used anymore these days
18:54 sri you've copied very old conventions
18:55 sri modern plugin would put it all in the plugin class
18:55 sri like the documentation says http://mojolicio.us/perldoc/Mojolicious/Guides/Rendering#Adding-your-favorite-template-system
18:56 buu =[
18:56 Kogurr joined #mojo
18:56 sri MojoX is actually pre-1.0 convention :o
18:56 buu sri: It actually provides a Mojolicious::Plugin::HandlebarsRenderer
18:57 sri right, the second class doesn't make much sense these days
18:57 buu Oh of course, I think I even knew that at one point.
18:57 dod joined #mojo
18:57 buu Ohwell =[
18:58 sri some old plugins have kept dragging these old conventions along since 2010
18:59 sri but people don't want to make a breaking change... and so the useless class never gets removed
19:00 sri and then other folks cargo cult it
19:00 sri little unfortunate
19:00 sri someone should poke the TT plugin maintainer
19:02 buu Yeah
19:22 klapperl_ joined #mojo
19:26 wingfold joined #mojo
19:37 klapperl joined #mojo
19:42 klapperl_ joined #mojo
20:00 klapperl joined #mojo
20:08 sh4 joined #mojo
20:13 klapperl_ joined #mojo
20:21 irq joined #mojo
20:24 bin_005 joined #mojo
20:25 CandyAngel Just to double check, I can't failed a minion job right after creating it, right?
20:25 CandyAngel Because it is inactive and it seems I can only fail it while it is active
20:26 sri that makes no sense
20:26 purl Chewbacca!
20:26 sri why would a job fail before it even started?
20:27 bpmedley CandyAngel: Are you trying to fail the job in the task or outside it?
20:27 CandyAngel Because there doesn't seem to be a way to create a job that isn't immediately performed by the next worker
20:27 CandyAngel bpmedley: Outside it, just after I have enqueued it
20:27 bpmedley https://metacpan.org/pod/Minion#enqueue <-- Note the delay?
20:28 CandyAngel Yeah, but it still gets executed eventually. I want it to not be executed until I change retry it :)
20:28 sri bpmedley: that makes no sense either, inside the task your job is active
20:28 CandyAngel until I retry*
20:28 sri CandyAngel: use a very big delay?
20:28 CandyAngel Basically, I want to queue job, but be able to manually start them
20:29 CandyAngel sri: Yeah, I could just make it wait for a year..
20:29 sri but honestly
20:29 purl honestly is, like, something sorta further down the road, unless some new people stepped up to work on it
20:29 sri jobs you don't want to run yet shouldn't be in the queue
20:29 sri make your own pre-queue or whatever
20:29 bpmedley CandyAngel: Why not enqueue when you want to manually start the job?
20:30 CandyAngel Because these jobs are being added by other jobs
20:30 CandyAngel I don't know what they are, a "discoverer" job finds them
20:30 CandyAngel For example, I start a job to process directories under /opt/some/dir
20:31 CandyAngel But when a subdir contains > 8000 entries, I don't want to descend, so it schedules a "suspended" job to process that directory
20:31 CandyAngel Which I can remove/retry etc. from the UI
20:31 CandyAngel That's what I want to do
20:32 sri for a moment i thought you wanted job dependencies... but that's not it
20:33 CandyAngel I was going to work around it by immediately failing it with a status of "requires manual retry" or something
20:34 howitdo joined #mojo
20:35 sri it's very easy to allow those state transitions, but i'm really not convinced this is a valid use case
20:36 sri especially since there's a clock on those failed jobs anyway
20:36 sri at some point they will get garbage collected
20:37 CandyAngel Docs only say 'Finished' get collected
20:37 sri so, no matter what, long delay is a better option
20:37 sri oh, that's true, i was conflating the states finished and failed
20:38 sri but that's a good point, making the state machine too complicated is bad
20:38 sri new transitions need a strong use case
20:38 bpmedley sri, are you totally against manually started jobs?
20:38 sri bpmedley: use cases!
20:38 purl use cases are evil. You start to question my sanity writing them out for non-work software
20:39 * CandyAngel shrugs
20:39 pink_mist use cases imply users! and everyone knows users are pure evil! :P
20:40 CandyAngel If you don't add it, I can redefine _update to be my ($self, $fail, $id, $result, $force) = @_;
20:40 CandyAngel :P
20:40 sri CandyAngel: well, it will break at some point
20:40 CandyAngel That's fine
20:40 CandyAngel Better for it to do what i want now and break later, than not do it at all
20:40 CandyAngel If I wanted to latter, I'd use Windows :P
20:41 sri if you think it's a reasonable feature you could also collect votes for it
20:42 CandyAngel Also, I can either spend ages implementing a system to hold "manual" jobs which may be buggy and not work properly.. or I can use the actual job system right here
20:42 bpmedley Perhaps a manual job could be started by a remote api call.  Say, for example from a social media callback.
20:46 CandyAngel It could also be used to allow users to request jobs to be run but get authorised by an admin
20:46 bpmedley Another example would be from DB stored procedures.  The stored procedure could update the minion job to go from state "manual" to state "inactive".
20:46 CandyAngel For example, a name change on a forum
20:47 CandyAngel Or to send them all their user data (social platform)
20:57 bpmedley sri, May I ask your thoughts on those use cases?
21:09 sri bpmedley: what use cases?
21:09 purl use cases are evil. You start to question my sanity writing them out for non-work software
21:12 CandyAngel The simplest way of implementing it would be to add a "suspended" state
21:12 CandyAngel Or something like that
21:12 purl something like that is totally possible
21:12 CandyAngel Thanks for your support purl
21:19 bpmedley sri: How specific does the use case have to be?  Do you want a list of steps, or just a high-level description?  How is the DB stored procedure lacking?
21:19 sri bpmedley: i feel like we are speaking different languages :S
21:34 mattastrophe joined #mojo
21:35 Grinnz bpmedley, a use case is an end goal to achieve, that this feature would support
21:41 buu As in "This is a case where I would use this feature"
21:53 klapperl joined #mojo
22:03 * sri nods
22:13 klapperl_ joined #mojo
22:19 mattastrophe joined #mojo
22:36 berov1 joined #mojo
22:44 CandyAngel Is my 2 use cases good examples?
22:47 klapperl joined #mojo
22:47 sri too abstract
22:48 sri the connection to the feature is not clear (at least to me)
22:51 CandyAngel I don't understand how it is too abstract? To implement it without doing it in Minion would pretty much require be just reimplementing minion..
22:54 CandyAngel would pretty muh be just reimplementing minion*
22:54 CandyAngel Gah, typing
22:57 klapperl_ joined #mojo
23:00 bpmedley Grinnz, buu: So, rather than saying "... from a DB stored procedure ..." I should say "A user runs a command-line program on the application server that needs to update the database in order to start a minion job that was inserted via the web server"?
23:02 klapperl joined #mojo
23:02 sri bpmedley: i don't see the connection to the feature
23:03 bpmedley The web server has Mojolicious and Minion, while the application server just has Perl and DBI.  Therefore, it would be difficult for the application server to insert a new job.
23:07 bpmedley Hrrm; I haven't mentioned why the web server would insert a manual job, though.
23:07 klapperl_ joined #mojo
23:08 sri wat?
23:08 purl Watt?  Watt?  I can't hear you, there's this buzzing noise where my brain should be. or https://www.destroyallsoftware.com/talks/wat
23:08 sri app->minion->enqueue()
23:08 sri nothing easier than adding a job from the app server
23:08 sri wait wat
23:09 bpmedley I agree with you, if they have Minion installed on the app server.
23:09 sri if you want to modify the storage backend manually, you're completely on your own
23:09 sri we will never ever support that
23:09 sri allright, that was easy
23:09 sri definitive no
23:10 sri (not that that is actually a real use case)
23:11 bpmedley That makes sense.  Hrmm.
23:12 sri to give an example for a real use case
23:12 zivester joined #mojo
23:12 klapperl joined #mojo
23:12 sri just recently the delay option was added for job restarting
23:13 bpmedley In order to properly move a job from "manual" to "inactive", the Minion api would need to be used - so why not just enqueue at that point?
23:13 sri the use case was very simple, jobs may depend on backend services, if one service goes down a lot of jobs will fail, now if the service comes back, and a lot of jobs get restarted at once, it would overload that service
23:13 sri that is basically the exmplanation i expect
23:15 CandyAngel Okay, so "you need to enqueue a job by regular users that requires manual authorisation due to resources required to complete the job" is one?
23:15 sri yes
23:15 CandyAngel Okie
23:16 sri and now we can argue about the validity of that use case
23:16 sri personally, i don't think such jobs belong in the queue
23:16 CandyAngel Same for jobs that enqueue jobs due to detecting that they are likely to use a lot of resources
23:16 CandyAngel How come? The only place else they are going to go is.. another queue
23:17 CandyAngel They are just jobs with another state
23:17 sri is there another job queue that supports this?
23:17 CandyAngel Not one that has all the tools Minion gives you, that I am aware of
23:18 CandyAngel e.g. command line access
23:18 sri so, technically, the idea of enqueueing a job and then making it fail right away so it won't get processed is flawed
23:19 CandyAngel Failing it right away was my idea for a workaround due to "suspended" not being a thing
23:19 sri those two operations are not atomic, depending on how fast the backend is, the job might already be active by the time you try to fail it
23:19 CandyAngel Or whatever you want to call a job that isn't processed, but otherwise inactive
23:19 klapperl_ joined #mojo
23:20 sri so that proposal is out
23:20 sri i imagine a proper implementation would use an enqueue option... like ->enqueue('foo' => {frozen => 1}) and then set the starting state to "frozen" or so
23:21 CandyAngel yeah
23:21 CandyAngel that's what i want
23:21 sri now, how would you start the job with the existing api?
23:21 CandyAngel retry, which would set it to inactive
23:22 sri the name retry doesn't really fit
23:22 CandyAngel You said existing API
23:22 CandyAngel If we can add a call 'unfreeze' :P
23:22 sri the feature would have to be pretty important for a new method
23:23 bpmedley CandyAngel: How will you find the particular frozen job to thaw?
23:26 CandyAngel bpmedley: My UI at the moment just list_jobs everything, and shows different controls based on the state
23:26 bpmedley So, you envision a two step process?  The user submits a job and the admin runs the job via your UI?
23:26 CandyAngel Yup
23:27 CandyAngel For the 2 use cases I have in mind, that'd be it
23:28 CandyAngel Well
23:28 CandyAngel For my forum, it would be that
23:28 CandyAngel In my filesystem thing, it would be a job submitting a job when it thinks it will be resource heavy
23:28 CandyAngel So say my scraper generates image gallery files, it would submit a frozen job if it contained > 200 images
23:29 mattastrophe joined #mojo
23:29 bpmedley What criteria would you use to thaw the job to process the images?
23:30 CandyAngel What do you mean?
23:30 CandyAngel Like.. when would I thaw them?
23:30 bpmedley Yes
23:30 klapperl joined #mojo
23:30 CandyAngel Could thaw them with cron when I'm not at the computer, or right now if I wanted that particular job doing now, even if it might slow my computer down
23:36 sri the use case does seem a bit weak to me, considering the implementation cost
23:36 sri perhaps there is something more generic?
23:36 sri with more use cases?
23:36 CandyAngel or, say a user submitted a job to rename their account on a forum, the admin would be authorising what they wanted it changed to
23:37 sri like ->enqueue('foo' => {error => 'this job requires admin review'})
23:37 sri perhaps there are more use cases for starting a job with an error?
23:37 sri CandyAngel: same use case
23:37 CandyAngel That'd be fine
23:38 sri as i see it, the use case here is "admin needs to review the job before it can get processed"
23:38 sri that's the one use case we have
23:38 sri everything else so far is just variants of that
23:39 CandyAngel Okies
23:40 meredith When I'm worried about a job on a single box using up resources, I've just had it renice or re-ionice itself so it doesn't disturb the other business going on.  Obviously that's not all the sorts of resources one can be worried about though.
23:43 klapperl_ joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary