The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2017-08-01

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:53 noganex_ joined #mojo
03:50 irqq_ joined #mojo
04:02 stryx` joined #mojo
04:16 tyldis joined #mojo
04:21 bianca joined #mojo
05:57 bianca joined #mojo
06:53 AndrewIsh joined #mojo
07:24 Vandal joined #mojo
07:51 bjakubski joined #mojo
07:56 Lee[home] joined #mojo
08:03 Lee[home] joined #mojo
08:20 dod joined #mojo
08:26 sri ok, this should be better https://github.com/kraih/minion/commit/23f6008c2e6cfc1ae70d75208dc8d0fceecba419
08:28 sri hope the id option for ->dequeue doesn't make some backends impossible
08:42 petru joined #mojo
09:02 sri here's the whole feature https://github.com/kraih/minion/compare/28afb4508f7a4a171665b7e518299ea63a9c0b94...a11d1ca907fe8263eed598f8b5be35aeadc3cd10
09:02 sri please review
09:02 sri (those of you not on vacation)
09:03 sri i think it's the right way to do the feature
09:05 sri the "and id = coalesce(?, id)" might seem a little sketchy, but does work and appears to make no difference in benchmarks
09:05 sri i can still dequeue 4k+ jobs/s on my laptop
09:17 irqq joined #mojo
09:59 coolo sri: where do I have to put the line to debug which worker gets which route?
10:01 sri coolo: entire connection with all requests gets processed by the same worker, so anywhere you like
10:02 coolo yeah, but which hook
10:03 coolo I need to get to know the POSTs before they hit the debug log
10:03 sri not sure i understand the question then
10:04 sri the debug log is as early as it gets, the log message showing the request path happens even before routes
10:04 sri routes only happen after the whole request has been received
10:05 stryx` joined #mojo
10:05 sri the after_build_tx hook code i gave you a few weeks back would be the earliest moment you can get the request path
10:06 sri looks like i never pasted it here... so it's not in the log :/
10:22 coolo sri: see my problem? :)
10:23 coolo it never was in git either
10:24 coolo we have tons of POSTs that end up in 502/406
10:25 coolo and this time with 15 clients against 20 mojo workers, so we should be fine - but we are not
10:32 tchaves joined #mojo
10:33 * coolo tried $app->log->debug("TX " . $tx->req->url); - but it's empty
10:36 coolo oh, I see. I need to add a hook to the $tx then
10:49 sri coolo: $app->hook(after_build_tx => sub {my $tx = shift; $tx->req->on(progress => sub { my $req = shift; return unless $req->headers->is_finished; $app->log->debug("Worker $$ has accepted the request for " . $req->url->to_abs) }) });
10:50 sri i remember posting it in #opensuse-factory, and as it turns out irccloud has logs from that channel back to last year :)
10:59 coolo sri: I remembered the on progress part after googling some mailing list posts
10:59 coolo but it does not help me :(
10:59 coolo something stinks in denmark
11:02 coolo sri: http://paste.opensuse.org/view/raw/54296912 - it does not look like a timeout anymore
11:03 coolo sri: and there is no worker whatsoever taking the request at 12:52:42: http://paste.opensuse.org/view/raw/59955372
11:06 sri doesn't apache only log finished transactions? wouldn't that mean the backend server had to have accepted the request earlier?
11:07 coolo sri: the timestamp in apache log is the start time of the transaction
11:07 coolo the 9591473 in the access_log is the request time in us
11:08 coolo so it was delayed 9 seconds
11:08 coolo but there was no prefork restart or anything strange in that time frame
11:09 coolo and we run 7.37 meanwhile
11:10 * coolo runs now a strace on the prefork
11:38 Lee[home] joined #mojo
11:51 bianca joined #mojo
11:53 * ashimema is confused again with Mojo::URL
11:54 ashimema I always expect $c->req->url->to_string to output an absolute url but it only output the relative path and query params
11:56 ashimema mwah.. I must just be dim
12:08 sri that's meant to be covered in the guides in the future https://github.com/kraih/mojo/issues/1109
12:08 ashimema oh.. cool
12:09 sri but it turns out it's hard to find volunteers for docs
12:09 ashimema I think my brain is just on a go slow today to be fair.. there's likely a good reason behind it
12:15 * ashimema wishes work was in a better place at the moment so he could devote some time to such a task himself
12:35 karjala_ joined #mojo
12:54 gizmomathboy joined #mojo
13:14 petru joined #mojo
13:25 sri well, i've released Minion 7.04 with foreground support
13:26 sri since i use it already for work
13:26 sri if it turns out something is really broken i'll just make an 8.0 release to fix it :p
13:27 genio sri: With regard to leaving issues open, please don't feel the need to leave the one I have in there open. It's not simply done and will likely introduce more issues than the one it wants to address
13:27 sri genio: then close it :P
13:28 sri it's been a few months without proposals
13:30 genio closed
13:36 gordonfish joined #mojo
13:37 gryphon joined #mojo
13:48 yukikimoto joined #mojo
13:54 ashimema is there a nice way to handle websockets gracefully on app restart?
13:54 ashimema i.e. should I be sending a connection close and how would I go about that
13:55 ashimema pointers to docs are always appreciated..
13:57 Pyritic joined #mojo
13:59 jeck joined #mojo
14:04 ashimema or do we properly close websocket connections as part of an app restart 'as if by magic' ;)?
14:04 ashimema in which cass it's my clientside code that's not handling things properly and re-initiating the connection.
14:07 ashimema hmm, looks like for morbo restarts for instance we get a 1006 - abnormally closed code
14:13 bianca joined #mojo
14:28 zivester joined #mojo
14:29 sri coolo: i'm starting to wonder if the errors we see in the techempower benchmarks might be related somehow to the interrupted requests you're seeing https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=db&d=1l
14:30 sri if only someone could replicate the problem in a smaller scale test case
14:34 stryx` joined #mojo
14:54 PopeFelix (LONG) I am updating one of my company's big web apps to work asynchronously. The app interacts with S3, and those interactions are moderated by a class that returns instances of Future::Mojo. I'm having trouble encapsulating these Futures inside of a controller, because by the time Test::Mojo calls the controller method, Mojo::IOLoop->singleton has already started. This is a problem for Future::Mojo, as under the hood, when you tell the Future to wai
14:54 PopeFelix t for a result, it does Mojo::IOLoop::one_tick.
14:55 PopeFelix I *think* that the ideal situation would be to hold off starting the IOLoop until everything has been properly queued.
14:55 PopeFelix So two questions. 1) Is my thinking correct? and 2) given #1, how do I hold off on starting the IOLoop until everything is queued?
15:04 sh14 joined #mojo
15:07 sri never use Mojo::IOLoop->one_tick
15:08 sri if something does it's prolly broken
15:09 itaipu joined #mojo
15:12 sri ah, which is why Future::Mojo croaks to protect you from yourself
15:20 PopeFelix And that makes perfect sense. What I want to know is what's the proper way for Future::Mojo to interact with the existing app?
15:22 zivester2 joined #mojo
15:26 petru joined #mojo
15:27 sri ok
15:41 PopeFelix So what is the proper way for Future::Mojo to interact with my existing app? Is there somewhere I can stop the IOLoop before I render back to the client?
16:11 kaare_ joined #mojo
16:14 gabiruh joined #mojo
16:27 PopeFelix joined #mojo
16:34 Grinnz no, if you are using futures in a web app you should never be awaiting them
16:35 Grinnz store the future somewhere so it doesn't get destroyed and let it complete when the event loop runs
16:37 Grinnz also keep the transaction alive and call render_later same as when doing anything else Async in controller methods
16:41 karjala_ joined #mojo
16:41 zivester3 joined #mojo
16:44 Grinnz hmm I just thought of a couple ideas for Future Mojo convenience modules
17:00 PopeFelix joined #mojo
17:05 PopeFelix Grinnz, thanks. i'll try that
17:29 stryx` joined #mojo
17:35 PopeFelix joined #mojo
17:59 PopeFelix why do i keep disconnecting, i wonder?
18:08 PopeFelix Also, what's a good phrase for the method associated with an endpoint that calls a particular method? "Calling endpoint" ?
18:09 petru joined #mojo
18:13 PryMar56 joined #mojo
18:16 preaction route handler?
18:17 Gedge joined #mojo
18:51 itaipu joined #mojo
18:57 irqq joined #mojo
19:23 PopeFelix joined #mojo
19:26 PopeFelix preaction++
19:30 irqq_ joined #mojo
19:31 PopeFelix So how are non-blocking Mojo requests going to behave with the Futures that I'm returning? Will they just get put into the IOLoop queue ahead / behind the stuff encapsulated in Futures?
19:55 itaipu joined #mojo
20:33 orev joined #mojo
21:02 iamb joined #mojo
21:21 Peppard joined #mojo
21:40 PopeF joined #mojo
22:20 rick_soc joined #mojo
22:20 rick_soc hello hello
22:20 purl o/` hello, hello? are you out there? huh? huh-huh? o/`
22:20 CandyAngel Welcome to Freddys!
22:22 rick_soc I'm playing around with form file uploads.  I'd like to get the md5 hash of what is uploaded - what is the cleanest way to do this?
22:22 rick_soc I see in Mojo::Upload there is a "asset", could I just run the md5 check against this
22:23 CandyAngel There are md5 functions in Mojo::Util
22:24 CandyAngel What are you getting the md5 for? If the files you are doing are large, you should probably background it (using Minion or something)
22:24 rick_soc Okay
22:24 rick_soc I just want to save the md5 of the file so later when I go to retrieve the file, I can check it hasn't been tampered with
22:25 CandyAngel You should be using a different hash function for that
22:26 rick_soc I see I could use md5_bytes against slurp if I'm willing to hold it all in memory
22:26 CandyAngel md5 is pretty much only for corruption detection, because intentional tampering means you can generate collisions
22:27 CandyAngel Mojo::Asset will hold it in memory until it reaches a certain size, then it'll get written to disk
22:28 rick_soc Right, so let's assume it gets big enough to be written to disk, what is the most efficient way to get the md5 through mojo
22:28 sri Digest::MD5 can generate a checksum without loading whole files into memory
22:28 sri we just don't have a friendly helper api for that
22:28 rick_soc Ah ok
22:28 rick_soc Then I know what to do, was just trying to stay within the mojo framework
22:29 CandyAngel ^_^
22:29 rick_soc CandyAngel: thanks for the suggestion, I'll consider sha as well since I need to use another module anyway
22:29 CandyAngel Just remember that SHA1 was recently "broken" too :P
22:30 rick_soc I'll go straight to SHA65536
22:30 CandyAngel :D
22:30 CandyAngel Despite the SHA1 collision people coming up with a special name for it, I can't remember it :P
22:30 CandyAngel Was it SHAttered?
22:31 sri the security circus
22:48 rick_soc uhoh, Digest::SHA isn't safe to use if there are wide characters
23:16 rick_soc Got it working, thanks for your help
23:20 karjala_ joined #mojo
23:22 karjala_ joined #mojo
23:53 Grinnz most digest functions expect bytes. since you said you are dealing with files, they are already bytes
23:54 Grinnz also Digest::MD5 and Digest::SHA are both core

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary