The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2017-03-16

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:20 Xyem- To be more precise, calling '$res->to_string' in 'redirect_to' or 'rendered' fixes the issue.
00:27 jberger ah
00:27 jberger don't do that
00:27 jberger wait, calling $res->to_string FIXES the problem?
00:27 jberger that usually causes problems
00:27 Xyem- Indeed.
00:28 Xyem- Something is causing the 'Location' header to gain the prefix ' : ' ("Location: : /")
00:28 Xyem- Happens between 'rendered' and what is sent to the browser.
00:28 preaction you might have to post a reproduction so we can see
00:29 jberger are you manipulating the header elsewhere?
00:29 Xyem- I don't believe so.
00:30 Xyem- This is only a "login form" expansion of the Mojolicious Lite template.
00:34 Xyem- Aha, I have managed to get Mojolicious to log what it is sending out and it is different to what the browser receives..
00:35 jberger you can use MOJO_DAEMON_DEBUG=1 in your environment
00:35 Xyem- If I understand this correctly, this is what I have when I output $res:
00:35 Xyem- "header_buffer" => "Location: /\r\n
00:35 Xyem- Which is correct, but the browser (sometimes) receives:
00:35 Xyem- "Location:: /"
00:36 jberger well, anyway, don't log/use to_string on the message objects
00:36 jberger that can have side effects you don't want/intend
00:36 jberger use MOJO_DAEMON_DEBUG=1
00:36 jberger and see what that is
00:36 Xyem- Ah, thank you.
00:36 jberger or you can make requests directly to your app
00:36 jberger myapp.pl get -v /path/to/whatevedr
00:37 Xyem- Doesn't answer why doing so fixes the problem, if the headers are coming from Mojolicious correctly.
00:37 jberger anyway, I have to step out, maybe someone else can take over
00:37 jberger good luck
00:37 purl You'll need it.
00:37 Xyem- Thank you.
00:37 preaction could you post the code you're using to a paste site?
00:42 Xyem- Modified to remove database things: https://gist.github.com/anonymous/2e6d81edd1fa1360d9ad5fafc658ec04
00:43 Xyem- Putting in any username/password will be "right". Sometimes, this redirects to /, sometimes, it goes to /action/:%20
00:47 Xyem- Ah, adding MOJO_DAEMON_LOG may do nothing, as this is being run under CGI.
00:48 preaction why are you putting the entire environment in the flash?
00:49 preaction it's not a good idea to do that: [Wed Mar 15 19:47:39 2017] [error] Cookie "mojolicious" is bigger than 4096 bytes
00:49 Xyem- Trying to debug this issue :)
00:49 Xyem- I thought that the environment may be different for the broken requests.
00:50 preaction you can't put the env in the cookie, as cookies have a limit in size. what version of mojolicious are you using?
00:50 preaction mine prevents me from making the cookie at all
00:51 Xyem- 7.28
00:51 preaction alright, i got 7.27, so i'll upgrade
00:52 preaction otherwise i can't reproduce this currently. i removed the env from the cookie and the flashes are now working correctly (they were not previously)
00:53 Xyem- It's quite possible that the CGI server is randomly mangling the header.
00:53 preaction i can't reproduce with 7.28 either
00:54 preaction so, yeah, i'd start checking there. run your app without the CGI server and see if you can reproduce it. if you can't, then it's the CGI server
00:55 polettix joined #mojo
00:56 Xyem- If it is, it can be a nice example to the client why I prefer deployments on VPS :)
00:57 Xyem- Though it isn't their fault, they don't know how to maintain a VPS or have the funds to pay someone who does.
01:00 Xyem- Running "./app.pl get -v M POST -c 'username=a&password=a' /action/login" always generates correct headers.
01:02 Xyem- I'll contact the hosting provider, their server may be mangling it. Thanks for the time and advice, much appreciated.
01:04 Xyem- I (very briefly, because of the shared hosting) ran "morbo ./app.pl" and it works correctly there.
01:34 aborazmeh joined #mojo
03:13 jberger just uploaded a release for my last cpan module with deprecated Mojo::Util/Mojo::Home methods!
03:13 jberger \o/
03:14 jberger at least according to grep.cpan.me
03:25 jberger nice to have that behind me
03:25 jberger I wonder how often grep.cpan.me is updated
03:37 noganex joined #mojo
04:10 asarch joined #mojo
04:19 perlpilot joined #mojo
05:04 alilles joined #mojo
05:04 dboehmer joined #mojo
05:10 irqq_ joined #mojo
05:13 irqq_ joined #mojo
05:27 inokenty-w joined #mojo
06:40 Vandal joined #mojo
07:14 dod joined #mojo
07:41 alilles joined #mojo
08:01 AndrewIsh joined #mojo
08:37 trone joined #mojo
08:56 rshadow joined #mojo
08:59 alilles joined #mojo
09:04 rshadow joined #mojo
09:31 alilles joined #mojo
09:44 alilles_ joined #mojo
09:49 alilles_ joined #mojo
09:52 alilles_ joined #mojo
09:59 alilles joined #mojo
10:02 kes joined #mojo
10:03 kes spicyjack: something was wrong ;-)
10:22 kes heh... no answer(
10:22 kes I rephrase the question and post it here: http://stackoverflow.com/questions/42831224/why-mojolicious-action-is-not-called
10:27 marcus joined #mojo
10:37 irqq joined #mojo
10:45 sri so, iron fist is getting very very bad reviews... real bummer what's happening with the netflix shows
10:56 coolo I have to admit I didn't get past episode 2 of luke cage either
10:59 * sri nods
11:09 gregf_ joined #mojo
11:17 alilles joined #mojo
11:17 rshadow joined #mojo
11:22 stryx` joined #mojo
12:02 foursixnine joined #mojo
12:14 aborazmeh joined #mojo
13:49 * vicash thinks that netflix has gone the way of hollywood management - one thing works, so keep repeating it
13:56 jnap I got halfway thru luke cage.
13:57 jnap OA was ok, and I'm making it all the way thru 'Travelers' (episode 10 now).
14:09 gizmomathboy joined #mojo
14:13 gryphon joined #mojo
14:17 itaipu joined #mojo
14:31 sri hmm, there is another bottleneck in Minion::Backend::Pg::repair
14:32 sri https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L155
14:32 sri that query takes a very long time too with many jobs with dependencies
14:33 sri not as devastating as the last one, but with our 500k jobs it takes a few minutes
14:37 jberger I wonder if a CTE that gets the jobs which are not finished and then subselect against that would be faster
14:38 mcsnolte joined #mojo
14:48 sri hmm
14:53 sri if anyone wants to try, i have the test data and can run explain against it
14:54 rshadow joined #mojo
14:56 sri plan looks weird https://gist.github.com/anonymous/9dae230bb9a9da0c831c9cb078233729
15:02 rshadow joined #mojo
15:05 asarch joined #mojo
15:21 PryMar56 joined #mojo
15:26 sri explain analyze select * from minion_jobs as j where finished <= now() - interval '1 second' * 172800 and state = 'finished';
15:26 sri that takes 7 seconds on my data
15:26 sri so even a sequential scan is not exactly cheap
15:26 sri (plain sequential scan)
15:27 sri i've got about 12k results that match the conditions
15:31 sri there has to be a way to get that fast :S
15:43 sri jberger: cte seems to make it worse
15:43 genio joined #mojo
15:44 jberger yeah, I wasn't sure
15:44 jberger CTEs don't seem to get optimized well by the query planner, but I was thinking perhaps the reduction in table size for scanning would make up for it
15:44 sri stats is also getting a bit slow
15:44 purl Since Fri Jan 20 15:59:37 2017, there have been 1443 modifications and 296 questions.  I have been awake for 54 days, 17 hours, 45 minutes, 0 seconds this session, and currently reference 988618 factoids. Addressing is in optional mode.
15:44 coolo did you say stats? :)
15:46 sri ok, stats get better with an index on state
15:46 purl Since Fri Jan 20 15:59:37 2017, there have been 1443 modifications and 296 questions.  I have been awake for 54 days, 17 hours, 46 minutes, 40 seconds this session, and currently reference 988618 factoids. Addressing is in optional mode.
15:46 sri grrr
15:46 sri i
15:46 miller joined #mojo
15:46 sri am not saying stats again
15:46 * sri kicks purl
15:46 * purl blocks!  Paint the fence!
15:53 CHYC_ "create index minion_jobs_finished_idx on minion_jobs (finished) where state = 'finished';"?
15:54 marty joined #mojo
15:59 sri gets used but the result is pretty much the same
15:59 sri https://gist.github.com/anonymous/61f89affbf9a3d2bd872eb24d58fec01
16:01 sri delete from minion_jobs as j where finished <= now() - interval '1 second' * 172800  and not exists ( select 1 from minion_jobs where j.id = any(parents) and state <> 'finished' ) and state = 'finished';
16:01 sri that;s the full query btw
16:01 jberger are query parameters evaluated in order?
16:01 jberger parameters is the wrong word
16:02 jberger conditions?
16:02 purl conditions is probably plural
16:02 jberger I wonder if checking state='finished' before not exists ( giant scan ) makes a difference
16:02 jberger hopefully the query planner is smart enough to get that
16:02 CHYC_ jberger: order does not matter.
16:06 sri the query planner will change order however it likes
16:06 jberger I thought so, and even as I mentioned it I felt silly
16:07 sri i learned that lesson when i tried to make advisory locks work
16:08 jberger I made a comment yesterday that quickly got buried, but its paining me again so I'll bring it up again
16:08 jberger it would be nice if there was a way to absolutize relative urls in the ua before they get requested
16:09 jberger and the start event happens too late
16:09 sri what do you mean absolutize?
16:09 sri for the test server?
16:09 jberger add a host, specifically
16:09 jberger I'm not using a test server
16:09 sri i mean $ua->server
16:10 sri it has methods to give you the url
16:10 jberger I'm doing redfish (effectively IPMI over REST/HTTP) and it gives me relative urls
16:10 sri oh
16:11 jberger I thought about making a subclass of Mojo::UserAgent::Server that would mangle things for me, but that seemed a bit evil too
16:11 sri ->base($tx->req->url)->to_abs
16:11 jberger right, the question is when to do that
16:11 sri ->to_abs($tx->req->url)
16:11 jberger and I'm having to do that on every query, which I suppose I can do
16:12 jberger but if there were an event early enough I could just set it up once
16:12 jberger not probably worth doing anything, this is the first time I've needed it
16:13 jberger it was a little surprising though because I thought I had used the start event to make absolute urls into relative ones so that they would get directed to the mock server
16:13 jberger but maybe not
16:14 sh14 joined #mojo
16:23 CHYC_ sri: http://pastebin.com/2cmNtYat <- I'd like a peer review that this actually does what you want, but I hope it's a starting point for discussion
16:25 sri similar execution time to the old one here
16:26 rshadow joined #mojo
16:27 bwf Use bind variables?  https://gist.github.com/bfaist/3f2ac127d95ec7f972a8d4f50bc6e0d2
16:27 CHYC Can I see your plan with the CTE query? I got a significant speed boost on my data.
16:29 CHYC Amirite it's because you have loads and loads of data in parents[]?
16:30 sri https://gist.github.com/anonymous/2fb3ea55731875df7f8599f0b41dba08
16:31 sri i do have lots of parents[]
16:31 sri almost all jobs have parents
16:32 sri things got faster though
16:32 sri i assume lots of problematic jobs got deleted in the meantime
16:32 sri i should have mentioned the exact stats... so it's easier to replicate :S
16:33 sri ->stats is funny too though, first run after a while always takes like 14s, and the next ones 2s
16:34 sri needs some warmup
16:52 cfedde joined #mojo
17:15 dod joined #mojo
17:25 dod joined #mojo
17:31 dod joined #mojo
17:43 rshadow joined #mojo
18:16 howitdo joined #mojo
18:22 msestak_ joined #mojo
18:24 irqq joined #mojo
18:24 msestak_ joined #mojo
18:26 rshadow joined #mojo
18:28 Xyem I found out what the problem is with the broken Location: headers.
18:29 Xyem Mojolicious is outputting the correct thing, but when the CGI server sees "Location: /\nStatus: 302 Found", it breaks :)
18:29 Grinnz there should be \r\n between each header
18:31 Xyem Ah, yes. Same thing happens
18:31 Xyem If dispatch.sh does: echo -e "Location: /\r\nStatus: 302 Found\n\n";
18:32 Xyem The headers the browser gets are: http://pastebin.com/imXvr0Uc
18:33 Grinnz location header looks broken, extra :
18:33 Xyem Yes, this is the problem.
18:33 Xyem I believe their server is breaking the Location header.. though why it only happens sometimes, I'm not sure.
18:39 Xyem Now I have built a minimal example, it happens every time.
18:41 Xyem Perhaps when it works, there is another header which prevents it from breaking :)
19:33 rshadow joined #mojo
19:46 preaction there's some rules about cgi header parsing and how big the header is allowed to be
19:47 preaction Xyem: also, there's a way to turn header parsing _off_, which you likely should. it's "Non-Parsed Headers" or "NPH"
19:48 * Grinnz has flashbacks
19:50 dmanto joined #mojo
19:57 Xyem preaction: I don't have control over the CGi server, this is on shared hosting.
19:57 preaction Xyem: you can turn header parsing off _from your script_
19:57 preaction by declaring it as a NPH script
19:58 Xyem Oh, I see. I shall look into that.
20:05 stryx` joined #mojo
20:12 Xyem Seems like NPH handling isn't enabled.
20:12 Xyem Perhaps I should just ask if they'll set up a reverse proxy :)
20:18 irqq_ joined #mojo
20:25 itaipu joined #mojo
20:50 gryphon joined #mojo
21:11 stryx` joined #mojo
21:12 jberger sri: is this a bug? kes found it on SO
21:12 jberger perl -Mojo -E 'my $r = app->routes; my $api = $r->any("/api/v(:api)", {api => 1}); say $api->render({api => 1})'
21:12 jberger renders
21:12 jberger /api/v
21:13 jberger note that the fact that the route defined the default value is important to it failing to render
21:13 jberger (correctly)
21:13 jberger here is the SO post http://stackoverflow.com/q/42204755/468327
21:40 rshadow joined #mojo
22:26 batman joined #mojo
22:28 itaipu_ joined #mojo
22:30 stryx` joined #mojo
22:38 pink_mist http://classicprogrammerpaintings.com/post/147044648074/gentle-technical-discussion-on-irc-channel
22:51 itaipu joined #mojo
23:30 veryrusty joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary