The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2014-11-22

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:17 sivoais joined #mojo
00:31 disputin joined #mojo
00:39 disputin joined #mojo
00:45 asarch joined #mojo
00:52 jberger o/
01:14 woz joined #mojo
01:22 sri \o
01:23 jberger oh man, what a week
01:23 jberger leaving a job is hard
01:23 jberger so much to clean up
01:27 bpmedley joined #mojo
01:50 tempire sri: can you send me the source for the cloud and crossbones artwork? I want to play around with some banners.
01:52 jberger oooooo
01:52 jberger thaljef++ # http://blogs.perl.org/users/jeff_thalhammer1/2014/11/perlcritic-has-new-home-and-new-look.html
01:57 sri tempire: sure
01:58 sri sent
02:03 * tempire received his new business cards from moo.com today
02:07 * tempire dances to taylor swift
02:07 tempire \o\
02:07 tempire /o/
02:07 tempire \o
02:07 tempire o/
02:09 sri don't care much for her music, but like reading her opinions on infosec
02:10 tempire lulz
02:11 sri https://twitter.com/SwiftOnSecurity # for those living under a rock
02:18 klapperl_ joined #mojo
03:03 woz joined #mojo
03:21 disputin joined #mojo
03:51 good_news_everyon joined #mojo
03:51 good_news_everyon [mojo] kraih tagged v5.63 at c28e3e6: http://git.io/2HyvQw
03:51 good_news_everyon left #mojo
03:52 good_news_everyon joined #mojo
03:52 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/nNccMw
03:52 good_news_everyon mojo/master ff8bd16 Sebastian Riedel: bump version
03:52 good_news_everyon left #mojo
04:03 s1037989 joined #mojo
04:04 basic6 joined #mojo
04:34 tempire crab!
04:34 tempire Or anyone who lives in India!
04:40 sri i eat curry every now and then, does that count?
04:41 sri and now i'm hungry :S
04:52 woz joined #mojo
05:24 hesperaux joined #mojo
05:28 s1037989 You can render not_found and exception...  But what's the way to not return anything, as in to completely ignore the request?
05:29 preaction what would you do then? hold the connection open until it times out?
05:29 s1037989 Is a response required?
05:29 preaction it's recommended
05:29 preaction what would you have the server do?
05:30 preaction better: why do you want to not respond?
05:30 s1037989 Because I'm building a chatbot for slack.com and not everything warrants a response and I don't want to send unnecessary bits of info.
05:30 s1037989 How could I keep the response the smallest?
05:31 preaction return "OK"
05:31 preaction or, return "K"
05:31 s1037989 I'm currently doing not_found, but it seems too large.
05:31 s1037989 Is that a 200 response status?
05:31 preaction ... not_found is an explicit response, with meaning
05:31 preaction yes, that would be 200
05:31 s1037989 Right, and too large.
05:31 preaction if that's too large, are you sure http is the right protocol for you?
05:32 s1037989 Ok, cool.   So 200 with a body of just OK.  I'm assuming then that the response will just be ignored?
05:32 s1037989 Yes, it's in the API.
05:32 s1037989 And it's not that just OK is too large, but the full Mojo 404 is, well, just unnecessary for a bot.
05:33 preaction will the response be ignored? who's reading the response?
05:33 preaction you don't choose a response code based on how big it is!
05:33 preaction if you really wanted to not have a response, why not use websockets?
05:36 s1037989 Because I don't think I have control over this.  I'm using slack.com's API.  It's not that I'm choosing a response code based on how big it is, but right now, to ignore the request, I'm using 404 and that sends all the pretty Mojo stuff.  Why bother sending that to a bot?
05:36 preaction so what does slack.com's API say you should return so that it doesn't do anything?
05:37 s1037989 It doesn't.  I just figured I'd be kind to the wire and to slack.com.
05:47 thowe joined #mojo
06:19 rem_lex|pivo joined #mojo
06:41 woz joined #mojo
07:16 marmez joined #mojo
07:42 ignacio_ joined #mojo
07:43 Vandal joined #mojo
07:51 jwang joined #mojo
08:30 woz joined #mojo
08:47 sugar joined #mojo
08:58 camelo joined #mojo
08:59 camelo Good morning!
08:59 purl Lies!
09:58 punter joined #mojo
10:19 woz joined #mojo
10:21 meshl joined #mojo
10:41 sh4 joined #mojo
10:52 denis_boyun joined #mojo
11:05 basiliscos joined #mojo
12:08 woz joined #mojo
12:25 marty joined #mojo
13:14 burtleton joined #mojo
13:57 woz joined #mojo
14:19 marty joined #mojo
14:25 hasan joined #mojo
14:25 hasan hi all
14:28 hasan I wonder how I can implement my API in such way that you can call http://.../api/v1/cars?vendor=bmw
14:29 hasan I thought of having a if ( defined $vendor ) { $query = "SELECT * FROM ... WHERE $vendorkey = $vendorvalue }
14:30 hasan else query is only "SELECT * FROM ...";
14:30 hasan does this make sense?
14:50 jberger hasan: however you structure your sql, that's model layer and is not really related to routing (which is controller)
14:50 jberger also, that looks like sql-injection vector when written that way
14:58 hasan jberger: I made the example like that. I am using it in my model of course.
14:59 hasan however I don't know how to say "WHERE ? = ?" and bind the $parametername as $name and the value as $value. $name fails.
14:59 hasan that's why I use "WHERE $name = ?" and bind only $value.
15:00 jberger that's ok (though you could use a sql builder if you wanted) because the parameter name is not sourced from the user
15:00 jberger s/user/client/
15:00 hasan jberger: but beside that strucuring, would that apporach be common practice?
15:00 hasan ok
15:02 sugar joined #mojo
15:02 jberger hasan: rereading your original question, $vendorkey looks like it might be sourced from the client. Is it?
15:02 hasan yes
15:03 jberger ok then I take my previous advice back partially
15:03 jberger you need to validate that against a list of valid parameters first otherwise you have an injection vector
15:03 hasan jberger: to strictly validate user/client sourced params I thought about using a hash validator
15:03 hasan oh. same thought :)
15:04 hasan I would use module for that. like Data-Validate-Struct or something
15:16 burtleton left #mojo
15:22 mib_nuujcm joined #mojo
15:26 * sri hates dynamic where clauses... still looking for a better solution there for Mojo::Pg... https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L62-L75
15:34 sri lol, and just as i say that i have a solution ;p
15:35 sri "where state = coalesce(?, state) and task = coalesce(?, task)"
15:35 sri gotta explain analyze it though
15:46 woz joined #mojo
15:48 sri hmm, the state = state and task = task clause is not optimized out
15:50 jberger http://i3.kym-cdn.com/photos/images/facebook/000/044/778/hatersgonnacat.jpg
15:54 sri hmm
15:54 sri sometimes i don't understand the query planner
15:55 sri where state = coalesce('inactive', state) gets to use the order by index... but state = coalesce(null, state) doesn't
15:55 moritz state = coalesce('inactive', state is the same as  state = 'inactive'
15:56 sri sure, but why wouldn't it use an index for the order by
15:57 sri full query "explain analyze select id from minion_jobs where state = coalesce('inactive') order by id desc limit 10 offset 5;"
15:57 moritz probably because it thinks too few rows are returned
15:57 moritz whereas state = coalesce(null, state) is always True
15:57 sri that one is optimized and uses the index
15:57 moritz so it might think more rows are returned
15:57 * sri shrugs
15:57 sri maybe it would even learn with time
15:57 moritz for few rows, using an index isn't always faster
15:58 sri without a where clause it also uses the index btw.
15:59 sri in fact even "where state = coalesce(null, 'inactive') and task = coalesce(null, task)" gets to use it -.-
15:59 * sri yells at the query planner
16:00 jberger oh no, I've gotten stuck in a google image search!
16:00 jberger http://www.dumpaday.com/wp-content/uploads/2013/01/haters-gonna-hate.jpg
16:01 * genio pours some coffee for jberger
16:01 genio wake up!
16:01 purl Huh? What...
16:02 jberger mmmmm coffee
16:03 sri anyway... question for you sql connoisseurs here... is "state = coalesce(?, state)" an ok thing to do?
16:06 genio I never use it in where clauses like that.  I only use it for the sake of default values on select
16:07 * jberger launches purl into space
16:07 purl Houston? HOUSTON?! ... shit.
16:08 genio I still like it when she bounces happily
16:08 zivester joined #mojo
16:08 * genio pushes purl down the stairs
16:08 purl Hey! *thump* ow! *bang* argh! *bam* son of a *thump* *crunch* whimper...
16:08 genio well, not that time
16:08 sri you monster!
16:09 sri botsnack!
16:09 purl :)
16:09 * genio lowers his head in shame
16:10 * jberger slaps genio with a fish
16:10 sri i would hate keeping the current solution :S https://gist.github.com/anonymous/17c40715f7c407b184e0
16:12 genio coalesce is definitely a prettier solution in that case.
16:14 jberger sri: so state=state gets optimized away?
16:15 jberger and if both are optimized away so is the where?
16:15 jberger pretty brilliant if that works
16:15 * jberger makes mental note
16:16 genio I'm assuming the where gets optimized away in the same sense as ...  where 1=1  etc.
16:17 jberger so I have an unrelated question
16:18 jberger when using die to control flow, it annoying to have to add \n to prevent "at ..."? does anyone else have a better solution?
16:19 jberger hmmm, several grammatical errors in the previous, please read as if I knew how to type
16:20 avenj my solution is usually 'throw exception objects + custom stringification'
16:22 jberger hmmmm, I guess so
16:22 jberger I thought of that but it seemed rather extreme
16:22 avenj it is for just that, if I'm using exceptions for control flow I have other motivations for doing that, usually I want access to other data relevant to the exception
16:23 avenj else there's always sub throw { die $_[0], "\n" }  # ;)
16:24 jberger the recipient still has to chomp then
16:24 jberger I think I'm liking the object solution more and more as I think about it
16:25 hasan joined #mojo
16:32 marty_ joined #mojo
16:48 sri jberger: it does not appear to get optimized away
16:50 sri not sure if my dataset might be too small, testing with 500k jobs atm.
16:52 mst sri: please ensure you're passing pg_server_prepare 0 to the DBI options
16:52 mst sri: otherwise you fuck over a bunch of optimisations
16:52 sri i'm testing different queries with psql atm
16:52 mst jberger: Return::MultiLevel Worlogog::Incident Worlogog::Restart
16:53 mst sri: ok. just the PREPARE stuff, which DBD::Pg uses by default, is evil
16:53 mst sri: because the optimiser has to plan without the values
16:54 mst jberger: I mostly just use R::ML directly, but it removes the need for exceptions for flow control
16:55 jberger mst: this is going to take some reading
16:55 mst jberger: R::ML is simple
16:56 sri results with psql are already not very encouraging at all
16:56 sri https://gist.github.com/anonymous/d26811f44a67dcd91860
16:56 jberger die is pretty common use in Mojo::IOLoop::Delay, I'm not sure how to integrate the two
16:56 mst jberger: with_return { my ($ret) = @_; ... };
16:56 sri state = state kills all the optimizations
16:56 mst jberger: then anywhere you need to return from with_return
16:56 mst jberger: you can call $ret->(...) and it JFDIs
17:02 sri really sucks that state = state doesn't get optimized away :S
17:11 sri hmm
17:12 sri this one might be optimizable "where (state = ? or ?::text is null) and (task = ? or ?::text is null)"
17:12 sri i have no idea what i'm doing though
17:13 crab (why are you generating a condition like that in the first place?)
17:15 sri yes, looks like it actually optimizes them out :)
17:16 sri explain analyze through DBI looks promising
17:16 sri crab: https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L62-L75
17:18 crab ok, state = ? seems sensible. but where does state = state or state = ? or ? is null come from?
17:19 sri this actually works https://gist.github.com/anonymous/937a2242bf539c1d5b4d
17:19 * crab reads
17:21 crab weird.
17:23 jberger mst: "It is an error to invoke $return after its surrounding BLOCK has finished executing."
17:23 sri this is how i did the explain analyze through DBI https://gist.github.com/anonymous/c2f9625dc0e26a38b3b1
17:23 jberger I don't know how to make use of that with non-blocking architecture, unless maybe we backed it into the IOLoop itself, but that's not my call
17:24 sri pretty much ideal for all my tested cases
17:24 sri crab: can you elaborate on that?
17:26 crab sri: probably not constructive. i just don't see much point writing such a weird query to avoid a few lines of code.
17:26 sri to avoid dynamically generated sql
17:26 crab but since you switched to single quotes, you can use $1 style placeholders instead of ?, which will save you from saying state state task task
17:27 mst jberger: er. you'd use 'local' to provide the thing
17:27 sri that's a good point
17:27 mst jberger: with_return { local $Return_Here = $_[0]; ... }
17:27 mst jberger: rather than eval { ... }
17:27 jberger mst: the eval isn't in my code, its in the IOLoop
17:28 mst I'm describing a general pattern that I find more elegant and more reliable than exceptions
17:28 jberger mst: ok, and that is interesting, but I don't think that it is relevant here in that case
17:29 mst if you're already stuck with the inferior design, then the superior design isn't directly helpful, no
17:29 mst you started off asking about exceptions for flow control in general, you never said "in a situation I don't control and can't change" until later :)
17:30 jberger mst: granted
17:30 sri crab: which one do you think is more elegant? https://gist.github.com/anonymous/1a159246c0bc25518956
17:31 sri or anyone else for that matter
17:32 sri (in my tests New gets all the same optimizations Old gets... so no performance difference)
17:34 woz joined #mojo
17:35 sri first one to say "just use SQL::Abstract" will get challenged to make a better version with it
17:35 * sri has yet to come across a truly elegant solution for dynamic where clauses
17:37 crab sri: actually, written using $1 it seems fine
17:37 meredith that looks so much nicer
17:38 meredith sql-abstract would be a mess similar to the top, just in a different dialect
17:41 dod joined #mojo
18:03 KCL_ joined #mojo
18:04 jberger hhahahaha: https://twitter.com/mlyar/status/536125518593413121
18:05 sugar_ joined #mojo
18:07 hasan joined #mojo
18:09 marty_ joined #mojo
18:10 sri lol
18:10 sri i like those barbie memes
18:12 sri btw. those ::text annotations are not necessary with $1 \o/
18:17 sugar__ joined #mojo
18:28 sri i wonder how those optimized out where clauses affect the query plan
18:29 sri (if postgres keeps multiple optimized code paths around)
18:31 mst eh?
18:31 sri as in... it seems like it would be rather bad if different input values invalidate the whole plan
18:31 mst if you have pg_server_prepare 0 set, then the planning is done at execute() time
18:32 mst so it's per-set-of-values
18:32 mst if it's set to 1, the planning is done at prepare() time, and pg simply won't perform any value-related optimisations
18:32 sri that seems odd
18:33 sri not my observation at all
18:33 sri unless explain analyze is a special case that doesn't get prepared
18:34 mst how are you running explain analyze?
18:36 sri https://gist.github.com/anonymous/e6eac6a9409800e5f20a
18:36 sri the plans are very different depending on values
18:37 mst yes
18:37 mst because, as documented, "queries that do not begin with SELECT, INSERT, UPDATE or DELETE are never sent as server-side prepared statements"
18:38 mst my superpower: reading the DBD::Pg documentation
18:39 * mst suggests using auto_explain server side so you get a log of the plans from a running actual minion instance
18:40 sri oh shit
18:46 mst honestly, set pg_server_prepare 0 and move on
18:46 mst it's a bad default, the author regrets it, but compatibility
18:46 sri i'll be damned, even inserts are faster with pg_server_prepare = 0
18:47 mst this is why I've mentioned it regularly since you started on Mojo::Pg :P
18:48 sri like wtf? 1986.155 inserts/s without prepare and 1796.300 with prepare
18:49 sri oh no, this one was my bad
18:49 * sri hangs head in shame
18:50 mst re-prepare()ing every time and thereby causing round trips?
18:50 sri yea
18:51 mst the nasty one is e.g. WHERE x = ?
18:51 mst if x is a boolean, and is only true, say, 10% of the time
18:51 mst if you pre-prepare it'll pick a table scan even if there's an index
18:52 mst if you prepare with values, it'll scan for false and use the index for true
18:52 mst well, not exactly *the* nasty one, but that'
18:52 mst s the case where I shot myself in the foot and learned about this as a result :)
18:55 sri does seem like a much better default
18:56 mst I suspect the planner used to be significantly slower
18:56 mst or at least, that's the only reason I can think of that the current default ever seemed like a good idea
19:00 bpmedley joined #mojo
19:12 hernan joined #mojo
19:18 zivester joined #mojo
19:23 woz joined #mojo
19:33 denis_boyun joined #mojo
19:35 hasan joined #mojo
20:03 marmez joined #mojo
20:18 denis_boyun joined #mojo
20:25 s1037989 preaction: this is what I was looking for to render no content: $c->render(text => '', status => 204);
21:01 alnewkirk joined #mojo
21:12 woz joined #mojo
21:20 bobkare joined #mojo
21:35 rem_lex| joined #mojo
21:38 meshl joined #mojo
21:58 hasan joined #mojo
22:07 Grinnz mst, probably true, postgres <9 was not so nice..
22:40 bobkare joined #mojo
22:53 basiliscos joined #mojo
23:01 woz joined #mojo
23:11 sri hmm, is no_ignore_case in Getopt::Long broken?
23:16 sri yea, looks like it
23:16 purl yea, looks like it is probably broken and doesn't report Z_BUF_ERROR http://pastie.org/8673657
23:18 sri mojo prefork -L http://*:5000
23:18 sri -L is supposed to be the lock timeout :S
23:19 sri -l is listen
23:19 sri i have a feeling this might have something to do with Mojolicious::Commands
23:20 bpmedley http://postgis.net/ <— What we need are minion workers that only process a job if they are “close” to a server… :)
23:21 davido_lt joined #mojo
23:22 davido_ joined #mojo
23:22 jberger purl,  forget yea, looks like it
23:22 purl jberger: I forgot yea, looks like it
23:26 sri shit, it was a bugfix in Mojolicious::Commands that broke it
23:27 AndroUser joined #mojo
23:28 good_news_everyon joined #mojo
23:28 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/8p9bWw
23:28 good_news_everyon mojo/master eb3d8ab Sebastian Riedel: fixed bug in Mojolicious::Commands where the global Getopt::Long configuration would be changed after a command had already been loaded
23:28 good_news_everyon left #mojo
23:28 sri that one kinda sucks
23:29 sri Getopt::Long is always causing trouble
23:29 jberger if only it wasn't clearly the right choice, I would highly recommend something else :/
23:30 bobkare joined #mojo
23:30 jberger sri: should we consider Mojo::OptionParser?
23:30 jberger meaning roll our own?
23:32 sri it is true that we are only using a very small subset of the Getopt::Long features
23:32 sri with a very opinionated configuration
23:33 jberger and its the global getopt options that keep biting us, which really should not be globals

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary