The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2014-11-09

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 bpmedley I’m confused.  Are you saying: AutoCommit=0 is the rigth way or AutoCommit=1 is correct?
00:01 sri AutoCommit=1 all the way
00:03 bpmedley I must be reading things incorrectly.  I thought that AutoCommit=1 would make a “COMMIT” or “BEGIN” rejected: https://metacpan.org/pod/DBD::Pg#Transactions
00:04 sri yes, you're reading things incorrectly
00:04 sri https://metacpan.org/pod/DBD::Pg#begin_work
00:05 sri transactions only work the way you would expect begin_work/stuff/commit with AutoCommit=1
00:06 bpmedley I didn’t notice that you had a begin_work in Mojo::Pg::Database.  My bad.
00:07 sri the DBD::Pg docs can be a little confusing
00:09 sri like this one is totally confusing https://metacpan.org/pod/DBD::Pg#rollback
00:09 sri gives the impression you can only rollback with a global AutoCommit=0
00:10 bpmedley It’s like: should echo default to adding a “\n”?
00:13 sri one more thing to note File works with the assumption of a global write lock
00:14 sri while Mango has to deal with the lack of transactions in MongoDB
00:14 bpmedley What we need is MySQL
00:14 sri so it's much more defensive
00:15 sri only the find_and_modify stuff is atomic
00:15 sri everything else uses a state machine to ensure consistency
00:15 sri no need for transactions really
00:16 sri just update with a state condition
00:17 sri if there's two conflicting updates the first one wins and we're still happy
00:17 sri i think there is only one case that will need a lock, and that's dequeue
00:18 bpmedley Let me try and get a proper dequeue and enqueue for review.  Then I’ll update other pieces of the code.
00:55 jamesw joined #mojo
00:55 rem_lex| joined #mojo
01:06 jberger sri: just an idea, now that there is Mojo::EventEmitter::catch, what about Mojo::EventEmitter::raise or throw
01:08 jberger I'm trying to figure out an interface that I like for a simple webservice api
01:10 jberger for the blocking interface, I'm trying to decide if I should: die $tx->error->{message} unless $tx->success
01:10 jberger or $self->emit(error => $tx->error->{message})
01:11 jberger or maybe just return nothing, or log or something
01:12 jberger I kinda want the ua/tx to be behind the scenes
01:41 amon joined #mojo
02:11 bpmedley https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm  <— How do dequeue and enqueue look now?
02:14 sri bpmedley: much better, but the ->dbh uses are still weird
02:15 sri you shouldn't have to call ->dbh
02:16 sri and for the insert i would use "insert ... returning id", not last_insert_id
02:17 sri https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm#L36
02:17 sri that one is actually pretty bad
02:17 bpmedley Moment
02:17 sri the db instance gets destroyed and the dbh cached again for reuse
02:18 sri and why would you insert in a transaction at all?
02:19 bpmedley Habit.  In this particular case, I don’t.. thanks for catching that.
02:19 sri i don't think that's a good habit
02:19 sri and i believe that _try is the only place you will need a transaction
02:20 bpmedley You haven’t seen some of the systems I’ve worked on.. :)
02:20 sri everything else is consistent because of the state machine
02:21 sri https://github.com/kraih/minion/blob/e9cf46ac872aa9fd4ec8ed397096fa54565de7c2/lib/Minion/Backend/Mango.pm
02:21 sri that file should be your blueprint
02:22 bpmedley I have downloaded that and perused it some.
02:22 sri queries are really pretty straight forward '$lt' and stuff are operators (lt = lower than)
02:22 sri or less than?
02:22 * sri shrugs
02:23 sri oh right, the notifications stuff won't work
02:23 sri let me look up a simpler version
02:24 hasan joined #mojo
02:24 hasan does anybody know and use the Mojolicious::Plugin::REST module?
02:25 sri hmm... older versions also have problems...
02:25 sri better just ignore the notifications stuff ;p
02:25 hasan I have a little problem: I can say: $r->rest_routes( name => 'Foo', controller => 'Controller::API::Foo' ); but not $r->rest_routes( name => 'Bar', under => 'Foo', controller => 'Controller::API::Foo::Bar' );
02:25 hasan because then mojolicious look for controller "MyApp::Foo" instead of "MyApp::Controller::Foo"
02:26 hasan +API
02:26 hasan because then mojolicious look for controller "MyApp::Foo" instead of "MyApp::Controller::API::Foo"
02:27 KCL_ joined #mojo
02:34 bpmedley sri: https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm <— How do enqueue/dequeue/_try look?
02:38 hasan looks like this is not implemented: https://metacpan.org/source/ABHIISNOT/Mojolicious-Plugin-REST-0.006/lib/Mojolicious/Plugin/REST.pm
02:38 hasan line 113: my $bridge_controller = ucfirst($under_name_lower);
02:39 hasan actually it should check for controller name if "under" is given and then use that instead.
02:39 hasan anyone here who wants to rewrite this module with me?
02:40 hasan s/rewrite/fix/
02:45 jberger joined #mojo
02:45 sri bpmedley: looks good, i wouldn't do the ->question thing though
02:45 bpmedley :)
02:45 bpmedley I assumed when we got to that it would be looking good.
02:45 sri :D
02:45 bpmedley I’m going to fixup the other methods.
02:45 sri bpmedley++
02:46 sri your code brings up an interesting point though
02:46 bpmedley What’s that?
02:46 sri i guess the transaction object should hold the db instance instead of the dbh
02:47 sri perhaps even allow calling ->do/query on $tx
02:48 bpmedley That might make things look less confusing.
02:48 sri do/query might be a bad idea, but i'll s/dbh/db/
02:55 sri or actually
02:55 sri might be better to keep it this way to avoid leaks
02:56 sri yea
03:01 hasan I fixed Mojolicious::Plugin::REST and it works for me. can anybody review before I pull a request? https://github.com/omani/Mojolicious-Plugin-REST/commit/1ea0e4796db0d66b4fdd732084e7e343bc19f0b5
03:03 jberger hasan: can you include the diff before running podweaver
03:04 hasan jberger: I've never worked with podweaver. let me check that one out first.
03:05 jberger I'm assuming you did `dzil build` or some such nonsense
03:05 * jberger dislikes dzil
03:06 hasan jberger: I just cloned the github repo of abhis and pushed the changes to my forked repo
03:07 hasan I don't want to upload on CPAN if you mean that.
03:07 jberger hasan: how did the new pod sections appear then?
03:08 jberger also, please don't use tabs if the author did not
03:08 hasan jberger: I guess because I cpanm'ed the module for my mojo app. and fixed it in there. but pushed the changes on a forked repo which appears to be newer
03:08 jberger and leaving the old code in place as comments is confusing
03:09 hasan ok. I will change that.
03:09 jberger hasan: dzil based dists might look very different in repo form
03:09 jberger podweaver takes some of the comments and weaves them into pod
03:10 jberger (one of the many reasons not to like it IMO)
03:12 bpmedley https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm <— How now?  Should list_jobs / list_workers / stats be pure SQL?  It doesn’t seem to be a huge issue the current way.
03:19 sri bpmedley: loading 100000000 jobs into memory no big issue? Oo
03:19 bpmedley Hrmm.
03:19 hasan jberger: fixed it https://github.com/omani/Mojolicious-Plugin-REST/commit/e1a24098e7bf41335ef56422df7fa3dca96b591f
03:19 hasan thanks.
03:22 mattastrophe joined #mojo
03:22 sri bpmedley: oh, i've only just now noticed that you're locking *all* new jobs
03:22 sri that doesn't seem like a good idea
03:23 sri in fact, that might select a million jobs too
03:24 sri i think all acceptable tasks need to be part of the query, and you only select one
03:24 bpmedley Understood
03:25 jberger hasan: I still see changes in the pod sections, including changing something to a typo
03:25 sri (postgres might have some nice array functions to make it easy to send a variable length list of values)
03:25 jberger beyond I am not qualified to comment on the changes you are making, as I have no idea what the module does or what your changes might do
03:26 sri DBD::Pg actually handles ['foo', 'bar', 'baz'] automatically, it's pretty nice
03:27 hasan jberger: I don't know why that happens but I informed the author about my changes.
03:28 jberger hasan: I would clone the repo again and make your changes directly there
03:28 hasan jberger: that plugin uses add_shortcuts to add some RESTful routes.
03:29 hasan jberger: yeah did that. clean fork of his repo and only changes pulled back.
03:29 sri hasan: pull requests that introduce typos tend to be rejected or ignored
03:29 hasan sri: well they aren't mine anyway.
03:29 bpmedley sri: It looks like the _jobs function needs to go, and be replaced by SQL.  And the _try lock needs a LIMIT 1.  Are those the main things you can see?
03:29 hasan and the author knows that (at least will see I didn't change anything in that PODs)
03:30 sri bpmedley: i don't see how a limit 1 would work, if your first result is an unsupported task you'll never get work
03:31 sri worker1 may only accept job A and C, while worker 2 only handles job B
03:31 jberger hasan: https://github.com/omani/Mojolicious-Plugin-REST/compare/abhishekisnot:master...master
03:31 jberger ^^ you really don't see any pod changes there? htt2crud?
03:32 sri hasan: https://github.com/omani/Mojolicious-Plugin-REST/compare/abhishekisnot:master...master#diff-1a87a08d7c22230814f0563a6ce7e8edL373
03:32 jberger changing C<> to ''
03:32 sri hasan: *you* made that change
03:33 hasan wtf
03:35 bpmedley sri: http://pastie.org/9706312 <— Something like this?
03:35 hasan I don't get this. If I diff my changes with https://metacpan.org/source/ABHIISNOT/Mojolicious-Plugin-REST-0.006/lib/Mojolicious/Plugin/REST.pm I only see my changes.
03:36 sri bpmedley: aside from it being horrible code, yes
03:36 hasan but when I do the actual changes in his forked repo, I see the typo changes. but I forked master.
03:36 bpmedley How to make prettier?
03:36 sri bpmedley: oops, my bad, it's not horrible
03:37 sri you just made a list of question marks again... i misread :)
03:37 bpmedley What we need is a “question” function.
03:37 sri funny thing
03:37 purl i guess funny thing is that the spam doesn't even link to real porn
03:37 * sri kicks purl
03:37 purl What? What?  Did i miss a cue?
03:38 sri few weeks ago i asked if Mojo::Pg needs support for (??)
03:38 sri which would automatically expand to the number of arguments
03:39 bpmedley I’m not aware of a way to do that in DBD::Pg or DBI; however, I haven’t read the man pages fully, lately.
03:39 sri you can't
03:39 sri it would be a Mojo::Pg feature
03:39 sri DBIx::Simple does it
03:39 bpmedley Understood, seems like cool feature to add.
03:40 sri https://metacpan.org/pod/DBIx::Simple#query-query-values
03:40 jberger hasan: the author has clearly fixed those typos in his repo, and podweaver generates the standard pod boilerplate
03:40 sri there's just the quoting problems
03:40 jberger this is why making changes vs the installed version and applying them to the repo is not going to work
03:40 bpmedley Can you elaborate on quoting problems?
03:40 sri DBIx::Simple makes sure not to expand (??) if it's surrounded by ''
03:41 bpmedley Aha
03:41 sri and once you do that you open a can of worms because of postgres dollar quoting
03:41 hasan damn. I was blind.
03:42 sri you need funny stuff like this https://metacpan.org/source/JUERD/DBIx-Simple-1.35/lib/DBIx/Simple.pm#L12
03:42 hasan jberger: you are right. my fault. I did my changes to my local module file and copied over because I thought it is the same file. but he his CPAN module is several commits behind the github repo.
03:42 jberger hasan: I'm sure that is usually the case for most cpan authors
03:43 bpmedley Gaw, craziness
03:43 jberger its not worth cutting a new release for a tiny typo fix
03:43 jberger always best to work from their cloned repo directly
03:43 hasan jberger: I should never expect that to be true (github repo == version on cpan) :D
03:43 sri sooo... the quoting problem keeps me on the edge about (??) support
03:43 hasan jberger: yes.
03:43 jberger (doubly so for dzil-based dists)
03:43 jberger hasan: cool
03:43 sri i guess Mojo::Pg is experimental... so we could try it
03:43 * sri shrugs
03:44 jberger and as I said before, I can't comment on your actual changes
03:44 bpmedley Would you also be against a convience function like “question”?
03:44 sri bpmedley: yes, i don't find such a function very convenient
03:44 jberger but now the author might actually look at them. I wouldn't have considered a patch like that to one of my repos personally (as I'm sure you would't either)
03:51 rem_lex| joined #mojo
03:52 hasan jberger, sri: https://github.com/abhishekisnot/Mojolicious-Plugin-REST/pull/4
03:52 hasan :>
03:53 jberger that's more like it
04:00 jberger what I've been working on so far: http://pastie.org/9706335
04:00 jberger a very simple api
04:32 bpmedley jberger: Looks good.  How would propose using it in a Mojolicious::Lite script?
04:33 mattastrophe joined #mojo
04:34 jberger bpmedley: I'm carving this into reuable components, but the idea is in there: http://pastie.org/9706371
04:35 bpmedley Woudl be cool to see lots of services on cpan.
04:35 jberger I'm trying to make an app that refuses to work if the weather is better for the client than for the server :-)
04:36 jberger I'm making a demo of non-blocking bridges, silly but interesting
04:36 jberger inspired by the discussion last night
04:36 jberger and yes, the more webservice apis the better
04:36 bpmedley Would make for a sweet lightning talk.
04:37 jberger the app got too big though, so I'm carving out the geoip and weather apis
04:37 jberger I'll put them on CPAN and then the demo will be easier
04:39 jberger I will say though, I'm already starting to get to the point where I want to use signatures on my CPAN modules
04:39 jberger it was so nice working on the app with signatures and postderef
04:40 bpmedley Signatures make things look up-to-date with respect to other languages.  Very cool, indeed.
04:43 basic6_ joined #mojo
04:47 jberger not just up to date
04:47 jberger one line callbacks can really be on one line
04:48 jberger <3
04:48 jberger Which is great for my code ocd
04:48 bpmedley :)
04:56 preaction Codesessive-Compulsive Disorder?
04:58 hasan sri: why is there no description to bridges in http://mojolicio.us/perldoc/Mojolicious/Guides/Routing?
05:02 hasan nvm.
05:04 hasan say I want to check if id in domain.tld/some/id/foo exists as an id in my database. should I use "over" or "under" for this route? ->to(foo#check) would be sub check { my $psql_check ... }
05:06 hasan with a bridge it does not work if I want to $self->render( text => "id not found in db" ) in my foo#check of the bridge
05:19 bpmedley https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm <— How does this look?
05:25 bpmedley jberger: Have you seen my attempt at a Mojo::Pg backend for Minion?
05:42 jberger bpmedley: I will take a look, but minion is not my strong suit either
05:48 hasan ok just figured it out. nvm.
06:21 Eke- joined #mojo
07:18 marcus pwd
07:18 marcus oopsie
07:34 cfedde /usr/local/wtf
08:17 jamesaxl joined #mojo
08:18 Vandal joined #mojo
08:36 damaya joined #mojo
08:37 damaya Hey, does anyone here run morbo as a daemon with systemd?
08:41 damaya I guess that doesn't really make sense
08:44 damaya Trying to figure out how to integrate systemd with a non-root user and Perlbrew
08:56 damaya man systemd is garbage
09:03 damaya It sucks that Poettering has such an influence in Linux since everything he creates is crap
09:17 sugar joined #mojo
09:20 basiliscos joined #mojo
09:24 sugar_ joined #mojo
09:37 amon joined #mojo
09:42 Eke- joined #mojo
09:59 ssm damaya: figure out how you would start it without systemd (which environment variables to set,  what command line to use, does it fork or not, does anything have to be done before or after start / stop, what to do at restart, and reload), and _then_ you should be able to use that for sysvinit, upstart, systemd, supervisor, daemontools, etc…
10:40 fhelmber_ joined #mojo
10:41 neyasov joined #mojo
11:20 denis_boyun joined #mojo
11:23 sh4 joined #mojo
11:23 dod joined #mojo
11:35 Vandal joined #mojo
11:47 denis_boyun joined #mojo
12:22 irq_ joined #mojo
12:37 basiliscos joined #mojo
12:38 sh4|2 joined #mojo
12:41 sh4|3 joined #mojo
12:56 jberger damaya: you might just want to use Perl::Build to build a deployment perl
12:57 jberger I never did find a great way to upstart perlbrew executables
12:58 jberger Perl::Build is the build logic of perlbrew
14:02 jamesaxl joined #mojo
14:20 * sri yawns
14:27 KCL joined #mojo
14:35 sri bpmedley: good start, but i still see many many questionable decisions
14:35 sri like https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm#L147-L165
14:36 sri why would you select and then delete instead of just delete with a condition?
14:38 sri oh, you're developing without tests
15:01 woz joined #mojo
15:26 sh4 joined #mojo
15:29 denis_boyun joined #mojo
15:34 denis_boyun joined #mojo
15:35 sh4 joined #mojo
15:59 Adura joined #mojo
16:14 bpmedley sri: I’m writing tests right now.
16:15 sri bpmedley: please try to replicate the mango tests (without the mango specific parts)
16:16 bpmedley Not sure why I have a SELECT and then DELETE.  I’ll chante to just a DELETE with a condition.
16:19 sri list_jobs is not quite correct either, limit and offset are always required
16:20 bpmedley Awright.  Thanks for understanding that is my first exposure to Mojo::Pg and Minion.
16:20 sri so the if $offset dance is unnecessary
16:20 basiliscos joined #mojo
16:21 sri required args are in the docs, which you skipped ;p
16:21 bpmedley I did skim them.
16:23 sri if you have to iterate through a large number of jobs, better use while (... ->hash) {...} instead of ->hashes
16:23 sri (just saw ->hashes in ->repair)
16:25 agregory joined #mojo
16:27 sri one more problem, you don't sort your lists for in (...) queries
16:27 sri specifically tasks
16:28 sri which result from keys %$tasks, which will be random in newer perls
16:28 sri so the sql will look different with each iteration, and the prepare/execute won't work
16:29 sri (won't work for automatic reuse)
16:29 bpmedley https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm#L185  <— So, ORDER BY created, task, priority  ?
16:29 sri oh wait... my bad again
16:30 sri you just make a list with question marks...
16:30 sri sorry!
16:30 bpmedley I’m confused; is the ORDER BY correct at line 185?
16:31 sri just ignore what i said, it's all fine
16:32 sri the "AND task in ($q)" stuff keeps tripping me up, since i keep expecting raw values in the sql when you interpolate variables, not just dynamic placeholders
16:33 bpmedley It looks a bit off and is a little ugly, compared to other parts.
16:37 sri i wonder if the in (...) stuff could be done nicer with some kind of array magic
16:38 sri so you could just do ->query($sql, \@tasks)
16:38 bpmedley It would make the code look nicer.
16:38 sri DBD::Pg knows how to handle array refs
16:39 bpmedley I skimmed that for array usage.  Didn’t see it - do you have a link?
16:39 sri so if you have one placeholder ? you get {'foo','bar','baz'} out
16:39 bpmedley Oh!
16:39 bpmedley One sec.
16:39 purl one sec is not a long time!
16:39 * sri doesn't know postgres operators well enough yet
16:41 * sri has to go running now
16:47 sugar_ joined #mojo
17:11 marcus run forrest, run
17:12 marcus if you have a chromecast, castnow is a must!
17:17 sri http://www.writersontheloose.com/writers/writer1323/photos/60271/Large_run%20forest%20run.jpg
17:18 denis_boyun joined #mojo
17:18 sugar__ joined #mojo
17:22 dotan joined #mojo
17:22 neyasov_ joined #mojo
17:26 jberger marcus: the npm package out phone app?
17:27 jberger s/out/or/
17:30 marcus jberger: the npm package. It's a command line app.
17:31 marcus so you can do like 'castnow <youtube url>' or 'cast now wat.mp4', or even castnow <sometorrent>
17:48 jberger cool
17:48 jberger so, yesterday I got close to happy with Mojo::WebService::FreeGeoIP
17:49 jberger and today the site has been down all day :(
17:49 jberger so I looked at the code, and saw that it just downloaded the MaxMind csv db into sqlite
17:49 jberger so I figured I could do that!
17:49 * jberger hacks
17:49 jberger working title GeoLiteLite
17:50 jberger :)
18:05 sri hmm, one test fail on cpan testers, seems pretty random though http://www.cpantesters.org/cpan/report/8f3fe01f-727e-1014-83cb-f4ae7c552f45
18:07 sri Net::DNS::Native 0.12 looks pretty good too http://www.cpantesters.org/distro/N/Net-DNS-Native.html?oncpan=1&amp;distmat=1&amp;version=0.12
18:08 jberger Oleg++
18:08 jberger sri: hmmm daemon failures on windows
18:09 Adura It works on Strawberry now? interesting.
18:09 jberger something to keep an eye on
18:09 Adura It kind of works? hah.
18:10 sri jberger: just one test and only windows, so might just be wonky sockets
18:12 sri bpmedley: you might want to share your schema too
18:13 sri think i'll start later with the migration part, then we can slowly merge and clean up the methods that look good
18:13 bpmedley sri: http://pastie.org/9708004 <— Pretty basic
18:14 sri hmm, all the timestamps look odd
18:14 bpmedley Right.  I didn’t know what to use for the data type in the beginning.  I haven’t gone back and adjusted.
18:15 sri timstamp i'd assume, and use postres functions for turning those into epoch when selecting
18:16 bpmedley That’s on the list.  After tests and some cleanup mentioned above.
18:16 sri ok
18:21 sri postgresql character types already feel very nosql-ish, with character limits not actually affecting performance
18:22 sri little funny how text is a little bit faster than character(n) actually
18:34 mst sri: varchar in postgres is implemented as text, with some overhead adding extra code to handle the length restriction
18:34 bpmedley https://github.com/brianmed/minion/blob/master/t/pg.t  <— Still several tweaks for the Pg.pm
18:34 mst sri: they optimised text to the point where there was no reason to keep a separate varchar impl.
18:36 sri mst: yea, just been reading the docs on it, cool stuff
18:38 sri bpmedley++ # not bad
18:39 bpmedley Thanks
18:41 sri this looks odd though https://github.com/brianmed/minion/blob/master/t/pg.t#L122
18:41 bpmedley Yes, I agree.  However, is it correct?
18:41 sri with minion that would have been a little hack abusing the schemaless nature of mongodb
18:41 sri s/minion/mango/
18:42 sri nope, there is no result field in the schema
18:42 bpmedley Is it above updated?
18:42 sri it's just to make sure the task actually ran
18:43 sri since it runs in a separate process
18:43 bpmedley Just as ec.
18:43 bpmedley *sec
18:43 sri file.t also use a schemaless hack there
18:45 sri i guess if you want to do it the cool way you make a results table for the test
18:46 sri and drop it once the test is done
18:47 sri uncool way would be to store the result in a temp file
18:49 bpmedley Hrmm.  A temporary table may work well.
18:50 * sri nods
18:56 bpmedley Are you saying that the “result” field should not be stored in the database?
18:56 sri i'm saying it's not part of the job table
18:57 sri it's an arbitrary test case
18:57 sri so a temporary table for that test alone makes perfect sense
19:05 bpmedley What would the scope be of a temporary table?  Aren’t they per session?  Would the dbh used to select be the same that created and updated the temp table?
19:19 davido__ joined #mojo
19:21 Eke- joined #mojo
19:30 punter joined #mojo
19:43 sugar joined #mojo
19:51 damaya joined #mojo
19:53 punter Not much talk happening here today
19:54 sri bpmedley: i'm not following i'm afraid, you might be thinking too complicated
19:55 sri anyway
19:55 bpmedley http://www.postgresql.org/docs/9.2/static/sql-createtable.html#SQL-CREATETABLE-COMPATIBILITY
19:55 sri showerthought... what if we renamed the error field to result and made it a structure (json)
19:56 sri would be much more static schema database friendly
19:56 bpmedley Hrmm
19:56 sri bpmedley: nooooo, i didn't mean that kind of temporary table
19:56 sri just a table for that one test file
19:56 sri create, test, drop
19:57 bpmedley Perhaps we can revisit if the error change doesn’t pan out.
19:57 bpmedley Right now I’m updating the schema to use more specific data types.  Can I get back to you?
19:57 sri guess i could give it a try later
19:58 * sri needs food now!
20:00 Vytas joined #mojo
20:03 sri allowing strucutres as errors will be good for other stuff, like preserving a stack trace
20:13 marcus joined #mojo
20:23 bpmedley http://pastie.org/9708228 <— Hopefully better schema
20:51 sri bpmedley: might be worth using bigserials ;)
20:51 bpmedley Do you think a create_scheme command is needed?
20:51 sri no, like i said, i'll use migrations
20:51 bpmedley Ok
21:00 bpmedley https://github.com/brianmed/minion/blob/master/t/pg_lite_app.t <— Thoughts on the tests?
21:01 sri oh, i didn't want a lite_app test
21:01 bpmedley Ok. Want me to delete?  Didn’t take very long.
21:01 sri yea, that only gets in the way of parallel testing
21:02 bpmedley https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm <— How does the code look?
21:06 sri bpmedley: timestamp should be unnecessary, postgres can almost certainly handle epoch
21:08 sri remove_job is a select/delete hybrid again
21:09 bpmedley Doh; sorry
21:10 sri looks pretty good otherwise
21:10 sri not sure about the _worker stuff... loading the entire table into memory always makes me nervous
21:11 bpmedley I debated changing that.
21:11 bpmedley How many workers do you expect, on average?
21:11 sri yea, that's the question
21:11 bpmedley With tests it should be easier for me to change.
21:11 sri in the end, i guess what matters more there is which version results in cleaner code
21:12 hertzhaft joined #mojo
21:12 bpmedley Did you notice the delayed data type in the schema?
21:13 sri nope
21:14 sri how come it's not a timestamp?
21:14 bpmedley delayed => $options->{delay} ? (time + $options->{delay}) : 1      <— It’s a VARCHAR because of that bit ‘o code.
21:14 sri oh
21:14 bpmedley Maybe make the “1” “time - 1” ?
21:14 sri ok, the idea was to give it a value in the past
21:15 sri hmm, that might conflict with server time
21:15 sri better make it 1
21:15 bpmedley Eww
21:15 sri as in 1970
21:15 sri not eww imo
21:15 sri safety
21:16 sri not depending on worker specific time is good
21:16 bpmedley Hrrmm.. maybe a: Gaw, time is always complicated
21:16 sri oooh
21:16 sri you're not using now()
21:16 bpmedley Where?
21:16 sri i'd always prefer server time if possible
21:17 sri retried = ?
21:17 sri was what i just saw
21:17 sri with timestamp(time) assigned
21:18 sri created too
21:19 sri even for delayed there might be a more clever way to calculate it on the server side :)
21:19 sri (or maybe not)
21:19 sri started is also not server side
21:19 purl okay, sri.
21:20 * sri slaps purl
21:20 purl o/` Hit me baby, one more time o/`
21:21 sri of course you ideally always have synchronized time between all workers and the database, but if there's a choice, i'd prefer database time
21:21 sri that actually really annoyed me about mongodb
21:21 sri it had no operator for that
21:23 sri i was afraid a Mojo::Pg backend would end up with more code, but this doesn't look so bad at all
21:23 stryx` joined #mojo
21:25 sri btw. epoch for input is supported http://www.postgresql.org/docs/9.4/static/datatype-datetime.html
21:26 denis_boyun joined #mojo
21:26 sri "...but the others are simply notational shorthands that will be converted to ordinary date/time values when read..."
21:28 sri so just throw your epoch timestamps at postgres (if any are left on the client side)
21:29 sri it's funny, i never liked postgres type annotation when i saw them... but since i've been learning more about postgres they just feel right
21:30 sri those crazy operators still freak me out though... @>...
21:32 sri punter: NO MORE SILENCE HERE ;p
21:32 bpmedley https://github.com/brianmed/minion/blob/master/lib/Minion/Backend/Pg.pm#L164
21:32 bpmedley DBD::Pg::st execute failed: ERROR:  invalid input syntax for type timestamp: "1414722612"
21:33 bpmedley Did I interpret what you said correctly?
21:33 bpmedley If so, it’s getting an error.
21:33 sri bpmedley: and if you do s/to_timestamp(?)/?/
21:33 bpmedley That error is without the to_timestamp()
21:33 bpmedley With to_timestamp it works.
21:33 sri ok, then try quoting the $after... as in "$after"
21:33 bpmedley Sorry fo the confusion.
21:34 sri ah
21:34 sri odd
21:34 sri it needs to be sent as a string according to the docs
21:35 sri i'd expect "$after" to result in a single quoted string, which postgres should recognize as a shorthand
21:35 bpmedley $db->query($sql, "$after");  <— No go, without to_timesamp()
21:35 sri gonna have to investigate that later
21:36 sri but to_timestamp(?) still looks nicer :)
21:36 sri bpmedley++
21:37 bpmedley Thanks;  sri++ # for the help
21:37 sri thanks for being so patient with my reviews :)
21:39 sri i'll try to get it all merged into master in the next few hours
21:39 sugar_ joined #mojo
21:39 sri first gotta check if turning the error field into result works out
21:39 bpmedley Do you want a pull request?
21:40 sri no, i'll merge it slowly by hand
21:40 sri migrations have to come first anyway
21:40 bpmedley Ok
21:40 bpmedley Be back in a bit
21:44 sri bpmedley: you could open a pull request so i can mention it in the commit message for auto close, then we have it all connected
21:44 sri (i guess that would result in the cleanest history)
21:47 alnewkirk joined #mojo
21:49 sri oh
21:50 sri btw. "select timestamp 'epoch' + '1';"
21:50 sri this works in psql
21:50 sri looks like to_timestamp() really is the cleanest solution
23:00 marcus joined #mojo
23:03 diegok joined #mojo
23:20 d4rkie joined #mojo
23:46 hahainternet sri: you're implementing migrations?
23:46 hahainternet ah, scrolled back up a bit and i see
23:47 hahainternet have yet to come across a particularly nice migration implementation so was interested :)
23:48 sri already have
23:48 sri https://github.com/kraih/mojo-pg#mojopg-
23:49 sri i went with the most minimalistic implementation possible, heavily relying on postgres features
23:49 hahainternet do you support transitions with things like added constraints? or do they have to be manually specified?
23:49 hahainternet that's where most of these systems fall apart
23:50 hahainternet especially when you have several parallel developers
23:50 sri please elaborate
23:50 hahainternet answered my own question nevermind :)
23:50 sri no you didn't
23:51 hahainternet i read the source, it's fine, i can explain what i was talking about if you like
23:51 sri what are "transitions with things like constraints"?
23:51 hahainternet it's more systems like south with django, which is particularly painful
23:51 hahainternet sri: for example, adding a not null constraint to a column
23:51 sri ah
23:51 hahainternet the forward migration obviously needs some data to add to it if it does not have a default
23:52 sri ok, it's pure psql, so no problem
23:52 hahainternet yeah i noticed
23:52 hahainternet probably the smartest approach really
23:53 sri goals were to make it really hard to screw up, and really easy to sync to the latest version automatically
23:54 cooper joined #mojo
23:54 sri migrations don't even run in steps, all intermediate steps get stitched together and run in a single transaction, so it's all or nothing
23:55 sri of course postgresql makes safety super easy
23:57 sri there's multiple named migrations, and only one migration can run at a time thanks to a super lock

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary