The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2015-02-28

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 riche lifesaver :: MOJO_MIGRATIONS_DEBUG=1 prove -I lib -v t/basic.t
00:00 riche man this is just awesome
00:02 pink_mist "-I lib" can be written as "-l"
00:02 Grinnz_ or -Ilib if you want to be especially nice to those with bad fonts
00:03 riche thanks ... old habits die hard
00:04 pink_mist Grinnz_: well, if you have a font where you can't distinguish between 1/l/I, you have only yourself to blame =)
00:10 sri migrations with postgresql are really cool, all i have to do is concatenate the sql for the versions you want and execute them with one big transaction
00:10 zackiv31 joined #mojo
00:11 sri if anything fails it just rolls back and you're right where you started
00:12 sri <3 transactional ddl
00:15 jberger riche: I need to go back and read the mct code again myself :-)
00:16 jberger oh batman didn't merge any of his changes, all in a branch still
00:21 franzkafka joined #mojo
00:21 franzkafka Is it possible to add a request cookie with Mojo::UserAgent?
00:22 franzkafka tempire: did you designe http://tempi.re?
00:23 pink_mist of course it is
00:23 jabberwok sri: Postgresql, do you have a good url describing those migrations? google is letting me down here. my last encounter was circa "Practical Postgresql" oreilly book (early 2000s) but that sounds amazing
00:23 pink_mist https://metacpan.org/pod/Mojo::UserAgent#cookie_jar
00:27 tempire franzkafka: yes
00:27 franzkafka I really like it
00:27 tempire It's the beard
00:35 riche jberger: take your time, I have to ramp up on Mojo::Pg
00:35 Grinnz tempire, those videos are fantastic
00:35 tempire :)
00:37 franzkafka I take it this '$ua->cookie_jar->collecting(0);' from documentation is no longer relevant?
00:42 protoCall7 joined #mojo
00:43 protoCall7 Hi All, I am playing around with Mojo::Pg, and from reading the docs, I don't see a method to close the connection to the database.  Is that handled automatically, or do I just need to close($pg) when I'm done or something?
00:46 riche $pg->db->disconnect
00:46 protoCall7 riche tyvm!
00:46 riche DBI is the superclass so just perldoc DBI to see what else you may need
00:47 protoCall7 Ahh okay, perfect
00:47 protoCall7 one other quick question, after doing a db->query, can I disconnect and still have the ->hash method from the results available to me, or do I need to wait until I'm completely done with my result set?
00:47 riche actually to be more accurate the Mojo::Pg doc says: Mojo::Pg is a tiny wrapper around DBD::Pg that makes PostgreSQL
00:48 riche protoCall7: not sure ... I usually don't disconnect during runtime
00:49 protoCall7 riche:  I'll experiment with it.  Thanks for your input
01:01 sri jabberwok: migrations are a Mojo::Pg feature
01:02 sri but they rely heavily on PostgreSQL features like transactional ddl
01:15 sri jberger: i think the blocking nature of Minion might be a problem for Mojo::Server::Prefork though
01:16 sri if you do a graceful shutdown of the manager process, it will send a QUIT to the worker, but the worker may be blocking for hours on a long grunning task
01:17 sri the graceful timeout will trigger, and the manager will send a KILL to the worker
01:17 d4rkie joined #mojo
01:17 sri then the manager and worker go down, leaving the job process behind
01:18 sri problem is the worker updates the database for successful jobs
01:18 sri so the job would stay active forever (until repair runs and marks it as failed)
01:20 davido__ joined #mojo
01:33 jberger in that case after thinks have been killed, our should run repair
01:33 jberger things
01:37 mattastrophe joined #mojo
01:38 sri that doesn't sound right
01:39 sri you have 3 layers of processes
01:39 sri manager -> worker -> job
01:39 sri if manager sends a KILL to worker, your manager has no clue what job is up to
01:40 sri remember, worker may be hanging in a waitpid() forever unable to do anything
01:44 davido___ joined #mojo
01:44 jnbek joined #mojo
02:04 sri jberger: perhaps consider a two tier architecture, one worker handling multiple jobs
02:05 sri and instead mess a bit with the waitpid code
02:06 franzkafka joined #mojo
02:06 sri that seems more realistic
02:11 sri jberger: on the plus side, i had an idea for how to do heartbeats properly
02:12 sri might have time later to give it a try
02:14 asarch joined #mojo
02:15 sri btw. for inspiration the sidekiq code is a much better read than resque
02:16 sri celery on the other hand makes me want to poke my eyes out
02:16 sri !@#$% python
02:19 sri i mean... it's kinda readable... but so damn unorganized
02:31 jabberwok thanks sri .
02:37 klapperl_ joined #mojo
03:29 jberger aaaaaannndd... I finally have time for open source
03:30 jberger sri: what're the plans for heartbeats?
03:36 noganex_ joined #mojo
03:44 sri jberger: just did a little proof of concept and it works pretty well
03:44 sri cleaning it up now
03:44 jberger sri++ I'm looking forward to seeing it
03:44 sri basically... allow $worker->register to be called over and over
03:45 jberger nice
03:45 sri if the worker is registered it updates a heartbeat column
03:45 jberger (guess I won't start my own attempt then :-P (
03:45 sri if not, the worker is added
03:45 jberger )) # ocd required two there
03:45 sri {
03:45 jberger makes sense
03:46 jberger AAAAAAAAAA }
03:52 sri jberger: the patch https://gist.github.com/anonymous/8ad41ab7ade21e169d6a
03:52 sri can't think of a better name than "active" atm :S
03:53 sri dead_after defaults to 1 day
03:53 sri which shouldn't be a big deal for most types of tasks
03:54 * jberger reads
03:54 sri especially considering that by default ->repair only runs when a worker is started or after 10 days :)
03:54 sri the timestamp should make manual monitoring pretty easy though
03:54 sri you'll see instantly if a worker has trouble
03:55 sri since you can see the type of task and when the worker last had a heartbeat
03:58 jberger nice
03:59 sri the fact that now workers from all servers can check on each other should be a big advantage too
04:06 jberger cool
04:08 sri still looking for a better name than "active" though :)
04:08 sri something fitting the ****ed theme would be nice
04:11 jberger I come back to "seen"
04:12 jberger but that doesn't fit ****ed
04:12 sri "poked" :)
04:13 sri "activated"
04:13 jberger tickled
04:13 jberger thpppppppt
04:15 sri notified
04:15 sri informed
04:15 sri apprised
04:16 * sri checks his favorite german/english dictionary
04:16 sri advised
04:17 sri posted
04:17 sri indicated
04:17 sri reported
04:18 sri registered
04:18 purl rumour has it registered is a rom that has some info that describes timing sequences for speeding up access, I believe
04:18 sri registered would be the natural choice... but might be misleading
04:19 sri every worker also has a started timestamp
04:20 jberger I don't dislike notified
04:21 marmez joined #mojo
04:24 sri notified it is then
04:25 protoCall7 joined #mojo
04:26 sri and committed https://github.com/kraih/minion/commit/5c4ffb24aa3b070e61caa8264e171e7ca55ba07a
04:27 Anon021 joined #mojo
04:28 sri Minion::Backend::Pg::repair is sooooo clean now https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L104
04:31 jberger avkhozov_: you might want to see this ^^
04:38 csson joined #mojo
05:00 protoCall7 joined #mojo
05:01 jberger nope I tried, reading that patch was the most effort I could conjure today
05:01 jberger tomorrow, I'll do better
05:01 jberger tonight I sleep
05:02 * jberger really just wants to release Test::Mojo::Phantom
05:02 preaction is $dom->at('urlset')->type, 'urlset'; <- something is wrong with this line :p
05:02 sri ->type became ->tag in 6.0
05:03 sri (correct dom api terminology and all)
05:03 preaction yeah, that's why the test fails, but the test would never fail otherwise ;)
05:03 preaction if i got an item back, it _must_ be a urlset tag, because that's what i was looking for
05:03 sri ah... yea :)
05:05 jberger it would error out if there were no urlset tag
05:05 jberger can't call method type on undefined
05:05 preaction sure, and that's perhaps even worse, because testing would stop there
05:06 jberger ok $dom->at('urlset')
05:06 d4rkie joined #mojo
05:06 preaction though i suppose what i just replaced it with would fail at the next line, that requires urlset to have existed (checking for children)
05:06 preaction yeah, that's what i replaced it with
05:07 preaction i use this idiom quite a bit, to try to get as much diags out of a single test run as possible: if ( my $elem = $dom->at( 'urlset' ) ) { ok $elem->text, 'derp' }
05:07 preaction sorry, if ( ok my $elem = $dom->at( 'urlset' ) ) { ... }
05:08 preaction so it fails if it doesn't exist, and then the block that depends on the element isn't run
05:08 jberger hmmmmm, clever
05:08 jberger hey does subtest have an implicit eval around it?
05:08 preaction nope
05:08 jberger too bad
05:09 preaction oh, wait. i dunno. i use Test::Pretty, which hijacks everything
05:09 preaction i mean, i wouldn't think it did. that'd be a bit out of scope i think
05:09 jberger we could build one that does ;-)
05:09 jberger custom subtest functions are a thing I know how to do now
06:07 marmez left #mojo
06:17 dotandimet joined #mojo
06:39 cpan_mojo Mojolicious-Plugin-WriteExcel 2.05 by Zak B. Elep - http://metacpan.org/release/ZAKAME/Mojolicious-Plugin-WriteExcel-2.05
06:59 dod joined #mojo
06:59 melo joined #mojo
07:03 dod joined #mojo
07:35 sh4 joined #mojo
08:15 Vandal joined #mojo
08:35 d4rkie joined #mojo
08:38 berov joined #mojo
08:51 trone joined #mojo
09:23 dotandimet joined #mojo
09:47 jontaylor joined #mojo
10:19 bjakubski joined #mojo
10:31 jontaylor joined #mojo
10:42 d4rkie joined #mojo
11:00 trone_ joined #mojo
11:21 melo joined #mojo
11:42 d4rkie joined #mojo
11:52 d4rkie joined #mojo
12:12 HtbaaPi joined #mojo
12:19 nicomen batman: re: your update of url. perhaps we could set up the subdomain thing that someone suggested? 2014.mojoconf.org 2015.mojoconf.org etc. and mojoconf.org pointing to the latest one
12:19 batman nicomen: we're on it.
12:19 batman just having a hard time making #act get it done...
12:20 batman nicomen: but we're trying to do /mojo2014 to avoid breaking existing links
12:20 batman but also /2014, /2015, ...
12:20 batman subdomains is more work
12:20 batman *are
13:13 kaare joined #mojo
13:14 d4rkie joined #mojo
13:20 d4rkie joined #mojo
13:21 kaare joined #mojo
13:33 HtbaaPi joined #mojo
13:55 tencendur joined #mojo
14:16 asarch joined #mojo
14:23 amon joined #mojo
14:32 d4rkie joined #mojo
14:34 trone_ joined #mojo
15:33 cpan_mojo Mojolicious-Plugin-TtRenderer 1.55 by PLICEASE - http://metacpan.org/release/PLICEASE/Mojolicious-Plugin-TtRenderer-1.55
15:36 riche so $r->bridge is gone?
15:36 riche since a long time?
15:37 batman riche: https://github.com/kraih/mojo/blob/master/Changes#L141 # 2015-01-24
15:38 riche sighs I have some old mojo code and I was wondering why it wsn't testing under newish mojo version
15:38 riche I should go thru that change list more carefully
15:39 riche also Mojolicious::Plugin::Authentication documentation needs work
15:40 riche anyways is there some other (better maintained) auth plugin to use instead of Mojolicious::Plugin::Authentication?
15:43 cpan_mojo Mojolicious-Plugin-Ident 0.31 by PLICEASE - http://metacpan.org/release/PLICEASE/Mojolicious-Plugin-Ident-0.31
15:47 mgrimes I understand Mojo::Dom->val has been deprecated and removed. What is the recommend way of getting the value of inputs,selects,etc?
15:50 batman mgrimes: good question... attr() will do for inputs, and ->text will do for textarea
15:51 batman i guess $dom->find('select')->find('input[selected], input')->first->attr('value') might do it for select
15:51 batman but i have to go now...
15:52 batman *bbl*
15:53 mgrimes thanks batman.
15:53 batman i'm probably wrong :P
15:55 mgrimes pretty complicated. wish ->val hadn't been removed.
15:57 jberger mgrimes: its removal was discussed at length
15:57 cpan_mojo Test-Clustericious-Cluster 0.20 by PLICEASE - http://metacpan.org/release/PLICEASE/Test-Clustericious-Cluster-0.20 (depends on Mojolicious)
15:57 jberger we begged for opinions, suggestions
15:58 mgrimes sorry i missed the discussion
16:05 mst mgrimes: ->val was impossible to make consistent
16:05 mst mgrimes: so it ended up getting filed under 'convenient, just like each() is convenient ... until it bites your genitals off'
16:34 maze joined #mojo
16:40 good_news_everyon joined #mojo
16:40 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/xGCF
16:40 good_news_everyon mojo/master ec40a50 Sebastian Riedel: more interesting Mojo::Template tests
16:40 good_news_everyon left #mojo
16:53 Vandal joined #mojo
16:58 d4rkie joined #mojo
17:04 Grinnz http://imgur.com/gallery/Nzn3vL2
17:07 jberger Grinnz++
17:09 sri \o/
17:12 sri guess this can't be fixed https://github.com/kraih/mojo/pull/748
17:14 sri riche: why are you randomly tweeting the mojoconf url but there is no official tweet campaign?
17:14 sri TELL ME WHAT TO TWEET!
17:19 sri if the site is ready i could have linked to it from the 6.0 blog post too and reached a few thoudand people
17:19 sri s/d/s/
17:19 mst "if the site is reasy" >?
17:19 Grinnz :)
17:20 sri http://www.mojoconf.org/
17:20 sri https://twitter.com/richardelberger/status/571717399092699136
17:20 sri where do we stand?
17:21 mst at least 100ft from the centre of the blast.
17:24 cpan_mojo Mojolicious-Plugin-DSC 1.003 by Krasimir Berov - http://metacpan.org/release/BEROV/Mojolicious-Plugin-DSC-1.003
17:28 riche sri, I was responding to brian d foy
17:28 riche he tweeted it was in nyc but w no url, and your blog has no url
17:30 riche imo if its live on a well known domain name, it's live and it's fair game.
17:30 sri that is my point, if the site is ready we missed a big opportunity
17:31 sri against all odds, he announcement tweet reached 50 retweets :o
17:32 riche this is exactly why kayi should have been coordinating the launch.  but things were happening without her knowing.  so, this is what we get.
17:32 sri so, the site has officially launched?
17:32 riche hell if i know!!
17:33 sri :S
17:33 riche you know, if people are putting things live without going through the coordinator, this is what we get.
17:33 sri allright... if i do a new version of this blog post next week, is there anything specific i should mention? http://blog.kraih.com/post/76535091594/mojoconf-2014
17:33 riche kayi is probably the most frustrated one.  that is why she scheduled the meeting
17:35 riche definitely if we can get a shortlist of core dev talks written in stone then you can certainly write that.
17:35 riche there have been several talked about on here.
17:36 riche kayi is going to put out a poll on the kind of keynote that's wanted - academic, motivational, perl hacker, etc.  cuz we need to nail the keynote asap
17:37 riche and for training - there needs to be more than one.  i was thinking of someone running a masterclass in addition to tempire's class.  but it's too close to yapc::na
17:39 riche anyways
17:43 riche sri: I think this is the confusion.  I asked for a splash, but the site is bigger than a splash.  so tempire did that, and that's awesome.
17:46 riche but the big confusion was mojoconf.com vs. mojoconf.org.  once it went live on mojoconf.com, we were OK to do anything.
17:47 riche look kayi needs to manage this because it also impacts seo.  anyways I need to get back to work.  but this should be a reminder that there's a coordinator for a reason.
17:51 sri cat herding ;p
17:53 Adura joined #mojo
17:57 riche yes.  that's what I told her
17:58 riche she hasn
17:58 riche she hasn't dealt with doing event with all volunteers that are really distributed
17:58 riche it boggles her mind how people are just going and doing things
17:59 riche but i told her that's the way people are used to doing things, so we just need to a) respect it, while b) organizing it.
18:00 maze joined #mojo
18:08 tempire riche: if kayi were in the channel, she wouldn't be so in the dark
18:09 tempire I haven't notified her of anything because she set the meeting on sunday, so until then, everything is still being developed
18:09 tempire In my mind, nothing really exists until sri tweets it, 'cause that's where everything usually starts for any mojolicious-related announcement
18:12 tempire sri: I thought you knew about the site and chose to wait for another post.
18:12 riche tempire: I respect that this channel is for mojolicious development.  but that doesn't mean it has to be the channel for conference communication.
18:12 tempire true enough
18:12 sri well, if you want to go public, historically late monday early tuesday has been the best choice
18:23 good_news_everyon joined #mojo
18:23 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/xGxs
18:23 good_news_everyon mojo/master 4cd34c5 Sebastian Riedel: mention that begin/end keywords are part of the surrounding tag
18:23 good_news_everyon left #mojo
18:25 riche just fyi I created a private google community so kayi can schedule the hangouts.  it will probably be better to discuss conference issues there instead of on here.
18:27 * sri could use the help on an postgresql expert
18:28 tempire riche: we could just use another channel as well
18:30 riche tempire: I am absolutely certain it's easier for you to use google community than for kayi to learn irc.
18:30 tempire k
18:30 sri yesterday i've made a mistake and used now() instead of current_timestamp as the default for a timestamp column
18:31 * tempire has never done that before (whistles)
18:31 sri and suddenly my dequeue query was fetching more than one row https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L192
18:31 sri it's almost like the "limit 1" there failed
18:32 sri the now() was one the "created" column
18:34 sri it's supposed to grab one row and set state = 'active', but it was updating more
18:35 sri is there some possible race condition with the limit 1 and my index? "create index on minion_jobs (priority DESC, created);"
18:39 sri and even if you're not an expert, the new monitoring feature in master could use more eyeballs
18:39 sri https://github.com/kraih/minion/compare/v1.08...master
18:47 btyler was poking around trying to figure out how to fix M::UserAgent::UnixSocket to work with 6.0, found a minor doc oversight: https://github.com/kraih/mojo/pull/749
18:50 good_news_everyon joined #mojo
18:50 good_news_everyon [mojo] kraih pushed 2 new commits to master: http://git.io/xZTm
18:50 good_news_everyon mojo/master f184428 Ben Tyler: POD: no custom socket support here either....
18:50 good_news_everyon mojo/master fdce625 Sebastian Riedel: Merge pull request #749 from kanatohodets/master...
18:50 good_news_everyon left #mojo
18:52 mattastrophe joined #mojo
18:53 good_news_everyon joined #mojo
18:53 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/xZkf
18:53 good_news_everyon mojo/master 5631020 Sebastian Riedel: $id is more appropriate now
18:53 good_news_everyon left #mojo
18:55 * sri wonders how often heartbeats should be sent to the database by workers
18:56 sri every 5 seconds does seem a bit much
18:56 preaction what's the socket timeout? somewhat less than that. maybe half that
18:56 d4rkie joined #mojo
18:57 sri not a keep-alive heartbeat
18:57 sri it's for monitoring
18:57 sri and it updates a column in the minion_workers table
18:58 mst I'd be pretty happy with every minute, I think
18:58 preaction depends on how many workers and how fast you expect to respond to failed heartbeats. maybe 60s instead?
18:58 Adura When do codenames change?
18:58 sri the heartbeat timeout is set at 1d by default :)
18:59 sri i guess i should just make it configurable :S
19:00 sri hate seeing the worker command get more complicated though
19:02 sri oh well, it's "worker -I 5" now, with a default of 60 seconds
19:02 sri and workers are considered missing after 1 day automatically
19:03 sri after which their jobs are set to failed
19:06 sri getting kind of big :/ https://github.com/kraih/minion/blob/master/lib/Minion/Command/minion/worker.pm
19:06 irq joined #mojo
19:09 riche sri: its a classic distributed computing problem, many dissertations out there on the subject -- look for 'grid' and 'heartbeat'.
19:09 riche imo it's in your best interest to have a very basic handler and allow people to extend it according to how they see the algorithm should work (many times in their own case)
19:10 sri oh, i did read the raft and paxos papers too ;p
19:10 sri this is a really trivial problem though
19:11 riche not if your workers are goegraphically distributed, for example
19:11 sri how does that make things more complicated?
19:11 riche but i think you're not covering that .... yet
19:12 riche because your failure cases increase, false negatives increase
19:12 sri how so?
19:13 sri that's a very vague statement
19:14 sri you'll certainly see more connection problems and time difference problems, but those are not really much of a concern here
19:14 riche if u have gdd workers many things can go wrong ... things ranging from dns to packet drops
19:15 riche shrugs ... i guess it depends on how much you care about what those workers are doing
19:16 sri it doesn't really add a new category of problems though
19:16 ZadYree joined #mojo
19:16 sri network trouble is normal everywhere
19:17 riche fair enough
19:17 sri time zones can be fun, but minion handles all time operations inside postgres
19:19 sri it is true that minion is rather vulnerable to network trouble in general though, database operations are not retried, connections are tested before every operation, but a worker will die if a failure happens at a bad enough time
19:19 ZadYree Hey! I'd like to know how to parse parameters that are passed in JSON form. Can't find it in guides even if I'm quite sure it is there :S
19:19 riche yah I guess its okay for non critical work
19:22 berov ZadYree: http://mojolicio.us/perldoc/Mojolicious/Guides/Cookbook#USER-AGENT
19:23 berov $ua->get($url)->res->json->{this}{that}
19:23 ZadYree berov, so I assume the Controller and the UserAgent have the same methods?
19:24 ZadYree Because I am talking about a validated form that sends a JSON object
19:25 berov just decode_json
19:25 berov http://mojolicio.us/perldoc/Mojo/JSON
19:26 berov or http://mojolicio.us/perldoc/Mojo/JSON#from_json
19:27 ZadYree Hum I may seem idiotic but what shall I pass it? Something like  values $self->req->body_params ?
19:27 dod joined #mojo
19:27 jberger ZadYree: $c->req->json ?
19:27 berov something which is json encoded :)
19:28 ZadYree jberger, wow, does it exist? <3
19:28 jberger does the form send JSON or form-encoded pairs where the values are json?
19:28 jberger ZadYree: http://mojolicio.us/perldoc/Mojo/Message#json
19:29 ZadYree jberger, it does. Thanks, and thanks berov too :)
19:56 mkrull joined #mojo
19:57 sri riche: "critical" is again a very vague term
19:57 punter joined #mojo
19:58 sri job results are stored for as long as you like... which may be success or failure
19:58 sri so yes, for certain kinds of "critical" tasks that is a perfectly fine choice
19:59 sri if you're dealing with "time critical" tasks, things are different
19:59 meshl joined #mojo
20:00 riche why is it vague?  because every organization has its own definition of it - based on responsibility to customers, regulation, and so forth.
20:00 sri yes
20:00 riche but I am sure that you want to publish some level of certainty
20:00 sri we make the guarantees i just mentioned
20:02 sri you made a very broad assumption earlier, "yah I guess its okay for non critical work", which i can't agree with
20:02 riche I don't want to agree with it.
20:02 riche in fact, the more that's said, the worse position I will be in to get projects implemented using it
20:04 sri jobs will fail, and workers will fail, the important part to me is that nothing gets lost when it happens
20:04 riche but in these large companies .. they have "enterprise architects" that will pick the shit out of stuff that *looks like* it doesn't have high reliability, even though it does
20:04 sri the system will eventually recover and report the failures
20:04 ZadYree joined #mojo
20:05 riche being in that position kills my pitch
20:06 riche shoot I am being "summoned" ... gotta go ... evening at the Met ;) ... which you guys will have the opportunity to do in a few months
20:06 mst what sri's describing is a model based on eventual consistency without transaction loss, which is basically the only set of guarantees you can provide at real scale
20:06 mst if your enterprise architect understands that, you should be fine
20:06 * sri nods
20:06 mst if you don't, they don't know what they're doing anyway and you can probably bury them under buzzwords until they submit
20:07 riche mst: that is fine, but the convergence needs to be spelled out
20:07 mst riche: then perhaps rather than making unfounded negative comments you'd be better spending your time writing a doc patch that does so
20:11 sri i would actually write the doc patch myself if i understood what it is you want
20:12 sri do you want a list of the architectual patterns used? :)
20:13 riche never mind I just talked it over with someone and it can be mitigated by running it through event root cause analysis and actioning self healing after that point
20:15 preaction lots of $10 words in that sentence
20:15 riche sri: it's just better understanding the lifecycle.  enterprise archs talk reliability, maint architecture, execution architecture, etc.  they don't care about development patterns
20:16 preaction sounds like "look at the log and re-run the job"
20:16 riche lol.... maybe that's the world you live in
20:16 riche root cause analysis -- yeah, i worked for a company who did that over a decade ago
20:17 riche self healing is just triggering actions based on event patterns
20:17 preaction i collect and store intraday fx market data for a major world bank :p
20:17 riche wow ... amazing.. because people have been doing that ever since cross border trading was on the rage in 1999
20:18 riche anyways going now
20:19 sri anyway, i'm wondering if fatal worker errors should be reported to stderr or the app log
20:20 sri (app log would mean that pretty much everything runs in a big eval block)
20:21 jnbek joined #mojo
20:21 * sri pokes jberger
20:22 jnbek joined #mojo
20:23 * jberger stirs
20:24 jberger I don't mind the idea of trying to keep the worker alive
20:24 jberger its not a server, but its easier to tell that a worker is alive or not if it is
20:24 sri haha, i was not planning to keep it alive ;p
20:24 jberger sorry, I'm mentally switching architecture patterns
20:25 sri just log the error and unregister the worker outside the eval
20:25 jberger I've been working on this: https://github.com/jberger/Test-Mojo-Role/blob/master/t/basic.t
20:25 jberger sri: ok, that works too, cleanly exitting
20:26 sri basically run this loop in an eval https://github.com/kraih/minion/blob/master/lib/Minion/Command/minion/worker.pm#L26
20:26 jberger then again, why throw the worker away, if the manager is going to just start a new one?
20:27 sri the new one will clean up after the old one
20:27 sri repair inconsistent state
20:27 jberger ah, I see
20:27 sri oooh
20:27 sri the unregister outside the eval makes that a lot faster
20:28 sri since we then know the old worker is gone for good
20:28 sri instead of waiting for the heartbeat timeout
20:28 * sri nods
20:28 sri that is good
20:30 jberger all this to me is like listening to french, I can pick out important words and make some sense of things, but I'm pretty sure I couldn't make conversation
20:31 preaction "mitigated" -> fixed, avoided. "event root cause analysis" -> "how did this happen?", "actioning" -> doing, "self-healing" -> re-run, because it knows how to heal itself
20:31 jberger (anymore that is, in the case of french)
20:31 jberger preaction: I don't try to understand enterprise speak, I'm talking about the patterns that sri is using :-)
20:31 jberger I let managers speak enterprise
20:32 preaction i swear to yhwh if i hear the word "leveraged" ever again it will be too soon... god i hate that word...
20:33 sri good thing about using pattern names is that you can just google them :)
20:33 sri although, i don't think i've used any today
20:34 jberger I have come to believe this about the software term "enterprise"
20:34 sri i may just be talking gibberish and using you for rubberduck debugging
20:35 * jberger can be a rubber duck, happy to in fact
20:35 sivoais joined #mojo
20:35 jberger enterprise means a piece of software that a company pays a lot of money for, from a company that pays a lot of money to another company to get some gold seal issued by people who have no idea about software
20:36 jberger what they get is usually bad software with a big price tag and a gold seal
20:36 mst purl: jberger is also Foreman
20:36 purl okay, mst.
20:36 preaction what they get is someone to blame when the inevitable problems happen
20:36 preaction because they can't be fixed, blame is the only available option
20:36 jberger and what good does that do them? they would be better off with good software
20:36 sri enterprise speak makes my head spin, the terms are always so damn ambiguous
20:37 mst that's not how the incentive structures play out
20:37 preaction "it wasn't our fault" is extremely valuable
20:37 cpan_mojo Mojolicious-Plugin-SemanticUI 0.13 by Krasimir Berov - http://metacpan.org/release/BEROV/Mojolicious-Plugin-SemanticUI-0.13
20:37 mst right, what preaction said
20:37 jberger RedHat doesn't absorb blame, neither does Oracle
20:37 mst yes... it does.
20:37 preaction just a few weeks ago, RHEL pushed a bad patch that broke tar
20:37 jberger the people who deployed bad software and lost a ton of money still get fired, stock holders and clients still lose money
20:38 mst it's really sweet that you think the first part is true
20:38 jberger if its not, then I need to add that to my list of concerns about enterprise
20:38 jberger it sounds like a thing that ought to be avoided like the plague
20:39 preaction they pushed the fix like right after, but 2 months later our admins picked up the bad patch and not the good one
20:39 preaction jberger: you worked on an enterprise platform already!
20:39 jberger and I left
20:39 jberger as quickly as I could
20:40 jberger and if you think that platform gets anyone's gold seal, you have another thing coming
20:40 jberger (I know you dont)
20:40 preaction but also, that platform's "vendor" doesn't accept blame :p
20:40 mst sure it will, they just need to give enough gold to whoever's issuing the seals
20:40 jnbek joined #mojo
20:40 jberger :-)
20:40 jberger they sure have put in a lot of gold
20:40 preaction it gets everyone above my manager's seal of approval, my manager listens to me and gets confused as to why his manager likes it, and i do not :p
20:41 jberger https://www.youtube.com/watch?v=lTOP_shhVBQ
20:41 preaction but he's newly my manager, so we're working on the trusting each other thing
20:41 preaction holy fuck what?
20:41 purl That's crazy!
20:41 preaction jberger: what?!
20:41 preaction WHAT?
20:42 preaction "Kirat Singh is the visionary behind Bank of America's Quartz platform" <- didn't he jump ship?
20:42 jberger you haven't seen that ;-)
20:42 jberger so did the guy who instituted it now too
20:42 sri http://images.dailydawdle.com/seal-of-approval.jpg
20:42 jberger no one wants to be there when people notice the giant holes drilled in the bottom of the ship
20:42 preaction i heard Chris Dean just "retired" too. things are getting interesting :p
20:43 jberger that's who I meant
20:43 jberger oh right there was that other guy too
20:43 jberger so three of them
20:43 preaction "Quartz alone comprises over ten million lines of Python code" <- this is not a _good_ thing...
20:46 preaction i just want to know how i get to have the label "visionary" applied to me
20:46 sri put it in your twitter profile?
20:47 preaction can one self-apply that without sounding like a tool?
20:47 sri lets find out!
20:51 jberger speaking of tools ;-)
20:51 jberger any thoughts? https://github.com/jberger/Test-Mojo-WithRoles
20:51 Adura Secondary definitions do make it kind of funny a term to use.
20:52 jberger I want a clean abstractions over adding behavior to Test::Mojo
20:52 jberger especially since I want to release one soon
20:52 Adura No one answered me on when codenames change, I'll assume it's tradition to wait till a .01 to change the codename in mojo version.
20:52 jberger and I'm getting tired of Test::Mojo::More::Most::AndMyThingToo
20:53 jberger Adura?
20:53 purl Adura is rewriting something that is in PHP, would like to have most of the code behave the same.
20:53 preaction Adura: the code name is clinking beer mugs, see /topic
20:53 sri looks quite reasonable
20:53 Adura Mojolicious (6.0, Tiger Face)
20:54 sri lol
20:54 sri nobody noticed that
20:54 * jberger notes to ack for $old_codename before releasing $new_codename
20:55 irq joined #mojo
20:55 d4rkie joined #mojo
20:56 good_news_everyon joined #mojo
20:56 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/xZQA
20:56 good_news_everyon mojo/master d8323bf Sebastian Riedel: fix code name in Mojolicious
20:56 good_news_everyon left #mojo
20:57 sri jberger: on second thought, i like the roles approach
20:59 sri and while we are at reviews ;p https://github.com/kraih/mojo/commit/4cd34c5a18536efe0a9348d377c5aadfb6a5f523
20:59 sri the begin/end description
21:02 jberger looks useful
21:02 jberger it would be nice if we could come up with some doc that would say "it has to be a single statement" too, even though "expression" should mean that, even I didn't think of it
21:05 jberger re roles: its been on my mind a lot lately
21:05 jberger for an object like Test::Mojo where chaining is so heavily used, this is going to be a problem
21:05 * sri just noticed that we don't use the fatal log level anywhere in mojolicious
21:06 * jberger hears a chainsaw running somewhere, suspects mst
21:06 sri fatal actually has a big advantage, it can't be disabled
21:10 dotandimet joined #mojo
21:12 jnbek joined #mojo
21:19 jberger is cpan author COOLMEN here?
21:20 jberger avkhozov_ you too, I'm curious about your test modules, with regard to Test::Mojo::WithRoles
21:21 dparry joined #mojo
21:23 dparry If I have $self->helper( conf => sub { state $conf = Sharpener::Config->new() } ); in my startup routine (running with daemon), what is the expected lifetime / scope of that object?
21:23 preaction from the first time you call the helper, until the end of the program
21:24 dparry where end of the program is termination of the process?
21:24 dparry and the scope is across all requests?
21:24 dparry so if request A updates the object, request B will see the persisted change?
21:24 mst well, yes. startup happens on startup, and the app object will go out of scope at the end
21:25 mst uh. only if they're in the same process
21:25 dparry interesting, is this different with morbo per chance?
21:25 bpmedley dparry: Do you have multiple processes answering requests?
21:25 mst which isn't going to be true in a daemon
21:25 mst morbo is a single process
21:25 preaction morbo restarts the mojo app regularly though (whenever it detects a change)
21:25 dparry interesting, I kinda thought when I first looked at this it seemed to be limited
21:25 mst but, yeah, it's global, but it's a per process global
21:25 dparry might have been morbo restarting
21:25 mst so don't mutate it
21:26 mst if you need persistent state, do actual persistent state :)
21:26 bpmedley dparry: Try starting morbo with -v
21:26 dparry I'm more thinking an in memory object cache that can be used across all requests
21:27 dparry to save fetching serealised objects from disk so often
21:27 bpmedley dparry: Something like memcached?
21:28 preaction or redis maybe?
21:28 dparry was thinking just a dict of id => object
21:28 marmez joined #mojo
21:30 dparry I thought this was not worthwhile due to the object being reload per request or some set of requests
21:30 bpmedley What would you use for the shared storage?
21:30 dparry but if it persists throughout then this is perfect
21:31 dparry just a reference to a perl hash
21:31 bpmedley Are you tring to share this across processes?
21:31 dparry don't believe so
21:31 dparry the app will run in just one process
21:31 bpmedley I see, sorry for my confusion.
21:32 dparry well, I have a couple of forks, but I get that it won't translate to there
21:32 dparry and that's fine
21:32 dparry thanks for clarifying this - appreciate your help
21:34 jberger dparry: see CHI
21:36 Grinnz yeah if you need something that can be shared between requests you need something external, redis is fine for that sort of thing
21:37 dparry thanks jberger
21:38 btyler sri: I've been messing around with trying to fix M::UserAgent::UnixSocket for 6.0 by creating a new M::IOLoop::Stream and passing its ID to the UA's $tx, but I'm discovering that I'd need to start hitting private methods in order to actually get data going in both directions, which is makes me think this approach is a dead-end. is there another approach I should chase?
21:38 dparry from what I could see Grinnz an update in one request could be seen in a different request
21:38 Grinnz dparry, yes thats the idea
21:38 Grinnz its a database essentially
21:38 dparry cool cool
21:38 btyler *which makes me think
21:39 Grinnz dparry, and redis keeps all data in memory (i think memcached is similar, never used it) so theres no disk I/O issue
21:39 btyler for context: I create the M::IOLoop::Stream with a unix socket, get an id for it from $ua->ioloop->stream, then set the $tx->connection to that id
21:41 dparry oh, wait I was talking about the helper object being find for sharing between requests
21:41 dparry fine
21:41 Grinnz dparry, that will only be seen by requests handled by the same process
21:41 Grinnz but if that's ok, then sure
21:41 dparry ah, yep fine cool
21:42 dparry redis is overkill for my use case
21:42 dparry but hopefully I can get everything in no more than 3 processes and then plan accordingly
21:43 dparry with each process doing its own thang
21:43 Grinnz if you use a prefork server like hypnotoad it's however many workers you set, but they will regularly be restarted
21:43 jberger again, CHI::Driver::FastMMap (I think that's it) should do that for all processes on the same host
21:43 preaction yes
21:43 Grinnz prefork processes don't do "their own thing", they are each a copy of the application
21:43 preaction and if you use CHI, you can scale to network caching very easily
21:44 dparry I'll stick with daemon prolly
21:44 Grinnz then you'll just have one process
21:45 dparry I fork twice then spawn the app, but yeah
21:45 jberger dparry: daemon doesn't prefork
21:46 Grinnz the other processes aren't involved in the application though?
21:47 dparry 1 process runs the app, one runs a mojo IOLoop server and the other 1 - n make lots of requests
21:47 dparry it's possible the latter could go in the same process as the app dependent on how performant the user agent turns out to be
21:48 dparry the IOLoop I've found I have to keep separate as it receives some blocking IO
21:48 Grinnz if you're using Mojo::UserAgent make them non-blocking requests and you should be fine
21:48 dparry yeah, I may roll the latter in if it looks like that's ok
21:48 dparry but I wanted to potentially fork n times for that to leverage more cpu
21:49 dparry but without testing not sure how that will play out
21:49 jberger dparry: sounds like an unorthodox architecture
21:50 jberger is the app a webapp?
21:50 jberger a mojo app?
21:50 purl it has been said that a mojo app is on a linux server. I want a process on another machine to issue a get/post to it to signal the process has completed, I then want that info to use SSE to tell the browser.
21:50 dparry mojo app
21:50 purl it has been said that mojo app is on a linux server. I want a process on another machine to issue a get/post to it to signal the process has completed, I then want that info to use SSE to tell the browser.
21:50 Grinnz lol
21:50 jberger purl forget a mojo app
21:50 purl jberger: I forgot mojo app
21:51 mst no, purl, mojo app is <reply>
21:51 jberger then yes, that architecture is strange
21:52 jberger or at least unorthodox, I don't say I know enough to say "strange"
21:52 sri btyler: not that i'm aware of
21:52 dparry like I say, it has to receive some IO that is blocking and I couldn't get that in the same process as the mojo app without it stalling the app so I keep that in a separate fork
21:53 dparry the other forks I created early on when I thought that was a good idea for leveraging cpu
21:53 btyler sri: ok, bummer. at least I learned more about UA internals chasing that down
21:53 dparry I get that the non-blocking route might wellbe better / good enough etc
21:54 sri btyler: there i something that almost works, $tx->connection(Mojo::IOLoop->stream(Mojo::IOLoop::Stream->new($socket))), but you can't get the event handlers attached https://github.com/kraih/mojo/blob/master/lib/Mojo/UserAgent.pm#L121
21:55 btyler sri: right, that's where I'm at
21:55 sri s/i/is/
21:55 btyler but indeed, I'd have to hit a bunch of private methods in UA (_read and friends)
21:56 * dparry heads off to code^H^H^H^Hsleep, thanks for your help!
21:57 btyler time for slightly insane plan B, proxying the unix socket via a TCP server for the UA to talk to
21:57 sri btyler: realistically, i think there are only two options for getting that ability back, a) we decide the old way was good and just bring it back, or b) connection management in Mojo::UserAgent gets a more generic abstraction layer... Mojo::UserAgent::ConnectionPool or whatever
21:58 sri keep in mind, i'm always in favor of abstracting stuff out of Mojo::UserAgent :)
21:58 sri not sure you'd want to put that kinda work into it though
21:59 btyler out of curiosity, what displeased you about the old way?
22:00 sri for a long time i even forgot it existed, but then preaction stumbled over a bug recently
22:01 sri for some reason the socket object persisted in ->connection and caused problems when it was used instead of a string id
22:01 sri Mojo::IOLoop->stream() accepts stream objects *or* string ids, and calls ref() to differentiate
22:02 btyler ah, I see
22:02 btyler and the socket was obviously a ref
22:02 sri the fact that this could happen bugged me
22:02 btyler right
22:02 sri right
22:02 btyler so, more forward-looking question: what kind of superpowers would you want UA to gain from a M::UA::ConnectionPool (or whatever)?
22:02 sri the problem was also impossible to replicate, since it happened in a locked down container if i remember correctly
22:03 sri which got me even more annoyed, and i started questioning how much trouble the feature is worth... you know how it goes :)
22:03 btyler heh, yep
22:04 sri in the end, storing sockets and connection id strings in the same attribute is not such a great idea
22:04 sri especially considering the assignment of the socket/id also triggers an event
22:04 btyler it did strike me as a bit odd when I was reviewing things closely, I totally understand wanting to disambiguate that
22:05 sri btyler: i'd expect a M::UA::ConnectionPool to make M::UA smaller, easier to manage
22:06 sri no big expectations besides that really
22:08 sri although, i'm not sure how exactly the api should look like
22:09 btyler I was just doing some poking around google to see if I could find some APIs to look at
22:09 sri that sounds like a good idea
22:09 sri perhaps the firefox or chrome repo
22:09 sri or netty
22:12 sri i guess you could say what bugged me most was the way we passed sockets to the user agent, not so much what we did with the sockets afterwards
22:12 sri that part is pretty sane, and we still do it for tls upgrades for example
22:12 sri or connect requests for proxies
22:13 sri in any case, having another person who knows the user agent internals is a big plus :)
22:14 * btyler flees from responsibility
22:14 * sri chains btyler to the channel
22:16 mst btyler++ # well volunteered
22:16 btyler karma for failing to complete my Jan. pull request challenge, I suppose
22:16 purl failing to complete my jan. pull request challenge, i suppose has neutral karma
22:18 good_news_everyon joined #mojo
22:18 good_news_everyon [mojo] kraih pushed 2 new commits to master: http://git.io/xnZm
22:18 good_news_everyon mojo/master 5d69a9b Stefan Adams: Fix spelling
22:18 good_news_everyon mojo/master 97923aa Sebastian Riedel: Merge pull request #750 from s1037989/patch-1...
22:18 good_news_everyon left #mojo
22:46 csson joined #mojo
22:53 sri a first attempt at explaining minion fault tolerance https://github.com/kraih/minion/commit/3b1bf06e9cef178eea26c3ddfb161406e0e55186
22:53 d4rkie joined #mojo
22:58 jberger "eventually"
22:58 purl i guess "eventually" is fine by me
23:06 sri ok, next try :)
23:06 sri https://github.com/kraih/minion/compare/v1.08...master#diff-b30d9d6fe78ad2d38daed2446ba66d92R159
23:10 tempire Oh that's interesting.
23:13 tempire Searching for mojolicious made the 6.0 release graphic show up in "In the News" under the googles
23:13 sri interesting
23:15 ZadYree joined #mojo
23:38 denny joined #mojo
23:47 jberger yapc::na talks submitted!
23:57 sri hmm, it looks like another subquery fixes the race condition i encountered yesterday

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary