The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2015-10-01

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:08 mattastrophe joined #mojo
00:10 genio Is there a doc somewhere that explains to an idiot when/why you might need to store an IOLoop->singleton vs just IOLoop->delay()ing everywhere?
00:11 bpmedley_ genio: May I ask what your implementing?
00:11 genio bpmedley_: This is more just an understanding line of questioning.  No real project behind the scenes
00:12 bpmedley_ I'd have to see example code.  I'm still not an expert in IOLoop, though.
00:13 meshl joined #mojo
00:14 genio say I create a NB method that does    sub method { my ($self,$cb) = @_; Mojo::IOLoop->delay( sub { die...},sub{never_gets_here()} )->catch(sub {return $self->$cb(pop)})->wait; return $self; }
00:14 bpmedley_ I'd need to see a complete Lite app in a pastie, sorry.
00:14 Grinnz genio, theres no reason to access IOLoop->singleton unless you actually need a loop object somewhere
00:15 genio I wonder (I saw some example that I can't even remember where) about things like $delay->stop; in the ->catch routine
00:15 Grinnz or if you need to access the singleton's reactor
00:15 Grinnz $delay does not have a stop method
00:15 Grinnz you mean $delay->ioloop->stop?
00:15 genio yes
00:16 genio sorry, a mistake on my typing
00:16 Grinnz Mojo::IOLoop creates a delay and passes the singleton as its ioloop automatically
00:16 Grinnz with Mojo::IOLoop->delay that is
00:17 genio So, should that loop be stopped on error like that, or no need? (wrapping my mind around IOLoop and Reactor isn't going very well)
00:17 Grinnz no, the loop should almost never be stopped on error
00:17 Grinnz unless it's an application that is using a loop only periodically
00:18 Grinnz i.e. for quasi-blocking calls
00:19 bpmedley_ genio: May I ask what you mean by "that" loop?  Are you saying that when delay is called a new event loop instance is created?
00:21 genio bpmedley_: err.  That's how I interpreted Grinnz's answer at first.  Rereading now and going back to docs
00:22 bpmedley_ genio: My understanding is that there is one IOLoop for the process unless a blocking operation happens.
00:23 meshl joined #mojo
00:23 Grinnz you can create multiple IOLoops, but the vast majority of the time, you want to use the singleton
00:24 bpmedley_ Grinnz: If a new IOLoop is created and then started, will the singleton be blocked?
00:25 Grinnz if you start it inside the running main loop, yes
00:25 Grinnz that's how the blocking Mojo::UA->get works
00:26 Grinnz essentially, any ->start on a reactor or IOLoop blocks on that particular statement until the reactor stops
00:26 bpmedley_ May I ask how an IOLoop can be started outside the main loop?
00:26 Grinnz and inside, whatever the reactor has been given to do, it will do
00:26 Grinnz start it before or after the main loop is started
00:27 Grinnz like before the app->start call in a Mojolicious::Lite app
00:27 Grinnz (or after)
00:27 bpmedley_ If started before or after, then technically, how many are running at once?
00:27 Grinnz it would be the only one running at that time
00:28 Grinnz it's just like starting the main loop, but it's a different loop which has been told to do different stuff
00:28 bpmedley_ What you say makes sense.  My statement could have been better, I think.
00:29 Grinnz thats also why you can't use different event loops together unless they're using the same singleton backend
00:30 genio That makes a lot of sense to me actually.  I think I have been overcomplicating something pretty simple
00:31 bpmedley_ genio: When I first started I was trying to shoehorn what I understood about the forking process model into an event loop model.  I made experience much more complicated than need be.
00:31 Grinnz yeah, forking is very different, forking is parallel while event loops are sequential
00:32 meshl joined #mojo
00:33 Grinnz you can wait for lots of things at once in an event loop, but you can never do two things at once
00:35 genio I'm glad I asked now.  The picture is much clearer to me
00:35 genio bpmedley_++ Grinnz++
00:36 Grinnz i'm still thinking about that IOLoop guide ive been wanting to make ;)
00:37 genio heh.  Anytime you need someone to spark ideas, just ask me to ask a stupid question and you'll see what people don't understand right away
00:38 aborazmeh joined #mojo
00:41 meshl joined #mojo
00:42 aborazmeh joined #mojo
00:45 Grinnz btw. i first learned about how event loops work by reading the Mojo::Reactor::Poll source code :P
00:51 disputin joined #mojo
00:53 meshl joined #mojo
00:54 mattastrophe joined #mojo
01:02 meshl joined #mojo
01:12 zivester joined #mojo
01:12 meshl joined #mojo
01:14 Zoffix PopeFelix, no, it wasn't something you said. I just have OCD with checking (and responding) to all IRC channels with messages and I was trying to stay focused on my Perl 6 module while staying on IRC and talking in a Perl 6 channel :)
01:15 Zoffix It's interesting to see Mojo mentioned in two places on P6 Most Wanted list :) https://github.com/perl6/perl6-most-wanted/blob/master/most-wanted/modules.md
01:15 * Zoffix would definitely add Mojo::DOM equivalent to that list
01:24 asarch joined #mojo
01:51 absolut_todd joined #mojo
01:53 disputin joined #mojo
02:05 davido_ joined #mojo
02:14 genio Now, on to understanding ->next_tick()
02:18 cpan_mojo Mojolicious-Plugin-ClientIP-0.01 by SIXAPART https://metacpan.org/release/SIXAPART/Mojolicious-Plugin-ClientIP-0.01
02:35 noganex joined #mojo
02:38 disputin joined #mojo
02:41 bpmedley_ genio: I believe next_tick is useful for when you wanted to invoke some code that will only run once immediately after the currently executing routine without using a timer.
02:43 sri oh, new os x release, lets break my macbook \o/
02:45 sri next_tick has a very simple use case, imagine you have a non-blocking method like ->foo(sub {...}) and in some cases you know there's an error right away, even before starting whatever non-blocking operation you wanted to start
02:45 sri now, you could just do "return $self->$cb('something went wrong')"
02:46 sri but that would invoke the callback right away
02:46 sri which can cause terrible timing errors
02:46 sri so you do "return Mojo::IOLoop->next_tick(sub { $self->$cb('something went wrong') });"
02:48 sri to the function that called ->foo(sub {...}) everything will look normal again, and only once the event loop is in control again, the callback will be onvoked
02:48 sri s/o/i/
02:58 kaare joined #mojo
03:02 jberger The problem is perspective, if you do something immediately you aren't inside the loop yet, so you use next_tick to be sure that you are
03:02 jberger Grinnz: "btw. i first learned about how event loops work by reading the Mojo::Reactor::Poll source code :P"
03:02 jberger ^^ me too
03:03 sri i wonder if that way of learning got harder with recent changes in the module
03:05 sri now there's rounding of timeouts and calls to IO::POll::_poll
03:06 sri which are huge performance improvements, but look a bit cryptic
03:06 jberger Zoffix: kinda three, JSON::Tiny is just Mojo::JSON
03:07 jberger sri: I don't know if I've read it carefully since that change
03:08 bpmedley_ sri: I wonder if the increase in examples, documentation, and support will offset the higher complexity in the IOLoop code.  Said another way, people won't have to read the code as much when starting out.
03:09 jberger Oh all the splicing of @poll and things
03:09 jberger Hmmmm
03:09 jberger Yeah I see what you mean sri
03:09 jberger bpmedley_: that's not what we mean
03:09 jberger Learning to use it is still just fine
03:10 jberger It's learning how it works and from that how all event loop patterns work
03:10 bpmedley_ Understood
03:10 Grinnz jberger, i don't think perl 6's JSON::Tiny is related to perl 5's at all, hopefully it's not related to JSON.pm either though
03:13 jberger Most ioloops are either implemented in C (EV) or are large and hard to read (POE)
03:14 genio sri: ah.  thanks!
03:14 jberger I was able to learn because Mojo::Reactor::Poll was so easy to read
03:15 genio sri: Be prepared to wait (non-SSD drive has been updating for over an hour -all while claiming only 20-something minutes remaining)
03:17 jberger I'll probably wait until the weekend
03:21 davido_ joined #mojo
03:24 sri i only have ssd \o/
03:26 jberger I <3 my SSD not for the speed but the battery life
03:27 jberger Seriously, I couldn't care less about this Mac stuff, but the battery life when just doing vim and the occasional dev server is unbelievable
03:29 jberger sri: not to stir the pot too much, but why are io and watch distinct?
03:29 jberger It seems to me that they could be merged
03:31 jberger I guess it might make for some strange looking method calls though
03:31 jberger But since io calls watch, and watch dies unless io had already been called
03:34 CromeDome joined #mojo
03:40 genio still upgrading.
03:40 genio I should really buy an SSD for this thing
03:44 bpmedley_ genio: What mac do you have?
03:45 davido_ I saw some code today that was using Mojo::UserAgent and Test::Mojo. There was a method call tap_post. Is that still a thing? Didn't find it in the pod.
03:45 genio 2012 MBP
03:46 davido_ (I wasn't able to investigate the code, it was in a presentation)
03:47 genio are you sure that wasn't just something they were calling their test suite? or a method for testing post in their test suite?
03:47 genio tap = test anything protocol
03:48 davido_ it was a method call that was chained in with some other Test::Mojo / Mojo::UserAgent calls, but it's entirely possibe that they had subclassed something from the Mojolicious distribution.
03:48 davido_ I didn't get a chance to ask.
03:48 genio Mojo::Base has a tap method, but I haven't run into anything called tap_post
03:49 davido_ yeah, that's what got my attention. I thought, hmm... I haven't seen that before.
03:49 genio bpmedley_: I should get a new one sometime soon, but I don't want the USB C
03:49 davido_ I tried to catch a glimpse of his inheritance hierarchy but there just wasn't time.
03:50 bpmedley_ genio: Why?  You've got 2-3 years left, probably.
03:50 genio bpmedley_: heh.  that was my timeframe for sometime soon :)
03:50 Grinnz jberger, well io updates the callback, watch can change the read/write flags without changing the callback
03:51 melo joined #mojo
03:52 bpmedley_ davido_: googling and grepping result in basically no hits.. may not be a thing
03:52 davido_ yeah, i'm thinking you're right. I'll have to ask him about it tomorrow.
04:08 forke joined #mojo
04:13 disputin1 joined #mojo
04:20 sri and done upgrading
04:20 sri pretty painless this time
04:21 sri jberger: the low level api is all about performance
04:22 sri if you can find a more elegant api that's not slower, go for it
04:23 sri and if you want to benchmark, examples/microhttpd.pl is not terrible
04:23 disputin joined #mojo
04:24 disputin joined #mojo
04:25 sri el capitan is noticeably faster
04:27 sri guess i'll release minion 2 later, unless there are new proposals
04:31 Adurah joined #mojo
04:33 irqq joined #mojo
04:43 davido_ joined #mojo
04:46 good_news_everyon joined #mojo
04:46 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/vc4Sg
04:46 good_news_everyon mojo/master 78792b2 Sebastian Riedel: fix small redirect bug in Mojo::UserAgent::Transactor
04:46 good_news_everyon left #mojo
04:47 sri the case is a little bit silly, but meh
04:57 damaya joined #mojo
05:21 fvox joined #mojo
05:29 PryMar56 joined #mojo
05:35 inokenty-w joined #mojo
05:56 Vandal joined #mojo
06:13 lsm joined #mojo
06:29 stephen joined #mojo
06:33 Dandre left #mojo
07:06 eseyman joined #mojo
07:34 trone joined #mojo
07:39 arthas joined #mojo
08:00 cono left #mojo
08:04 vanHoesel joined #mojo
08:22 vanHoesel1 joined #mojo
08:30 Mattes joined #mojo
08:34 mattastrophe joined #mojo
08:46 gaunt joined #mojo
08:49 Lee joined #mojo
08:56 gaunt Hi, I'm looking for way to configure morbo to listen on a unix domain socket. Does morbo support this?
08:57 * nic doesn't think so, but can't say for sure
09:01 gaunt The Changes suggests, it does since version 0.999920: https://github.com/kraih/mojo/blob/78792b2ba8008c7f7e1f279b03a5f567f0579c5a/Changes#L3783
09:01 nic interesting
09:02 gaunt But I cannot find anything in the documentation.
09:02 * nic thinks so, but can't say for sure
09:02 irqq joined #mojo
09:05 sri it has been removed again around 1.x i believe
09:05 nic Changes just says added and then fixed
09:06 meshl joined #mojo
09:07 gaunt Ah, fount the commit: https://github.com/kraih/mojo/commit/ad8d32401a790e60dca46903ee01aafedd2b7dd8
09:07 sri makes sense, big refactoring
09:08 sri it was never very good
09:09 sri think someone else asked about it a few months ago
09:09 sri and i said we could bring it back if there was a very very good patch
09:13 gaunt unfortunately I do not consider myself to be competent enough for that
09:14 gaunt so back to a tcp socket
09:14 gaunt non the less - thanks!
09:14 sri it's actually rather complicated, since there's no prior art for a good user agent api
09:14 sri server side is not too bad
09:15 sri (yes, we would need both)
09:21 cpan_mojo CallBackery-0.3.0 by OETIKER https://metacpan.org/release/OETIKER/CallBackery-0.3.0
09:30 melo joined #mojo
09:32 gaunt_ joined #mojo
09:40 Martin90 joined #mojo
09:40 Lee_ joined #mojo
09:41 Martin90 when you say: use Mojo:Base unless you need  $class->meta, what you really mean by meta method ?
09:45 nic look at Moose for meta
09:46 arthas_ joined #mojo
09:49 Martin90 I wonder whether Mojo:Base will be enough as a only OO system for whole projects
09:49 Martin90 project*
09:50 Martin90 so that "metal" is generic method that enable to inspect object in detail, right ?
09:53 nic Given your starting point, I'd recommend using Mojo::Base while reading about Moo
09:53 Martin90 "meta"*
09:53 nic Once you have Mojo::Base projects under your belt and Moo in your head, then look at what additional is available from Moose
09:54 Martin90 yes but I would like to avid case where i have to rewritte whole code
09:54 Martin90 but it sounds reasonable ;)
09:54 cpan_mojo Mojolicious-Plugin-ReCAPTCHAv2-0.3 by HJANSEN https://metacpan.org/release/HJANSEN/Mojolicious-Plugin-ReCAPTCHAv2-0.3
09:55 nic If you don't know you want mode, having discussed the questions this long, then you don't need it
09:55 nic s/mode/moose/
09:55 Martin90 right
09:55 Martin90 ok thanks for advice ;]
09:56 nic Another way of finding your answer...
09:56 nic Read Role::Tiny
09:57 nic If you don't get why you'd want it, then definitely stick with Mojo::Base
09:57 Martin90 ok ;)
09:58 nic If you do want it, you can use it with Mojo::Base, but by then you're on your way to understanding why you might want Moo
10:02 sue joined #mojo
10:04 sri "I wonder whether Mojo:Base will be enough as a only OO system for whole projects" <- lol!
10:04 sri you know, it kinda works for the mojolicious project
10:07 sri it's much more important to know design patterns
10:09 sri choosing an oo system is easy (any will do!), but you have to learn how to use them properly
10:12 sri mojolicious uses a super minimalistic oo system, but there's dozens of design pattterns begin used, delegation, factories, decorators, singletons, observer, visitor... and so on
10:18 nic true, the use of those patterns has a far greater power on the value of the code
10:27 Martin90 and all those patterns are implemented with Mojo:Base ;)
10:32 sri guess i can release minion 2
10:33 Flying_Squirrel joined #mojo
10:35 nic woohoo!
10:36 nic "Soon available on DVD!"
10:36 nic oh
10:37 nic it's a shame that Mozilla have their Minion Backend project
10:38 ashimema joined #mojo
10:42 dvinciguerra joined #mojo
10:45 sri and just as i say that i'm considering support for multiple queues ;p
10:46 asarch joined #mojo
10:48 mattastrophe joined #mojo
10:51 nic I'm imagining what that might entail, and wondering if it means I wouldn't need task groups
10:51 sri yes, that what it means
10:51 sri +'s
10:51 nic Instead of dequeue according to group, simply dequeue from the appropriate queue
10:51 sri it would replace the -t option
10:52 sri all jobs are in a queue named "default" or so by default, and all workers fetch jobs from there
10:52 sri but you can add ->enqueue(foo => {queue => 'premium_workers'})
10:53 sri and do "./myapp.pl minion worker -q premium_workers"
10:53 sri easier to distribute jobs to different worker groups
10:53 nic thus making it orthogonal to priority
10:54 nic (ie you don't have to hijack priority for non-priority logic)
10:54 sri right
10:54 sri or have tasks with the same name to distribute them to different workers
10:54 sri like "foo" and "premium_foo"
10:55 nic yes, so also fulfilling a namespace role if that's what you need
10:56 melo joined #mojo
10:56 nic and I _think_ it satisfies what I need re throttling, cos I'd control how many workers each queue had
10:57 nic sri: I'd like to be able to disable a queue without upsetting the others
10:57 nic B is down for maintenance (no dequeues allowed) while A & C continue as normal
10:57 nic no enqueues either, obviously
10:58 nic Wow, that would take Minion to another level
11:04 Dandre joined #mojo
11:16 asm35 joined #mojo
11:19 rwp joined #mojo
11:20 asm35 left #mojo
11:32 Dandre left #mojo
11:37 jontaylor joined #mojo
11:50 asm35 joined #mojo
11:51 asm35 left #mojo
11:57 sri think i have a patch :)
12:04 berov joined #mojo
12:06 jberger sri: is pg 9.5 out?
12:07 sri nope
12:07 jberger Weren't you waiting on that for 2.0?
12:07 sri that will be 3.0
12:07 jberger isn't it in beta now?
12:08 sri yea, i've been using it for development for a few weeks now
12:08 sri you might have noticed the SKIP LOCKED benchmarks i've been running recently
12:08 lluad joined #mojo
12:10 jberger That's what I thought, I'm not asking blind
12:10 jberger So why not wait and do it all at once?
12:11 sri why wait?
12:13 berov joined #mojo
12:14 berov joined #mojo
12:15 jberger Punchier release blog post
12:15 jberger It doesn't matter
12:15 jberger I was just surprised to hear you say 2.0 before 9.5
12:16 ZoffixWork joined #mojo
12:16 ZoffixWork Lol. Have you seen this? Unicorn meat! http://www.amazon.com/ThinkGeek-Thinkgeek-Canned-Unicorn-Meat-circled/dp/B004CRYE2C/?tag=unicode-unicorn-20
12:17 sri and committed https://github.com/kraih/minion/commit/74bddca37e02743744145a4431a22040151a03d5
12:17 * sri pokes nic
12:17 nic that's ridiculous, it's no wonder they're going extinct
12:17 jberger ZoffixWork: that's old
12:17 * nic tries to 'fetch' commit, but is too excited to type properly
12:17 ZoffixWork Aw :(
12:17 jberger There's a funny story about it too
12:18 jberger The nation pork council sent them a cease and desist
12:18 nic It's a little too close to the bone in Europe after the mislabelled meats debacle
12:19 ZoffixWork heh
12:19 sri jberger: the little projects get no blog post
12:19 jberger http://www.thinkgeek.com/blog/2010/06/officially-our-bestever-cease.html
12:22 ZoffixWork Heh. the PORKBOARD promo is a hilarious touch :D
12:23 jberger :D
12:26 sri performance in my benchmark seems not to be affected by queues
12:26 hernan605 joined #mojo
12:29 mattastrophe joined #mojo
12:30 nic sri: I'm wondering if not specifying a queue should mean 'all/any queue' rather than ['default']
12:32 nic The scenario where a queue is later added; hmm, maybe workers should only interact with queues they explicitly know about
12:32 nic Yeah, I think what you have is more useful to me
12:33 nic cos I'm contemplating the possibility of using a queue to park jobs
12:33 nic So an alternative to having a status 'blocked' would be having a queue 'blocked'
12:34 nic (assuming we can move jobs between queues, which probably requires a job remembering which queue it came from)
12:34 sri interesting thought, you could ->finish; ->retry({queue => 'some_other_queue'}); them
12:34 nic oh yes
12:35 sri CandyAngel: *poke*
12:35 sri this might interest you
12:51 sri jberger: postgres is a "when it's done" project, so 9.5 might not even happen this year
12:52 jberger oh, I see
12:52 jberger I assumed they were on some rough timeline
12:53 pink_mist all my favourite projects seem to be "when it's done" projects ... except Mojolicious .. or maybe it's just that that's done so quickly I hardly notice :P
12:55 sri we are totally a "when it's done" project, never had planned release dates
12:55 pink_mist alright then, scratch the exception :P
12:56 sri perl has pretty strict release dates now
12:57 ajr_ joined #mojo
12:57 pink_mist good point; not quite sure I'd rate it among my favourite projects though ... it's not so much perl itself that I enjoy but rather all the stuff around it
12:58 CandyAngel Would the queues replace priority too?
12:58 sri nope
12:59 CandyAngel Well the queue idea sounds nice, especially if it can replace the freeze/thaw workaround
13:00 CandyAngel (also, I like the term "parked jobs" :P)
13:00 nic sri: That (v2.0) is awesome
13:01 ZoffixWork joined #mojo
13:01 CandyAngel So instead of having dequeue by task name, it would be dequeue from specific queues (which might be the task name, depending on how the user implements?)
13:02 sri it's already implemented https://github.com/kraih/minion/commit/74bddca37e02743744145a4431a22040151a03d5
13:03 sri everything works just like it does now, you can just put jobs in arbitrary queues
13:03 sri and have workers for individual queues
13:04 CandyAngel Niiiice
13:04 aborazmeh joined #mojo
13:07 sri my $id = $minion->enqueue(foo => {queue => 'garage'}); my $job = $minion->job($id); $job->finish; $job->retry({queue => 'actual_workers_here'});
13:07 nic sri: What are your thoughts on the ability to move a job, eg $job->requeue($queue) and $minion->requeue($source_queue, $target_queue) ?
13:07 sri that was the idea for a parked job
13:08 nic oh yeah, retry rather than requeue
13:08 sri i like to keep the strict state machine
13:09 sri ->retry makes sure there's a new retries number, which we now use as a version basically
13:09 Flying_Squirrel joined #mojo
13:09 nic yeah, was forgetting about that
13:09 mattastrophe joined #mojo
13:10 sri CandyAngel: the garage queue would have no workers, so they stay forever inactive
13:10 nic moving from 'garage' to 'default' is obviously safe
13:10 * sri nods
13:11 nic I'm just thinking through moving in the other direction, in a distributed setup
13:11 sri if you can do it then it should be safe
13:12 CandyAngel If "move to another queue" would be the only thing that you can modify, I'm fine with having it so you have to (fail/finish)-retry to move it
13:12 CandyAngel Rather than it be an exception
13:12 nic I have in mind an operation which is inherently tricky (and the reason I gave up on resque)
13:13 nic to disable a whole queue, I can block the thing(s) doing the adding and then move everything from 'default' to 'garage'
13:14 nic I guess I could then wait a period of time then check if 'default' is still empty
13:14 nic cos additions to it may be trickling in from other nodes due to lag
13:14 sri that's too weird for me ;p
13:14 gryphon joined #mojo
13:16 CandyAngel Could you restart the workers so they only process tasks queued after a set time (e.g. 15 minutes after they have been started)?
13:17 CandyAngel Then if they dequeue one they aren't meant to process, that fail-retry-move to the garage
13:17 CandyAngel they*
13:18 bpmedley_ CandyAngel: Why not just defer the start of the worker for 15 minutes?
13:21 CandyAngel It might only need to ignore some task types..
13:21 CandyAngel But true eough :P
13:22 sri you'd make your own worker command, but that's all totally possible
13:22 nic While our minds are together... is there a way to tell a worker "complete what you have, but don't dequeue any more" ?
13:23 Mattes joined #mojo
13:23 nic 'graceful stop' is what I'm trying to say
13:23 sri nic: SIGINT
13:23 sri worker will never kill a running job
13:23 zivester joined #mojo
13:24 sri http://mojolicio.us/perldoc/Minion/Command/minion/worker#SIGNALS
13:26 nic ah yeah, just found it ($self->{finished}), v nice
13:28 nic So I don't need complicated queue operations, just gracefully stop affected workers till maintenance is completed
13:33 nic Now dependencies... if those jobs are enqueued onto 'blocked' and something periodically checks each of them and ->retry() when satisfied
13:34 nic I'm expecting there'd be no performance hit
13:34 * sri is over dependencies, too problematic
13:35 sri it's mostly linear anyway, so you can just have your jobs enqueue the followup, or abstract out shared logic with helpers
13:36 genio sri: It took over 3 hours to upgrade to El Capitan for me.
13:38 sri ouch, i was done in like 40 minutes
13:39 sri just the time machine backup afterwards took forever
13:39 genio spinning disks--
13:39 sri never going back
13:40 genio Yea, I really need to find an affordable 512GB SSD
13:40 sri re minion features, two that still intrigue me are auto restarts and rate limits
13:41 sri (auto restarts = auto retry for jobs)
13:41 sri like 20 times in incremental intervals
13:42 sri rate limits are super hard to do, i don't even know where to start
13:42 sri celery handles rate limits per worker, that's not too bad
13:43 sri but if you wanted to do it across the cluster...
13:44 vervain joined #mojo
13:55 nic I think I might prefer per-worker; it leads to smoother degredation
13:56 nic I have 4 workers, and they hit their limit at different times, so number active might go 4, 3, 2, 1, 0, 1, 2, 3, 4
13:56 nic rather than 4, 0, 4
13:57 nic maybe add a random element like you did in Mojolicious
14:04 jberger sri: still time for a 2.0 feature request?
14:05 jberger I'd like a column on the jobs table for binary data associated with the jo
14:05 jberger job
14:06 jberger I have a job where we realized that we had to b64 encode some binary data to be stored in the job args because it was being corrupted round tripping to JSON
14:11 Onigiri genio: the 512GB EVO 850 keeps going on sale for 150$ USD every few weeks
14:11 genio That's not bad.  I'll keep an eye out.  Thanks
14:12 jberger sri: also is there any reason not to use JSONB for the args?
14:12 Onigiri jberger: Does JSONB work on 9.3?
14:12 jberger 9.4
14:13 Onigiri Then that's probably a good reason to not use it
14:13 jberger (would JSONB actually be able to store binary data directly?)
14:13 jberger Onigiri: going to use 9.5 soon enough anyway
14:13 Onigiri I don't think so. I think it just allows faster access etc.
14:13 jberger s/use/require/
14:14 Onigiri jberger: that's kind of sad.
14:14 Onigiri Going to have to jump from 9.3 to 9.5 here then.
14:14 jberger its been known since SKIP LOCKED was announced
14:14 jberger or just don't upgrade
14:14 jberger Carton++
14:14 Onigiri Well, I want to go to 9.5 anyway... just
14:19 vervain joined #mojo
14:28 jberger from an author: http://pgeoghegan.blogspot.com/2014/03/what-i-think-of-jsonb.html
14:28 jberger “ In contrast, jsonb is a datatype that will only ever output valid textual JSON, and will only ever accept valid textual JSON”
14:28 jberger “In any case it's important to note that the protocol or on-disk binary representation of jsonb is an implementation detail”
14:28 jberger so JSONB doesn't solve my binary values need
14:31 sue joined #mojo
14:34 reneeb joined #mojo
14:37 sri jberger: why use jsonb instead of json here?
14:38 sri we never touch the actual data
14:38 sri some binary blob field seems a bit random
14:39 jberger JSONB isn't useful given what I found
14:39 jberger my use case is this
14:39 jberger I have a job that does printing
14:40 jberger sometimes I get HTML to print, which I then run though wkhtmltopdf to get pdf, then print, all in memory in the worker
14:40 jberger but sometimes I get pdf (from a fax service for example)
14:40 kaare joined #mojo
14:40 jberger most of the pdfs printed just fine, but some would corrupt
14:41 * sri shrugs
14:41 sri not really a job queue concern
14:41 jberger we finally got hold of one that we could repeat and tested the heck out of it, and it really was the fact that it was stored as JSON
14:41 jberger no, and we solved it another way, as I said
14:41 jberger it was just a question
14:41 jberger my $job_data = $job->data;
14:41 cpan_mojo Mojo-Webqq-1.5.1 by SJDY https://metacpan.org/release/SJDY/Mojo-Webqq-1.5.1
14:42 sri and what if i want two binary blobs?
14:42 jberger ->enqueue('task', \@args, {data => $data})
14:42 jberger hmmm
14:42 sri it's bad ;p
14:43 sri just make your own pdf table
14:43 sri or use a cache
14:44 abra joined #mojo
14:46 Onigiri How about that -T? heh
14:47 abra joined #mojo
14:47 sri ?
14:47 Onigiri On minion
14:47 Onigiri More just being silly while drinking a coffee
14:47 sri you seem to have missed a lot
14:47 Onigiri Yeah, I'm a tad out of date
14:48 sri https://github.com/kraih/minion/commit/74bddca37e02743744145a4431a22040151a03d5
14:48 sri -t is gone
14:48 sri there's now named queues
14:49 bpmedley_ Onigiri: Could you use multiple queues to achieve what you want?
14:49 sri you attach workers to arbitrarily named queues, and then just distribute jobs however you like
14:49 Onigiri bpmedley_: frankly, I don't know. I'm not even sure what this multiple named queues stuff is.
14:49 jberger specifically a queue name "dont_start_a_worker_here"
14:49 PryMar56 joined #mojo
14:50 jberger which is then a bucket for parked jobs
14:50 Onigiri So the one node would pick off all the X jobs, start running # of them and moving the others into a holding pattern?
14:51 jberger sri: you could make it so that there is an unstartable queue, {queue => undef}
14:52 sri lets not
14:52 jberger I thought you'd say that, but just in case :-P
14:53 stephan48 i like the queues idea but i dislike that -t is gone. i think they could happily coexist
14:54 bpmedley_ I'm personally debating if we would need that much fine grained control at work.  Said another way, I'm not sure if we do or do not need a -t.
14:54 stephan48 can a job be in multiple queues and can a worker match to only jobs being in certain queues(AND instead of OR relationship)?
14:55 sri just one queue
14:55 stephan48 i had a usercase but it broke down while describing it.
14:56 stephan48 thanks for clarifying
14:56 sri i very much doubt folks need that much control
14:57 Martin90 joined #mojo
14:57 sri the most common use cases are separating paying/non-paying users, and tasks with resource limits, both are covered nicely by queues
14:57 stephan48 priority controls job run order?
14:57 Onigiri Thinking about it, yes, this queue setup very much solves my need to easily maintain it when additional tasks are added.
14:58 sri yes, priority controls run order
14:59 bpmedley_ That makes sense.  Without the -t it's much simpler.  Kudos for queues, and thank you.
15:00 stephan48 my irrelevant objection wanished
15:00 sri it's basically the resque model
15:01 CromeDome joined #mojo
15:01 disputin joined #mojo
15:05 vervain joined #mojo
15:08 sri jberger: i've added a note to the docs though http://mojolicio.us/perldoc/Minion#enqueue
15:10 hernan606 joined #mojo
15:11 MartinR joined #mojo
15:14 neilhwatson joined #mojo
15:14 cfedde I'm getting an undefined value error from Mojolicious/Controller.pm. https://gitlab.com/snippets/9092 Care to point out where I went off the rail?
15:15 sri http://mojolicio.us/perldoc/Mojolicious/Controller#param
15:16 cfedde apparently there was a time when it returned a list of keys?
15:16 vervain joined #mojo
15:18 sri before the security issue
15:19 cfedde ah. that's probably when I generated this legacy issue.  Is there a way to list all the parameters passed?  I don't see an obvious way.
15:21 bpmedley_ cfedde: Perhaps https://github.com/kraih/mojo/wiki/Upgrading (Parameter name listing)
15:22 cfedde awesome!  thanks!
15:24 jberger sri: yeah, that's good
15:35 bd using mojo::ua to get a tinyletter.com archive, parse it with m::dom, look for <li>s, extract links, get those with m::ua, convert that to rss entries, write atom feed file, let that get picked up by rss2email, read stuff offline. profit.
15:58 vervain joined #mojo
16:05 PopeFelix I'm getting this error: 'Mojo::Reactor::EV: I/O watcher failed: Can't call method "build_tx" on an undefined value' in relation to this code: https://github.com/PopeFelix/Mojo-UserAgent-Mockable/blob/master/lib/Mojo/UserAgent/Mockable.pm#L224
16:06 PopeFelix What's happening here is that the mock UA has received a request that it doesn't recognize. It's supposed to respond with a null value (when configured to do so) in that case.
16:08 Mattes joined #mojo
16:11 batman PopeFelix: the problem is probably that your object goes out of scope
16:11 PopeFelix batman, which object, the app?
16:11 batman the ua, i would guess
16:12 batman i'm not sure
16:14 PopeFelix I don't see how the UA would go out of scope. The object _is_ the UA.
16:17 sh4 joined #mojo
16:19 jberger both the app and the ua have build_tx methods
16:19 jberger and you are messing with both types of object
16:19 jberger I have no idea what is being lost there
16:19 jberger something weakened is going out of scope
16:19 jberger but which one, who can say
16:20 * jberger said all that in the voice of Ser Davos Seaworth and prefers it be read that way too
16:20 jberger :-P
16:21 PopeFelix Who is Ser Davos Seaworthy?
16:21 jberger A character in A Song of Ice and Fire (also known as Game of Thrones)
16:21 jberger I might be reading too much
16:21 PopeFelix jberger, no such thing. :)
16:21 jberger (rereading for the third time, in anticipation of the new book)
16:21 * PopeFelix never got round to that series.
16:21 jberger so good
16:23 jberger if you generally like LOTR but find its inconsistencies and deux ex machinas too hard to stomach
16:23 jberger this is your series
16:23 jberger deus
16:24 PopeFelix I'm attempting to read one of the Amber prequels right now.  It hasn't gripped me tremendously.
16:24 jberger bigger world, more characters, at least as much history
16:24 jberger and more "realism" or as much realism as you can expect in a fantasy series
16:25 lluad If you like historical fiction, but wish it had more giant walls made of ice...
16:27 PopeFelix The weird thing is that same logic works when the request matches.
16:27 PopeFelix That is, in MUA::Mockable
16:27 PopeFelix The only difference is that I'm adding headers in one case.
16:29 jberger lluad++
16:32 reneeb left #mojo
16:33 jberger PopeFelix: have you tried with Carp::Always
16:33 jberger I'd say you need a stack trace
16:33 PopeFelix jberger, Probably a good idea.
16:34 PopeFelix I'm going to try reworking the logic a bit first, though - I see no good reason to set $self->server->app every time I start a new TX.
16:38 jberger yeah, that sounds wrong
16:38 jberger not that I've read your code to see what its doing, but it does sound strange
16:39 PopeFelix OK, so moving the app out of $ua->on(start) fixed that problem.
16:39 jberger cool
16:40 PopeFelix Testing against a local app, however, failed miserably.  Probably because playback is dependent on an internal app.
16:42 PopeFelix I do wonder if I'm going about things right by storing Transactions instead of individual Request/Response pairs.
16:43 PopeFelix Although in practice, I'm just taking the Response instance out of the stored TX and putting it into the response.
16:44 PopeFelix How does Mojo do it under the hood?  Create a new Response instance and parse the server response into it?
16:45 mattastrophe joined #mojo
16:50 jberger PopeFelix: https://github.com/kraih/mojo/blob/master/lib/Mojo/Transaction.pm#L10-L11
16:51 PopeFelix jberger, that would seem to indicate parsing the server response in.
16:54 jberger all I was trying to say is that a transaction is basically just a pair of messages
16:54 PopeFelix Oh, sure.
16:54 PopeFelix So it doesn't really matter if I'm storing a transaction or a pair of messages.
16:54 jberger perhaps better proof of that is https://github.com/kraih/mojo/blob/master/lib/Mojo.pm#L22
16:55 jberger and Mojolicious.pm's isn't too different: https://github.com/kraih/mojo/blob/master/lib/Mojolicious.pm#L80-L85
16:55 jberger just emits a hook
16:58 PopeFelix Right.
17:02 CromeDome joined #mojo
17:13 abra_ joined #mojo
17:15 marty joined #mojo
17:16 fvox joined #mojo
17:34 Flying_Squirrel joined #mojo
17:38 hernan605 joined #mojo
18:00 PopeFelix How do I get to https://metacpan.org/pod/Mojo::Server#request from inside my UA? I'm thinking that might be a better route to take for playback than creating an app to do so.
18:01 Grinnz_ $ua->server->on(request => ...) ?
18:01 PopeFelix Nope. Tried that.
18:02 PopeFelix $ua->server contains an instance of Mojo::Server, but is not itself an instance of same.
18:02 Grinnz_ what?
18:02 Grinnz_ oh, Mojo::UserAgent::Server has the app
18:02 PopeFelix Yes.
18:05 fvox joined #mojo
18:17 CromeDome joined #mojo
18:26 Martin90 joined #mojo
18:31 mattastrophe joined #mojo
18:37 PryMar56 joined #mojo
18:37 cpan_mojo Minion-2.0 by SRI https://metacpan.org/release/SRI/Minion-2.0
18:40 sri \o/
18:43 bpmedley_ Woohoo
18:43 trone joined #mojo
18:44 CandyAngel Yay \o_
18:45 CandyAngel Now to fix everything :P
19:09 jb360 joined #mojo
19:19 Mattes joined #mojo
19:27 marty joined #mojo
19:31 disputin joined #mojo
19:33 mattastrophe joined #mojo
19:57 vanHoesel joined #mojo
20:04 neilhwatson joined #mojo
20:13 mattastrophe joined #mojo
21:00 disputin joined #mojo
21:10 CandyAngel Hmm
21:10 CandyAngel Are there any modules or techniques for running a step of an algorithm at a time?
21:15 ZoffixWork joined #mojo
21:15 preaction the perl debugger: perl -d
21:16 CandyAngel I want to visualise it with a renderer
21:19 CandyAngel Acutally, something like MVC might do.. have the controller execute a step and return the next controller in the algo
21:20 preaction visualize what? the thread of execution?
21:20 CandyAngel Visualise the algorithm running
21:21 CandyAngel So say all it does is generate 10 random points, then selects 2 random points and joins them until they all have connections
21:22 preaction you mean you want to save every step. you don't care what perl code is being run, you just want to see each selection as it happens?
21:22 CandyAngel It would stop at each point drawn, then at selecting 2 points, then at each check for existing connections
21:22 CandyAngel So I can draw it (in OpenGL)
21:22 CandyAngel LIke umm
21:23 CandyAngel http://bost.ocks.org/mike/algorithms/
21:23 CandyAngel (second example, where it draws the circles)
21:24 preaction yes, that person wrote the code that makes the animation. you can save each step and then use something like imagemagick to make an animation out of it
21:25 CandyAngel It wouldn't have animation
21:25 CandyAngel Just render the current state
21:25 CandyAngel The only sane way I can think of doing it is have the model run a callback whenever it is modified
21:26 preaction it seems like you might get better help if you step back and explain more what you're trying to do with this
21:27 CandyAngel Random dungeon generator
21:27 CandyAngel And I want to draw it as it is being generated (at slower speed than it actually takes to generate)
21:28 preaction so the model is generating itself? i'd think a model is just the state of the dungeon, and that some other function is generating it by updating the model
21:29 preaction and then you can have that other function write out the model's state to a picture or what-not
21:29 preaction once the dungeon is generated, you don't need to generate it, nor do you need any pictures of the generator's progress, so likely i'd make my generator into another class
21:30 preaction (the Controller, if one wants MVC)
21:30 CandyAngel it'll eventually be going into an ECS-structured game
21:31 CandyAngel But this is just me sandboxing :P
21:32 CandyAngel Hmm
21:32 preaction right, but the ECS part isn't in Perl, unless you're using SDL::Game, and if so, very nice
21:32 CandyAngel Yeah, the entire thing is coded in Perl
21:32 CandyAngel (with some thin C wrappers for libspnav and openhmd)
21:36 preaction either way i'd likely make the model not know how to generate itself, that seems like a job for something more specialized than a model
21:36 CandyAngel Yeah
21:36 CandyAngel So like
21:36 preaction bonus: if your model has an api, you could re-use your generator with anything that conforms to that aapi
21:36 CandyAngel Create a controller and pass it in a blank model to generate in?
21:37 preaction basically, or it could make its own blank if you don't give it one (though further extending an existing dungeon might be an interesting feature)
21:38 CandyAngel The thing is, once I'm in the generator controller, how do I prompt a drawing and stop-for-a-second? callback?
21:38 CandyAngel Ooo
21:38 CandyAngel Hadn't considered that.. seeded dungeons..
21:38 CandyAngel Could be useful for boss-levels (set up the boss room, let a dungeon be generated around it
21:38 preaction you don't. have a setting in the controller that says "draw the dungeon after every step"
21:39 CandyAngel Ohhhh
21:39 CandyAngel Wait.. you give the controller a view and it calls that?
21:39 CandyAngel So the controller doesn't know if the view in an OpenGL window or output-to-file
21:39 preaction i wouldn't get too attached to the pattern, as i don't think it's a good fit
21:41 preaction ah, okay, yeah, you could have an event/callback get called after every step, wherein you take the model and do something with it
21:41 jberger CandyAngel: it sounds to me like almost everything you are mentioning is really more "business logic" rather than "model" though in MVC they are basically the same thing
21:41 preaction if you need to do more than just "draw it in a picture to test how it generates"
21:42 CandyAngel Right now, it's just to see it working
21:42 CandyAngel But it might be used in the game as a loading screen or something
21:42 CandyAngel Or to show loading progress or something
21:42 murre joined #mojo
21:42 CandyAngel "or something" < favourite suffix to any sentence :P
21:43 preaction i'd go with YAGNI. do what you need right now, as long as you don't make what you might need in the future impossible
21:44 CandyAngel YAGNI? not heard that one :P
21:45 CandyAngel You are going nowhere.. innit?
21:45 Grinnz_ you ain't gonna need it
21:46 CandyAngel Ohhh, nice
21:46 CandyAngel Right-o, I'll just code *something* then
21:48 preaction the idea is you can always go back if it turns out this something isn't enough. constant refactoring is how things get better
21:48 CandyAngel Trueeee
21:48 Grinnz_ design for what you need right now, and in a modular way so that refactoring is easier later
21:50 preaction yeah. good design ends up being more about how you can respond to changing requirements, because requirements are always subject to debate
21:50 preaction and yes, eventually patterns emerge
21:50 Grinnz_ and because management changes their minds on a weekly basis if not faster
21:52 Grinnz_ "so you know that thing people can change? can you show a list of all the changes that have ever been made?" - my favorite "additional requirement"
21:52 nic_ joined #mojo
21:53 CandyAngel Grinnz_: As someone who makes Excel spreadsheets, I've heard that one a lot :|
21:53 preaction yeah... that sounds like a fun undertaking... make a log file!
21:53 Grinnz_ preaction: it's especially fun when the "changes" could be modification of one of 30 different fields
21:53 CandyAngel "No, Excel doesn't do that" "But it does on this other spreadsheet" "That spreadsheet is used by *one* person, on *their* machine, not by 50 people across the country :|"
21:54 CandyAngel The latest one I was asked to make was pretty much *any* issue tracking system (e.g. MantisBT)
21:54 CandyAngel Implemented in Excel
21:54 CandyAngel :|
21:54 nic_ bpmedley: When you update minion::backend::mysql, be sure to s/postgresql/mysql/g :)
22:26 jb360 joined #mojo
22:28 jberger or don't :-P
22:28 jberger jk
22:28 marty_ joined #mojo
22:33 disputin joined #mojo
22:47 bpmedley_ nic: Gaw.. :P
22:47 PryMar56 joined #mojo
22:54 shadowpaste "PopeFelix" at 217.168.150.38 pasted "Why is this hanging?" (44 lines) at http://paste.scsys.co.uk/499743
22:55 PopeFelix Test runs successfully, but hangs for a few seconds before exiting.
23:27 mattastrophe joined #mojo
23:30 dabudabu joined #mojo
23:35 meshl joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary