The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2017-10-20

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:09 * jberger is having fun :D
02:09 jberger \m/
02:11 mohawk jberger, what you up to?
02:11 mohawk i am implementing reading/writing schemas from text in GraphQL
02:19 jberger almost ready for an initial commit
02:26 noganex joined #mojo
02:31 jberger https://github.com/jberger/Mojo-ChromeHeadless/blob/master/ex/google_news.pl
02:48 mohawk cor
02:53 mohawk the name of it suddenly makes me think of william gibson
02:59 jberger not only do I not get the reference, I have no idea if that will be the final name
02:59 jberger indeed it doesn't actually have to run headless, so perhaps Mojo::Chrome is more correct
02:59 mohawk https://en.wikipedia.org/wiki/Burning_Chrome_(short_story_collection)
03:00 jberger yeah, I'm a terrible scifi fan
03:00 jberger I like it, but I tend to reread/rewatch things I already like rather than finding new stuff
03:05 mohawk there's no wrong way to consume fiction ;-)
03:46 zivester joined #mojo
03:55 jberger yeah, I think I'm renaming it to Mojo::Chrome
03:55 jberger (at least for now)
03:55 jberger nothing is set in stone until I release to CPAN, which is still a ways off
03:57 jberger new link: https://github.com/jberger/Mojo-Chrome/blob/master/ex/google_news.pl
04:04 mohawk even more gibson-tastic :-)
04:04 dboehmer_ joined #mojo
04:04 mohawk albeit the austin powers version
04:04 mohawk baby
04:10 hesperaux_ joined #mojo
04:14 jberger :D
04:42 inokenty-w joined #mojo
06:04 dod joined #mojo
06:18 itaipu joined #mojo
06:40 hesperaux__ joined #mojo
07:03 AndrewIsh joined #mojo
07:09 dod joined #mojo
07:57 dod joined #mojo
08:02 trone joined #mojo
08:03 rba joined #mojo
08:33 rba_ joined #mojo
08:37 jamesaxl joined #mojo
08:45 hesperaux joined #mojo
08:46 jamesaxl joined #mojo
08:54 rshadow joined #mojo
08:55 prg joined #mojo
08:57 hesperaux_ joined #mojo
09:00 rba joined #mojo
09:01 karjala_ joined #mojo
09:04 itaipu joined #mojo
09:09 hesperaux joined #mojo
09:14 hesperaux_ joined #mojo
09:15 hesperaux__ joined #mojo
09:20 hesperaux joined #mojo
09:21 good_news_everyon joined #mojo
09:21 good_news_everyon [mojo] kraih pushed 1 new commit to master: https://git.io/vddFc
09:21 good_news_everyon mojo/master e81e5f2 Sebastian Riedel: try testing with Strawberry again
09:21 good_news_everyon left #mojo
09:28 hesperaux_ joined #mojo
09:34 aborazmeh joined #mojo
09:58 margeas joined #mojo
09:59 itaipu joined #mojo
10:14 hesperaux__ joined #mojo
10:15 CandyAngel Wait.. Mojo wasn't being tested with strawberry?
10:16 Vandal joined #mojo
10:16 sri nope, it didn't work for a time
10:18 CandyAngel Oh.. it's been working fine for me :P
10:18 CandyAngel (it's how I use Mojo at work)
10:20 pink_mist CandyAngel: pretty sure he meant the automated testing wasn't working
10:21 CandyAngel Ohh, that makes sense
10:23 sri correct
10:46 kes joined #mojo
10:48 tchaves joined #mojo
10:49 itaipu joined #mojo
11:16 rba joined #mojo
12:01 plicease joined #mojo
12:26 vicash jberger: it looks good. i might use it for crawling react.js sites in the future
12:43 perlpilot joined #mojo
13:09 jberger Of course I intend to tie it into testing, though because this is more flexible than phantom I don't quite know what the interface will look like
13:14 batman jberger: i wouldn't expose so many methods. I would make connect() and send() internal
13:15 batman why would i want to call connect()? it should just happen on the first time i try to send a command imo
13:16 batman also, must i start the server myself? could it be started by Mojo::Chrome if it's not already running?
13:19 gizmomathboy joined #mojo
13:24 jberger yeah, I'm going to add starting the server
13:24 jberger and yes many of those methods may become private before then
13:29 jberger the hardest part about this is trying to divine how it all works
13:29 jberger they aren't even apologetic about it, there is basically no usage documentation
13:30 jberger there is now some api documentation https://chromedevtools.github.io/devtools-protocol/
13:30 jberger but they still are basically like "sniff the protocol to see how it all works together"
13:32 rba joined #mojo
13:33 gryphon joined #mojo
13:44 sri ohoh, minion locks can apparently cause a deadlock somehow
13:45 sri https://gist.githubusercontent.com/anonymous/ff2b93e54d6c4714bb6b40635c3c214a/raw/dfb2b0cc3c0129e93c7d7efb4bab430d4d5e3f57/gistfile1.txt
13:46 sri this line https://github.com/kraih/minion/blob/master/lib/Minion/Backend/Pg.pm#L965
13:52 batman jberger: cool
13:52 genio odd timing issue where several minion jobs are updating the locks table at roughly the same time since you can't insert into an exclusive locked table?
13:53 batman i would like to use it in Test::Mojo::Role::Selenium
13:53 genio sri: would locking the table before deleting old records make that go away?
13:54 gizmomathboy joined #mojo
14:00 gryphon_ joined #mojo
14:02 ChmEarl joined #mojo
14:04 zach joined #mojo
14:05 genio nevermind. that's just a waiting game, that shouldn't create a deadlock.
14:05 genio if process 1 has a lock on Foo and is waiting for a lock on Bar, and process 2 has a lock on Bar and waiting for a lock on Foo, a deadlock happens.  I'd have to look at all of the queries in Minion to see where that might happen
14:09 rba joined #mojo
14:22 dod joined #mojo
14:26 dod joined #mojo
15:03 sri genio: nothing else accesses the minion_locks table
15:03 sri it's a bit weird
15:04 jacoby joined #mojo
15:04 pink_mist my gut feeling is that it's a bug in pg
15:07 sri return $job->retry({delay => 30}) unless my $guard = $minion->guard('import', 10800, {limit => 1});
15:07 sri the actual code causing it
15:07 sri unique job
15:09 sri genio: the delete should be the only operation that happens on minion_locks without the exclusive lock
15:11 dantti_laptop joined #mojo
15:22 jberger batman: automagic connect now in master
15:24 sri doesn't look like i can replicate the deadlock, i only see it for a few failed minion jobs
15:24 sri happened twice this week
15:26 sri wonder if there's a way to avoid the lock completely
15:37 gryphon joined #mojo
15:58 sh14 joined #mojo
16:12 dod1 joined #mojo
16:21 gryphon joined #mojo
16:41 dod joined #mojo
16:45 gizmomathboy joined #mojo
17:15 batman jberger: cool!
17:28 sh14 joined #mojo
17:30 jberger I can even kinda-sort spawn the process automagically now
17:30 jberger though I can't automagically kill it because I can't figure out when that should happen
17:30 jberger I might have to make those manually taken actions for now
17:32 mohawk jberger, on DESTROY?
17:32 jberger I suppose
17:32 jberger I haven't given it enough though
17:33 sh14 joined #mojo
17:33 jberger I chased a stupid bug for far longer than I should have
17:33 mohawk i guess making a few likely use-cases as tests and seeing what feels most "smooth" is the way to find out
17:33 jberger CHECK YOUR ERROR SLOT IN NONBLOCKING CODE PEOPLE!!
17:33 jberger (I'm saying this to myself)
17:34 mohawk ha ha! i've been fighting with a Pegex grammar to make it treat comments (which *were* whitespace) as something to be captured into the next rule-group
17:34 mohawk you only *think* you know what frustration is ;-)
17:34 jberger hehe, yeah
17:34 jberger anyway when you spawn the process yourself, you have to disable gpu. I'm assuming this has to do with perl not being built with threads
17:35 mohawk gawd
17:35 mohawk jberger, would that sort of error-checking be more natural/idiomatic if you used Future (i'm told Promises is not the best way for perl)
17:35 jberger well when I made connecting automagic, I missed adding an error slot somewhere
17:35 mohawk ?
17:35 mohawk oh no
17:35 jberger which then meant I was not getting the result there, because it was showing up in the missing error slow
17:35 jberger slot
17:36 jberger and THEN, because I didn't check the error higher up, I missed that it was erroring out
17:36 jberger BUT
17:36 jberger all this was doing was controlling the timing of waiting for the navigation request to complete
17:36 mohawk gawd
17:36 jberger so when the gpu was enabled, ie when I started chrome in its own terminal, it was fast enough to complete in time
17:37 jberger so this ownly showed up as the spawning code running to completion, executing all steps and yet returning different (empty) output
17:37 jberger THAT was fun
17:37 mohawk wow
17:38 jberger mohawk: in essence no, it wouldn't be any different
17:38 jberger it would just be say that I didn't attach an error handler to the Future when I needed to
17:38 jberger same thing
17:38 mohawk sure
17:39 mohawk i guess i was thinking Future would bubble errors up till they got handled though?
17:39 jberger well maybe, I don't know Future that well
17:40 mohawk once i've got this schema read/write thing cracked and made the graphql-js->perl tutorial translation notes, promises is next
17:40 jberger my goof was not having this line: https://github.com/jberger/Mojo-Chrome/blob/master/ex/google_news.pl#L27
17:40 mohawk it's going to be async-tastic - mojo, baby!
17:40 mohawk one line? oh no :-)
17:41 jberger well, that's why I couldn't see it
17:41 jberger the error was actually caused by not having $err in https://github.com/jberger/Mojo-Chrome/blob/master/lib/Mojo/Chrome.pm#L34-L35
17:41 jberger well not having it in the signature
17:41 mohawk jberger, why isn't the thing calling that slot just doing the die() itself?
17:41 jberger which pushed $payload back
17:41 jberger it can't
17:41 mohawk shurely that's the idiomatic way?
17:42 mohawk couldn't Mojo::Chrome do it?
17:42 jberger in the io loop if it dies before calling the callback the error goes to the wrong location
17:42 mohawk oh
17:42 mohawk ok
17:42 jberger you have to pass the errors into the callback
17:42 jberger then dying sends the error to the containing delay
17:42 dmanto joined #mojo
17:42 mohawk you could wrap the user's callbacks in yours and exceptionalise them?
17:42 jberger yeah, I've tried that about a half dozen times
17:42 mohawk don't let the user do dumb stuff like not handling errors?
17:43 jberger it gets messy really fast
17:43 jberger oh, just wrapping
17:43 trone joined #mojo
17:43 mohawk yes
17:43 jberger i mean, no, because they need to get the errors out in the same way
17:43 mohawk so the user-space callback doesn't get errors
17:43 mohawk ok
17:43 jberger the nature of the beast
17:43 mohawk i'd suggest having an experiment with Future but it's your trainset not mine :-)
17:43 jberger Future does hide that for you
17:44 jberger I trade this monotonous beast (always have these three lines of boilerplate everywhere) vs the complexity of when to make which Futures and when to close over what
17:44 jberger that bends my mind
17:44 jberger and I've already bent mine this way just the way I like it ;-P
17:45 mohawk when i made this beast: https://github.com/mohawk2/sw-turnkey i started out not properly understanding promises
17:45 jberger maybe I should say it a different way, nonblocking code ALWAYS passes the error to callbacks
17:45 mohawk the heart of it was realising i just needed to handle the cases with little functions that returned the right type of promise
17:46 jberger the combination of delay and die just makes it easier for me
17:46 jberger but notice that the delay's error handler still does that for me
17:46 jberger it just means that dying is a funnel that anything in that scope that dies goes into a callback with that same slot
17:47 mohawk it seems to me the wrapping with a "die if" then call onwards if no error would work
17:47 mohawk ok
17:47 jberger "little functions that returned the right type of promise" <-- see that's what I don't get and in my pattern I don't need to
17:47 jberger I'm not saying its wrong
17:48 jberger but my way, I just always have to pass the error up
17:48 jberger and eventually it gets to the user's callback
17:48 jberger (assuming I make my signatures correctly, which I didn't)
17:48 jberger very wrote
17:48 jberger rote?
17:48 purl i heard rote was something like catechism: a tautology.
17:48 jberger yes, rote
17:51 mohawk rote = following a very fixed process
17:52 mohawk i will advocate for what i promise (ha!) is the very last time:
17:52 mohawk Dr StrangeFuture: or how i learned to stop worrying and love promises
17:52 mohawk it's a valuable idiom to get one's head round - it gives one extra ways to deal with this async craziness
17:52 mohawk anyway, nuff said ;-)
17:53 jberger in the end I probably am going to have to
17:53 jberger seems that the world has gone that way
17:53 jberger but I find that delays are so simple (once you grok them) that it's hard for me to find the urge to do so
17:54 mohawk callback hell is bad, mm'kay
17:54 jberger this isn't callback hell
17:54 pink_mist mohawk: that's why delays are good
17:54 jberger you don't see even one nested callback .... ah right
17:54 jberger gotcah
17:55 jberger whatevs I'm not correcting ^^
17:55 mohawk grin
17:55 jberger :D
17:55 mohawk i'm just going to read the "delay" doc to make sure i slightly know what we're talking about
17:55 jberger I think of it as callback flow-control
17:56 jberger it linearizes callbacks and waits on them
17:56 mohawk as i see
17:56 mohawk Future has idioms for this sort of thing too
17:56 jberger sure and they are more expressive
17:56 jberger but that means they have more primatives
17:57 jberger this essentially only has one primative, begin
17:57 mohawk my googling a number of days ago didn't seem to indicate (to me) that mojo and Future have really got a lot of visible co-usage
17:57 Grinnz future has a lot more options. they may or may not be useful for any particular task
17:57 jberger there is a Mojo::Future
17:57 Grinnz Future::Mojo*
17:57 jberger ^^ what the author said
17:58 Grinnz the two things I really prefer futures for are 1. error handling boilerplate goes in one place, ever, 2. you can cancel them which makes setting timeouts trivial to do at the top level
17:58 jberger Grinnz: so (1) confirms that exceptions bubble automagically?
17:59 jberger all the delay boilerplate is to manually bubble exceptions
17:59 Grinnz depends how you combined the futures, but yes in a regular chain, one failure fails the whole chain
17:59 mohawk boom
17:59 Grinnz or for example a needs_all fails if any of the passed futures fail
18:01 jberger interesting conversation, but it strikes me that it is lunchtime :-P
18:01 jberger (past actually)
18:01 Grinnz as for automagic handling, a 'failed future' doesn't throw an exception by itself but depending how the user retrieves its result it might. ->get on a failed future throws an exception for one
18:01 Grinnz or the user could attach an on_fail handler to do whatever
18:01 jberger ^^ see that's already more complexity than I want
18:02 Grinnz maybe, but it's all about giving agency to the top level to handle things directly
18:02 Grinnz this is something that always annoyed me working with delays
18:02 jberger I think perhaps the delays you've used do too much
18:02 Grinnz it's more to learn, but more helpful, imo
18:03 mohawk when jberger's a bit closer with M::Chrome we could maybe PR a Future-y version that i believe would be able to zap all the boilerplate die $err stuff
18:03 jberger I'm a big proponent of using Delays as sequences (with a few parallels if necessary) and then calling methods (which might themselves have delays) from the outer layer
18:03 mohawk and jberger's tests that he'll have there will help us validate the Future-y stuff still does the correct things
18:03 jberger in that way, the top level has all the agency it needs
18:04 jberger mohawk: that would be interesting
18:04 mohawk i'm already looking forward to having a go
18:04 mohawk i need to get my head round Future as i intend to make it fundamental to graphql-perl
18:04 jberger my example script already works, consider it the test case for the time being
18:04 mohawk as it (or rather promises) is/are for gql-js
18:04 jberger feel free to play with it
18:05 mohawk jberger, if you could just wrap it into a .t file...? :-)
18:05 mohawk and then travis it up too :-)
18:05 jberger the only reason I can't is because the content changes, it is reading the headlines from google news
18:05 mohawk that way a PR will show in the most unambiguous way whether it works right ;-)
18:05 jberger what I would need to do is have it point back to a tiny mojo app, which is indeed what I will do once I wrap it in a .t
18:05 mohawk jberger, make a mock and/or real, local server?
18:06 jberger Mojo doesn't do mock servers, all of them are real
18:06 mohawk i gather there's a web framework that makes that sort of thing relatively easy
18:06 Grinnz :)
18:06 mohawk #trolling
18:06 jberger even if you USE them as a mock, they still do everything as a real server
18:06 gizmomathboy joined #mojo
18:06 mohawk oh, you're saying it would be easy to make a real server for test purposes, then?
18:06 mohawk huh
18:06 Grinnz you dont need to actually run it as a server. the app is executed by the user agent requests
18:06 jberger sure, I get that, I'm just saying that unlike PSGI tests, Mojo tests DO use a real server
18:07 jberger right, it just-in-times
18:07 jberger mohawk: yes, it would be very easy
18:07 mohawk well then...
18:07 jberger I just haven't done it yet :-P
18:07 mohawk :'(
18:07 Grinnz the only time it needs to be run as a server is if you need to connect to it with a client that's not Mojo::UserAgent. in which case i tend to use Test::TCP
18:07 jberger until now it was mostly figuring out "how the heck does this undocumented protocol work?!"
18:08 jberger Grinnz: hunh?
18:08 jberger its a real server, you can talk to it with anythign
18:09 jberger maybe I'm not understanding
18:09 pink_mist jberger: I think he means the just-in-timing only happens for M::UA?
18:09 pink_mist maybe
18:09 jberger but that isn't true
18:10 Grinnz jberger: HTTP::Tiny would need an actual tcp port to connect to
18:10 jberger sure, so ask it what tcp port its using
18:10 jberger $t->ua->server->port
18:10 jberger or nb_port
18:10 rshadow joined #mojo
18:10 jberger depending on what loop you're using
18:10 Grinnz it listens on a port when you haven't started it?
18:10 jberger constructing the Test::Mojo object starts it
18:11 Grinnz this is not using Test::Mojo
18:11 jberger I'm confused, what are we talking about?
18:11 pink_mist I'm also confused, I thought Test::Mojo was a given
18:11 Grinnz i was talking about when i use mojo apps to test things that aren't using mojo
18:11 Grinnz because it's easier to set up than some other way to make a mock http server
18:12 jberger right, that's possible too, but then you just Mojo::Server::Daemon it
18:12 jberger I don't see what the Test::TCP is for
18:12 Grinnz https://metacpan.org/source/DBOOK/WWW-OAuth-0.006/xt/author/request_ua.t
18:13 jberger (if that sounds combative, it isn't meant that way)
18:13 Grinnz if you can make that work without Test::TCP be my guest
18:15 jberger so the problem there is that you are testing non-io-loop clients?
18:15 Grinnz yes, as i said
18:15 jberger sorry, I must have missed/misunderstood that
18:16 jberger I was assuming either you were testing some external service or some external process
18:16 jberger I don't use too many other perl web clients these days
18:17 jberger what you're doing makes sense
18:17 jberger anyway, really lunch time now ;-p
18:19 * Grinnz too
18:19 mohawk nom nom nom
18:19 purl I eat your head!
19:46 gizmomathboy joined #mojo
19:46 dmanto Hi, I am trying to read several Redis queues, using Mojo::Redis2, but when I use (for instance) blpop method more  than once, it seems to block until previos blpop are finished
19:47 jberger dmanto: I don't remember for sure, but you might have to make multiple instances
19:49 petru_ joined #mojo
19:49 marcus redis blocks on blpop? https://redis.io/commands/blpop
19:50 jberger marcus makes a good point. To clarify, you do realize that the "bl" stands for blocking right?
19:50 jberger (which I why I suggested using multiple instances (and thus multiple connections))
20:04 dmanto jberger: yes I realize that
20:06 dmanto I dont understand the "using multiple instances". You mean fork different connections?
20:06 dmanto or you mean like defining multiple helpers, redis1, redis2 and so on
20:10 jberger fork is the wrong word, you wouldn't be forking
20:10 jberger well, hmmm
20:11 dmanto hehe yes I know
20:11 jberger so just instantiate another Mojo::Redis2 yeah?
20:11 jberger its essentially connection pooling
20:11 dmanto yes like $s->helper(redis1  => sub { state $r= Mojo::Redis2->new() });
20:11 dmanto and $s->helper(redis2  => sub { state $r= Mojo::Redis2->new() });
20:12 jberger why not just a helper that returns a redis instance?
20:12 jberger rather than caching it?
20:12 jberger or whatever
20:12 jberger yes, in essence that's what I mean, I just wouldn't structure it that way
20:12 dmanto mmm I don't know
20:13 dmanto yes that could be a quick fix
20:13 dmanto s/state/my/
21:02 sri why doesn't Mojo::Redis2 have connection pooling built in?
21:04 Grinnz it does
21:04 Grinnz sorta
21:04 Grinnz normally for things that required pooled connections, you would have a new object for that, like Mojo::Redis2::Bulk
21:05 Grinnz also, blpop and brpop use their own connection, according to the docs
21:05 Grinnz ... if you call them non-blocking
21:06 Grinnz it looks like it just queues it as a normal non-blocking command though
21:08 Grinnz ah, it queues it separately from other commands, but within the blpop command each command is queued on the same connection
21:08 Grinnz wonder why
21:09 Grinnz brpop and brpoplpush each have their own queues as well
21:14 hrupp joined #mojo
21:15 sri so, i've been able to replicate the minion deadlock problem
21:15 sri it's the delete, if i move it below the lock everything is fine
21:26 zivester joined #mojo
21:26 sri this should be the fix https://github.com/kraih/minion/commit/624d68ff3fb37af90b2474834c41135f57020317
21:27 sri really like that i can just use create or replace function and move the declaration down a version
21:29 Grinnz wonder if the sqlite version has the same problem https://metacpan.org/source/DBOOK/Minion-Backend-SQLite-2.003/lib/Minion/Backend/SQLite.pm#L89-93
21:33 sri test case is simple
21:33 sri just run two instances of this https://gist.github.com/anonymous/75f40e9263935d930a8701e1ea057d9d
21:35 dantti_laptop joined #mojo
21:36 Grinnz i just get endless streams of numbers from both
21:36 sri then you should be fine
21:37 sri it died with the error message from earlier for me
21:37 sri ....deadlock...
21:41 rba joined #mojo
22:09 trone joined #mojo
22:38 jberger so it isn't easy to tell if chrome has started up
22:38 jberger even the official chrome-loader for nginx just polls it
22:39 jberger but I figured, one thing you can tell it to do is to open AND load a page immediately
22:39 jberger well, if that page is just **another** mojo server, it can act as the trigger when it is called ;-D
22:39 pink_mist lol
22:41 jberger https://github.com/jberger/Mojo-Chrome/blob/spawn/lib/Mojo/Chrome.pm#L132-L143
22:41 jberger kinda insane, but it seems to work
22:43 CandyAngel Is there a.. style(?).. with UserAgent, of having it attempt to log in if you make a request and it fails due to not being logged in?
22:44 preaction wrap it in a function that does that
22:45 Grinnz or an object that keeps track of that maybe
22:45 preaction if object, could subclass UA :p
22:45 Grinnz well if you're going to do that might as well use the new role support :P
22:45 jberger then again, presumably this is a client to something
22:46 jberger and that client delegates to the UA
22:46 jberger so just have the client object take care of that stuff rather than putting it on the UA, which is essentially transport not business logic
22:47 jberger anyway, lots of patterns and thus no recommendation
22:47 diegok CandyAngel: we just do what jberger just said :-)
22:47 Grinnz basically, whatever you feel is best
22:47 preaction making a model is probably the best bet
22:48 CandyAngel It's a Mojolicious::Lite app which is a frontend for a forum
22:48 CandyAngel Remembers post content and highlights changes
22:48 Grinnz i use the role now for our tests to login to our app
22:48 Grinnz much cleaner than what i was doing before
22:49 CandyAngel Some topics require being logged in, but you can get logged out at any time
22:49 sri neat, mojo now has over 700 reverse dependencies on cpan
23:19 gretchen joined #mojo
23:26 jberger and now it auto spawns chrome (assuming you've told it where your chrome lives)
23:26 * jberger wipes brow
23:53 Grinnz cpan-dependents --dist Mojolicious gives 780 dists recursively
23:53 Grinnz i wish more dists would set up their metadata properly so this sort of analysis was more reliable

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary