The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2017-08-04

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 arcanez anyone using Mojo::mysql::PubSub?
00:00 arcanez batman seems to like to include ' The implementation should be considered an experiment and might be removed without warning! '
00:00 arcanez in his modules :)
00:11 preaction i haven't, but if it's a message broker you need, i did write Mercury
00:36 tchaves joined #mojo
00:38 mattp_ Single Database connection waits for notification by executing SLEEP on server. connection_id and subscribed channels in stored in mojo_pubsub_subscribe table. Inserting new row in mojo_pubsub_notify table triggers KILL QUERY for all connections waiting for notification.
00:38 mattp_ id probably ... not use that in production
01:26 gizmomathboy joined #mojo
01:55 arcanez mattp_: live a little
02:11 bryan joined #mojo
02:24 preaction i've got a problem where sometimes a websocket through a proxy can get disconnected, but not every part of the connection knows that it's disconnected. i think that i can help fix this problem by sending pings regularly. will that work? how can i make sure that every websocket sends a regular ping?
02:32 Ptolemarch joined #mojo
02:58 noganex_ joined #mojo
03:09 jberger hmmmm, this might finally help me push my minion ui over the finish line
03:09 jberger http://epicmax.co/vuestic
03:09 jberger h/t mst for the link
03:12 preaction huh. that might work well for that "simple" cms i want too
04:50 karjala_ joined #mojo
05:04 arcanez wow. that's nice.
05:57 ashimema That does look nice
05:57 ashimema interesting to see mention of Mercury and message brokers again just as I'm about to embark on that..
05:59 ashimema I had a nice simple setup using my existing Mojo::Pg connections and Mojo::Pg::PubSub.. but I've found it really doesn't scale well :( (unless I'm doing somthing wrong)
06:00 ashimema I basically learnt you can't use PgBouncer in transaction mode if you want to use `Listen` which kinda means you quickly require allot of connections
06:01 arcanez "Mojo::Pg::PubSub is a scalable implementation of the publish/subscribe pattern used by Mojo::Pg."
06:02 ashimema mm.. I 'think' it works asynchronously which is where it's scalability comes in..
06:02 ashimema it's more my use of it that's flawed I think. ;)
06:02 ashimema comes back to an earlier irc conversation here.. I mop up allot of connections fast in my structure
06:05 ashimema https://irclog.perlgeek.de/mojo/2017-07-06#i_14835404 if your interested arcanez
06:06 ashimema sri actually made some improvements to help improve connection sharing in Mojo::Pg after I brought it up.. hense 4.0 of ::Pg :)
06:36 prg joined #mojo
06:38 batman arcanez: it's because that module is a hack.
06:39 batman MySQL doesn't really support pubsub
06:39 batman and yes... I do like to include that message in early releases :)
06:42 trone joined #mojo
06:53 VVelox joined #mojo
06:53 Vandal joined #mojo
07:17 AndrewIsh joined #mojo
07:18 karjala_ joined #mojo
07:52 bianca joined #mojo
08:29 dustinm joined #mojo
09:18 irqq joined #mojo
09:34 Lee joined #mojo
10:59 noganex joined #mojo
11:15 tchaves joined #mojo
11:17 ashimema anyone fancy helping me understand Mojo::Pg connections in relation to pubsub and running an app under Morbo?
11:17 ashimema as soon as I connect two clients I seem to start loosing postgres connections
11:29 ashimema aha!
11:29 * ashimema goes to read the pubsub code
11:33 ashimema the problem appears to go away if I up the default max_connections to 2 or more
11:33 sh14 joined #mojo
11:46 ashimema oh.. no it doesn't
11:46 ashimema just becomes less apparent
11:53 trone joined #mojo
12:15 bianca joined #mojo
12:40 Vandal joined #mojo
12:43 gizmomathboy joined #mojo
12:53 tchaves joined #mojo
12:54 Pyritic joined #mojo
13:08 sri oh, this is funny, irccloud actually highlights `$foo->bar` as code
13:10 sri http://i.imgur.com/Ih8pkbu.png
13:10 sri guess i've never seen it because nobody uses backticks on irc
13:14 sri jberger: agreed about http://www.epicmax.co/vuestic
13:14 sri looks perfect for a minion ui
13:18 genio vuestic does look nice
13:52 mcsnolte joined #mojo
13:54 itaipu joined #mojo
14:36 bianca joined #mojo
14:38 ashimema right.. I'm totally confused
14:38 ashimema should I be checking that the connection still exists before calling pg->pubsub->notify ?
14:39 ashimema I seem to often get 'DBD::Pg::db do failed: server closed the connection unexpectedly'
14:39 ashimema as I try to call the notify
15:08 itaipu joined #mojo
15:14 PopeFelix Still trying to figure out this Future business, and trying to reproduce the problem outside of my controller. When does a controller start the IOLoop?
15:14 petru joined #mojo
15:15 genio What does your controller method look like?
15:16 nic joined #mojo
15:17 genio You're making use of $c->render_later;  ?
15:17 genio http://mojolicious.org/perldoc/Mojolicious/Controller#render_later
15:17 PopeFelix a few lines of text, some brackets. the usual. ;)
15:18 PopeFelix I think I've tried it both with and without, but it see that it's commented out, so let me try it with. just a mo
15:19 PopeFelix no, i take that back. render_later is in my controller method, just before I have a debug line telling me if the ioloop is running or not
15:21 PopeFelix But oddly, that debug message shows up 3 times when I call the controller method
15:22 genio Well, sorry. the delay helper makes using render_later helper.  I should have pointed to the proper location: http://mojolicious.org/perldoc/Mojolicious/Plugin/DefaultHelpers#delay
15:22 genio err. easier
15:24 genio you were right to remove the render_later() if you're using the delay helper.  *should get some sleep*
15:25 PopeFelix lol
15:25 PopeFelix I'm not using the delay helper. but I suppose i can try that.
15:25 sri for some reason most people who want to use futures/promises seem to misunderstand them completely
15:26 PopeFelix and I'm sure i'm no exception.
15:26 sri like they think promises make async code look sync
15:27 sri but it's just attaching more callbacks at different places
15:31 pink_mist PopeFelix: the loop is already running when you're in your controller method. there is absolutely no way for you to have entered your controller method without the ioloop already running. it is the the ioloop that makes it possible to enter your controller method in the first place.
15:32 pink_mist at least as far as I understand
15:34 sri the web server is running the event loop
15:34 sri the framework doesn't really care at all
15:35 sri that's why morbo and hypnotoad are so important for us
15:35 sri and why plack is not so nice for us
15:41 ashimema sri any pointers why I'm seeing these db disconnects when using pubsub->notify?
15:41 ashimema sri any pointers why I'm seeing these db disconnects when using pubsub->notify?
15:41 ashimema oops
15:41 ashimema sorry for the double post there
15:42 sri no
15:42 ashimema I'm listening in one controller and notifying in another
15:43 ashimema ok.. always worth asking..
15:43 PopeFelix pink_mist, OK, thanks.
15:43 PryMar56 joined #mojo
15:47 PopeFelix To replicate this, then, it would probably be easiest to set up a toy controller method, I think
15:56 ashimema am I right in thinking this effectively servers a cached dbh: https://metacpan.org/source/SRI/Mojo-Pg-4.01/lib/Mojo/Pg/PubSub.pm#L40
15:56 ashimema in which case.. should there not be a ping somewhere to make sure it's still alive?
16:01 sivoais joined #mojo
16:01 * ashimema goes back to re-reading this code to make sense of it..
16:27 Grinnz sri: its a new `feature`, you can disable it in settings
16:27 Grinnz ``` it also does
16:27 Grinnz multiline blocks```
16:27 Grinnz which is ... odd, in irc land
16:27 Grinnz > and this, which i disabled cause its annoying
16:29 pink_mist what does that one look like? (and what do the multiline blocks look like? 0_o
16:29 pink_mist )
16:29 PopeFelix It seems to come down to convergent Futures not playing well inside of the controller. If I return a simple Future from an operation, the IOLoop picks it up, does the thing, and marks it done. But if I return a convergent Future (e.g. Future::wait_all, Future::needs_any, Future::needs_all) it looks like the individual Future is marked done, but the convergent Future isn't.
16:30 Grinnz https://usercontent.irccloud-cdn.com/file/svDncjRd/irccloud_text_features.png
16:32 Grinnz PopeFelix: no reason those shouldn't work the same as in any other situation
16:32 PopeFelix Grinnz, That was my thinking as well.
16:39 pink_mist Grinnz: thanks, yeah, that looks horrible :P
16:43 kaare joined #mojo
16:43 PopeFelix No, I think it really is the interaction between the controller and the convergent Futures
16:43 purl okay, PopeFelix.
16:43 PopeFelix okay, purl
16:43 purl Whatever you say, PopeFelix.
16:43 pink_mist purl: I think it really
16:43 purl pink_mist: sorry...
16:44 pink_mist guess not
16:44 PopeFelix let me work up a paste to be sure.
16:45 ashimema joined #mojo
17:18 PopeFelix http://paste.scsys.co.uk/564777
17:19 PopeFelix Is there something I'm doing fundamentally wrong here, or are the convergent Futures really not playing well with Mojo?
17:19 Grinnz PopeFelix: you need to use Future::Mojo->needs_any etc, and you need to keep a reference to the futures until they complete
17:20 PopeFelix Grinnz, thanks, I'll try that.
17:25 PopeFelix Grinnz, like so? http://paste.scsys.co.uk/564778
17:26 Grinnz you are still not storing the futures anywhere
17:26 PopeFelix Where do I need to store the futures?
17:27 Grinnz anywhere that will exist until they complete
17:27 Grinnz in the stash, perhaps
17:27 Grinnz also you need render_later
17:27 PopeFelix OK
17:27 PopeFelix oh, you're right. I do need render_later. oops
17:28 Grinnz you only need to store the topmost future. the convergent futures, for example, keep a reference to all the futures they are waiting on
17:29 Grinnz or similarly the combined future that results from ->then
17:29 * PopeFelix nods
17:32 PopeFelix Grinnz, that did it, thank you. But why did it work without the convergent Future?
17:33 Grinnz the future created by Paws was already a Future::Mojo. as for why it worked without render_later or storing the future, no idea
17:33 Grinnz that sounds like it wasn't actually executing async
17:33 PopeFelix ok
17:33 Grinnz i.e. the future completed before it was returned
17:35 PopeFelix I don't know if that's true or no. I know that under the hood, Paws is just doing Mojo async calls. https://metacpan.org/source/JLMARTIN/Paws-0.34/lib/Paws/Net/MojoAsyncCaller.pm#L54
17:35 PopeFelix For the case without a convergent Future, I mean
17:35 Grinnz yeah i don't see why it would be running sync
17:35 Grinnz but thats the only explanation i can think of
17:35 * PopeFelix nods
17:36 Grinnz hmm, it might not have needed to be stored because it's creating a memory cycle with the ioloop, so the future is never cleaned up
17:36 PopeFelix For the convergent Futures, I think it has to do with (e.g.) https://metacpan.org/source/PEVANS/Future-0.35/lib/Future.pm#L1571
17:37 PopeFelix Grinnz, the non-convergent Future, you mean?
17:37 Grinnz the future Paws is creating
17:37 PopeFelix oh, i see
17:37 Grinnz that could be a memory leak, it should probably do something similar with weaken
17:38 PopeFelix Paws should weaken $self there?
17:38 Grinnz same as in https://metacpan.org/source/DBOOK/Future-Mojo-0.003/lib/Future/Mojo.pm#L28-29
17:38 Grinnz no, it should create a weak copy of the future, and use that in the callback
17:38 PopeFelix oh, i see
17:38 Grinnz (don't weaken the original ref, because that's the only ref you have at that point)
17:38 PopeFelix hm. let's see what happens when we do that
17:43 PopeFelix well, if I weaken $future at https://metacpan.org/source/JLMARTIN/Paws-0.34/lib/Paws/Net/MojoAsyncCaller.pm#L60 it doesn't change anything.
17:44 PopeFelix oh, but what about the convergent futures...
17:44 dmanto joined #mojo
17:44 sri futures sure are fun
17:45 Grinnz you have to create the weak copy outside of the closure
17:45 sri you just don't get pages and pages of backlog about plain callbacks
17:45 PopeFelix "Make everything async," I said. "Make it Futures all the way down," I said. "It'll be cool," I said.
17:45 PopeFelix oy
17:46 Grinnz right after "my $future = Future::Mojo->new;" do "weaken(my $weak_f = $future);" and then only use $weak_f in the callback
17:46 PopeFelix righto
17:47 PopeFelix sri, plain callbacks can lead to callback hell, though
17:48 PopeFelix i hear, anyway
17:48 * PopeFelix is only starting to do async stuff
17:48 PopeFelix Grinnz, oh, hey, that broke things. neat!
17:49 Grinnz the expected breakage or a different breakage :)
17:49 PopeFelix "Mojo::Reactor::Poll: I/O watcher failed: Can't call method "done" on an undefined value at /Users/kipeters/.perlbrew/libs/perl-5.16.3@p5-bean-ats/lib/perl5/Paws/Net/MojoAsyncCaller.pm line 74."
17:49 PopeFelix lol
17:49 PopeFelix oh, sorry. The expected breakage
17:49 Grinnz right, thats cause the future wasnt stored
17:49 PopeFelix yeah
17:50 PopeFelix neato
17:50 PopeFelix I would never have guessed that was the fix
17:52 pink_mist PopeFelix: Mojo::IOLoop's delay avoids callback hell
17:52 * PopeFelix nods
17:54 PopeFelix I've got to have Futures for my S3 interaction. But maybe I *shouldn't* do the rest of it as Futures all the way down.
17:54 Grinnz it's easier when it's consistently futures imo
17:55 PopeFelix yeah, and mixing futures and not is going to be its own special kind of hell
17:55 PopeFelix which i don't need.
17:56 Grinnz the convergent futures aren't being closed over so you dont need to worry about weakening them btw
17:56 PopeFelix nods
18:08 CandyAngel Trying to port something from SQLite to Pg, not going well. The import into Pg made the column just filled with binary (1010101010) rather than text >.<
18:09 CandyAngel Probably because the export from SQLite made the column X'HEXCHARS' which I don't think Pg liked :P
18:12 Grinnz yeah, the Pg format is E'\\xHEXCHARS'
18:22 CandyAngel Bleh, some of the data it is not happy with (says invalid byte sequence for UTF8) and it only did the first character right.. so I guess it is E'\xDE\xAD..' ?
18:26 CandyAngel Yeah, looks like I have to escape string each character.. hmm..
18:26 CandyAngel Grinnz++ thanks for the lead
18:53 CandyAngel Yeah, found a short one and manually escaped each character and that imported correctly
18:54 Grinnz i only have needed to do one \\x at the beginning.. what version of pg?
18:55 CandyAngel postgresql 9.6.3-3
18:55 Grinnz maybe it's different when not a bytea col
18:56 irqq joined #mojo
19:27 bianca joined #mojo
19:30 karjala joined #mojo
19:31 irqq_ joined #mojo
19:52 bryan joined #mojo
20:05 stryx` joined #mojo
20:06 PopeFelix I have a synchronous action that may throw an error, followed by an asynchronous action. Should I just do eval { $action->(); 1; } or $c->render( 'error' ) and then $c->delay( ... ) ?
20:11 raytestinger joined #mojo
20:12 jabberwok greetings raytestinger !   first time on IRC
20:12 preaction second-ish, i think ;)
20:14 PopeFelix I think putting something that doesn't render inside delay() is screwing things up.
20:14 raytestinger yes, that's correct, I can't make the meeting Thur, will be on the road.
20:14 preaction no worries, enjoy your trip!
20:19 marty joined #mojo
20:22 PopeFelix OK, so I *should* eval / render and *then* delay. putting eval/render inside delay causes things to hang.
20:26 raytestinger left #mojo
22:00 VVelox joined #mojo

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary