The web in a box - a next generation web framework for the Perl programming language

IRC log for #mojo, 2014-11-12

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:05 hasan purl: netstat shows more than 300 established connections. I don't think I have so many workers
00:05 purl hasan: sorry...
00:05 hasan do I?
00:08 bpmedley Moment
00:11 hasan purl: ?
00:11 purl wish i knew, hasan
00:12 preaction purl: botsnack
00:12 purl :)
00:16 Averna joined #mojo
00:21 hasan ok. so here my results for mojolicious with psql on a usb stick: ab -k -c 300 -n 300 -H "Accept-Encoding: gzip,deflate" 'http://localhost:8080/api/v1/foo
00:22 hasan Requests per second:    381.84 [#/sec] (mean)
00:22 hasan :D
00:25 bowtie_ joined #mojo
00:25 bpmedley I can’t say for sure.  I would have to spend more time looking at the connection pooling that PgAsync does; however, I don’t have that much time right now.
00:25 hasan bpmedley: would appreciate. I want to debug this more but since I'm in the company right now, I can't.
00:26 bpmedley hasan: There are several people in the channel that are more interested in development time, rather than execution time.
00:26 sri umm, those rps values could mean anything
00:26 hasan bpmedley: with development time you mean?
00:27 hasan sri: in my case it is just a simple SELECT query.
00:27 hasan so not much surprised
00:27 bpmedley Yes, I’ve heard the tale that reducing development time can make a larger impact that increasing rps.
00:27 sri they could be awesome on a shitty box, or terrible on a super fast box
00:28 sri bpmedley: wtf? how can you deduce anything from the information provided?
00:28 bpmedley I didn’t say I could.
00:28 preaction is 400rps not acceptable? are you running the benchmarks on the same box? is the hardware exactly the same as production hardware?
00:28 hasan I can tell you from where I work. there are several customers facing troubles with their boxes due to load impacts of their business web sites.
00:28 bpmedley I have no information on the box it ran on, the load, and no code.
00:29 jzawodn <sarcasim>sounds awesome</sarcasim>
00:29 hasan average to big customers one can say. and I wonder how they have designed their application.
00:29 preaction i prefer siege to ab, and i had thousands of rps using, iirc, just the standard prefork daemon. i think i had peak efficiency on my hardware around 50 workers, but i'd have to drag up all the tests i did to know for sure
00:30 bpmedley I wasn’t trying to be negative.
00:30 hasan preaction: no this is my shitty workstation. the db is even on a memstick (for me carrying around so I can develop everywhere I go)
00:30 preaction the hard part was finding enough different systems to saturate the server with enough clients to make it break down
00:31 preaction i think i had 5 different servers of the same size making requests to the one server. i don't have root, so there were file descriptor problems more than any other problem...
00:31 hasan preaction: yes. I played around with worker settings, too. unfortunately on my dev box, I can't go beyond 5 workers due to the kernel oom killer killing my daemon process
00:31 hasan that's why I did this app pure non blocking.
00:31 preaction so you can't really measure performance on this box :p
00:32 hasan there is no route which blocks so far.
00:32 hasan preaction: no :(
00:34 preaction i was able to get staggering performance results. the bottlenecks i identified were all in my own code (i wrote a bad XS wrapper around some threaded C++ library), not in mojolicious, and not in the other team's messaging system i'm using
00:34 Lee joined #mojo
00:34 bpmedley I’ll also put this out there that I believe it’s better to focus on decreasing development time rather than to focus on increasing rps.
00:35 preaction turns out because of that messaging library, i could just spawn more crappy workers using my crappy XS implementation, but there was a limit to that too. i think it was something like 400. i couldn't get anything to break, and by the end of the night i was getting angry that i couldn't break more ;p
00:36 hasan bpmedley: please explain development time. I have a hard time to understand this in context.
00:36 preaction how long does it take to write the code
00:36 hasan I guess you do not mean the development time of the whole application.
00:36 hasan hmm.
00:36 bpmedley No, how long does it take to implement a feature and beat the competion and at what cost.
00:36 hasan I see.
00:36 preaction it doesn't matter if your code is infinitely scalable and performance if you can never deliver it
00:37 bowtie joined #mojo
00:37 preaction 1) make it work 2) make it right 3) make it fast
00:37 hasan and speaking of rps. isn't it far better to have a low mean time per request rather than a high rps with also high mean time
00:38 preaction you have to have an acceptable level for both. what you need is "what is the acceptable mean time when the server is dealing with X rps?"
00:38 hasan short example: here I have 300 concurrent reqs with 300 requests each: 252.26 [#/sec] (mean) and 1189.244 [ms] (mean).
00:38 hasan but 3 concurrents with 3 requests each: 181.93 [#/sec] (mean) 16.490 [ms] (mean)
00:38 hasan of course it isn't linear. but you know what I am saying.
00:51 bowtie joined #mojo
00:59 disputin joined #mojo
00:59 bowtie_ joined #mojo
01:05 laouji joined #mojo
01:18 sugar__ joined #mojo
01:18 basiliscos joined #mojo
01:18 bowtie_ joined #mojo
01:25 HtbaaPi joined #mojo
01:30 good_news_everyon joined #mojo
01:30 good_news_everyon [mojo] kraih tagged v5.60 at 8e03261: http://git.io/225Leg
01:30 good_news_everyon left #mojo
01:31 good_news_everyon joined #mojo
01:31 good_news_everyon [mojo] kraih pushed 1 new commit to master: http://git.io/gZKV9A
01:31 good_news_everyon mojo/master 8697869 Sebastian Riedel: bump version
01:31 good_news_everyon left #mojo
01:36 sugar joined #mojo
01:43 sugar_ joined #mojo
01:46 bowtie joined #mojo
02:02 sujithm joined #mojo
02:03 bowtie joined #mojo
02:13 bowtie_ joined #mojo
02:19 bowtie_ joined #mojo
02:23 * jzawodn presenting Perl in Space in ~30 minutes: http://www.meetup.com/San-Francisco-Perl-Mongers/events/215733252/
02:28 KCL joined #mojo
02:30 bpmedley http://bmedley.org/mojo.pdf <— I converted to LaTeX for the book I was writing.  There are my style improvements to be had.  Thoughts on content?
02:30 bpmedley https://github.com/brianmed/mojo_book
02:31 bpmedley jzawodn: Looks really cool
02:31 jzawodn should be fun
02:40 bowtie joined #mojo
03:01 zivester joined #mojo
03:13 bowtie_ joined #mojo
03:28 bowtie joined #mojo
03:55 ua joined #mojo
03:58 doublelel joined #mojo
04:03 bowtie joined #mojo
04:03 Eke- joined #mojo
04:10 bowtie joined #mojo
04:31 basic6_ joined #mojo
04:54 bowtie joined #mojo
05:01 rem_lex| joined #mojo
05:01 romel hey guys. how to use command line interface with full app?
05:05 Snelius what u mean?
05:07 romel i want to add custom command to my app so i have added Myapp::Command::deploy and added Myapp::Command to namespaces. but i can't figure out how to start my app with this new command. examples contain myapp.pl which is mojolicious lite app so i'm confused a bit
05:10 sujithm joined #mojo
05:14 tianon ./script/myapp ?
05:19 jberger romel: it can be done, but I don't recommend it
05:19 jberger make a command that does all the setup, then just use existing deployment options
05:23 romel all i need at the moment is to call 'deploy' method of my app's schema
05:24 romel so i thought it would be nice to add some 'db_deploy' command to app
05:24 jberger right, so make a command that does that
05:24 jberger Galileo does that
05:25 romel thank you
05:25 jberger but to confirm, that shouldn't leave your app in a running state
05:25 jberger it just should deploy the schema and exit
05:26 romel ok, i see. thank you very much
05:26 nicomen also, I believe you need to use the Mojolicious::Command namespace, and not Myapp::Command namespace
05:26 crab wait, why not use script/myapp deploy?
05:26 jberger crab: how does that work with hypnotoad?
05:27 crab works fine, in the sense that hypnotoad has nothing to do with it?
05:28 crab it doesn't interfere or interact with hypnotoad in any way
05:28 jberger sure, a deploy command is great, but it shouldn't start the app (in the sense that it shouldn't attempt to start the server)
05:28 jberger which was the time of the original question
05:28 jberger s/time/tone/
05:29 jberger maybe I misread the original question
05:31 romel no, you didn't. that's what i meant
05:31 sujithm joined #mojo
05:32 jberger romel: cool
05:33 jberger Galileo LOOKS like its setup command starts the app, but it doesn't :-P
05:34 jberger it does some trickery with the router and starts a setup app that runs with the same configuration and helpers but with different routes and controllers
05:34 laouji joined #mojo
05:37 romel well, that's pretty smart :)
05:46 doublelel joined #mojo
06:00 sri i do automatic schema stuff on app startup with Mojo::Pg migrations
06:01 sri https://github.com/kraih/mojo-pg#mojopg-
06:01 sri no command, just a schema sync on startup with migrations
06:04 sri although i might be misunderstanding the question... if the solution is a setup application
06:05 sri on a related note, i'd really like to release a stable Mojo::Pg 1.0, but need more feedback!
06:06 sri features i'm unsure about atm. are notifications and if hard connection limits should be managed automatically somehow
06:09 sri the topic of crazy numbers of active database connections came up earlier, and made me wonder
06:09 sri in Mango i could just have a queue for non-blocking operations, and perform them round robin on a connection pool with a static size
06:10 sri PostgreSQL connections are stateful though, so you always need your own connection
06:10 sri only real solution would be a callback for getting your db handle
06:11 sri $c-pg->db(sub { my $db = shift; ... });
06:13 marmez joined #mojo
06:14 sri it's a vicious circle, folks don't want to use Mojo::Pg because it's experimental, and i can't make it stable because folks don't give me feedback :)
06:15 sri btw. i'm worried about notifications because i don't see a way to use them in Minion for pub/sub
06:15 sri (with the current API)
06:16 sri replicating the old mongodb pub/sub stuff https://github.com/kraih/minion/commit/2dced8910d56521b6cc3513139cc6504445bd339#diff-4cb82f2377d9246cda487335e38ea990L23
06:17 sri ok, i'm done now, you can talk again! :)
06:35 dotandimet joined #mojo
06:45 tempire I tried notifications, but couldn't get them working
06:46 tempire may have been my postgres noobness
06:46 tempire I'll try again
06:55 Lee joined #mojo
06:55 Eke- joined #mojo
07:01 basiliscos joined #mojo
07:02 dod joined #mojo
07:40 xevz joined #mojo
07:53 bjoernfan joined #mojo
07:56 doublelel joined #mojo
08:13 trone joined #mojo
08:14 woz joined #mojo
08:15 Vandal joined #mojo
08:20 basiliscos joined #mojo
08:22 Dandre joined #mojo
08:24 batman joined #mojo
08:26 batman any assetpack users here? i would like to get some feedback on https://github.com/jhthorsen/mojolicious-plugin-assetpack/pull/32/files
08:29 gatitskiy joined #mojo
08:32 gatitskiy Mojolicious supports validation the requests with JSON in body?
08:50 gatitskiy joined #mojo
09:01 arthas joined #mojo
09:10 dotan joined #mojo
09:11 vytas joined #mojo
09:26 fhelmber_ joined #mojo
09:48 neyasov____ joined #mojo
10:05 basiliscos joined #mojo
10:12 rawler joined #mojo
10:20 arthas joined #mojo
10:20 dp_ joined #mojo
10:21 sugar joined #mojo
10:31 rawler1 joined #mojo
10:40 dod joined #mojo
10:47 dod joined #mojo
10:48 Dandre Hello,
10:51 irq joined #mojo
10:52 Dandre I have some issues with file upload. I have followed this: http://mojolicio.us/perldoc/Mojolicious/Lite#File-uploads and every things work fine with small files. But when I try to upload a 100M+ file I get this error: Out of memory!
10:52 Dandre I have set $ENV{MOJO_MAX_MESSAGE_SIZE} to 0
10:52 Dandre in the docs there is:
10:52 Dandre And you don't have to worry about memory usage, because all files above  250KB will be automatically streamed into a temporary file.
10:52 Dandre and I have suffisant disk space.
10:54 denis_boyun joined #mojo
10:54 Dandre where does this temporary file written? In memory wich could explain my issue or on disk?
11:06 gatitskiy joined #mojo
11:08 Dandre I have found the downloaded file in /tmp. can the out of memory error come from $file->move_to?
11:19 Averna joined #mojo
11:27 bpmedley Do you have a smallish Mojolicious::Lite example?
11:28 doublelel joined #mojo
11:36 zivester joined #mojo
11:47 gatitskiy joined #mojo
11:55 gatitskiy joined #mojo
11:55 doublelel joined #mojo
12:04 bpmedley http://bmedley.org/mojo.pdf and https://github.com/brianmed/mojo_book  <— Will someone comment this?   I would like to get some feedback before I go further.
12:21 neilhwatson joined #mojo
12:33 vytas bpmedley, wow that's awesome ! I'll proof read it and try all examples.
12:33 bpmedley Thank you.  The LaTeX code includes the perl code verbatim.
12:37 vytas bpmedley, do you have a twitter handle?
12:37 bpmedley At present, no.
12:45 risugg joined #mojo
13:21 vytas bpmedley, I mentioned your book here: https://twitter.com/vytasdauksa/status/532517982925504512
13:22 vytas though it's first time I got annoyed by twitter characters limit..
13:33 ignacio_ joined #mojo
13:46 basiliscos joined #mojo
13:54 aleksey joined #mojo
13:57 dotan1 joined #mojo
14:00 bpmedley vytas: Cool
14:05 zivester joined #mojo
14:10 sujithm_ joined #mojo
14:14 amon joined #mojo
14:29 KCL_ joined #mojo
14:30 Dandre bpmedley: Ok I have found, it was elsewhere (mime type guessing)
14:32 bwf joined #mojo
14:37 doublelel joined #mojo
14:43 GillesM joined #mojo
14:44 maze joined #mojo
14:55 gatitski_ joined #mojo
15:05 jb360 joined #mojo
15:16 hernan604 mojo migrations is awwsomeeeee!
15:16 hernan604 much much thanks sri
15:16 hernan604 i dont think i can live without this feature anymore
15:16 hernan604 plus Mojo::Pg
15:16 hernan604 no going back /
15:18 neyasov____ joined #mojo
15:18 sh4 joined #mojo
15:19 sh4 joined #mojo
15:24 sri anyone know if this is a feature or a bug? https://github.com/kraih/mojo-pg/issues/2
15:27 Lee joined #mojo
15:30 Ptolemarch joined #mojo
15:34 genio I can't test that right now, but at a first glance, it seems like the problem would be in DBD::Pg
15:38 mst sri: might be worth asking on #postgresql
15:38 mst LISTEN/NOTIFY aren't used that often so the number of people who know what they're doing is limited
15:41 sri haha, just as i write this, i know what's going wrong :)
15:41 mst sri: transaction?
15:41 purl transaction is probably a set of database operations that can be undone if one or more of them fails
15:42 sri nope, the file descriptor never becomes readable while the event loop is active, because the notify was sent blocking
15:42 mst ooo
15:42 sri so the notification is received during the blocking operation and cached somewhere in DBD::Pg
15:43 sri i need to poll for notifications after every operation
15:45 sri the fix https://github.com/kraih/mojo-pg/commit/8e0ab647a0550db2ef0b08bc17968d542a3aee91
15:55 mst ah, of course
15:55 doublelel joined #mojo
15:57 Eke- joined #mojo
16:00 neyasov____ joined #mojo
16:26 sri ohoh, it gets more complicated with transactions
16:30 tempire batman: were there any photos taken of the mojoconf training?
16:46 gatitskiy joined #mojo
16:53 sri this got complicated pretty fast :o https://github.com/kraih/mojo-pg/commit/5697dd548188bbb94f33a1209ae662ff73379f54
16:54 sri transaction scope guards and notifications are a complicated combination
17:01 alnewkirk joined #mojo
17:01 ua left #mojo
17:09 sri hmm, i wonder if notifications could just be sent with sth operations and we might miss them too then
17:21 sri ok, looks like fetch operations do not steal notifications
17:25 basiliscos joined #mojo
17:29 dparry joined #mojo
17:31 dparry Hi, any tips on best way to implement an http proxy with basic auth using mojo. Mojolicious-Plugin-Proxy? Mojo/UserAgent/Proxy? ...?
17:35 Ptolemarch joined #mojo
17:36 bpmedley dparry: Do you want a proxy in front of your mojo app?  Or, do you want to send the response from a different host as the response to a request?  Or, something else?
17:36 dparry the latter
17:36 purl the latter is, like, better
17:37 dparry the thing behind mojo is only listening on localhost:port and I want to expose its dashboard through mojo at a particular url
17:38 bpmedley dparry: http://mojolicio.us/perldoc/Mojo/UserAgent#proxy <— Have you seen this option?
17:40 dparry so I have a route foo/admin to pass off to a handler that is something like http://pastebin.com/tzcSp9U4
17:41 dparry is that vaguely on track?
17:41 bpmedley Just a sec.
17:47 sri would be nice if someone used Mojo::Pg notifications in a real app
17:47 bpmedley dparry: http://pastie.org/9714911 <— Maybe something like this?
17:48 bpmedley If you would excuse line 6, pls.
17:49 dparry right but when I render that doc there are links within that I would like to handle as if were proxied through to it
17:50 dparry so I guess I could pull the paths through to the ua->get
17:50 bpmedley Hrmm.  I also forget to put a comment for $c->ua->proxy
17:50 sri bpmedley: use ->render(data => ...) when dealing with raw content
17:50 ua joined #mojo
17:50 bpmedley Thanks
17:50 ua left #mojo
17:51 dparry but I wondered if I could basically set up the equivalent of an apache / nginx reverse proxy within mojo...
17:52 bpmedley dparry: http://pastie.org/9714920 <— Somewhat better.  Give me a few minutes.
17:53 dparry thanks bpmedley that certainly looks like something to try with, might be good enough for what I need!
17:54 dparry gonna have to run catch a train in a few
17:57 sri wow, .Net is not open source :o http://tirania.org/blog/archive/2014/Nov-12.html
17:57 sri s/not/now/
17:57 sri haha
17:57 dparry looking promising though, thanks very much bpmedley!!
17:58 jzawodn Mojo.NET
17:59 Oleg joined #mojo
18:03 bpmedley https://gist.github.com/brianmed/9efcb96574221e4837d8 <— Wish they there still here
18:05 Oleg bpmedley: you need to do this in non-blocking style to save a tree and a kitten :)
18:06 bpmedley That would be better, I’ll try later on
18:08 aghbas joined #mojo
18:08 ignacio_ joined #mojo
18:23 disputin joined #mojo
18:38 dotandimet joined #mojo
18:48 neyasov____ joined #mojo
18:55 Eke- joined #mojo
18:56 irq joined #mojo
19:03 dotandimet joined #mojo
19:04 dod joined #mojo
19:11 denis_boyun joined #mojo
19:28 igorsutton joined #mojo
19:29 ilbot2 joined #mojo
19:29 Topic for #mojo is now ???? hear me roar | http://mojolicio.us | http://irclog.perlgeek.de/mojo/today
19:29 dotandimet joined #mojo
19:33 disputin joined #mojo
19:35 disputin1 joined #mojo
19:56 disputin joined #mojo
20:01 dotandimet joined #mojo
20:02 jamesaxl joined #mojo
20:06 jamesaxl joined #mojo
20:13 basiliscos joined #mojo
20:18 dotandimet joined #mojo
20:20 gatitskiy joined #mojo
20:31 Lee joined #mojo
20:32 denis_boyun joined #mojo
20:33 vytas joined #mojo
20:47 dotandimet joined #mojo
20:58 disputin joined #mojo
21:12 dotandimet joined #mojo
21:15 dod joined #mojo
21:26 dotandimet joined #mojo
21:28 disputin joined #mojo
21:31 dotandimet joined #mojo
21:53 sugar_ joined #mojo
22:13 sugar__ joined #mojo
22:13 disputin joined #mojo
22:21 disputin joined #mojo
22:21 disputin1 joined #mojo
23:10 disputin joined #mojo
23:27 bpmedley https://gist.github.com/brianmed/9efcb96574221e4837d8 <— Is this a decent way to do a non-blocking “proxy” for an alternate host?
23:51 sri bpmedley: depends if you just want to get it to kinda work, or work correctly
23:52 sri i guess this is the least bad implementation so far... ignoring the fact that marcus still mentions Mojo::Client in the docs -.- https://metacpan.org/pod/Mojolicious::Plugin::Proxy
23:53 sri real proxy would rewrite the hop-by-hop headers though
23:54 sri there is a huuuuuuge difference here between getting it working and doing it correctly
23:55 s1037989 joined #mojo
23:55 Averna joined #mojo
23:55 sri the plugin doesn't even preserve end-to-end headers
23:56 s1037989 Any advice where I should look or post for hiring freelancers or agencies specializing in Perl web app development, ideally well versed in Mojolicious?
23:57 sri bpmedley: this might be of interest http://tools.ietf.org/html/rfc7230#section-2.3

| Channels | #mojo index | Today | | Search | Google Search | Plain-Text | summary