Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-03-19

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:08 timotimo i was sidetracked by people around me
00:09 timotimo samcv: the problem is that we're joining a whole bunch of things and we have to check every seam where we join stuff
00:11 timotimo i have probably hallucinated the result where i had a tenth of what i thought i'd have
00:14 timotimo okay, i probably have to assume my problem is that i recurse instead of doing my stuff flat?
00:14 timotimo one of the strings has 7201 pieces
00:15 timotimo so it could very well be that at its peak it's 7k calls down
00:15 timotimo in its heap
00:17 timotimo okay, cool, now i'm "down" to 4 gigs
00:18 MasterDuke look at those gains!
00:18 timotimo dem gains, brah
00:19 timotimo parse-string is just not so clever in general
00:20 Ven joined #moarvm
00:20 timotimo it may want a partial rewrite
00:21 MasterDuke do you know what "\n" or "\r\n" are as MVMGrapheme32's?
00:22 timotimo yup
00:22 timotimo MVM_nfg_crlf_grapheme(tc)
00:23 timotimo i made it work much faster, but now it b0rks multiple \ in a row
00:28 timotimo no, something more basic was actually wrong
00:28 timotimo grrr
00:28 timotimo i think i'll drive home instead of continuing to bang my head against this
00:28 timotimo no doubt this can be done much easier
00:28 timotimo wat, i just made it pass?!
00:28 timotimo haha. nope.
00:31 MasterDuke may have made it a bit faster and use less memory, nqp tested fine, seeing what a spectest looks like
00:33 MasterDuke in MVM_nfg_is_concat_stable, changed this `if (last_a < 0 || first_b < 0)` to this `if ((last_a != MVM_nfg_crlf_grapheme(tc) && last_a < 0) || (first_b != MVM_nfg_crlf_grapheme(tc) && first_b < 0))`
00:34 MasterDuke spectest passed, hope something would have failed if my change wasn't correct
00:35 timotimo i expected something like that would be right
00:36 timotimo but it could very well be that our test coverage of this is ... meager
00:37 MasterDuke if i printed the .perl of the from-json result, would that be different if i'd messed something up?
00:37 timotimo not necessarily
00:38 timotimo it's probably only bad in cases we're not thinking of right now
00:39 timotimo there's also the comment above that if that says that the rest of the code relies on the last_a and first_b being >= 0
00:39 MasterDuke but it says that's an over-estimate, right?
00:40 timotimo yes
00:40 timotimo but maybe we have to put checks for < 0 into the other pieces of code
00:40 timotimo bbiab
00:40 MasterDuke well, i'll PR it for people to think about
00:41 MasterDuke fwiw, it was about .5s faster and .3g less ram
01:07 Geth ¦ MoarVM: MasterDuke17++ created pull request #556: Special case "\r\n" in MVM_nfg_is_concat_stable
01:07 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/556
01:21 timotimo hm. that's not tremendously much
01:21 timotimo the memory usage is still pretty amazing
01:23 timotimo in a bad way
01:30 timotimo dat profile.sql ...
01:34 MasterDukeMobile joined #moarvm
01:35 MasterDukeMobile Yeah, the qt viewer choked when I created a .json profile
01:35 MasterDukeMobile Too deeply nested or something like that
01:35 timotimo could be, yeah
01:36 timotimo the plot sqlitebrowser shows for retained/promoted/cleared bytes is ... amazing
01:36 timotimo cleared is extremely low all the time
01:36 MasterDukeMobile Hm, haven't tried a profile after my change
01:36 timotimo and the other two make a trumpet
01:36 timotimo oh, your change. i should pull that in
01:38 timotimo the BEGIN and COMMIT in the .sql file make sqlitebrowser quite confused
01:38 timotimo it seems to want to put its own transaction around the code
01:39 timotimo which then explodes because you can't have nested transactions in sqlite
01:39 timotimo the other way to do it makes it fail to create it somehow and it just waits forever
01:41 timotimo well, sqlite3 /tmp/profile.sqlite -init /tmp/profile.sql does the trick just fine
01:41 MasterDukeMobile Please feel free to edit that code (sql profile stuff) as you see fit
01:43 timotimo i was still hoping you could do that one thing for me :3
01:43 timotimo anyway, the ping-ponging between promoted and retained is a clear indicator of having almost completely long-lived objects
01:46 timotimo i feel like i did something today
01:46 timotimo that's nice
01:49 MasterDukeMobile Yep
01:50 timotimo i think it'd be really valuable if parse-string wouldn't recurse
01:53 MasterDukeMobile Recursion can be so clean a solution and yet not so great for performance
01:54 timotimo mhm
01:54 MasterDukeMobile Btw, did you pull my change? If so, did it change the profile much?
01:59 timotimo really difficult to judge
01:59 timotimo were you able to spot if the difference in memory usage was mostly re_nfg?
02:00 MasterDukeMobile Not sure. But I think it dropped to approx 35% in the perf report
02:01 timotimo it used to be almost 50%?
02:01 MasterDukeMobile I think I compared at the same commit of Json::fast if that's what you mean
02:01 MasterDukeMobile Yep
02:03 MasterDukeMobile 49 -> 46 with the first j::f fix, then to 35 with my change
02:03 MasterDukeMobile Something like that anyway
02:33 MasterDuke just looked at the previous heaptrack report. 890m allocated in re_nfg, 432m after my change
02:48 ilbot3 joined #moarvm
02:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
07:24 domidumont joined #moarvm
07:26 samcv yay
07:26 samcv gonna pull MasterDuke
07:26 samcv merge i mean
07:27 Geth ¦ MoarVM: f273133979 | (Daniel Green)++ | src/strings/nfg.c
07:27 Geth ¦ MoarVM: Special case "\r\n" in MVM_nfg_is_concat_stable
07:27 Geth ¦ MoarVM:
07:27 Geth ¦ MoarVM: "\r\n" can never combine with anything else, so we can special-case it
07:27 Geth ¦ MoarVM: in the check whether the relevant graphemes are synthetic.
07:27 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f273133979
07:27 Geth ¦ MoarVM: fe1dc84ac7 | (Samantha McVey)++ | src/strings/nfg.c
07:27 Geth ¦ MoarVM: Merge pull request #556 from MasterDuke17/special_case_crlf_in_MVM_nfg_is_concat_stable
07:27 Geth ¦ MoarVM:
07:27 Geth ¦ MoarVM: Special case "\r\n" in MVM_nfg_is_concat_stable
07:27 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/fe1dc84ac7
08:54 KDr2_c joined #moarvm
09:06 domidumont joined #moarvm
09:11 domidumont joined #moarvm
09:28 domidumont joined #moarvm
09:31 domidumont joined #moarvm
09:35 agentzh joined #moarvm
11:03 Ven joined #moarvm
11:13 Ven_ joined #moarvm
11:33 timotimo i wonder if we can only re_nfg in join when we're finished
11:33 timotimo instead of over and over and over
11:35 lizmat jnthn: re https://docs.google.com/spreadsheets/d/1kpSb8LoskHSbM1FQvWdQ269rlRkU8vh5A_ElpN3Qay4/edit#gid=0 , why is .elems delegating to .cache, end end to .list ?  I mean, isn't end just .elems - 1 ?
12:08 MasterDuke timotimo: MVM_string_join does only do it to the final result `return MVM_nfg_is_concat_stable(tc, a, b) ? result : re_nfg(tc, result);`
12:13 timotimo oh
12:13 timotimo strange
12:14 timotimo why then would it take so long? :\
12:16 MasterDuke timotimo, jnthn: slight change in topic, in native_ref_fetch (src/6model/containers.c:234) or native_ref_fetch_i, is there any way to tell if we're dealing with a signed vs unsigned native? it switches on a MVMNativeRefREPRData->primitive_type for int vs num vs str, but i can't see any way to find out about the signedness
12:16 MasterDuke timotimo: is parse-string doing more joins than it needs to?
12:16 timotimo it used to
12:17 timotimo it might still do that
12:17 timotimo it also does concats every now and then
12:17 timotimo i really must measure this more precisely
12:18 MasterDuke yeah, maybe construct a very small test case that you can calculate how many joins/concats it should only have to do and compare to how many it actually does?
12:21 timotimo yeah, it'd just be like "\r\n foo \r\n bar \r\n zap \t yoink \t blubb \t omg"
12:21 timotimo that's the perfect test case right there
12:28 MasterDuke timotimo: somewhat related question. https://github.com/MoarVM/MoarVM/pull/551 fixes/addresses https://rt.perl.org/Ticket/Display.html?id=126450, but i'm not sure how to write a test for memory use. could your vmhealth branch help with that?
12:29 timotimo hm. bad idea, i think
12:29 timotimo ... just write the test so that if the bug is there, it'd consume 64gigs of ram, and if the user's computer crashes during testing, the test failed
12:30 timotimo (yeah probably don't do that)
12:31 MasterDuke heh, i'd want to reach through the internet and slap someone who merged a test like that
12:32 timotimo :D
12:35 samcv sounds like a great test
12:35 samcv fullfills the criteria that when ran, the user will know whether or not the test failed
12:35 MasterDuke i'm not even sure there are many (any?) performance tests in roast, i wonder if it makes more sense to have a separate suite for that
12:35 timotimo yup, nothing wrong with that at all
12:36 samcv MasterDuke, we have the nqp tests
12:36 samcv and rakudo has some tests but i don't think any moar specific ones?
12:38 MasterDuke doesn't jnthn have a page of moar/rakudo daily benchmarks?
12:38 timotimo yup
12:38 timotimo http://moarvm.org/measurements/
12:38 timotimo we can also put more different kinds of measurement there
12:38 MasterDuke any sort of alerting when values change?
12:39 MasterDuke hm, seems to have stopped at 3/8/17
12:39 timotimo because there's a level of folders in between
12:39 timotimo no alerting
12:40 timotimo interesting, it did stop
12:40 timotimo probably ran out of memory on the server! :P
12:40 MasterDuke are there any plots over time of those benches?
12:41 timotimo don't think so
12:41 timotimo i believe jnthn's daily script nukes the results every day so we don't have histories on a per-day basis
12:42 MasterDuke oh, the *-releases and *-releases-history have some results over time
12:42 MasterDuke but those seem to have died on 1/31/17
12:45 MasterDuke and those graphs are a little odd, they always have a sharp up or down turn at end, like there's always some bad value
12:45 MasterDuke anyway, afk for a while &
12:46 Geth ¦ MoarVM: cac2a57c04 | (Stefan Seifert)++ | Configure.pl
12:46 Geth ¦ MoarVM: Do not set use rpath if installing into proper system locations
12:46 Geth ¦ MoarVM:
12:46 Geth ¦ MoarVM: This is a hopefully better attempt than commit
12:46 Geth ¦ MoarVM: f777352ebae29349d1bfa9e6c03c12f32f2ec689.
12:46 Geth ¦ MoarVM:
12:46 Geth ¦ MoarVM: Removing because rpath is prohibited on Fedora and strongly discouraged on
12:46 Geth ¦ MoarVM: openSUSE. It's also not needed if libmoar.so is installed into the default
12:47 Geth ¦ MoarVM: library path.
12:47 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/cac2a57c04
13:37 agentzh joined #moarvm
15:44 ggoebel joined #moarvm
15:50 zakharyas joined #moarvm
15:50 domidumont joined #moarvm
15:59 IRCFrEAK joined #moarvm
16:20 IRCFrEAK joined #moarvm
17:38 agentzh joined #moarvm
17:54 agentzh joined #moarvm
18:36 agentzh joined #moarvm
19:18 Geth ¦ MoarVM: MasterDuke17++ created pull request #557: Correctly detect+handle overflow in mp_get_int64
19:18 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/557
19:29 MasterDuke nine: my PR ^^^ was inspired by/based off of the attempted patch you gave me a little while ago, how does it look to you?
19:45 agentzh joined #moarvm
20:01 zakharyas joined #moarvm
20:10 spebern joined #moarvm
21:24 nine MasterDuke: looks good to me! I wonder how it does on the performance front?
23:52 MasterDuke nine: slightly faster. `for ^10_000_000 { my uint64 $a = 2**64-1; my int64 $b = 2**62; my int64 $c = -2**63 }` took an average of 13.4s before, 13.1s after

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary