Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-04-06

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:49 ilbot3 joined #moarvm
01:49 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:21 geekosaur joined #moarvm
03:37 vendethiel joined #moarvm
05:48 domidumont joined #moarvm
05:53 domidumont joined #moarvm
06:34 zakharyas joined #moarvm
08:11 leont joined #moarvm
08:39 zakharyas joined #moarvm
08:40 btyler joined #moarvm
08:44 brrt joined #moarvm
08:58 brrt ohhai #moarvm
09:12 jnthn o/ brrt
09:20 brrt \o jnthn
09:26 brrt i'm kind of hoping i can get the register allocator finished within a shortish timeframe... and after that, i'd think the expr jit would become mergeable
09:26 brrt or, otherwise asked, what more would be necessary / desirable to have prior to merging when that is done?
09:27 brrt we'd like / want an optimizer, but that doesn't have to be present prior to merging, i'd think
09:28 jnthn brrt: What will we have in terms of regressions upon a merge?
09:29 brrt breaking tests? well, currently, none, but currently, the expr jit is really weak
09:30 brrt with the register allocator, it will be considerably more useful, and then the answer is 'hopefully none'
09:31 brrt we won't get any cases where we could compile something today that we won't be able with the expr jit merged, because of the piecemeal way the expr jit works
09:31 brrt we'll spend more work on JIT compiling, obviously, but I have no idea whether that will be significant
09:32 brrt as far as i can see, it's still a pretty cheap JIT
09:41 jnthn Yeah...guess we can do a few small perf tests to make sure we don't get notably *worse* at some things.
09:41 jnthn But if it's piecemeal introduction that seems less likely
09:42 jnthn I was mostly meaning breaking stuff though.
10:13 * jnthn is free to do Perl 6 / MoarVM bits the rest of the day o/
10:19 nwc10 please remember to eat
10:21 jnthn :-)
10:21 jnthn My wife is good at reminding me about that also :)
10:25 jnthn timotimo: Am reviewing/tidying/merging stuff from your describe refs branch
10:29 dalek MoarVM: 7eb7835 | timotimo++ | src/6model/ (44 files):
10:29 dalek MoarVM: give all REPRs a "describe_refs" function
10:29 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/7eb7835f4d
10:29 dalek MoarVM: 83a0ce6 | timotimo++ | src/profiler/heapsnapshot.c:
10:29 dalek MoarVM: actually call describe_refs for things
10:29 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/83a0ce6e53
10:29 dalek MoarVM: 4df1fd9 | timotimo++ | src/6model/reprs/MVMHash.c:
10:29 dalek MoarVM: give MVMHash an unmanaged_size
10:29 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/4df1fd98d5
10:40 brrt (lets try not to merge when it is still breaking :-))
10:40 brrt i mean the expr jit, not timotiomo's work :-)
10:51 jnthn :)
10:54 dalek MoarVM: 7d78b8d | jnthn++ | src/profiler/heapsnapshot.c:
10:54 dalek MoarVM: describe_refs is an alternative to gc_mark
10:54 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/7d78b8d3aa
11:42 brrt joined #moarvm
12:27 dalek MoarVM: 2c415d5 | jnthn++ | src/6model/reprs/Lexotic.c:
12:27 dalek MoarVM: Implement describe_refs for lexotic.
12:27 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/2c415d518e
12:27 dalek MoarVM: 0b95607 | timotimo++ | src/6model/reprs/MVMArray.c:
12:27 dalek MoarVM: describe_refs in MVMArray
12:27 dalek MoarVM:
12:27 dalek MoarVM: i wasn't able to see if it works, actually ...
12:27 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/0b956075cc
12:27 dalek MoarVM: 40ae855 | timotimo++ | src/6model/reprs/MVMCompUnit.c:
12:27 dalek MoarVM: describe refs of MVMCompUnit
12:27 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/40ae855cc5
12:27 dalek MoarVM: d78a5e3 | timotimo++ | src/6model/reprs/SCRef.c:
12:27 dalek MoarVM: Describing SCRef's refs.
12:27 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/d78a5e3404
12:27 dalek MoarVM: 550d3c3 | jnthn++ | src/6model/reprs/MVMStaticFrame.c:
12:27 dalek MoarVM: Implement describe_refs for MVMStaticFrame.
12:27 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/550d3c3d5c
12:28 jnthn timotimo: Got everything except your abortive P6opaque attempt, I think. Plus I did tweaks, cleaned up warnings, etc.
12:49 jnthn Those make much more clear why some of the leakage
12:49 jnthn spesh slots and guards
12:49 jnthn However, even with spesh disabled it still leaks
12:52 * nwc10 wonders if there is some correlation between nice hats (that are black) and not eating
12:52 nwc10 [ilmari may have to explain this :-)]
13:01 ilmari nwc10: what, did you get a black hat and forget eating?
13:01 * ilmari has already had lunch today
13:03 nwc10 ilmari++
13:03 nwc10 no, it was more that IIRC jnthn has a nice black hat, and sometimes forgets to eat
13:03 ilmari ah
13:03 nwc10 and I forget whether DrHyde has a had
13:03 nwc10 er hat
13:03 nwc10 I have hte bobs
13:18 masak joined #moarvm
13:19 masak for those interested in/curious about SSA, I found http://www.cs.indiana.edu/~achauhan/Teaching/B629/2006-Fall/CourseMaterial/1998-notices-appel-ssa_fnprog.pdf to be very approachable and wirth my reading time
13:19 masak I don't know if it's a "classic paper", but maybe it should be.
13:23 moritz masak: it's funny that you misspelled "worth" as "wirth" in this context, 'cause one of my former CS profs or TAs was named "Wirth" :-)
13:29 masak oops :)
13:29 masak ...er, not *that* Wirth, though -- right? :P
13:29 * masak .oO( I made Pascal, then I worked as a TA for a while )
13:31 geekosaur (call by value?)
13:33 masak clearly what moritz was holding was a *reference* to Wirth, but that *reference* is still passed *by value*. yes, I know the terminology is confusing.
13:37 moritz masak: not *that* Wirth, no :)
13:38 lizmat .oO( I'm not Wirthy )
13:48 * timotimo "wakes" up
13:48 brrt joined #moarvm
13:49 brrt masak++
13:49 dalek MoarVM: e016fef | jnthn++ | src/ (4 files):
13:49 dalek MoarVM: Include inter-generational roots in heap snapshot.
13:49 dalek MoarVM:
13:49 dalek MoarVM: That means snapshots will include things kept alive only by their
13:49 dalek MoarVM: presence in the inter-generational roots set.
13:49 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e016fefde4
14:09 zakharyas joined #moarvm
15:05 ilmari joined #moarvm
15:16 ilmari joined #moarvm
15:22 ilmari joined #moarvm
15:26 jnthn Damn, hope everyone will close their eyes while I push an embarassing patch in a little bit... :)
15:26 ilmari joined #moarvm
15:28 timotimo huh, will inter-gen-roots ever keep things alive that are supposed to die already?
15:28 timotimo i thought they get cleared out when a major collection happens, or something?
15:30 ilmari joined #moarvm
15:30 jnthn Yes, and we weren't clearing them out properly.
15:30 timotimo !!
15:31 jnthn The full analysis I've been doing today has turned up various other intresting things we'll need to address, but that one was the whopper.
15:31 timotimo also ... *facepalm* of course spesh candidates will keep old stuff alive via logging, spesh slots, stuff like that
15:31 jnthn Right, that's the other one.
15:31 jnthn Though that does at least saturate.
15:31 timotimo yeah, but do we actually want it to saturate?
15:31 timotimo when types disappear, we'll never get them for our guards
15:31 timotimo well, they don't disappear; i mean when they *could* disappear
15:32 timotimo and they'll be blocking candidates
15:32 jnthn Indeed.
15:32 timotimo ugh, cache invalidaiton
15:32 timotimo even typing it is hard
15:32 masak what I'm hearing from this is that things will get better after the patch, right? because that's all I care about ;)
15:32 jnthn :P
15:33 jnthn masak: Yeah, they will... Since this is a patch touching the GC, it's getting some careful testing before it lands :)
15:33 masak yes, of course
15:33 timotimo jnthn, did i show you what the profiler looks like for splitting a huge string into small-ish chunks? in particular the GC tab ...
15:33 jnthn I've been writing up my investigations as I go, so they'll land in a 6guts post at the weekend or so
15:33 masak whee
15:33 jnthn timotimo: Don't remember so
15:33 timotimo OK ...
15:34 timotimo basically it's a few hundred runs of 1/3rd red, 2/3rd orange and then 2/3rds red, 1/3rd orange over and over
15:34 timotimo we may want to have some kind of adaptive tuning for the case where there's more than half orange
15:35 timotimo because all those string objects ended up going straight to gen2, but they cause a lot of space in the nursery to get blocked (every second time, at least)
15:38 ilmari joined #moarvm
15:38 timotimo actually, the person who owns the benchmark that did that suggested "can we turn off the GC somehow?" and that might even be a good idea for our implementation of split when it notices it's making a gigantic amount of Str objects
15:46 jnthn Yeah, everything builds OK with my patch
15:46 jnthn CORE.setting builds a little slower ('cus we're doing proper full collects, presumably), but with around 40MB less memory use
15:47 jnthn And the EVAL in a loop platteaus.
15:47 timotimo \o/
15:47 timotimo it plateaus how soon?
15:47 timotimo didn't it use to plateau, too, just after a surprisingly long time?
15:47 jnthn No
15:48 jnthn It kept growing forever because of the GC bug I'm about to push a patch for
15:50 dalek MoarVM: 7f2a6fa | jnthn++ | src/ (2 files):
15:50 dalek MoarVM: Fix a full vs. partial collection detection bug.
15:50 dalek MoarVM:
15:50 dalek MoarVM: We detected it in two different places, repeating the same, factored
15:50 dalek MoarVM: out, calculation. Unfortunately, the outcome could change between the
15:50 dalek MoarVM: two places, leading to us failing to clean up the inter-generational
15:50 dalek MoarVM: roots set. This could in turn lead to ever-growing memory use for
15:50 dalek MoarVM: various kinds of program.
15:50 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/7f2a6fa841
15:51 timotimo oh, is_full_collection looked at the "size of stuff added since last full collection"?
15:51 timotimo and that was cleared in between?
15:51 jnthn Yeah
15:51 timotimo well, i implemented that counter, so it's probably partially my fault :)
15:51 jnthn So we never did all aspects of a full collect :/
15:51 timotimo man, i'm glad you found that
15:52 jnthn Well, found a few things along with it too, but yeah, that was the worst of it :)
15:52 jnthn I wonder if this fixes Skarsnik's leaking program too
15:53 japhb jnthn: I wondered why you focused on leaks so heavily first.  Now I know why.  Thank you for making the right decision.  :-)
15:55 jnthn japhb: Also 'cus long-running concurrent things that juggle mostly IO-bound stuff *is* a place where Perl 6 performance isn't really an issue (and Perl 6 has nice things for doing that), but leaks would be.
15:57 jnthn One other thing that's probably worth doing now that we have unmanaged_size is pondering factoring it in to the count that decides "when to gen2"
15:57 timotimo ah, that's like my previous attempt to have "string pressure"
15:57 jnthn An EVAL loop can create fairly large data structures that then go away, so we'd want to full collect more often to do so.
15:58 jnthn Whereas compilation is mostly producing a lot of small objects that get promoted
15:58 jnthn So we can't raise the "enough promoted to collect" threshold too high at the moment for fear of large unmanaged size causing a problem
15:59 jnthn While if we factor that in, we can bump it a bit further
15:59 jnthn timotimo: Yes, that'd be an example :)
15:59 jnthn So I may work on that next. :)
16:00 ilmari joined #moarvm
16:04 jnthn Actually I'm getting a mild headache so it's probably time for a break and some dinner :)
16:04 jnthn So will continue with that later or so :)
16:05 timotimo have a good one! :)
16:07 vendethiel joined #moarvm
16:41 leont joined #moarvm
17:09 ilmari joined #moarvm
17:12 ilmari joined #moarvm
17:19 ilmari joined #moarvm
17:20 zakharyas joined #moarvm
17:52 leont joined #moarvm
18:59 dalek MoarVM: e95ed88 | jnthn++ | src/gc/collect.c:
18:59 dalek MoarVM: Factor unmanaged size into promoted bytes.
18:59 dalek MoarVM:
18:59 dalek MoarVM: So that objects that hold onto a lot of memory indirectly will have it
18:59 dalek MoarVM: accounted for, so we can manage memory more smartly.
18:59 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e95ed88414
18:59 dalek MoarVM: 6112aaa | jnthn++ | src/gc/ (2 files):
18:59 dalek MoarVM: A (hopefully) smarter when-to-full-collect scheme.
18:59 dalek MoarVM:
18:59 dalek MoarVM: This allows for a slightly lower bound on small heaps, and better
18:59 dalek MoarVM: throughput on big heaps, by using percentage growth to decide when
18:59 dalek MoarVM: to collect.
19:15 jnthn So the fix cost us a bit on CORE.setting build time, but decreased the memory it took a bit.
19:16 jnthn The above two patches between them get us that time back, without giving much of the memory win
19:16 jnthn And also makes my leaking example from before now top out around 100MB lower. :)
19:16 jnthn Enough for today... :)
19:18 timotimo mhm mhm
20:34 colomon joined #moarvm
20:42 brrt joined #moarvm
21:33 colomon joined #moarvm
21:35 cognominal joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary