Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-05-18

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:48 ilbot3 joined #moarvm
01:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
01:57 zakharyas joined #moarvm
05:04 lizmat_ joined #moarvm
05:40 domidumont joined #moarvm
05:44 domidumont joined #moarvm
06:03 domidumont joined #moarvm
06:16 lizmat joined #moarvm
07:13 tokuhirom joined #moarvm
07:59 dalek MoarVM: 62ef65c | jnthn++ | src/gc/worklist.h:
07:59 dalek MoarVM: Fix typo in debug output.
07:59 dalek MoarVM:
07:59 dalek MoarVM: "Null STable in time" would make a good song title, though...
07:59 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/62ef65c273
08:02 dalek MoarVM: a183e38 | jnthn++ | src/profiler/heapsnapshot.c:
08:02 dalek MoarVM: Teach heap snapshotter about frame collectables.
08:02 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/a183e38912
08:02 dalek MoarVM: 85375aa | jnthn++ | src/profiler/heapsnapshot.c:
08:02 dalek MoarVM: Don't put callstack frames on snapshot worklist.
08:02 dalek MoarVM:
08:02 dalek MoarVM: We'll visit them specially, as the GC itself does.
08:02 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/85375aaa78
08:04 dalek MoarVM: 15b6a0f | jnthn++ | src/profiler/heapsnapshot.c:
08:04 dalek MoarVM: Ensure we always produce a heap snapshot.
08:04 dalek MoarVM:
08:04 dalek MoarVM: Force a GC run at the end of heap profiling, so we get an "exit
08:04 dalek MoarVM: snapshot". This also means that we'll never spit out empty snapshot
08:04 dalek MoarVM: sets if we snapshot a program that runs for a very short time (as we
08:04 dalek MoarVM: might when examining the base memory use of NQP/Rakudo).
08:04 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/15b6a0f534
08:06 timotimo aha! i implemented a183, but i didn't think of the changes in 85375
08:06 dalek MoarVM: 088dc71 | jnthn++ | src/profiler/heapsnapshot.c:
08:06 dalek MoarVM: Walk the correct thread's inter-gen root set.
08:06 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/088dc71059
08:22 dalek MoarVM: 6805576 | jnthn++ | src/profiler/heapsnapshot. (2 files):
08:22 dalek MoarVM: Include thread local callstacks in snapshots.
08:22 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/680557684c
08:43 jnthn timotimo: I pulled various of your heap analyzer improvements into mine also :)
08:45 timotimo ah, neat
08:46 timotimo the implementation of "more" is less than awesome, but it surprisingly works! :P
08:46 jnthn That seems to get it working nicely again and lets you see what was on the thread local call stack at each snapshot, anyways
08:46 zakharyas joined #moarvm
08:47 dalek MoarVM: 494661e | jnthn++ | src/profiler/heapsnapshot.c:
08:47 dalek MoarVM: Remove "we're snapshotting now" debug output.
08:47 dalek MoarVM:
08:47 dalek MoarVM: Don't really want to dump it all over the user's program output.
08:47 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/494661e708
08:47 jnthn Yes, I didn't take `more` for now 'cus of the code dupe and also it's got at least one bug (keeps returning the last page of results when you hit the end!) :)
08:49 timotimo ah, hehe
08:49 timotimo yeah, i'd rather re-implement it in a saner fashion
08:49 timotimo maybe gather/take would actually make sense there
08:50 ilmari joined #moarvm
08:58 MadcapJake joined #moarvm
08:58 jnthn timotimo: Do you plan to look further at the utf8-c8 thing at some point?
08:59 timotimo i could try
08:59 timotimo the way it works messes with my brain :P
09:07 jnthn :)
09:08 timotimo yeah, forgot to "make install"
09:08 timotimo that's not going to help
09:10 jnthn OK, will leave it with you for now and work on other bits
09:10 jnthn Though going to do my other MoarVM perf bits today in branches, because The Release Is Soon
09:14 * timotimo twiddles lots of < vs <=, + 1, - 1, ... :P
09:40 brrt joined #moarvm
10:10 timotimo i seem to have stumbled upon something that works. now to stress-test it.
10:16 * jnthn seems to have gotten a couple of seconds off Rakudo's build time
10:16 timotimo cool. it's not even parallelized! :)
10:18 timotimo i've not fixed the thing, but it at least doesn't invalid-read-of-size-1 any more
10:18 dalek MoarVM/jnthn/opts: ff710c7 | jnthn++ | src/ (6 files):
10:18 dalek MoarVM/jnthn/opts: Avoid VMNull setup memcpy/loop in spesh'd frames.
10:18 dalek MoarVM/jnthn/opts:
10:18 dalek MoarVM/jnthn/opts: Just shove in `null` instructions for all object registers when we
10:18 dalek MoarVM/jnthn/opts: compute the CFG, but before forming SSA. Then, let dead instruction
10:18 dalek MoarVM/jnthn/opts: detection rip out all that we don't need, so only read-before-use
10:18 dalek MoarVM/jnthn/opts: registers get the VMNull put in them, and we do nothing for the
10:18 dalek MoarVM/jnthn/opts: others.
10:18 dalek MoarVM/jnthn/opts: review: https://github.com/MoarVM/MoarVM/commit/ff710c7803
10:18 timotimo nice
10:39 arnsholt Yeah, definitely!
11:03 brrt \o/
11:03 brrt very clever... jnthn++
12:12 zakharyas joined #moarvm
12:13 dalek MoarVM/jnthn/opts: 6ada706 | jnthn++ | src/core/exceptions.c:
12:13 dalek MoarVM/jnthn/opts: Use existing function, to ease upcoming refactor.
12:13 dalek MoarVM/jnthn/opts: review: https://github.com/MoarVM/MoarVM/commit/6ada706b5d
12:40 * jnthn is really quite confused about how his Rakudo build is busted
12:40 jnthn Apparently, by above patch
12:41 timotimo the in_caller_chain uses the wrong tc perhaps?
12:41 * timotimo didn't look closely enough at all
12:41 nwc10 will test once valgrind has finished what it's currently doing
12:41 jnthn It doesn't look at tc at all
12:41 timotimo ah, hm
12:41 jnthn It's literally this:
12:41 jnthn static MVMuint8 in_caller_chain(MVMThreadContext *tc, MVMFrame *f_maybe) { return f_maybe->tc ? 1 : 0;
12:41 jnthn }
12:43 * timotimo shrugs
12:43 jnthn Going back one before that commit fixes things
12:43 timotimo must be some nondeterministic problem that just gets shuffled around?
12:44 jnthn Well, could be I guess but...wtf
12:44 timotimo yes wtf
12:47 nine jnthn: rakudo builds just fine here with that patch
12:48 jnthn um, and it did here too after I went back to it o.O
12:48 timotimo o_O
12:48 jnthn ...wonder if I had a really weirdly busted build
12:48 timotimo time to buy a new machine. oh and tear down your apartment building and build it anew
12:49 moritz yes, give the Computrons some time to recover
12:49 jnthn I think some repair guy did that to our boiler today and I'm not sure it's better off now :P
12:49 timotimo the magic smoke may have been shook too hard somehow?
13:24 jnthn It's really, really odd. A seemingly innocuous change causes a bizzare CORE.setting compilation failure
13:25 jnthn (Default invocation handler is not invokable)
13:25 nwc10 use more valgrind?
13:25 nwc10 or GC bug?
13:25 jnthn Disabling JIT/spesh doesn't hide it
13:25 jnthn Not sure.
13:25 jnthn It's hard to tell
13:28 dalek MoarVM/jnthn/opts: 8ed7ffc | jnthn++ | src/ (4 files):
13:28 dalek MoarVM/jnthn/opts: Stop caching MVMContext on a frame.
13:28 dalek MoarVM/jnthn/opts:
13:28 dalek MoarVM/jnthn/opts: We rarely request them anyway, and it should be fairly uncommon to
13:28 dalek MoarVM/jnthn/opts: ask for it more than once. And even then, it's cheap to create, and
13:28 dalek MoarVM/jnthn/opts: while it costs an extra allocation or so we already are using less
13:28 dalek MoarVM/jnthn/opts: space in the nursery for all the other frames (due to losing 8 bytes
13:28 dalek MoarVM/jnthn/opts: thanks to not needing the cache slot), so will very likely come out
13:28 dalek MoarVM/jnthn/opts: ahead even in those cases.
13:28 dalek MoarVM/jnthn/opts: review: https://github.com/MoarVM/MoarVM/commit/8ed7ffc728
13:28 jnthn That's the commit that triggers it
13:28 jnthn Building in my Linux VM too now, to see if it happens there
13:32 jnthn Yup, I get it there too
13:33 jnthn valgrind time
13:34 jnthn valgrind: the 'impossible' happened: Killed by fatal signal
13:34 jnthn o.O
13:36 masak jnthn -- makes the 'impossible' happen in code, sometimes in a good way! :P
13:37 jnthn It manages to spew this out:
13:37 jnthn Thread 1: status = VgTs_Runnable
13:37 jnthn ==915==    at 0xD2E5A05: ???
13:37 jnthn ==915==    by 0x502B668: MVM_jit_enter_code (in /home/jnthn/dev/MoarVM/install/lib/libmoar.so)
13:37 jnthn And the usual stuff below
13:38 jnthn Trying again with MVM_JIT_DISABLE=1...
13:38 jnthn But that didn't help locally
13:39 jnthn uh, off the VM I mean
13:41 jnthn Curious if anyone else building the branch gets a similar failure.
13:42 jnthn With JIT off, valgrind is happy...and it explodes
13:48 moritz building now
13:48 nwc10 Stage parse      : Default invocation handler is not invokable
13:48 nwc10 ASAN says nothing
13:48 jnthn Yup, that's the one
13:48 jnthn I struggle to see how the patch in question could lead to that...
13:50 moritz I get the same error as nwc10
13:52 jnthn Running both a valgrind with GC debug turned on and FSA debug turned on, and also a build on Windows with a small GC threshold, to see if either of them give a clue.
13:52 jnthn Really weird.
13:55 jnthn The first of those two didn't show anything different.
13:59 jnthn Nor did the second
14:00 JimmyZ How about disabled spesh too ?
14:01 jnthn Already tried that. No difference.
14:03 * JimmyZ tries build rakudo with moarvm master and only with that patch
14:03 jnthn That combo I didn't try
14:03 jnthn Without that patch things seem fine
14:04 JimmyZ ok build a core setting
14:04 JimmyZ but ./perl6-m tools/build/install-core-dist.pl /home/cloud/OpenSource/rakudo/../MoarVM/install/share/perl6
14:04 JimmyZ getlex: outer index out of range
14:05 jnthn hmm
14:06 JimmyZ hmm, looks like this error is not about that patch
14:06 JimmyZ without patch I also get this error
14:07 JimmyZ but I build a core setting with that patch
14:07 jnthn That's even more odd...
14:08 jnthn The only thing that branch has over Moar master is a spesh patch (but disabling spesh makes no difference whatsoever) and re-using an existing function that's really simple
14:13 jnthn Stranger yet, if I log when we re-use contexts while building CORE.setting, to see where we're allocating another one...that code doesn't seem to fire
14:14 jnthn Not until stage mast, anyway
14:15 jnthn ohhhh
14:15 jnthn Darn it.
14:15 jnthn Serialization relies on them being interned.
14:17 jnthn And that's why it seemed so action-at-a-distance
14:17 moritz serialization-at-a-distance
14:18 JimmyZ oh , that's why I buld a core setting, beacase i use a old *.moarvm code to build the CORE  setting.
14:19 JimmyZ I only rebuild the CORE setting.
14:19 JimmyZ and make realclean I got the same error
14:19 jnthn aha :)
14:19 jnthn yeah, guess that's why I had funny stuff before too
14:20 jnthn Interesting. I didn't realize serialization did that :)
14:20 jnthn Or forgot at least
14:21 JimmyZ and the other error I see because I use a new CORE setting :P
14:26 jnthn Now that frames themselves are MVMCollectable, they have an SC slot...so we could probably eliminate the use of the context wrapper altogether during serialization
14:29 jnthn We also do some linear scans of context lists
14:30 jnthn Seeing how long...
14:33 nine I'm all for everything that makes serialization simpler ;)
14:33 JimmyZ jnthn: the pacded SC + object_data_offset is about 1MB in the CORE setting size , FYI :)
14:34 JimmyZ *packed
14:35 JimmyZ which is use 8 byte per object
14:36 JimmyZ and it is big int and can't use varint
14:37 jnthn Yeah, 'cus we need to be able to index into that table :)
14:38 JimmyZ yeah, and another part about SC don't have a index
14:38 JimmyZ which uses varint now
14:39 jnthn iirc, we also only look at that table rather than copying it into memory
14:40 jnthn In which case it's mmap'd and shared over all processes
14:40 JimmyZ aye
14:40 jnthn So it's not quite so bad
14:40 JimmyZ yes, just FYI ;)
14:41 jnthn Yeah, it's interesting to know :)
14:41 jnthn In CORE.setting compilation, that thing I mentioned us linear scanning can get up to around 1600 things long
14:41 jnthn So not ideal but not a huge deal
14:42 jnthn But still, I'd not mind getting rid of using context wrappers there
14:58 jnthn Hm, seems like an overall simplification. :)
14:58 jnthn Now to see if it'll work
14:59 JimmyZ Segmentation fault
14:59 jnthn :P
15:00 jnthn It passed the serialization tests in the nqp repo at least :)
15:02 jnthn The NQP build is good, Rakudo one is working on CORE.setting
15:04 jnthn it made it :)
15:04 jnthn So question is...what if I try it on top of my context caching removal patch :)
15:07 JimmyZ still a good patch to push to master ;)
15:08 jnthn This lot will all make it into master shortly :)
15:08 jnthn Just not before the monthly release :)
15:08 jnthn Since they're all optimizatins, and commiting those on the eve of a release is kinda needless risk :)
15:08 jnthn *optimizations
15:08 JimmyZ yes
15:09 jnthn I think the Rakudo release is scheduled for tomorrow, so I'll probably do the Moar release this evening, 'cus I may end up a little busy tomorrow.
15:11 hoelzro jnthn: I thought the rakudo release was on saturday?
15:12 hoelzro I hope so, since I'm the release manager this time =)
15:12 hoelzro I wouldn't mind a MoarVM release as a head start, though!
15:12 jnthn hoelzro: oh... :)
15:12 jnthn Right, I still had Thursday in my head
15:12 jnthn If you're release manager then it's whenever you want it ;-)
15:13 jnthn I'll probably hold off the Moar release until Friday in that case, though today's changes can still wait until after it.
15:13 jnthn But gives chance to fix any urgent things noticed in the next day or so
15:13 jnthn Hopefully nothing :)
15:14 jnthn yay, everything built :)
15:27 hoelzro Friday works for me!
15:27 hoelzro I'll probably do most of the work for it on Friday night my time
15:27 hoelzro so very early Saturday for $everyone-else (-) $other-americans
15:33 tokuhirom joined #moarvm
15:48 dalek MoarVM/jnthn/opts: 6329eee | jnthn++ | src/6model/s (3 files):
15:48 dalek MoarVM/jnthn/opts: Use frames directly when serializing closures.
15:48 dalek MoarVM/jnthn/opts:
15:48 dalek MoarVM/jnthn/opts: Before, we obtained a context ref, just so we could mark the SC on it.
15:48 dalek MoarVM/jnthn/opts: However, now that frames are collectables we can mark an SC directly
15:48 dalek MoarVM/jnthn/opts: on them. This avoids a bunch of allocation and indirection. It also
15:48 dalek MoarVM/jnthn/opts: eliminates the need to have context objects be interned, which was
15:48 dalek MoarVM/jnthn/opts: removed in the previous patch.
15:48 dalek MoarVM/jnthn/opts: review: https://github.com/MoarVM/MoarVM/commit/6329eee1ae
15:50 jnthn m: say 9152 / 9176
15:50 camelia rakudo-moar e39ce3: OUTPUT«0.997384␤»
15:50 jnthn A little bit off NQP base memory
15:50 jnthn m: say 46808 - 46736
15:50 camelia rakudo-moar e39ce3: OUTPUT«72␤»
15:50 jnthn 72KB off Rakudo's too
15:51 jnthn But since it makes frames a little smaller it also means a tad less GC pressure also :)
15:51 jnthn s/also$//
15:58 jnthn Enough for now :)
16:34 tokuhirom joined #moarvm
17:03 domidumont joined #moarvm
17:59 nwc10 jnthn: I assume by its aloof silence that ASAN considers your fix to be tolerable.
18:02 nwc10 jnthn: I assume by its aloof silence that ASAN considers your fix to be tolerable.
18:02 nwc10 er, oops, that "up" was supposed to be for the other window (with the build)
18:49 vendethiel joined #moarvm
19:44 Ven joined #moarvm
19:58 camelia joined #moarvm
20:01 MadcapJake joined #moarvm
20:36 tokuhirom joined #moarvm
21:00 tokuhirom joined #moarvm
22:01 vendethiel joined #moarvm
22:37 vendethiel joined #moarvm
23:01 tokuhirom joined #moarvm
23:32 vendethiel joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary