Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-02-18

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:54 Geth joined #moarvm
01:38 agentzh joined #moarvm
02:18 agentzh joined #moarvm
02:48 ilbot3 joined #moarvm
02:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:53 ggoebel joined #moarvm
05:18 pyrimidine joined #moarvm
05:57 agentzh joined #moarvm
07:52 pyrimidine joined #moarvm
09:11 pyrimidine joined #moarvm
09:22 pyrimidine joined #moarvm
09:59 agentzh joined #moarvm
10:23 pyrimidine joined #moarvm
11:44 Geth ¦ MoarVM: 824644ef04 | (Jonathan Worthington)++ | docs/ChangeLog
11:44 Geth ¦ MoarVM: ChangeLog for 2017.02 release.
11:44 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/824644ef04
11:49 Geth ¦ MoarVM: f319a3659a | (Jonathan Worthington)++ | VERSION
11:49 Geth ¦ MoarVM: Bump VERSION.
11:49 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f319a3659a
11:52 timotimo the changelog looks kind of short, huh.
11:53 timotimo but the big crash-fixes don't get more than one line, so ... that's probably why
11:53 timotimo wait, jnthn
11:53 timotimo isn't the data section for jit code only in the exprjit?
11:53 timotimo or did i misunderstand the original commit?
11:54 jnthn No
11:54 jnthn It's in master
11:54 timotimo oh!
11:54 timotimo neat
11:54 jnthn The changelog comes from a git log output :)
11:54 jnthn It may be used more widely in exprjit, but it fixes a memory leak
11:54 jnthn (in master)
11:54 timotimo i didn't even know! awesome!
11:54 timotimo (about the leak)
11:55 jnthn Which is nice, 'cus I also nailed the other missing bit of Rakudo cleanup this month (in repossession)
11:55 jnthn So a perl6-valgrind-m -e '' is now clean with no missing cleanup reported :)
11:55 jnthn yay, clean spectest
11:55 timotimo <3
11:56 dogbert17 new release, cool
11:57 timotimo yup yup
11:58 jnthn https://www.moarvm.org/releases/MoarVM-2017.02.tar.gz
11:58 dogbert17 does any of you have any theories about the harness6 SEGV which was discussed yesterday?
11:59 dogbert17 or any ideas as how to debug this further
11:59 jnthn Been pondering how we might prove/disprove the double-release theory
11:59 jnthn Or hunt down how it happens
11:59 * dogbert17 listens
12:00 jnthn It occurred to me we could set a bit in the object header of decode streams when we free them
12:00 jnthn And then whine if we see that added to the worklist
12:00 jnthn (Something with that header bit set, that is)
12:00 dogbert17 like MVM_panic (for whining that is)
12:01 jnthn Yeah
12:01 jnthn Just like the other checks we do in add_to_worklist when GC debug mode is turned on
12:01 dogbert17 if that works do you think enough info will be available or is it 'too late' in the process
12:03 dogbert17 I don't know how/where to set that bit I can volunteer to test it
12:04 jnthn If it finds something, it'll be at the very least a further clue of where to look next
12:04 dogbert17 cool
12:06 dogbert17 so where should that fix be made (the extra bit and the whine :)
12:06 timotimo did you know that valgrind lets you find pointers to a memory location?
12:06 timotimo i think we can even do that from code
12:06 jnthn timotimo: No? Hmm :)
12:06 timotimo so we could do something insane like "when some place frees a pointer, find everything else that points at it"
12:06 timotimo ... "and set the lowest bit to 1 for later complaining"?
12:07 timotimo having that bit set is probably more trouble than it's worth?
12:07 jnthn Well, if there are any pointers to an object we are about to free then something is already wrong
12:07 timotimo true
12:08 timotimo okay, maybe you can only find things pointing to a location with the gdbserver part of valgrind
12:10 timotimo where the heck did i read about that feature? i can't seem to find it
12:11 timotimo i must have hallucinated that?!
12:11 travis-ci joined #moarvm
12:11 travis-ci MoarVM build errored. Jonathan Worthington 'ChangeLog for 2017.02 release.'
12:11 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/202912049 https://github.com/MoarVM/MoarVM/compare/542baec75630...824644ef04e5
12:11 travis-ci left #moarvm
12:12 timotimo bad changelog! bad, bad changelog!
12:17 dalek moarvm.org: ad92bd5 | jnthn++ | / (2 files):
12:17 dalek moarvm.org: Update website for release.
12:17 dalek moarvm.org: review: https://github.com/MoarVM/moarvm.org/commit/ad92bd5aca
12:17 yoleaux2 24 Jan 2017 15:38Z <AlexDaniel> dalek: Please switch this bot to Geth
12:17 yoleaux2 28 Jan 2017 16:40Z <AlexDaniel> dalek: Please switch this bot to Geth
12:18 jnthn https://www.moarvm.org/ # look, it's secure :)
12:19 jnthn Failure above is just a wedge while doing it pull, it seems
12:19 timotimo yeah
12:21 Geth ¦ moarvm.org: ad92bd5aca | jnthn++ | 2 files
12:21 Geth ¦ moarvm.org: Update website for release.
12:21 Geth ¦ moarvm.org: review: https://github.com/MoarVM/moarvm.org/commit/ad92bd5aca
12:21 jnthn There we got, another dalek straggler updated :)
12:21 jnthn *go
12:26 dogbert17 wrt the 'double-release' bug, I got a different MVM_backtrace yesterday, dunno if it provides any clues though: https://gist.github.com/dogbert17/6cb41a1dad22e7a80158ad8ee886bfe4
12:27 timotimo huh, that's not a double-free or anything, that straight-up crashes
12:28 timotimo you don't have that session or the core dump any more, do you?
12:28 dogbert17 nope :( but I can rerun and hope for the best
12:29 dogbert17 but yes, this was a SEGV
12:29 jnthn lunch; bbiab
12:30 moritz http://hhvm.com/blog/2017/02/17/region-jit.html
12:30 timotimo it'd be interesting to see what pointer was 0 (or whatever other bad address)
12:31 dogbert17 would that have been possible if I had the gdb session running?
12:33 dogbert17 timotimo: if you have some cool hardware nearby you try a 'make spectest HARNESS_TYPE=6 TEST_JOBS=5+'
12:33 timotimo yeah, sure
12:33 timotimo i will
12:34 timotimo you just attach gdb to the process via pid after starting it?
12:34 dogbert17 yes
12:34 Geth ¦ MoarVM/vmhealth: 60e3096605 | (Timo Paulssen)++ | src/moar.c
12:34 Geth ¦ MoarVM/vmhealth: vmhealth: guard against lack of event loop thread
12:34 Geth ¦ MoarVM/vmhealth: review: https://github.com/MoarVM/MoarVM/commit/60e3096605
12:35 timotimo when you run it via valgrind, do you do anything in particular to prevent the test files themselves to be valground?
12:38 dogbert17 haven't figured out a way to do that :(
12:39 timotimo hehe.
12:39 timotimo well, a clean way would be to allow the harness to get passed a different command to invoke than $*PROGRAM-NAMe or whatever it uses
12:40 timotimo i wanted to propose using "instrumentation off at program start", but i think that option is only available for callgrind, not for memcheck
12:46 dogbert17 I guess I'll start another run myself as well, can always do something alse while it runs
12:59 timotimo i put in a super evil hack
12:59 timotimo class SourceHandler::Perl6 does SourceHandler::Proc {
12:59 timotimo -        submethod BUILD(:@incdirs, Str:D :$!path = ~$*EXECUTABLE) {
12:59 timotimo +        submethod BUILD(:@incdirs, Str:D :$!path = ~$*EXECUTABLE ~~ s/-valgrind//) {
12:59 timotimo in lib/Tap.pm6
12:59 timotimo oops, wants quoting
13:00 timotimo takes a long time to reach the syntax error stage because it precompiles before doing anything much
13:04 timotimo no, it still runs all tests through valgrind
13:07 timotimo aha!
13:07 timotimo harness6 has a "perlpath" commandline argument
13:13 timotimo oh wow
13:13 timotimo it basically immediately exited with MoarVM panic: use of invalid eventloop work item index -1
13:14 timotimo but it shows 0 errors for all those processes it started ...
13:18 timotimo oh, it is because my perl path was wrong
13:18 timotimo the file doesn't exist and therefor it panics
13:18 timotimo pretty bad
13:19 Ulti joined #moarvm
13:19 Ulti left #moarvm
13:23 timotimo hah
13:23 timotimo it's sloooow
13:24 timotimo i have test jobs set to a high number, but apparently the harness is so slow that it only runs one process at a time :D
13:24 timotimo since valgrind causes threads to be serialized (if i recall correctly) it might be a better idea to reduce the number of test jobs a bit
13:30 timotimo i expect it still leads to a massive decrease in total cpu time used anyway
13:31 timotimo and hopefully it also still provokes the failures
13:35 dogbert17 I have been able to run many times with TEST-JOBS=2,3 without anything going wrong
13:38 timotimo OK
13:38 timotimo i'm at 6 right now
13:38 timotimo already is S14
13:38 timotimo in*
13:39 dogbert17 my first run with 5 worked without a hitch
13:39 dogbert17 insidious bug
13:50 * dogbert17 crash dammit
14:01 agentzh joined #moarvm
14:37 camelia joined #moarvm
15:05 zakharyas joined #moarvm
15:15 lizmat joined #moarvm
15:25 dogbert17 timotimo: got a SEGV again
15:27 dogbert17 https://gist.github.com/dogbert17/71bd542c007241aa10c45e0bf44dae61
16:19 timotimo how often has it been the decodestream so far?
16:20 dogbert17 timotimo: every time
16:20 timotimo OK, so next step might be to put debug stuff in there ...
16:21 dogbert17 any suggestions, i.e. what to look for
16:21 timotimo not really, no :(
16:21 dogbert17 and jnthn has fallen asleep :)
16:21 timotimo at the point of the free, most of the interesting data has been thrown away
16:21 pyrimidine joined #moarvm
16:21 timotimo (well, not really, just marked as "to be reclaimed", but still ...)
16:25 dogbert17 does that mean we have to back up a step on the stack, to gc_free?
16:27 timotimo i'm interested in the chunks that it deallocates there
16:27 timotimo these while loops follow a linked list of chunks that each hold a pointer to a buffer of bytes
16:28 dogbert17 yes
16:39 dogbert17 I wonder how often MVM_string_decodestream_destroy is called
16:39 moritz add a printf("42\n"), recompile, run, count :-)
16:40 timotimo yeah %)
16:40 timotimo also, be extra sure to only count it in the harness, not the testing processes
16:45 timotimo argh!
16:45 timotimo i got it to error, but tmux kept less than 2000 lines of scrollback around >:(
16:45 timotimo oh, that's just global destruction
16:45 timotimo OK, that's no problem
16:47 timotimo ok, turned off --full-cleanup and upgraded the backscroll to 4000 lines
16:47 timotimo well, i asked for 5000, but whatever
16:48 dogbert17 it's too often for comfort :)
17:50 ggoebel joined #moarvm
18:02 agentzh joined #moarvm
18:34 agentzh joined #moarvm
18:53 timotimo it's not crashing ...
18:59 dogbert17 quite annoying
19:01 dogbert17 are you using gdb or valgrind?
19:01 timotimo valgrind
19:02 dogbert17 what about ASAN?
19:03 timotimo *shrug*, could try that
19:03 timotimo in theory i could probably run one spec test per core
19:03 dogbert17 at least it's faster
19:05 timotimo uuuh, weird
19:05 timotimo when i pressed ctrl-z, it landed in some process, but not the main one
19:06 timotimo now i'm running two in parallel
19:06 dogbert17 weird
19:06 timotimo and a third for good measure
19:11 geekosaur odd. it should go to all of them
19:12 geekosaur hm, depends. all processes in the foreground process group, but if something is starting them in distinct process groups then all bets are off
19:13 timotimo :D
19:49 zakharyas joined #moarvm
19:49 pyrimidine joined #moarvm
21:54 timotimo ah, global destruction
21:54 timotimo curses!
21:55 timotimo i don't know why. it's not in the valgrind script .. oh!
21:55 timotimo i didn't copy that script, i don't think
21:59 timotimo since vmhealth is part of a pull request, i feel entitled to a rebase upon latest master
22:00 Geth ¦ MoarVM/vmhealth: 9 commits pushed by (Timo Paulssen)++
22:00 Geth ¦ MoarVM/vmhealth: 56644d6d3f | Count spesh_produced even without spesh_limit.
22:00 Geth ¦ MoarVM/vmhealth: 48c0ce30d9 | introduce vmhealth op
22:00 Geth ¦ MoarVM/vmhealth: 61c3bede1e | also report page counts and free items in thread's gen2
22:00 Geth ¦ MoarVM/vmhealth: be7899e5df | committing WIP on "blocked threads" and "per-stage" threads
22:00 Geth ¦ MoarVM/vmhealth: cf143392c3 | WIP: add timings for GC and also major gc count
22:00 Geth ¦ MoarVM/vmhealth: 0a56801ec2 | set maj/min flag in gc prof when finished
22:00 Geth ¦ MoarVM/vmhealth: e2449994a8 | WIP on threads-per-stage and threads blocked
22:00 Geth ¦ MoarVM/vmhealth: 808561bd53 | vmhealth: guard against lack of event loop thread
22:00 Geth ¦ MoarVM/vmhealth: a00f0a87d9 | hook up threads_per_state list to vmhealth hash
22:00 Geth ¦ MoarVM/vmhealth: review: https://github.com/MoarVM/MoarVM/compare/60e3096605...a00f0a87d9
22:53 timotimo what combinations of thread state and "is blocked for gc" do we have?
23:12 dogbert17 timotimo, jnthn: got it with ASAN, looks slightly different/better/worse: https://gist.github.com/dogbert17/eaba7dfc536b7a65114a52b1748fbe23
23:14 timotimo oh, huh, that's interesting
23:14 timotimo maybe we have some logic error in discarding bytes from a decodestream
23:14 timotimo like, maybe an off-by-one somewhere
23:15 timotimo or an edge case like when we had the memory regions for frames
23:16 dogbert17 looks quite suspicious, so part of decodestreams are freed outside of gc?
23:17 timotimo well, a decodestream owns a bunch of buffers
23:17 timotimo remember that linked list w etalked about?
23:19 dogbert17 yes
23:20 timotimo i would assume that's what got freed in discard_to first, then later in freeing the object itself
23:25 dogbert17 could be that something isn't nulled?
23:26 timotimo potentially
23:34 dogbert17 well at least it feels as if we're zooming in on the right piece of code
23:44 timotimo yup
23:45 timotimo i couldn't look closer, people around me were distracting me
23:47 timotimo ugh, internet connection is dropping all the time
23:47 dogbert17 uh oh
23:49 timotimo so yeah, you looking at this code, too?
23:49 timotimo line 76 of decode_stream.c?
23:50 dogbert17 I have it in front of me
23:50 dogbert17 that's the line
23:51 timotimo yeah
23:52 timotimo discard->bytes somehow survives
23:52 dogbert17 but how
23:53 timotimo yup
23:55 timotimo i mean, we can set it to a bogus value and see what happens

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary