Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-05-05

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:45 timotimo the first stab at teaching decont_* about get*ref_* shows me zero hope for both my particles benchmark as well as json_fast
00:45 timotimo but i do see two instances of the generated code trying to decont_n a wval
00:46 timotimo which can with a high probability be constant-folded, which would be nice
00:46 timotimo but somehow a goto became the writer of some register, which is ... clearly not what we want
00:47 dalek MoarVM: f51ddf8 | timotimo++ | src/spesh/optimize.c:
00:47 dalek MoarVM: missed optimizing decont_s and decont_u reprops
00:47 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f51ddf8b9e
00:47 dalek MoarVM: 1088538 | timotimo++ | src/spesh/facts.c:
00:47 dalek MoarVM: there were a bunch of "break" statements missing.
00:47 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/1088538fe4
00:47 timotimo those two commits were just sensible in general, though they give us practically nothing.
06:15 domidumont joined #moarvm
06:22 domidumont joined #moarvm
06:40 Ven joined #moarvm
06:51 zakharyas joined #moarvm
09:13 jnthn nine_: Code-reviewed your Moar PR :)
09:23 dalek MoarVM/reframe: 96c8b15 | jnthn++ | src/core/threadcontext.c:
09:23 dalek MoarVM/reframe: Initialize/destroy call stack region per thread.
09:24 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/96c8b15421
10:12 Ven joined #moarvm
10:42 brrt joined #moarvm
10:43 dalek MoarVM/reframe: 736a38f | jnthn++ | src/core/interp.c:
10:43 dalek MoarVM/reframe: Eliminate duplicate MVMContext creation code.
10:43 dalek MoarVM/reframe:
10:43 dalek MoarVM/reframe: Just use the function that does it, rather than repeating it a bunch
10:43 dalek MoarVM/reframe: of times in the interpreter.
10:43 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/736a38f808
10:53 brrt oh, hey, i have a better solution for the write barrier thingy
10:54 jnthn o/ brrt
10:54 brrt \o jnthn
10:54 brrt hmm, not a perfect solution though
10:54 dalek MoarVM/reframe: a129c2e | jnthn++ | src/ (7 files):
10:54 dalek MoarVM/reframe: Insert frame stack -> heap promotion where needed.
10:54 dalek MoarVM/reframe:
10:54 dalek MoarVM/reframe: We don't allocate any frames on the call stacks yet, and the code to
10:54 dalek MoarVM/reframe: actually move the frames is not yet implemented. This just inserts the
10:54 dalek MoarVM/reframe: calls to do the promotion, if needed, in (hopefully all of) the places
10:54 dalek MoarVM/reframe: where a GC-able object is about to come to reference a frame.
10:54 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/a129c2e9b3
10:58 brrt it does fix the SEGV in install-core-dist.pl, so that's something
10:59 dalek MoarVM/reframe: e910664 | jnthn++ | src/core/continuation.c:
10:59 dalek MoarVM/reframe: Remove out-dated comment.
10:59 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/e910664ba1
11:02 jnthn lunch &
11:11 dalek MoarVM/reframe-jit: 2a89927 | brrt++ | src/jit/emit_x64.dasc:
11:11 dalek MoarVM/reframe-jit: write-barriers should short-circuit
11:11 dalek MoarVM/reframe-jit:
11:11 dalek MoarVM/reframe-jit: This fixes the SEGV in rakudo/tools/install-core-dist.pl; it also
11:11 dalek MoarVM/reframe-jit: fixes the write barrier in bindlex, which requires both a check_wb
11:11 dalek MoarVM/reframe-jit: as a hit_wb.
11:11 dalek MoarVM/reframe-jit: review: https://github.com/MoarVM/MoarVM/commit/2a89927b1c
11:13 dalek MoarVM/reframe-jit: d7b2218 | brrt++ | src/jit/graph.c:
11:13 dalek MoarVM/reframe-jit: Fix declaration after code
11:13 dalek MoarVM/reframe-jit: review: https://github.com/MoarVM/MoarVM/commit/d7b2218e54
11:16 brrt slowly creeping towards correctness...
11:26 * brrt afk
12:07 Ven joined #moarvm
12:09 dalek MoarVM/reframe: 46a9395 | jnthn++ | src/core/frame.c:
12:09 dalek MoarVM/reframe: Remove now-unrequired zero assignments.
12:09 dalek MoarVM/reframe:
12:09 dalek MoarVM/reframe: We simply zero the frame as a whole now upon allocation, so these are
12:09 dalek MoarVM/reframe: no longer required.
12:09 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/46a9395344
12:15 domidumont joined #moarvm
12:39 dalek MoarVM/reframe: 096a45a | jnthn++ | src/core/ (3 files):
12:39 dalek MoarVM/reframe: Implement stack to heap promotion of frames.
12:39 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/096a45a278
12:42 dalek MoarVM/reframe: 7a12456 | jnthn++ | src/gc/roots.c:
12:42 dalek MoarVM/reframe: Tiny code simplification.
12:42 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/7a12456ffe
12:48 masak ooh, stack-to-heap promotion of frames
12:48 masak that sounds interesting
13:04 dalek MoarVM/reframe: f9fd956 | jnthn++ | src/gc/ (2 files):
13:04 dalek MoarVM/reframe: Update GC to be aware of frames on call stacks.
13:04 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/f9fd9561b2
13:06 JimmyZ jnthn: https://github.com/MoarVM/MoarVM/commit/096a45a278#diff-520b036258bc63b0834970478a950b64R604  # looks like forgetting clear .
13:07 dalek MoarVM/reframe: 775efea | jnthn++ | src/core/frame.c:
13:07 dalek MoarVM/reframe: Correct a couple of thinkos in promotion.
13:07 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/775efeabe2
13:07 jnthn JimmyZ: It was even wronger than that :)
13:22 jnthn ooh, sneaky, the active handler chain can get outdated
13:22 timotimo o/
13:35 dalek MoarVM/reframe: 920ec05 | jnthn++ | src/core/frame.c:
13:35 dalek MoarVM/reframe: Update active handlers when promoting frames.
13:35 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/920ec0519d
13:35 dalek MoarVM/reframe: 8af6f7e | jnthn++ | src/gc/roots.c:
13:35 dalek MoarVM/reframe: Add missing NULL check.
13:35 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/8af6f7eafb
13:40 jnthn Can build/test NQP successfully with new callstack thingy switched on
13:40 jnthn (Enabled in local commits)
13:43 jnthn And Rakudo gets to building its CORE.setting and...it gets up to allocating 8 GB before I killed it :(
13:43 timotimo uh oh
13:43 jnthn I didn't update perl6_ops.c, mind. Maybe something in there wants attention :)
13:46 timotimo *shrug* it'd be interesting to know how the allocation happens. like, is it stuck in a loop of some kind that just allocates and allocates?
14:03 jnthn Found a few places that perl6_ops.c needed updates :)
14:03 jnthn Trying now with those
14:10 dalek MoarVM/reframe: 69322c6 | jnthn++ | src/core/frame.h:
14:10 dalek MoarVM/reframe: Make a couple of functions public.
14:10 dalek MoarVM/reframe:
14:10 dalek MoarVM/reframe: So they can be used in fixing up Rakudo's extops.
14:10 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/69322c6cab
14:15 jnthn Didn't do it. But...uh...MVM_dump_backtrace(tc) is printing a heck of along stack trace
14:17 timotimo along of stack trace
14:17 jnthn a long :P
14:18 timotimo clearly we aren't popping stacks off the stack when we return out of code
14:18 jnthn We are
14:18 jnthn :)
14:18 jnthn If that didn't happen a lot more would be wrong :)
14:19 nwc10 jnthn: I think it might well plateau at 8Gb, as it completed for me
14:19 jnthn nwc10: You don't have my local patch that actually enables the new call stack, though :)
14:20 nwc10 ah right. good point.
14:21 jnthn oh, hah
14:21 jnthn But good to know all the changes *without* that work :)
14:22 jnthn oh...it's not what I suspected
14:22 jnthn Hmm
14:24 jnthn Seems that somehow it looks up $*ACTIONS and gets a bogus object
14:24 jnthn I thought that was going to be because of the dynlex cache holding onto a frame
14:24 jnthn But no
14:25 jnthn Anyways, somehow it seems to end up with a Perl6::Grammar instance instead of a Perl6::Actions instance and then recurses endlessly
14:25 jnthn Taht's the first guess from the stack trace, anyways.
14:26 timotimo does the dynlex cache have a simple on/off switch?
14:28 jnthn Dunno, but it's not clear if it's to blame
14:29 timotimo 'k
14:33 jnthn no
14:33 jnthn Somehow we have...a circlular call stack o.O
14:37 timotimo the wonders of modern computing architecture
14:40 jnthn Yeah, now to figure out how the heck that happens
14:40 jnthn tc->cur_frame->caller->caller->caller->caller->header.flags
14:40 jnthn 4
14:40 jnthn tc->cur_frame->caller->caller->caller->caller->caller->header.flags
14:40 jnthn 0
14:40 jnthn That should never happen :)
14:51 jnthn ooh, unlinlining is doing a naughty
15:03 dalek MoarVM/reframe: ead7190 | jnthn++ | src/spesh/deopt.c:
15:03 dalek MoarVM/reframe: Uphold no heap->stack pointers rule in deopt.
15:03 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/ead7190024
15:05 zakharyas joined #moarvm
15:08 jnthn That wasn't The Bug causing the hang though
15:09 jnthn In fact, totally disabling spesh doesn't unbust it
15:19 ggoebel114 joined #moarvm
15:20 nebuchadnezzar joined #moarvm
15:20 jnthn oh, I think it did fix the circular thingy...and now it runs and leaks a ton
15:22 jnthn Just forgot to free the ->work and ->env, that's all :P
15:26 jnthn OK, current status: NQP builds/test passes. Rakudo builds. :)
15:26 nwc10 it compiles? ship it!
15:26 jnthn A bunch of test fails, though they fail a sanity check I put in.
15:27 jnthn Or at least, the first I checked did
15:27 jnthn Well, recompiling 'cus I note I didn't compile some bits of Rakudo with said sanity check
15:33 dalek MoarVM/reframe: be0ca8f | jnthn++ | src/core/continuation.c:
15:33 dalek MoarVM/reframe: Missing stack->heap force in continuation invoke.
15:33 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/be0ca8f5d0
15:33 jnthn Better. Guess it's spectest time...
15:39 jnthn Some issues, but guess it's "close enough" so will flip it on in the branch for others to join in with the testing.
15:39 timotimo \o/
15:39 timotimo liking the sound of that very much
15:41 dalek MoarVM/reframe: 354ed2c | jnthn++ | src/core/frame.c:
15:41 dalek MoarVM/reframe: Allocate frames on the per-thread callstack.
15:41 dalek MoarVM/reframe:
15:41 dalek MoarVM/reframe: This means we get simple add/subtract the pointer cleanup of the
15:41 dalek MoarVM/reframe: frame memory itself. A few remaining issues, but gets through the
15:41 dalek MoarVM/reframe: NQP build/test and Rakudo build/test, with remaining few issues in
15:41 dalek MoarVM/reframe: the spectests.
15:41 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/354ed2c3b7
15:42 jnthn A "further down the line" thing will be moving ->env and ->work onto that callstack too, but that'll be problematic in a few places.
15:42 jnthn So want to get this part of it straightened out first.
15:43 timotimo that'd be swell :)
15:43 jnthn The weird thing is that I'm getting some hangs in spectests, but every moar process is stuck at zero CPU
15:44 dalek MoarVM/decont_assign_nativerefs: 3181590 | timotimo++ | src/spesh/facts.c:
15:44 dalek MoarVM/decont_assign_nativerefs: set KNOWN_BOX_SRC on all the native ref ops in fact discovery
15:44 dalek MoarVM/decont_assign_nativerefs: review: https://github.com/MoarVM/MoarVM/commit/31815905be
15:44 dalek MoarVM/decont_assign_nativerefs: a87daa3 | timotimo++ | src/spesh/optimize.c:
15:44 dalek MoarVM/decont_assign_nativerefs: try rewriting decont_* into set or getlex
15:44 dalek MoarVM/decont_assign_nativerefs:
15:44 dalek MoarVM/decont_assign_nativerefs: didn't manage to trigger it even once in user code, though,
15:45 dalek MoarVM/decont_assign_nativerefs: so 100% untested. also lots of debug spam.
15:45 dalek MoarVM/decont_assign_nativerefs: review: https://github.com/MoarVM/MoarVM/commit/a87daa30d5
15:45 nwc10 jnthn: before your fixes (and hence still on master) I can see one of the spectests hanging, with all threads sitting waiting for a mutex
15:45 nwc10 (peacefully, until the end of time)
15:46 jnthn I've got a test with a single thread that's sat waiting on a condvar...
15:46 jnthn oh wait, is this gonna be...
15:46 nwc10 so it might be that something like that is happening, to get everything idle but unending
15:52 jnthn I think it's that threads were premature-exiting
15:54 timotimo ./perl6-m tools/build/install-core-dist.pl /home/timo/perl6/install/share/perl6
15:54 timotimo Internal error: zeroed target thread ID in work pass
15:54 timotimo known?
15:54 dalek MoarVM/reframe: 4503bd6 | jnthn++ | src/gc/roots.c:
15:54 dalek MoarVM/reframe: tc->thread_entry_frame may be on the stack.
15:54 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/4503bd6b9c
15:54 dalek MoarVM/reframe: 5301caf | jnthn++ | src/core/frame.c:
15:54 dalek MoarVM/reframe: Update thread_entry_frame if promoted.
15:54 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/5301caffdb
15:55 jnthn timotimo: One of the above coulda potentially just fixed that.
15:55 timotimo doesn't seem so
15:55 jnthn But I didn't try make install yet
15:55 timotimo ah, ok
15:56 jnthn Seeing just a small handful of spectest fails
15:57 jnthn And one just gave "Internal error: invalid thread ID 107110992 in GC work pass"
15:57 timotimo that's more threads than we could start in a lifetime!
15:58 jnthn One explodes with Trying to unwind over wrong handler also, which is interesting.
16:02 timotimo i accidentally ran the primes code from the channel the other day with jit and it didn't asplode
16:03 timotimo oh btw, is there any sense in resolving the "real data"/"replaced" pointer of P6Opaque during gc runs? or do we perhaps already do that?
16:04 jnthn Resolving?
16:04 jnthn Surely we follow it when GC-marking?
16:04 timotimo making it not needed for future uses of the object, i mean
16:05 timotimo i don't 100% understand how the mechanism is implemented, fwiw :)
16:05 jnthn ah
16:05 timotimo but it seems like an indirection that could be removed when we copy the object and its data around anyway
16:05 jnthn maybe but...not sure it's common enough to be worth it
16:06 jnthn It'd need the GC to allocate it a bigger spot when moving it
16:06 timotimo OK, maybe i'll whip up a measurement tool for that
16:06 jnthn It's not clear how to factor that so it doesn't slow everything down for the sake of a few
16:22 jnthn Time for a break...can't concentrate any more...
16:23 jnthn But that unwind bug seems to be at the heart of many problems.
16:23 timotimo time for you to unwind a bit :)
16:43 domidumont joined #moarvm
18:23 dalek MoarVM/reframe: 830f246 | jnthn++ | src/gc/roots.c:
18:23 dalek MoarVM/reframe: Active handler frames may be on the call stack.
18:23 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/830f246d41
18:32 jnthn grr, the unwind bug is tricky indeed
18:33 jnthn Shows up when attempting to resume after an exception
18:33 jnthn And with a junk active handler record
18:35 jnthn Let's see if valgrind can give a hint...
18:38 timotimo food time /o/
18:40 nwc10 that's not the usual hint that valgrind gives
18:41 jnthn Yeah...it's output is more likely to send me to the beer fridge or the whisky cupboard...
18:54 jnthn Unfortunately, though, no additional insight from valgrind.
18:55 jnthn Seems the failing sanity check is catching the problem before there's a chance for corruption...
18:55 nwc10 it it one of the regular test scripts that you're running here?
18:56 jnthn t/spec/S10-packages/basic.rakudo.moar
18:56 jnthn Is the current one
18:56 nwc10 ah OK.
18:56 jnthn Though a number of them do it
18:56 nwc10 ASAN build not yet finished.
18:56 nwc10 have you got something locally enabled, or is the code in git good enough?
18:57 jnthn Nothing that should matter locally
18:58 jnthn I have https://gist.github.com/jnthn/38fbd0a7628465f79c3ea3386fce8fec
18:58 jnthn But not found anything that trips over those two yet
18:58 nwc10 ASAN goes SEGV
18:58 nwc10 regular boring SEGV
18:59 jnthn Oh?
18:59 nwc10 NULL pointer dereference
18:59 nwc10 #0 0x7f132a41f5f1 in MVM_interp_run src/core/interp.c:433
18:59 jnthn You've got MVM_JIT_DISABLE seet also?
18:59 nwc10 pants
18:59 nwc10 no
18:59 nwc10 not in that window
18:59 nwc10 thanks
18:59 jnthn :)
18:59 jnthn Yeah, I did the exact same thing 5 mins ago :)
19:01 nwc10 reaches the end of the test (I think)
19:01 nwc10 ok 81 - .WHO of nested subset definition stringifies to long name
19:01 nwc10 # FUDGED!
19:01 jnthn huh, so it does for me too on Linux
19:01 jnthn Boom with a GC error on Windows
19:02 nwc10 Odd, and annoying
19:02 jnthn t/spec/S03-sequence/nonnumeric.rakudo.moar I did get to blow up under valgrind
19:02 nwc10 (that it can't be replicated on Linux)
19:02 nwc10 ok 25 - 'A' ... 'ZZ' does not go on forever
19:02 nwc10 Trying to unwind over wrong handler
19:03 nwc10 no ASAN barfage
19:03 jnthn Right, and same on Windows too
19:03 jnthn Yeah :(
19:03 jnthn That'll be some fun
19:03 nine_ jnthn: thanks! Updated the pull request
19:03 nwc10 I'm not sure if I can be of much help in the next $soon, so I might be AFK for a bit
19:04 jnthn Yeah, I'm getting tired to the point of totally useless for today also :)
19:04 jnthn Though glad that the refactor got far enough along today that we're already onto worrying about spectests.
19:05 jnthn Disabling the fixed size allocator didn't make things any differently faily
19:06 jnthn Mildly suspect continuations given where it fails, but can't see anything obviously missed.
19:08 * jnthn gives up for the day
19:09 jnthn nine_: Cool, I'll look tomorrow :)
19:09 nine_ Thanks :) Have a very nice rest :)
19:09 jnthn Thanks! :)
19:14 brrt joined #moarvm
19:15 brrt oh wait, jnthn, you mean that ->work can move too?
19:15 jnthn brrt: Not in this set of refactors, no
19:16 brrt hmmmm... how much further down the line will that be, then?
19:16 brrt also, nwc10, where do you get a SEGV with jit enabled... oh wait, you probably don't run reframe-jit
19:17 brrt ehm, we already compile with -fno-omit-frame-pointer
19:18 brrt ok, that's interesting to know
19:18 jnthn brrt: Not sure, but when it happens it will only be a small handful of ops where it might happen.
19:18 brrt hmmm...kay
19:18 jnthn Rather than with FRAME where anything that allocates can make it happen.
19:18 brrt we'll add a check to those ops, then
19:18 jnthn Yeah
19:19 jnthn Invocation and continuation invocation are two of the paths, when we're already falling out of the JITted code.
19:19 brrt oh... of course
19:20 brrt what about making a continuation?
19:20 jnthn reset, or control?
19:21 jnthn reset no, control yes
19:21 brrt from the current frame? i'm fuzzy on continuations
19:21 jnthn Ah, there's 3 operations
19:21 jnthn reset = mark the root of the continuation, control = take it (slicing everything off the stack down to the reset), invoke = restore those things we took onto the current stack
19:22 brrt aha
19:23 * jnthn goes for a walk before he gets too tired to do that also :)
19:40 brrt negative jit entry label, again
19:42 moritz maybe it's a synthetic codepoint instruction :-)
19:42 brrt more probably, it's a jit entry label that doesn't belong to the function
19:42 brrt question is, how does that happen
19:45 zakharyas joined #moarvm
19:48 nebuchadnezzar joined #moarvm
19:49 brrt answer, we're really in the callers jit code
19:49 brrt so evidently something went wrong somewhere
19:52 brrt sequence number 924
19:52 brrt wow....
19:52 brrt 924 frames before this goes wrong
19:52 brrt figure that
19:56 brrt and there's the thing; our callee is seq nr 405
19:59 brrt .. interesting
19:59 brrt plot thickens
19:59 brrt but, that's for tomorrow
20:56 timotimo huh, i don't see any assign_i, assign_n or assign_s be emitted ...
20:57 timotimo ah, ok, in json_fast it does happen
20:58 timotimo but the writers of those registers turn out to only getlex
20:58 timotimo to only be getlexes, i mean
20:58 timotimo and that's not something spesh could improve, i don't think
21:17 avar joined #moarvm
21:21 avar joined #moarvm
21:23 Ven joined #moarvm
23:44 dalek MoarVM/decont_assign_nativerefs: 154eb7b | timotimo++ | src/spesh/optimize.c:
23:44 dalek MoarVM/decont_assign_nativerefs: copy-paste of try_optimize_decont_native for assign_* ops
23:44 dalek MoarVM/decont_assign_nativerefs:
23:44 dalek MoarVM/decont_assign_nativerefs: doesn't do anything, also doesn't trigger often.
23:44 dalek MoarVM/decont_assign_nativerefs: review: https://github.com/MoarVM/MoarVM/commit/154eb7be8a

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary