Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2014-08-17

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:24 avuserow joined #moarvm
01:26 FROGGS_ joined #moarvm
01:48 ilbot3 joined #moarvm
01:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:12 klaas-janstol joined #moarvm
06:06 timotimo speshing iscont ends up with sad spectests :\
06:13 dalek MoarVM: 5dccd58 | (Timo Paulssen)++ | src/jit/ (2 files):
06:13 dalek MoarVM: add an impl of callercode to the jit
06:13 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/5dccd5808d
06:13 timotimo spectests with this are clean :)
06:14 timotimo (except for the usual)
06:44 timotimo my implementation of objprimspec seems to lead to a segfault; gdb's disassemble command doesn't seem helpful
06:49 timotimo https://gist.github.com/timo/a9c5a2476b63a9afe842
06:50 timotimo i don't see anything wrong with it, which doesn't mean much :)
07:25 brrt joined #moarvm
07:25 brrt timotimo: ARG1 might be TMP1, but it might not
07:25 timotimo oh!
07:26 brrt depending on ABI
07:26 timotimo i didn't know that they are double-booked
07:26 brrt TMP5 == FUNCTION
07:26 brrt TMP6 == always free :-)
07:26 timotimo OK, let me try re-naming
07:26 brrt well, i should comment it
07:26 timotimo how many can i freely use?
07:31 brrt TMP5 and TMP6 are safe from the ARG1,2,3,4 set
07:31 brrt ARG5 and ARG6 are /not defined/ on windows, so you shouldn't use them in hand-written code
07:31 brrt RV is also safe, so if you can't deal with it in any other way, you could use that
07:46 timotimo actually, i'm going to commute for about 1.5h or so
07:47 timotimo if you want, you can feel free to adapt the code to use different registers and commit&push that if it works
07:47 timotimo but if not, feel free to work on whatever :)
07:51 brrt oh, yeah, i'll change it and test it
08:02 brrt uhm, darn
08:03 brrt get_storage_spec returns an object (11 bytes wide as far as i can tell) not a pointer
08:12 brrt can i change the signature for that?
08:20 brrt i'd like to pass a pointer directly
08:24 brrt also, it crashes :-(
08:25 * brrt wonders why we suddenly have so many issues
08:25 brrt what limit are we reaching?
08:28 dalek MoarVM/moar-jit: b1b7dc9 | (Bart Wiegmans)++ | src/jit/ (2 files):
08:28 dalek MoarVM/moar-jit: Add an implementation of objprimspec.
08:28 dalek MoarVM/moar-jit:
08:28 dalek MoarVM/moar-jit: This add the space for the struct as a third argument.
08:28 dalek MoarVM/moar-jit: I'd like to change the get_storage_spec signature to
08:28 dalek MoarVM/moar-jit: make this formal so it'll work on all platforms.
08:28 dalek MoarVM/moar-jit:
08:28 dalek MoarVM/moar-jit: rakudo build fails with this patch.
08:28 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/b1b7dc988d
08:53 brrt joined #moarvm
09:11 nwc10 brrt: moar-jit causes ASAN barfage in Rakudo setting
09:13 nwc10 hangon, PEBKAC. Barfage is real, but I'm not sure what version I'm building
09:14 nwc10 bother, no, it's master, but with JIT
09:14 nwc10 This is MoarVM version 2014.07-407-g5dccd58 built with JIT support
09:17 nwc10 OK, I can't repeat it.
09:17 nwc10 that's not normal
09:18 brrt ugh
09:18 brrt it's not surpising either
09:18 nwc10 http://paste.scsys.co.uk/416285
09:18 nwc10 retrying with a complete clean build
09:19 cognome joined #moarvm
09:23 brrt uhm, findmethod is allocating?
09:23 nwc10 can reproduce with a clean  build
09:24 nwc10 I don't know enough to answer that
09:29 brrt yeah, it does
09:35 dalek MoarVM/moar-jit: 2a4ce16 | (Bart Wiegmans)++ | src/jit/ (2 files):
09:35 dalek MoarVM/moar-jit: Read only word of storagespec. Doesn't work at all
09:35 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/2a4ce16219
09:38 brrt seems like that frames misses a refcount
09:49 brrt we've got issues, man
09:49 nwc10 MoarVM master^ is unhappy
09:50 brrt yeah
09:55 brrt i'm unhappy, i might add
09:58 FROGGS jnthn: here is what valgrind thinks about my problem: https://gist.github.com/FROGGS/446932521a4283eaca71
09:59 FROGGS errrm
10:00 FROGGS jnthn: this makes me think we add cu to gen2 twice: https://github.com/MoarVM/MoarVM/blob/master/src/core/compunit.c#L11
10:03 FROGGS yeah, I am pretty sure that patch is wrong
10:03 FROGGS question is: did it really help to solve an issue?
10:04 brrt yeah, it did, CU was moved in during allocations
10:05 brrt and the JIT continued to look at the wrong CU
10:05 FROGGS hmmm
10:07 klaas-janstol joined #moarvm
10:14 nwc10 2014.07-405-g669a171 looks to be good
10:31 brrt which one is that?
10:37 brrt such mystery, much unhappy
10:40 brrt also.. i can't repeat it now, maybe i can repeat it some other time (yay)
10:43 brrt can assign be invokish? i may have asked this before
10:43 brrt since decont can be
10:45 brrt and the cur_op is incremented before calling store
10:45 brrt yeah, assign can be invokish
10:45 brrt ok
10:45 brrt we probably need to change that
10:55 brrt i think it's fair to say that made a lot of diffference
10:56 brrt since well, not dealing invokish well can lead to freaky states
10:56 brrt btw, can make spectest be parralelized?
11:01 dalek MoarVM/moar-jit: 85b3b14 | (Bart Wiegmans)++ | src/ (3 files):
11:01 dalek MoarVM/moar-jit: Made assign invokish
11:01 dalek MoarVM/moar-jit:
11:01 dalek MoarVM/moar-jit: Not dealing with invokishness makes for some freaky states.
11:01 dalek MoarVM/moar-jit: Enabled iscont, disabled objprimspec. Spectests pass for me
11:01 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/85b3b14417
11:01 lizmat brrt: I have TEST_JOBS=8 in my ENV to parallelize spectest
11:02 lizmat I *think* you should be able to do the same with the hardware you have :-)
11:04 brrt yeah, i think so too
11:04 brrt although i usually limit it to 4
11:05 brrt but, spectests are clean for me
11:05 brrt anyway, lunch &
11:20 sergot hi o/
11:24 klaas-janstol joined #moarvm
11:38 brrt joined #moarvm
11:38 brrt \o sergot
12:16 brrt basically, from a jit perspective, returning structs between 8 and 16 bytes wide are a major pain
12:16 brrt with some emphasis on /major/
12:16 jnthn brrt: I'm guessing storage_spec is the pain here?
12:16 brrt yes
12:17 brrt i'm busy changing that into void (ThreadContext*, MVMStable*, MVMStorageSpec *s)
12:18 brrt or rather, changing that, and i'm thinking 'what if we return the pointer as well as having it passed, that'd be handy in functions that take the result directly
12:18 brrt note that there is virtually no difference between the earlier signature and the win64 behavior
12:20 dalek MoarVM: eaf46f5 | jnthn++ | src/ (2 files):
12:20 dalek MoarVM: Avoid bogus mark_special_return_data calls.
12:20 dalek MoarVM:
12:20 dalek MoarVM: Hopefully fixes use-after-free found by nwc10++ with ASAN.
12:20 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/eaf46f585c
12:21 jnthn brrt: I wonder if we should just return a pointer always. (more)
12:21 jnthn For some REPRs the data is basically constant, so we can just have it as static data and return its address.
12:21 jnthn For the REPRs more complex than that, we can hang it off their REPR_data struct and build it just once
12:22 jnthn That'd save building it each time also.
12:23 brrt jnthn: note that the use-after-free can potentially also be caused by assign being invokey and this being not handled
12:24 brrt hmm
12:24 jnthn brrt: It showed up in code I added yesterday
12:24 jnthn brrt: I assume we're considering findmethod invokey already?
12:24 brrt ehm... yeah, but i should check
12:25 brrt ehm, no
12:25 brrt not invokish
12:25 brrt probably lost in merge?
12:25 brrt wtf
12:25 jnthn findmeth            w(obj) r(obj) str :pure :invokish
12:25 jnthn findmeth_s          w(obj) r(obj) r(str) :pure :invokish
12:25 jnthn That's in master
12:25 brrt hmm let me see, i may be in the wrong branch
12:25 jnthn And master also has:
12:25 jnthn assign              r(obj) r(obj)
12:25 jnthn assignunchecked     r(obj) r(obj)
12:25 jnthn Both of those are invokish
12:26 brrt sp_findmeth in master?
12:26 brrt yeah i fixed that in moar-jit
12:26 brrt i should probably merge that back
12:26 jnthn sp_findmeth      .s w(obj) r(obj) str sslot :pure
12:26 jnthn That wants marking invokish too
12:26 brrt uh, that's invokish indeed
12:26 jnthn Though that one is interesting
12:26 brrt that wants a refactors
12:26 jnthn In so far as if we hit the monomorph cache we can jump straight over the invokish guard too.
12:26 brrt refactor
12:26 zakharyas joined #moarvm
12:26 brrt hmm yeah
12:26 jnthn So we can save the check.
12:27 brrt wait
12:27 brrt sp_findmeth is implemented in such a way as to deal with the invokish itself
12:27 brrt as in MVM_6model_find_method_spesh returns 1 if invokish, 0 if not
12:27 brrt so yeah
12:27 brrt it does what you suggest
12:27 jnthn OK, cool :)
12:27 brrt i should comment it in the oplist, though
12:27 jnthn So we needn't mark it invokish? :)
12:28 brrt no
12:28 jnthn ok
12:28 brrt as in, it won't do any harm, but it won't do any good either
12:28 jnthn Heh, maybe the mark is as good as the comment ;)
12:29 jnthn I guess if_o and unless_o already have their invoke potential handled by the desugaring?
12:32 jnthn assertparamcheck    r(int64) :noinline
12:32 jnthn I think I saw that mentioned recently. It's invokish.
12:33 brrt yeah
12:33 brrt oh is it?
12:33 jnthn yeah. That's what it does
12:33 jnthn If the bind fails, it invokes something to produce and throw a decent error.
12:34 jnthn We often eliminate it during spesh.
12:34 brrt yeah, i see
12:34 brrt if it pre-increments cur_op, good chance it's invokish :-)
12:34 jnthn Some of the continuation ones will be invokish (control and invoke)
12:34 jnthn But I don't think we've tried to JIT those yet :)
12:34 jnthn They'd just be calls anyway.
12:35 jnthn Think that's all I see wrt invokish marks.
12:39 brrt ok, merged moar-jit nqp tests pass, at least
12:39 jnthn :)
12:42 brrt parrallel tests make a lot of difference
12:42 FROGGS jnthn: please backlog here when you have time
12:44 jnthn FROGGS: Did already; that's where I found the double-free thing I fixed a moment ago...
12:44 jnthn FROGGS: Sadly, the ASAN output for the bug you're seeing is a bit less informative about the problem there :(
12:46 jnthn As for "adds it twice": that's not really what's going on here. MVM_gc_root_gen2_add isn't about allocating in gen2, it's about saying "this gen2 object may point to nursery objects"
12:46 FROGGS hmmm
12:49 dalek MoarVM: 7dad12b | (Bart Wiegmans)++ | src/jit/ (2 files):
12:49 dalek MoarVM: Commit branching iscont as well as extra labels
12:49 dalek MoarVM:
12:49 dalek MoarVM: Both these changes break spectest / nqp. So something is off.
12:49 dalek MoarVM: Commit on moar-jit for the time being.
12:50 dalek joined #moarvm
12:50 brrt adding a few extra labels, btw, still breaks about everything
12:50 brrt but why? no idea
12:53 jnthn FROGGS: My guess was that somewhere we'd be not properly protecting storing an SC reference in the CU, e.g. with MVM_ASSIGN_REF. But having looked through all the places that happens, I can't find anywhere it's an issue. :S
12:53 brrt on the good news side, we now have both iscont and assign
12:53 jnthn (There's actually only a few places.)
12:53 FROGGS :/
13:12 ggoebel11111110 joined #moarvm
13:27 * brrt is starting on the get_storage_spec thingy
13:28 timotimo o/
13:37 timotimo brrt: i assume you're aware of what the "firm pencils down" and "1. final evaluation deadline" mean?
13:40 timotimo (because i forgot the exact stuff again, but i know where to look)
13:40 jnthn timotimo: I think "firm pencils down" means "any work beyond this point can't be taken into account for GSoC evaluation", not "please stop contributing" :)
13:41 timotimo *that* part i recall vividly :P
13:43 jnthn The key thing is that on 19th, 20th, or 21st, brrt and I ensure we file evaluations.
13:44 jnthn And on that a little beyond that, some code samples are submitted.
13:44 jnthn s/on that//
13:44 timotimo good, i see you already have this thing planned out perfectly and will likely not require further semi-educated nagging
13:45 brrt eh, yeah, i'm aware of what it mean
13:46 timotimo while digging a few finger's widths into the jit code, i thought at first that making the register allocation stuff will be super hard, then realized it may actually end up being somewhat easy
13:47 jnthn I think brrt++ and I will be in the same place when the eval deadline arrives, anyway. :)
13:47 brrt means
13:48 timotimo i'm sure i'll actually stumble upon the thing that makes it seem not easy if i do any more work inside the emit.dasc file
13:49 brrt basically, we need to add stuff to dynasm to make it work, and i haven't gotten too that yet
13:49 brrt and by 'stuff', i mean 'select all registers on x64 dynamically'
13:49 timotimo yup, my evaluation of the situation presumed that feature already existing
13:50 brrt ah
13:50 brrt well, yeah, i agree
13:50 brrt when that happens - sketching ahead into the future right now - we'll probably make some very simple nodes like conditionals and stuff
13:51 timotimo to make register allocation work properly across jump borders?
13:51 brrt and try to move calls that are now implemented in asm back into the graph
13:51 brrt no, ehm, to be honest, that would require some work with the ssa form rather than the original registers i think
13:52 timotimo OK
13:52 brrt but i haven't really thought that out
13:52 brrt yet
13:53 timotimo so we have proper working iscont and assign (and assignunchecked i guess too?) on moarvm's master branch?
14:05 brrt yeah
14:05 brrt it seems that way
14:05 brrt and i'm working on objprimspec
14:06 brrt but that means refactoring get_storage_spec
14:06 brrt which has /many/ implementations
14:06 brrt so that's taking a while
14:06 brrt (bbiab, errands &)
14:06 timotimo oh well :\
15:21 nebuchadnezzar joined #moarvm
15:27 brrt joined #moarvm
15:33 nwc10 jnthn: sadly not http://paste.scsys.co.uk/416320
15:35 nwc10 http://paste.scsys.co.uk/416321
15:54 nebuchadnezzar joined #moarvm
16:21 zakharyas1 joined #moarvm
16:27 brrt joined #moarvm
16:27 brrt left #moarvm
18:06 brrt joined #moarvm
18:32 timotimo i'd like to build something like "jvisualvm" for moarvm, some program to give access to all kinds of stats, performance numbers, spesh/jit dumps, ...
18:33 timotimo is there a known design pattern/thingie that'll allow me to put data gathering stuff into moarvm without impeding non-instrumented execution?
18:34 diakopter my thought (and parrot's technique) was to render specialized interpreter "runcores"
18:35 timotimo right, moarvm already has this trace thingie that'd put an ifdefd piece of code into the beginning of the interpreter loop
18:37 timotimo i could build something based on ptrace or something similar, but that'd be linux-only
18:37 diakopter right, but you could munge the file to have several versions of the same interpreter loop in the same binary
18:37 timotimo and using a gdb from python is ridiculously slow (for example the code that scans the gen2 and nursery for types and such)
18:37 timotimo i could; it'd be more or less copy-pasting the whole interpreter loop, though :\
18:38 diakopter yeah, but it's fine if you don't mind a few hundred more KB in the binary for such frankenbuilds
18:38 timotimo IIUC the "local state" in the interpreter loop isn't too complex, so you could even imagine longjmping from a normal interpreter loop into an instrumented one back and forth
18:38 diakopter obviously it wouldn't be the default to build more than one variant
18:38 timotimo of course
18:38 timotimo do you know something that'd make the copy-pasting stuff automated and somewhat robust-ish?
18:39 diakopter methinks a perl script could do it
18:39 diakopter robust? what's that?
18:39 timotimo :))
18:39 timotimo AFK
18:43 diakopter if you *really* wanted to, you could put the loop variants in the same routine so you could jump between them as you crossed tracing/non-tracing boundaries
18:43 diakopter you'd want special instructions/ops to switch runcores :)
18:43 diakopter (so you didn't have to run a check with each instruction)
18:45 timotimo aye, thought of that as well
18:47 jnthn timotimo: I've a few ideas on this, though am a bit tied up worrying about fixing bugs and doing my YAPC::EU talks and also preparing to be away from home a while these next days...could happily look at it together in a few days...
18:48 timotimo but then i won't have the "lazy sunday evening" effect working for me any more ;))
18:48 timotimo i think you once said spesh would be a fantastic opportunity to do instrumentation of regular bytecode
18:48 jnthn Yes
18:49 jnthn I still think that'd be a decent approach.
18:49 timotimo in principle we could also change spesh to just spesh *every single* frame and instead of optimizing stuff it'd put instrumentation in
18:50 timotimo hmm. but maybe for instrumentation it'd be enough to just instrument frames that end up being speshd in the regular way
18:50 timotimo as in: "hot" frames
18:52 timotimo i wonder if i can use hardware breakpoints to trap calls to the relevant pieces of the gc
18:55 timotimo hmm, with CGOTO there's not really a single spot where i could plop a function call that'd check for MVM-bytecode-level breakpoints
18:56 diakopter in the macro itself?
18:57 diakopter OP?
18:58 timotimo oh
18:58 timotimo that's actually a better idea than at the goto NEXT spot.
18:59 zakharyas joined #moarvm
18:59 diakopter that's not to say I'm supportive of such checking for breakpoints; I think dynamically injecting the special ops is a better idea (as you mentioned doing with spesh, and as my earlier idea was assuming you'd have to do)
19:03 timotimo hm, so every annotation for a line number would get a little "line number changed" op called that'd be hooked into for things
19:04 diakopter maybe, but why not use those markers instead to just know where to inject your "breakpointbreak" op
19:04 diakopter that reminds me to watch Point Break again.
19:04 diakopter afk&
19:22 nwc10 jnthn: I can see why ASAN is still barfing. I'm not sure what the right fix is
19:24 nwc10 there's a GC run while attempting to throw the exception in late_bound_find_method_return(), and the GC mark routine assumes that frame->special_return_data is still pointing to something valid, because mark_special_return_data is not 0
19:24 nwc10 somehow the free-ing and the NULL-ing need to end up in the same function
19:30 jnthn ah...
19:31 jnthn Or we NULL before we ever call the function...
19:31 jnthn Since we're passing the data into it anyway
19:31 jnthn And the only reasonable thing for it to do is clear up
19:31 jnthn Since it's not like it can be called again.
19:32 brrt ahah... reprs can be assigned from deserialization too
19:32 brrt much great
19:34 brrt ugh
19:34 brrt this is harder than it looks
19:34 brrt (this = refactoring all get_storage_specs)
19:35 timotimo got a hot tip how i could instrumentalize the GC to collect stats like how long does it take, how many objects (of what sizes) got promoted to {the next nursery, gen2}, what kind of allocations cause gc runs most often, ...
19:37 timotimo hm. i should probably look into perf some more.
19:37 timotimo it may just be able to do half of that stuff by itself
19:38 nwc10 jnthn: if that works, yes, sounds right. I wasn't sure if it was how it worked
19:41 timotimo oh wow
19:41 timotimo i can just put a perf probe into any currently running program
19:41 timotimo all it needs is debug symbols, it seems
19:45 FROGGS Program received signal SIGSEGV, Segmentation fault.
19:45 FROGGS 0x00007ffff79c78ec in gc_mark (tc=0x603650, st=<optimized out>, data=<optimized out>, worklist=0x379d700) at src/6model/reprs/SCRef.c:75
19:45 FROGGS 75        MVM_gc_worklist_add(tc, worklist, &(sc->sr->root.sc));
19:45 timotimo interesting, there's a -ggdb flag for gcc
19:45 FROGGS sc->sr->root is accessible, sc->sr->root.sc points to invalid mem
19:45 jnthn ooh
19:45 jnthn That's a good find
19:45 FROGGS coincidence :o)
19:46 nwc10 same bug, or different?
19:46 jnthn Will look in a moment, trying my second attempt at fixing the nwc10++ reported thing first... :)
19:46 FROGGS nwc10: related to what I saw yesterday me thinks
19:46 FROGGS :o)
19:47 itz_ joined #moarvm
19:49 brrt y have i an all zero p6num repr_data
19:49 brrt where are repr_data's created
19:52 jnthn brrt: Allocated in compose or deserialize_repr_data normally
19:52 brrt hmmm
19:53 brrt doesn't seem like they're called
19:55 jnthn It's possible (though maybe a bit odd) that get_storage_spec somehow is called before those...
19:55 jnthn I think get_storage_spec has a code-path to handle that
19:55 jnthn And hands back a fairly "default" answer in that case.
19:56 brrt yeah
19:56 brrt but still i get a p6num with a zeroed-out storage spec
19:56 brrt which is just
19:56 brrt weird
19:56 brrt imho
19:58 jnthn It is a bit
20:00 dalek MoarVM: 92e3f76 | jnthn++ | src/core/frame.c:
20:00 dalek MoarVM: Clear special return data more eagerly.
20:00 dalek MoarVM:
20:00 dalek MoarVM: This prevents access to it if the special return handler frees it,
20:00 dalek MoarVM: then goes on to do something that allocates nwc10++ for identifying
20:00 dalek MoarVM: the problem.
20:00 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/92e3f764ad
20:00 jnthn I don't quite see how that can happen.
20:00 jnthn Apart from a serialization oddity
20:00 jnthn Like, we're asked for the storage spec before having deserialized.
20:01 jnthn Do you know what is asking for the storage spec at this point?
20:01 dalek MoarVM/moar-jit: cb9e151 | jnthn++ | src/6model/6model.c:
20:01 dalek MoarVM/moar-jit: Complain properly about missing late-bound methods.
20:01 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/cb9e151ecf
20:01 dalek MoarVM/moar-jit: 5dccd58 | (Timo Paulssen)++ | src/jit/ (2 files):
20:01 dalek MoarVM/moar-jit: add an impl of callercode to the jit
20:01 brrt yeah, smart numify
20:01 brrt the thing is in fact deserialzed ahead of time
20:02 jnthn Ah
20:02 dalek joined #moarvm
20:02 jnthn Then...odd.
20:02 brrt yes
20:02 brrt anyway, this should be the bulk of the work, i'll investigate the rest tomorrow
20:02 brrt good night :-)
20:02 jnthn 'night o/
20:02 timotimo jnthn: i can just inject a probe into any given process and then i can use perf to get counters for that!
20:02 brrt left #moarvm
20:03 timotimo like, i can select a function and then an intra-function spot (like @return)
20:06 timotimo or per line number
20:06 jnthn timotimo: Sounds useful to know...
20:06 timotimo i just don't know what to try it out on ...
20:07 jnthn Full GC collects? Deopts? Lexical vivifies? :)
20:08 timotimo mhm, mhm
20:19 timotimo bugger, i have to do these as root it seems
20:19 timotimo otherwise i get a permission denied for the trace events i added
20:19 timotimo which is understandable, i guess
20:24 timotimo oooooh!
20:25 timotimo perf_events has JIT support to solve this, which requires the VM to maintain a /tmp/perf-PID.map file for symbol translation. Java can do this with perf-map-agent, and Node.js 0.11.13+ with --perf_basic_prof. I'll write up instructions for these when I get a chance.
20:49 timotimo perf also has a "trace" command that is like strace, but with less overhead and prettier output
20:51 timotimo *wow*, you can apparently ask perf to give you a list of variables for a given functions and then you can trace the values ...
20:57 jnthn o.O
20:58 timotimo like ... wow.
20:59 timotimo so with a simple set of commands we can get a histogram of what sizes get allocated in the nursery or gen2 or fixed-size-allocator or ... stuff
21:01 timotimo i don't know exactly what this perf map is supposed to look like on the inside
21:01 timotimo oh, it's just "startaddr size name"
21:02 jnthn FROGGS: Am currently ding a v5 build
21:02 FROGGS ohh
21:02 jnthn FROGGS: I guess nqp_to_perl6 is the correct branch?
21:02 FROGGS correct
21:02 jnthn hmm
21:02 jnthn Compiling (43) warnings::register to mbc
21:02 jnthn ===SORRY!===
21:02 FROGGS but gimme a sec to push something
21:02 jnthn Unable to write bytecode to 'lib/Perl5/warnings/register.pm.moarvm'
21:03 jnthn oh, that came after an error
21:03 jnthn "The syntax of the command is incorrect."
21:03 jnthn I don't suppose you've got a mkdir -p in there? :P
21:03 FROGGS ohh
21:03 FROGGS *g*
21:04 FROGGS jnthn: I have :o(
21:04 FROGGS Build.pm line 73
21:06 jnthn Well, creating it manually worked for now
21:06 jnthn I just want to get to your SEGV.
21:07 FROGGS when you have worked around that, you can test with: perl6 Build.pm summary, then abort and run: perl6 -Ilib t/spec/op/read.v5
21:07 FROGGS (to get the fudged test file)
21:08 jnthn Well, I'm up to module 335 by now... :P
21:08 FROGGS you can abort that too
21:08 FROGGS you normally only need to build up to Config.pm
21:09 jnthn C:\Perl64\bin\perl.exe t/spec/fudgeall --add_use_v5 v5 t/*/*.t t/*/*/*.t
21:09 jnthn Can't open perl script "t/spec/fudgeall": No such file or directory
21:09 jnthn That's what I get from summary
21:10 jnthn Indeed, I've no t/spec
21:11 FROGGS jnthn: pull and run the summary again
21:12 jnthn Did that
21:12 jnthn bah!
21:12 jnthn C:/consulting/perl6/v5/t/spec/fudge: No such test file 't/*/*.t'
21:12 jnthn C:/consulting/perl6/v5/t/spec/fudge: No such test file 't/*/*/*.t'
21:12 jnthn so fail!
21:12 FROGGS damn
21:12 FROGGS :o/
21:12 FROGGS jnthn: I'll fix all issues on windows and will tell you then
21:13 FROGGS but this won't be today most likely :/
21:14 jnthn OK. I tried to manually fudge it
21:14 jnthn But making a .v5 copy and putting "use v5" as the first thing after the shebang
21:15 jnthn When I do that I get
21:15 jnthn ===SORRY!===
21:15 jnthn This type does not support associative operations
21:15 FROGGS hmmm, I don't see that here... but perhaps on windows...
21:16 FROGGS ohh, there are some "#v5 emit #"
21:16 jnthn I don't see them?
21:16 FROGGS hmmm, weird
21:17 jnthn Do I need some branch of spectest repo too?
21:17 FROGGS no
21:17 FROGGS I'll check that too
21:17 jnthn origin  git://github.com/rakudo-p5/roast5.git (fetch)
21:17 jnthn Is that the right thing?
21:17 jnthn (for my t/spec)
21:18 FROGGS okay, pushed the fudge lines
21:23 FROGGS Build.pm is fixed, will fix now t/test_summary
21:24 jnthn Yeah, I got a fudged one now. Same error.
21:25 FROGGS need to build a fresh rakudo on my windows box first...
21:25 jnthn This type does not support associative operations at src/Perl6/Actions.nqp:4675  (C:\consulting\MoarVM\install\languages\nqp\lib/Perl6/Actions.moarvm:circumfix:sym<{ }>:180)
21:26 jnthn Further back down is a
21:26 jnthn from src/Perl5/Grammar.pm:1456  (C:\consulting\perl6\v5\lib\Perl5\Grammar.pm.moarvm::135)
21:26 jnthn from src/Perl5/Grammar.pm:1449  (C:\consulting\perl6\v5\lib\Perl5\Grammar.pm.moarvm:FOREIGN_LANG:108)
21:27 FROGGS jnthn: you are using an installed v5
21:27 FROGGS the old nqp version
21:27 FROGGS jnthn: -Ilib should help
21:29 jnthn perl6-m --ll-exception -Ilib t\spec\op\read.v5
21:29 FROGGS that should do
21:29 FROGGS do you have a lib\Perl5\Actions.pm.moarvm file?
21:30 jnthn Yes
21:30 jnthn Should I just do a git clean -fdx and build stuff again? :)
21:31 FROGGS no, let me fix this first, so you don't waste more time :o)
21:31 jnthn OK
21:41 avuserow joined #moarvm
21:57 timotimo how should i detect whether or not MoarVM should write a /tmp/perf-PID.map file?
21:57 timotimo (in its jit compiler)
21:58 timotimo also, i'd like to force a flush, but not make the moarvm process wait for it to complete ...
22:00 FROGGS op/read.v5 passes on my windows box :/
22:05 timotimo can we do anything to make inc/dec frame less costly?
22:05 timotimo inline more stuff? :)
22:07 jnthn timotimo: How costly does your profiling thing they are, ooc?
22:07 jnthn inc should be really cheap
22:07 jnthn dec includes the logic to release/clean up things
22:07 timotimo in "say [+] rand xx 100000" i get 14% interp_run, 9.1% dec_ref, 4.88% inc_ref
22:08 jnthn There are various things we could do, though. Inlining is one. Detecting frames that don't need an ->outer and not bothering with it is another.
22:08 jnthn Really? An atomic increment is so costly?
22:09 timotimo 95.65 times (whatever that means?) it hits the pop op after the lock xadd in MVM_incr
22:09 jnthn Would be worth taking a look at what libatomicops is doing there
22:09 jnthn Ah, so it is using lock xadd
22:10 timotimo yes
22:10 jnthn I know that's not *free*, but I'm surprising it's showing up so highly.
22:10 timotimo yeah
22:11 timotimo since we now always have a second thread for the libuv event thread, we can't even switch out inc_ref and dec_ref to use non-atomic ops instead :\
22:11 jnthn You only get that thread started if you do something that needs it.
22:12 timotimo oh
22:12 jnthn But still, I'm surprised it comes out so costly.
22:12 timotimo so would it be sensible to swap out frame_inc_ref and dec_ref? that would prevent inlining and such :\
22:12 timotimo and gives an indirection for ... all the time
22:12 jnthn It'd be interesting to compare what other profilers make of it on the same benchmark.
22:13 timotimo right; perf is probabilistic
22:14 timotimo say [+] rand xx 100000 is the exact code i used
22:15 timotimo i thought it was kinda interesting that free from libc only ended up at about 0.4%
22:15 timotimo so the win from a separate thread to do the free() calls is not going to help this very benchmark much
22:15 timotimo but it's one of these microbenchmark things
22:16 timotimo i suppose this is just a case of having a ridiculous amount of calls and returns when we do [+] rand xx ...
22:16 timotimo maybe gather/take makes things jump back and forth all the time?
22:17 jnthn Oh...gather/take does some inc/dec, yes
22:17 jnthn xx being implemented in terms of gather/take is silly.
22:17 timotimo mhh
22:17 jnthn Should really do it with a loop iter
22:17 jnthn Apart from we don't have those yet.
22:18 timotimo oh, interesting
22:18 timotimo [+] do rand for ^100000 gives me dec_ref in the 2nd place, but inc_ref is way lower
22:18 timotimo interestingly, __lll_lock_elision from libpthread shows up high-ish
22:30 diakopter my guess is cache thrash since there are *so* many atomically incremented/decremented values.. might be worth centralizing them all in an array accessed viaan offset instead of inside each frame's struct..?
22:31 jnthn Well, the more general issue is that MVMFrame carries around a bunch of stuff that many frames don't need.
22:31 diakopter since the cleanup-related thingies could be handled in a background thread eventually, could actually farm out all the increment/decrement actions to another thread
22:32 jnthn Except in a tight invoke loop we'd like to re-use the same frame right away for the next invocation.
22:33 diakopter o yeah
22:33 jnthn I count 12 or so fields in MVMFrame that could easily be pushed out to an allocated-if-we-need-it data structure hanging off frame.
22:35 timotimo could we just keep the reference count of a frame the same and if we know it's one we'll just re-use it for the next invocation? or something like that?
22:36 timotimo i haven't actually looked at what's inside such a MVMFrmae
23:02 avuserow joined #moarvm
23:13 jnthn 'night o/

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary