Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-03-13

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
03:13 vendethiel joined #moarvm
08:36 vendethiel joined #moarvm
08:38 harrow` joined #moarvm
10:33 vendethiel joined #moarvm
11:12 pyrimidine joined #moarvm
11:23 brrt joined #moarvm
11:27 brrt good * #moarvm
11:27 brrt i come to ask your oracle for one more question
11:28 brrt how can i better deal with the problem of pieces-of-the-inteprreter-that-want-to-know-where-we-are
11:29 brrt in the JIT, that is, where the current state of the art is 'store our position at the start of every *** basic block'
11:30 brrt redundantly, because we treat the JIT 'graph' as a dumb stack
11:31 brrt yes, the graph is a stack, and the tree is a graph. i know
11:41 mojca joined #moarvm
11:53 brrt the pragmatic approach is to store the jit 'node' belonging to the label somewhere
11:53 brrt that way we can check if a label has already been added and prevent them from being added twice
11:54 brrt and then i can factor the dynamic-control-label sequence out as well
12:08 brrt .. the pragmatic approach seems more reasonable now i think about it more
12:08 brrt but i still feel icky about checking the jit entry label everywhere
12:27 jnthn .tell brrt We might want to make a list of the things that do need to know, so we can get an overview of them.
12:28 jnthn ah, no teller here :)
12:52 vendethiel joined #moarvm
12:53 mojca joined #moarvm
13:45 FROGGS joined #moarvm
13:52 vendethiel joined #moarvm
14:09 nine Where do SC names like 1EDBBBE0B7DA22E0D938F7A73B4CAD9D9D472056 in mbc files come from?
14:09 FROGGS they are stored in the string heap
14:09 FROGGS moar --dump foo.moarvm should show them at the top
14:10 nine Yes, that's where I got it from. But what do they mean? How are they generated?
14:11 nine I noticed that the "Missing or wrong version of dependency 'p6scalarfromdesc'" only occurs when loading a Panda::Common that was precompiled during installation. If precompiled on first load, everything's fine.
14:12 nine The precomp file created on installation has one more SC compared to the other one: SC_0 : 1EDBBBE0B7DA22E0D938F7A73B4CAD9D9D472056
14:13 patrickz joined #moarvm
14:15 nine All other differences between the dumps of the precomp files seem innocent enough.
14:15 FROGGS every literal string and every CU-id of a dependency as well as their paths go in there
14:15 FROGGS when precomping stuff obviously
14:16 vendethiel joined #moarvm
14:17 nine That means it picks up an additional dependency by referencing objects of an additional SC when being precompiled as part of compilation?
14:20 FROGGS when you precompile stuff and something extra is in memory, such a precompiled file will eagerly collect these strings
14:20 FROGGS I had the same problem when doing the precomp stuff in CURLI
14:21 nine Ok. The odd thing is that precompilation runs in an external process. The good news is that this could only mean that there's a difference in how this process is run, i.e. the command line or ENV variables.
14:21 nine Ought to be easy enough to find out the differences there.
14:21 moritz aw
14:21 moritz sorry, typo
14:21 FROGGS good luck
14:43 vendethiel joined #moarvm
14:52 nine Ok, there's definitely something weird going on with the repository chain
14:54 mojca joined #moarvm
14:55 lizmat joined #moarvm
15:07 vendethiel joined #moarvm
15:53 vendethiel joined #moarvm
15:56 zakharyas joined #moarvm
16:19 mojca joined #moarvm
16:26 lizmat joined #moarvm
16:33 vendethiel joined #moarvm
16:38 jnthn nine: I think those IDs are the handle of the Serialization Context; look for handle in TOP or maybe comp_unit in Perl6::Grammar.
16:53 mojca joined #moarvm
16:59 vendethiel joined #moarvm
17:06 nine Ok, one step further: it doesn't seem to be Panda::Common at all. I can now repro the bug even with a Panda::Common compiled on first load. So it's probably somehow related to its dependencies.
17:07 FROGGS does it boil down to a 'use lib' or a PERL6LIB env var?
17:07 nine No. And I've already fixed the bug in setting up the repo-chain for precompilation, so the process to recompile Panda::Common is run in exactly the same way
17:17 geekosaur joined #moarvm
17:47 dalek MoarVM: 79dce11 | FROGGS++ | src/io/dirops.c:
17:47 dalek MoarVM: get directory listing in utf8-c8 encoding
17:47 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/79dce1101b
17:53 vendethiel joined #moarvm
18:02 nine I can still reproduce a Missing or wrong version of dependency '<unit-outer>' after stripping Panda down to the Panda::Common module, this module down to "use Shell::Command;", Shell::Command down to "use File::Find;" and File::Find down to "unit module File::Find;"
18:09 domidumont joined #moarvm
18:10 domidumont joined #moarvm
18:15 vendethiel joined #moarvm
18:15 domidumont joined #moarvm
18:15 cognominal joined #moarvm
19:55 brrt joined #moarvm
20:14 timotimo every time i see find_best_dispatchee show up in my code i get in the mood to flip tables
20:15 timotimo and the way in which the profiler sometimes shows times and entry counts as "NaN" doesn't make it any better
20:15 jnthn Why? It normally means you've a good opportunity to make a big saving :)
20:15 jnthn Yes, why does it do that...
20:15 brrt don't work on a late bound dynamically typed language with multiple dispatch then :-P
20:15 timotimo the last time i looked, it showed up in dispatching infix:<+>
20:16 timotimo tell me, how does that make any sense?
20:16 timotimo i think i want an nqp op that does the backtrace dump and nothing else
20:17 jnthn timotimo: Well, remember the first call will always go through find_best_dispatchee
20:17 timotimo and just throw that into the find_best_dispatchee function
20:17 jnthn timotimo: So it's only an issue if you have a *lot* of those calls for a given function
20:17 timotimo hm. it accounted for 90% of time spent in infix:<+>, so maybe the rest of infix:<+> is so fast that it won't ever make up for that?
20:17 timotimo i don't remember how often it invoked that, tbh
20:17 timotimo i'll try to reproduce that case
20:21 timotimo 357642 entries in total
20:22 timotimo 356241 to find_best_dispatchee, 356240 to infix:<+>
20:22 timotimo gen/moar/m-CORE.setting:9195
20:22 timotimo and 1 entry to infix:<+>
20:22 timotimo gen/moar/m-CORE.setting:9198
20:23 jnthn Do you have an overlord of it?
20:23 timotimo none that i'm aware of
20:24 timotimo definitely none in the code i'm compiling here
20:25 timotimo do you have a working SDL2 for perl6 locally, or should i try to golf the SDL parts out of it?
20:25 timotimo i probably should do it just because
20:26 jnthn timotimo: The other thing to check is if we're somehow filling up the multi dispatch cache
20:27 timotimo is it a global thingie?
20:31 timotimo because i could imagine that my code does fill that up if it's global
20:50 jnthn timotimo: No, it's per routine
20:50 jnthn But has a limited number of entries
20:51 lizmat joined #moarvm
20:56 timotimo so how do we get entries in there for infix:<+>?!
20:57 timotimo also, i can't reproduce it with the same code on consecutive runs. perhaps because i just recompiled rakudo and therefor it has to re-precompile
21:13 timotimo yeah, after rm-ing all files under ~/.perl6/precomp, i get the find_best_dispatchee being called a whole bunch of times. the next time i run it, it's gone.
21:20 timotimo i feel like we want an op to invalidate all multi dispatch caches and fire that when precomp is and other start-up-business cases are handled
21:23 jnthn timotimo: Doesn't need an op
21:23 jnthn timotimo: You can just null it in the Routine
21:23 timotimo oh
21:24 jnthn You could also try increasing the multi cache size to see if it makes the issue vanish, then we'll knwo that's what is going on :)
21:24 timotimo in which of our things does that magic number live? moar, nqp or rakudo?
21:26 jnthn MoarVM
21:26 jnthn MVMMultiCache.h or so
21:26 timotimo 'k
21:27 timotimo er, huh. 1482070 BOOTHash allocations in AT-POS, in sink it's 1068153 and in Bridge there's another 718436
21:28 timotimo as that from making the profiler aware of signatures allocating stuff?
21:28 timotimo another sink, another Bridge
21:28 jnthn Probably slurpy hashes, yeah
21:29 timotimo could that be because methods have *%_ ? or because of proto methods?
21:30 jnthn Could be *%_
21:30 jnthn onlystar protos make a callcapture instead
21:35 timotimo i'm not sure how exactly to investigate this
21:35 timotimo the AT-POS in question is the numarray role's AT-POS(array:D: int $idx) is raw { nqp::atposref_n(...) }
21:36 jnthn I think if we can lexical-to-local lower the %_s then it may just go away anyway
21:36 timotimo m: my num @foo; say @foo.^find_method('AT-POS').cando(:(@foo: 1)).perl
21:36 camelia rakudo-moar b6f3ec: OUTPUT«===SORRY!=== Error while compiling /tmp/z8mconLC89␤Can only use the : invocant marker in the signature for a method␤at /tmp/z8mconLC89:1␤------> o.^find_method('AT-POS').cando(:(@foo: 1⏏)).perl␤    expecting any of:␤        constra…»
21:36 timotimo m: my num @foo; say @foo.^find_method('AT-POS').cando(:(@foo, 1)).perl
21:36 camelia rakudo-moar b6f3ec: OUTPUT«Type check failed in binding $c; expected Capture but got Signature (:(@foo, Int $ where {...)␤  in block <unit> at /tmp/GmOZhrB5ml line 1␤␤»
21:36 timotimo m: my num @foo; say @foo.^find_method('AT-POS').cando(\(@foo, 1)).perl
21:36 camelia rakudo-moar b6f3ec: OUTPUT«(method AT-POS (array:D $: Int $idx, *%_) { #`(Method|60354656) ... }, method AT-POS (Any:D $: Int:D \pos, *%_) { #`(Method|44421168) ... }, method AT-POS (Any:D $: Any:D \pos, *%_) { #`(Method|44421472) ... }, method AT-POS ($: **@indices, *%_) { #`(Metho…»
21:37 timotimo hard to tell what exactly is going on there
21:38 timotimo um ... something's gone horribly wrong with moar --dump
21:38 timotimo it seems to crash
21:38 timotimo let me see ...
21:38 jnthn urgh
21:39 jnthn I wonder if it was anything to do with the string decoding changes...
21:39 timotimo Callsite_144 :
21:39 timotimo num_pos: 2[Inferior 1 (process 13099) exited normally]
21:39 timotimo probably not flushed stdout?
21:39 jnthn yeah, that was perhaps not the last thing really
21:40 timotimo strace -e write makes it work 100%
21:40 timotimo because ... fuck you, that's why
21:40 jnthn o.O
21:40 jnthn valgrind it :)
21:42 timotimo valgrind also makes it complete. i saw one invalid write, which resulted from writing 2 out of 4 bytes beyond the end of "lineloc"
21:42 timotimo hehe
21:42 timotimo looks like a sizeof multiplication was missed
21:46 timotimo is "fflush" the right function to use when using "printf" and friends?
21:46 timotimo like, "fflush(stdout);"?
21:46 timotimo actually, maybe fsync
21:47 jnthn fflush is probably right here
21:47 timotimo 'k
21:48 timotimo it still just aborts in the middle of business o_O
21:56 timotimo oh, i was wrong
21:56 timotimo valgrind doesn't let it finish properly
21:56 timotimo it's also not reaching the end
22:02 timotimo the dump output weighs in at about 32 megabyte, according to strlen
22:19 lizmat_ joined #moarvm
22:21 orbus not really news, but nice to know - I just built moar and nqp on a raspberry pi 3 running fedora 23 with no issues
22:21 orbus it passes the nqp test suite - building rakudo now
22:22 timotimo neat
22:22 timotimo probably a whole bunch faster than on a rpi2, too
22:23 orbus I wasn't really watching the time, but I think so
22:23 timotimo jnthn: i found the frame in the CORE.setting dump, but i can't see it write to or even read from its lex_Frame_7847_%__obj
22:23 orbus it's clocked 300 MHz higher
22:24 timotimo jnthn: would "takedispatcher" ever allocate a BOOTHash?
22:25 jnthn timotimo: No, it never allocates nowt
22:25 orbus I don't have a heatsink or fan or anything on there - with all 4 cores cranking 100% the case is slightly warm to the touch
22:25 timotimo the speshed result of that AT-POS is just sp_getarg_o, sp_getarg_i, takedispatcher, atposref_n, return_o
22:26 timotimo orbus: allegedly the rpi3 has a real heat problem if you don't put a heatsink on it
22:26 orbus so far so good
22:26 orbus I wonder if there's a heat sensor
22:26 orbus hmmm
22:27 orbus it doesn't seem all that hot just based on touching the case
22:28 timotimo the sink (of class List), which is really an empty body and --> Nil in the signature, which is speshed to takedispatcher, wval, return_o, also allegedly allocates a huge amount of BOOTHash
22:28 timotimo 1068153 invocations of that sink method, which turn into 1068153 allocations of BOOTHash
22:28 orbus looks like temperature sensor is exposed in /sys... let's see here...
22:28 timotimo so at least the numbers match up in a 1:1 fashion
22:29 timotimo it's a bit annoying that i can't get at the post-profile bytecode
22:31 orbus looks like it's at 54C
22:31 orbus that doesn't seem that hot
22:37 orbus hrm... well, I'm running headless too so the gpu is doing nothing - that probably helps
22:37 timotimo the profiler may be a bit borked actually
22:40 jnthn timotimo: Well, noticed it doesn't like to close its windows sometimes...
22:40 timotimo yeah, well, that's just, you know, a UI issue
22:40 timotimo (you can click outside the window instead)
22:41 timotimo what i mean is maybe i broke it when i moved the name out of the allocations array
22:46 timotimo i don't know what's wrong :\
22:46 timotimo it doesn't make sense to see all those BOOTHashes
22:49 timotimo json_xs undoes the sorting of hash keys when pretty printing -_-
22:49 timotimo it can, however, do it by itself somehow
22:50 timotimo it doesn't expose that option to the commandline, though
22:54 * timotimo has a local copy of json_xs that sets "canonical"
23:00 vendethiel joined #moarvm
23:03 timotimo i think i'll give the profiler IDs starting at 0
23:12 timotimo cool, only three-digit IDs are in this profile now
23:12 timotimo down from about 8 digits per ID
23:12 timotimo er, up-to-three-digit IDs of course
23:15 jnthn timotimo: Well, the BOOTHash is perhaps coming from the slurpy hashes
23:15 jnthn It'd make sense there'd be plenty of 'em if we're not eliminating them
23:15 jnthn Sleep time...'night o/
23:16 timotimo well, i don't see an op to generate these hashes
23:16 timotimo so where do they actually come from? what allocates them?
23:16 timotimo and if i don't see an op to generate them, how does the profiler know?
23:24 Unavowed joined #moarvm
23:32 timotimo ... ?!?
23:32 timotimo i added debug spam to add_instrumentation to output the frame's name and its cuuid
23:33 timotimo AT-POS doesn't show up at all, neither in its name nor do the cuuids i've logged
23:33 timotimo but they do show up in the spesh log

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary