Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2014-09-02

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:04 FROGGS_ joined #moarvm
06:14 sergot hi o/
06:59 itz_ joined #moarvm
07:07 zakharyas joined #moarvm
07:40 brrt joined #moarvm
07:51 ggoebel1111112 joined #moarvm
08:12 kjs_ joined #moarvm
08:29 timotimo ohai sergot
08:29 timotimo (i'm a bit late to the party, but enh.)
08:31 brrt \o timotimo
08:31 brrt what was your issue yesterday again?
08:32 nwc10 brrt: have you built with ASAN recently?
08:32 brrt .. no don't think i have
08:32 brrt why? very broken?
08:33 timotimo brrt: i'm having difficulty matching up the output of the profiler against the jit log
08:33 brrt uhm... why is that timotimo?
08:33 nwc10 t/spec/S32-list/uniq.rakudo.moar t/spec/integration/99problems-51-to-60.t and sometimes t/spec/integration/99problems-11-to-20.t are failing
08:33 timotimo as in: i see routines in the routines tab that have ~100% spesh, but i see no bails whatsoever in the jit log
08:33 brrt jit log is lying
08:33 timotimo yeah
08:33 nwc10 for the same thing, and jnthn is stuck on why
08:33 timotimo i think it's because of inlining
08:33 brrt hmm
08:33 brrt darn
08:34 brrt i don't really have time to look at that right now
08:34 brrt sorry :-(
08:34 timotimo that's okay
08:34 nwc10 (git submodules)-- # not transparent
08:35 brrt yeah submodules are problematic
08:38 brrt nwc10: happen to know when they started breaking?
08:39 nwc10 not offhand, accurately
08:40 timotimo brrt: do you have hints where i should look next to advance moarvm and/or perl6?
08:40 nwc10 brrt: the IRC logs here remember better than me, and it's not conclusive: http://irclog.perlgeek.de/moarvm/2014-08-29
08:40 nwc10 it seemed to be a rakudo change
08:42 odc hi, I just installed the latest rakudo star and i'd like to do some async stuff but I can't find find IO::Socket::Async anywhere
08:43 odc and google doesn't really help
08:43 FROGGS m: say IO::Socket::Async
08:43 camelia rakudo-moar a640aa: OUTPUT«(IO::Socket::Async)␤»
08:44 FROGGS odc: you do not need to import it, it is built in
08:44 FROGGS into rakudo
08:45 odc err, but in the repl, when i type use IO::Socket::Async it says it cannot find it
08:45 FROGGS odc: do not 'use' it
08:46 FROGGS like you do not need to 'use' Int or Str
08:46 FROGGS it is there
08:46 odc oh
08:46 odc i see
08:46 odc thx FROGGS
08:46 FROGGS you're welcome
09:19 kjs_ joined #moarvm
09:54 itz joined #moarvm
10:02 * brrt afk for lunch
10:02 brrt left #moarvm
10:04 itz_ joined #moarvm
11:55 jnthn evening, #moarvm
11:57 nwc10 heresy!
11:59 lizmat heresy?
11:59 jnthn China! :P
11:59 lizmat ah, that  :-)
11:59 jnthn Also, I read the IRC logs in the morning in the taxi to work, which kinda counts as "on IRC" :P
12:00 carlin evening jnthn
12:00 jnthn Apart from this morning I forgot to grab 'em before setting off 'cus I got up too late. D'oh :)
12:00 carlin (in one minute it will be morning)
12:03 jnthn Still several hours to go until morning here :)
12:03 jnthn Which is good 'cus I didn't dinner yet. :)
12:07 brrt joined #moarvm
12:07 nwc10 you've done this before haven't you? Forget to eat
12:20 jnthn nwc10: This ain't forgetting, just that the taxi driver took an insane route this evening, on top of leaving a bit later form $dayjob anyway...
12:21 jnthn Going out soon :)
12:21 nwc10 have fun, but not "fun"
12:25 jnthn Will try :)
12:25 jnthn bbl &
12:45 diakopter typical taxi driver scam
14:29 brrt joined #moarvm
14:36 jnthn diakopter: No, I think this one was genuinely incompetent. There are plenty of scammer ones, though. :)
15:17 jnthn carlin: It feels a little odd putting the malloc wrappers in src/gc/allocation.* when they're actually nothing to do with GC-able allocation at all...
15:18 jnthn Also, while the #define MVM_free free is expedient, another static function would be more consistent (and easier to hack for those who want to instrument allocation for some reason)
15:18 jnthn Other than that I'm in favor, I think.
15:29 * brrt nods that adding a real function for MVM_free is good for instrumentation (breakpoitns)
15:29 brrt breakpoints
15:42 carlin should I make a new file, src/core/alloc.{c,h} perhaps?
15:43 jnthn carlin: Thta works fo rme.
15:44 jnthn wow. so typing...
15:45 carlin ok, I'll get onto that in a few hours
15:45 jnthn Cool. I'll take a look at the PR that results tomorrow and all being well can merge it :)
15:47 timotimo jnthn: do you know of a somewhat-LHF i could try to tackle in the coming days?
15:51 jnthn timotimo: If you fancy diving into the profiler code, it appears to me that the number of deopts reported on the front page doesn't equal the number seen on the OSR/Deopt page in all cases.
15:51 jnthn timotimo: I don't know which one is correct. I do know they can't *both* be correct. :)
15:52 jnthn timotimo: For a second LHF-ish that involves the GC but is maybe not terrifying but should be tried in a branch first: when I put in the profiler I started tracking the number of bytes promoted.
15:53 jnthn timotimo: Task: try to use that data in order to decide how often to do full gen2 collects as opposed to nursery ones.
15:53 timotimo ah
15:53 timotimo should the code also try to increase/decrease the nursery size in response perhaps?
15:53 jnthn timotimo: Today it's a fixed count. If you take a bunch of perl6-bench profiles that allocate a lot of short-lived things, you'll noticed we promote jack all, but still do a full collect every 25 runs, which is wasteful.
15:54 jnthn No, I don't think so. Not yet at least.
15:54 timotimo OK
15:54 jnthn Just play wiht having a megabytes promoted threshold for gen2 rather than a number of nursery collects threshold.
15:54 timotimo do you have an intuition what a good target value for "promoted bytes until a full collection makes sense" is?
15:55 jnthn I'd run a few benchmarks on promote-heavy things
15:55 timotimo all right
15:55 jnthn --profile-compile is pretty good for that
15:55 jnthn JSON parsing --profile is also good for that
15:55 jnthn It seems to me we often promote around 1MB per collect in those caess
15:55 jnthn Meaning that a 25MB threshold would be about equivalent to what we do today.
15:57 jnthn nine: FYI, I have an increasingly concrete design for DESTROY stuff on Moar forming in my head, which will get the thread affinity you need for the Perl 5 stuff. I'll likely work on a first-cut impl something within the next week.
15:57 jnthn *some time
15:57 timotimo hm, did i merge the string allocation pressure thing for the GC back then?
15:57 jnthn Don't think so.
15:58 jnthn Not sure we've missed it terribly since the strings refactor made us a lot less copy-heavy though.
15:58 timotimo right
15:58 jnthn For extra credit, you may want to look at the libuv API; there is a function that tells how much virtual memory the host system has.
15:59 timotimo that seems like it wouldn't require much code; the other LHF you mentioned doesn't seem that enticing to me at the moment :S
15:59 jnthn We might like to set the gen2 collect threshold based on that...
15:59 timotimo i've been dreaming about writing an analysis tool thet allows you to inspect the SSA form used in spesh in much more detail and with helpful tools like highlighting versions of the same register you're hovering or something like that
15:59 jnthn timotimo: Hm. It would turn a bunch of 25ms things into 5ms things in longer-running benchmarks, which could add up.
16:00 timotimo you mean reducing the number of gen2 collections, yes?
16:00 jnthn Yes
16:01 timotimo i'd imagine so
16:01 jnthn I mean, in benchmarks like for 1..lol-so-big-number { } we can reduce the numbber to zero with what I'm suggesting
16:01 jnthn 'cus they promote nothing.
16:01 timotimo that'd be nice indeed
16:01 jnthn Uh, or the $i++ one for non-native $i, at least.
16:01 jnthn there's really no point doing a full collect when we promoted nothing.
16:01 timotimo how do you feel about reacting to "too much stuff promoted" by upping the nursery size?
16:02 jnthn I'm not sure. It's a *very* mixed blessing.
16:02 timotimo it could lead to "more time between collections gives more chance for things to die"
16:02 timotimo but also "longer nursery collections"
16:02 jnthn It may be less collects but (a) they'll be longer, and (b) make it bigger than it is now and you make hte active set of memory to consider larger than most folks CPUs have in on-chip cache.
16:03 timotimo oh
16:03 jnthn And once we miss beyond the L3 it's ugly.
16:03 timotimo right, caches are *way* important
16:03 jnthn I tuned it on my quite-cache-rich box and suspect we're already pushing our luck.
16:04 timotimo does windows give you a tool that shows you how many cache misses you get?
16:04 timotimo because time -v and perf somethingsomething gives you things like "percentage of branches mispredicted" and cache stuff and such
16:04 jnthn I'm also thinking of games like tadzik++ makes, which likely also promote very few objects, want fast collects often, and might also get to avoid gen2 collects if we change this...
16:04 timotimo mhm
16:05 jnthn those things on Windwos are typically exposed as performance counters
16:06 timotimo OK, good
16:07 jnthn Dunno if it's worth it or not to try and obtain such numbers to include in the profile. Will need platform specific code. Can always say N/A when we can't get 'em too...
16:07 jnthn OK, time for me to sleep
16:07 jnthn 'night
16:08 timotimo gnite jnthn!
17:10 nine jnthn: awesome! Many thanks already!
17:48 Ven joined #moarvm
18:15 kjs_ joined #moarvm
18:38 nine joined #moarvm
18:42 lizmat joined #moarvm
18:46 PerlJam joined #moarvm
18:51 brrt joined #moarvm
18:51 brrt \o
18:51 [Coke] o/
18:57 brrt uhm, i have a bit of a theory wrt to the ASAN / valgrind dependent breakage
18:58 brrt although no experimental evidence obviously
19:00 brrt and maybe a quick disproving
19:17 Ven joined #moarvm
19:17 vendethiel joined #moarvm
19:32 kjs_ joined #moarvm
19:36 timotimo hum.
19:39 timotimo i want to put a "collect all 'promoted_bytes' from the threads that have gc'd, sum them up and store them in the instance", but there's threads that are blocked and thus have their work stolen from them
19:39 brrt \o timotimo
19:42 timotimo how big is an AO_t?
19:42 timotimo like, is there a guarantee?
19:43 brrt uhm, yeah
19:43 brrt i think an AO_t should be exactly one register wide
19:43 brrt so 8 bytes on x64
19:44 dalek MoarVM: 21c5507 | (Bart Wiegmans)++ | src/jit/graph.c:
19:44 dalek MoarVM: Add JIT logging of inlines
19:44 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/21c550765d
19:44 brrt present for you!
19:45 timotimo sweet!
19:47 timotimo that means an AO_t is enough to store how many bytes we promoted since the last time we full-collected
19:47 timotimo then i can just MVM_add
19:47 brrt uh
19:47 brrt i think so :-)
19:49 timotimo cool.
19:55 timotimo there's no good spot to reset the "bytes since last time we full-collected" to 0, because run_gc wants to inspect that for "do we want to do a full inspection?" before we kick stuff off and all the threads want to add stuff to it after they're done GCing
19:55 timotimo maybe i should just remember the value before we started and decrement the value by that amount afterwards
19:55 timotimo that should be robust
20:00 brrt i can't help you tonight, sorry :-)
20:00 * brrt afk
20:00 brrt left #moarvm
20:00 timotimo oke :)
20:02 nine Do I understand this correctly that MoarVM's GC moves objects in memory? What happens to NativeCall callbacks? Those pointers are somewhat out of reach for MoarVM.
20:08 timotimo i think callbacks are referred to with their CUID and stuff
20:09 timotimo the callback_data is malloc'd, so it doesn't move
20:16 nine ah, good to know, thanks
20:16 PerlJam random questions:  a) Is MoarVM written entirely in C?  b) What standard is it written to?
20:16 PerlJam I ask because github thinks there's some C++ code in there and there are some .c files that have comments that start with //
20:17 PerlJam (I think C99 groks //-style comments, but C89 does not)
20:18 timotimo MSVC doesn't like //
20:18 nine MSVC doesn't support C99 and never will
20:18 timotimo so if we commit // comments, jnthn has to fix them before being able to compile it
20:21 PerlJam Well ... lines 32 and 33 of src/strings/utf8.c look to have //-style comments and have been around since June 2012.   I don't know what to make of that.
20:22 timotimo weird.
20:23 PerlJam In any case, I was just reading the code and became curious.
20:25 PerlJam timotimo: there are even some lines that you added back in Feb in src/math/bitintops.c that have //-style comments  :)
20:26 PerlJam (git blame)++  :)
20:26 PerlJam er, src/math/bigintops.c even
20:27 timotimo d'oh!
20:27 timotimo in that case ... i guess it doesn't bother jnthn's msvc
20:27 timotimo okay, moar's gc now tries to only full-collect if enough stuff has been promoted
20:27 timotimo i've yet to test this with multi-threaded workloads.
20:29 timotimo i'll have a look if it does the spec tests without being sad
20:33 japhb_ Regarding CPU cache info, I found a number of sources for doing autodetection.  There's an API for Windows, and for Linux you have (of course) a selection of methods: shelling out to lshw, parsing from the pseudo-files /proc/cpuinfo and/or /sys/devices/system/cpu/cpu0/cache/index*/*, or in C using sysconf: https://gist.github.com/jesstess/797876
20:34 japhb (Just in case someone wanted to explore the auto-tuning angle, the info is there -- no need to benchmark and detect steps in the performance curve, which was the old way to do it)
20:36 lizmat decommute&
20:38 timotimo well ... that's no good.
20:40 timotimo ah, just forgot an &
20:47 dalek MoarVM/dynamic_gen2_tuning: 9ebdb71 | (Timo Paulssen)++ | src/ (3 files):
20:47 dalek MoarVM/dynamic_gen2_tuning: dynamically tune full collection interval
20:47 dalek MoarVM/dynamic_gen2_tuning:
20:47 dalek MoarVM/dynamic_gen2_tuning: based upon how many bytes have been promoted to the old generation since
20:47 dalek MoarVM/dynamic_gen2_tuning: the last time we did a full collection.
20:47 dalek MoarVM/dynamic_gen2_tuning: review: https://github.com/MoarVM/MoarVM/commit/9ebdb713cb
21:06 carlin jnthn: https://github.com/MoarVM/MoarVM/pull/134
21:41 kjs_ joined #moarvm
22:26 timotimo brrt: it doesn't seem like any "Entering" lines end up in the jit log at all
22:26 timotimo also, it seems like is_entry is set to 1 on both calls, so it'd also say "Entering" when it should say "Leaving"
22:31 TimToady it seems to run the parser a bit faster
22:33 timotimo "the parser"? like stage parse of core setting fame?
22:34 TimToady yes
22:35 TimToady maybe from running gen2 moar often?
22:35 TimToady so keeps a smaller set?
22:35 TimToady I dunno
22:35 timotimo should be "less often", no?
22:36 TimToady depends on how much intermediate stuff, I suppose
22:36 timotimo we still can't get a --profile-compile from the core setting :P
22:36 timotimo does somebody have a machine with lots and lots of ram to spare handy?
22:36 TimToady oh, to try to compile with only nursery?
22:39 timotimo no, to get the --profile-compile to find out how it's doing :)
22:39 TimToady slightly faster on spectest, 987 -> 980 cpu sec
22:39 TimToady oic
22:39 timotimo it'll compile fine and then use LOTS of ram to build up the json that it wants to put into the profile html page app thingie
22:39 TimToady how much memory we talking about?
22:39 timotimo on my weak 12gb of ram laptop, it won't get far
22:40 timotimo can't really tell how far it gets, though
22:40 TimToady .oO(12gb as weak, sigh)
22:40 timotimo so maybe 32 gigs? 64 gigs?
22:40 timotimo 128 gigs? :P
22:40 timotimo the browser will likely take a whole bunch of ram to run the app :D
22:40 timotimo ideally, we'd build some code to prevent the call graph from becoming OMGhuge without ruining the numbers
22:41 TimToady well, way outa my league...
22:42 TimToady but wait five years or so...
22:43 TimToady they've got a Cray sitting down the street in the Computer History Museum...oh wait, my phone is more powerful than that...
22:44 timotimo :))
22:44 TimToady we just need to convince one of our computers that an SSD is really its main memory...
22:44 cognome joined #moarvm
22:45 * TimToady wonders off to see if he can figure out why .base(3) has burk
22:51 lizmat joined #moarvm
23:03 TimToady oh, base.t is the 1..57 bug agaain
23:04 TimToady maybe...
23:05 TimToady anyway, last line of Rational.base is:$s ~= (0..9,'A'..'Z')[@f].join;
23:05 TimToady m: 0.1.base(10)
23:05 camelia rakudo-moar 83a5f5: ( no output )
23:05 TimToady m: 0.1.base(10).say
23:05 camelia rakudo-moar 83a5f5: OUTPUT«0.ABCDEFGHIJKLMNOPQRSTUVWXYZ␤»
23:06 TimToady or something is making a range sequence there
23:07 TimToady oh, wait, no, that's probably just a parcel needs flattening, dur
23:07 * TimToady slinks back to #perl6

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary