Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-05-11

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 diakopter ilmari: I'm confused
00:01 diakopter maybe you thought I wrote KB/s
00:03 geekosaur no, ilmari is pointing out that in many areas 2.4G is so congested that you can't get any kind of wifi performance out of it
00:03 diakopter ohhh. well I'm in Mountain View, CA at the moment XD
00:03 diakopter there's around 30 visible SSIDs in range
00:04 TimToady at least it looks like we'll get 1Gb Google fiber in the next three years
00:06 TimToady (in MV, I mean)
00:07 diakopter I'm at $work at Castro & Evelyn; but yes, I was on the secret backup wifi network, not the normal throttled one for the worker masses
00:08 diakopter then again, to be throttled at 10Mb/s per user isn't that bad
00:10 diakopter in other news, my dumb Google Fi phone (Nexus 6P) hasn't received the May 2016 OTA security update yet :( :( :(
00:10 diakopter it's a conspiracy
00:16 geekosaur fwiw I haven't gotten it on my 5X yet either
00:18 timotimo oh diakopter is here!
00:19 timotimo did you see i did the thing where you suggested you would make the qast operations mappers take less memory with their closures and whatnot
00:22 cognominal joined #moarvm
00:31 timotimo you can find the commit in a branch in nqp
00:31 timotimo in case you were wondering
00:31 timotimo there's more wins to be found there, i suppose. there's still more than 300,000 bytes used by frames that are closures in that region. i expect we could store the closed-over values compactly in a hash and have a simple lookup-thingie installed for every mapped op
01:48 ilbot3 joined #moarvm
01:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
01:49 cognominal joined #moarvm
02:15 zakharyas joined #moarvm
05:36 domidumont joined #moarvm
05:40 domidumont joined #moarvm
06:04 domidumont joined #moarvm
09:47 jnthn moarning o/
09:47 timotimo hey jnthn
09:47 timotimo how are you today?
09:54 jnthn Not so good, but maybe it improves with more tea...
09:57 timotimo ah, tea. the nectar of the gods
09:57 timotimo or something
09:59 jnthn My body is doing that "oh you went to bed earlier to deal with your lack of sleep...how cute...let's lay awake some hours" thing :/
10:00 timotimo i lay awake for a long time yesterday and today, but there was no "go to bed earlier" trigger to that
10:02 nine Yeah, bodies are funny like that. I always sleep tworst when I'd need it most.
10:02 timotimo ugh, i can imagine that
10:03 * jnthn wonders if he's smart enough to improve reframe at all today...
10:07 nine jnthn: if not, a review of https://github.com/MoarVM/MoarVM/pull/364 is probably much less exhausting :)
10:08 jnthn :)
10:08 jnthn Yeah, especially given turning on a little more debugging seems to suggest that somehow an MVMArray is ending up containing references to past fromspaces...
10:08 timotimo i don't see a/the commit that addresses jnthn's line comment there
10:09 timotimo but it's probably in there somewhere
10:10 nine timotimo: the commit changed. I don't know why github still displays the old version. Fix is here: https://github.com/MoarVM/MoarVM/pull/364/commits/326bb9dfd3e0ecf9af4caa32adf37a7894408797#diff-98ce06798cc6f1f992daed4e81890460R39
10:10 timotimo good!
10:18 arnsholt joined #moarvm
10:21 jnthn nine: Did some review
10:30 konobi jnthn: did you have a peek at libck at all?
10:34 jnthn konobi: Had a quick look...it has some useful things
10:37 konobi ah, cool... there's also an academic paper or two behind it too. Also on the website i believe
10:38 konobi goes into details like multicore effeciency, lock tradeoffs, etc.
10:45 jnthn argh...why oh why does a bug have to show up exaclty in the intersection between gc, continuations, and stack->heap callframe promotion...
10:45 nwc10 because it has shares in whisky distilleries?
10:45 jnthn The first two are a headache individually :P
11:05 dalek MoarVM/reframe: a3d01de | jnthn++ | src/gc/collect.c:
11:05 dalek MoarVM/reframe: Fix crash when GCing an ended thread.
11:05 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/a3d01de163
11:05 dalek MoarVM/reframe: 6d4e5d9 | jnthn++ | src/gc/roots.c:
11:05 dalek MoarVM/reframe: Cope with MVMROOTing a call stack frame.
11:05 dalek MoarVM/reframe:
11:05 dalek MoarVM/reframe: This fixes various crashes when adding temproots to the worklist.
11:05 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/6d4e5d90e1
11:05 jnthn Those two we rather easier to find/fix at least :)
11:18 * jnthn sets off another spectest run over lunch to see how much those two fixes fixed
11:18 jnthn bbiab
11:54 TimToady joined #moarvm
12:13 dalek MoarVM/even-moar-jit: 85e1f7c | brrt++ | tools/jit-comparify-asm.pl:
12:13 dalek MoarVM/even-moar-jit: use labels instead of offsets in prettyfing asm
12:13 dalek MoarVM/even-moar-jit:
12:13 dalek MoarVM/even-moar-jit: jit-comparifiy-asm.pl is intended to make jit frames diffable.
12:13 dalek MoarVM/even-moar-jit: This is easier and more readable using labels rather than code
12:13 dalek MoarVM/even-moar-jit: offsets.
12:13 dalek MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/85e1f7c8aa
12:22 * jnthn back
12:45 brrt joined #moarvm
12:46 brrt good afterlunch jnthn
12:48 jnthn :)
12:48 * jnthn found a continuations problem
12:48 jnthn Hopefully *the* continuations problem
12:49 jnthn gah, alas not the only one
12:49 moritz :/
12:49 moritz continuations are like politics: not just a single problem
12:51 jnthn Well, the "poli" is a clue there's gonna be more than one... :P
12:52 brrt lets not discuss polytics in polyte compani
12:54 dalek MoarVM/reframe: da178fb | jnthn++ | src/core/continuation.c:
12:54 dalek MoarVM/reframe: Optimize away a loop we no longer need.
12:54 dalek MoarVM/reframe:
12:54 dalek MoarVM/reframe: Was needed when we had to fiddle with refcounts, which we no longer
12:54 dalek MoarVM/reframe: do.
12:54 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/da178fb2ce
12:54 dalek MoarVM/reframe: a06891f | jnthn++ | src/core/continuation.c:
12:54 dalek MoarVM/reframe: Missing MVMROOTs in continuation invoke.
12:54 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/a06891fb9a
13:03 dalek MoarVM/reframe-jit: 6c6acd2 | brrt++ | src/jit/compile.c:
13:03 dalek MoarVM/reframe-jit: Add entry label out of range check
13:03 dalek MoarVM/reframe-jit:
13:03 dalek MoarVM/reframe-jit: This should allow us to spot weirdness somewhat earlier.
13:03 dalek MoarVM/reframe-jit: review: https://github.com/MoarVM/MoarVM/commit/6c6acd261a
13:04 brrt this lets the JIT crash at the expected place, just checked rather than within the code segment itself
13:09 jnthn :)
13:12 dalek MoarVM/reframe: c6b8c93 | jnthn++ | src/core/exceptions.c:
13:12 dalek MoarVM/reframe: Missing MVMROOT in exception throwing.
13:12 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/c6b8c93b8c
13:12 brrt i already know, fwiw, that the jit entry label points into the caller
13:13 jnthn Turns out the second issue wasn't continuations themselves.
13:13 jnthn Just that gather/take happen to make a ton of exceptions.
13:13 jnthn In loops
13:14 jnthn Can't believe I missed that one when putting the stack -> heap promotion code in :/
13:18 jnthn Into S06 in spectest and looking good.
13:18 * jnthn crosses fingers
13:30 brrt hmmpf, all of this looks bog-standard
13:30 brrt hmm, waitaminute
13:30 jnthn OK, that spectest runs is waaay better :)
13:31 brrt nope, its not OSR
13:41 dalek MoarVM/reframe: 48df567 | jnthn++ | src/core/frame.c:
13:41 dalek MoarVM/reframe: Add a missing heap force operation.
13:41 dalek MoarVM/reframe:
13:41 dalek MoarVM/reframe: Seems we don't hit it in the whole spectest, and perhaps can't, but
13:41 dalek MoarVM/reframe: better to just do the promotion rather than error out in case we end
13:41 dalek MoarVM/reframe: up being able to do this at some future point.
13:41 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/48df567d8b
13:48 nwc10 jnthn: without that least MoarVM patch, ASAN gets all spectests to pass, apart from  t/spec/S17-supply/syntax.t which bails out with "Aborted" after ok 52 - did we throws-like X::ControlFlow?
13:48 nwc10 1..3
13:48 nwc10 which I think is an assert() failure about mutexes or somesuch
13:48 jnthn nwc10: hm, odd
13:48 jnthn Oh...
13:48 jnthn Yeah, that's not new.
13:48 nwc10 and I've seen it before
13:49 nwc10 I think it will be timing related, because ASAN ends up alterning race conditions
13:49 jnthn On my "to fix" list, though not in this branch...
13:49 nwc10 yes, IIRC it's a master bug, and not new.
13:50 nwc10 oh, except if I run it under gdb it passes
13:50 dalek MoarVM/reframe: de11e29 | jnthn++ | src/ (4 files):
13:50 dalek MoarVM/reframe: Clean up GC debugging support.
13:50 dalek MoarVM/reframe:
13:50 dalek MoarVM/reframe: Bring various debugging support bits under one flag, and adds what I
13:50 dalek MoarVM/reframe: have been using locally to debug some bits, so it's there to turn on
13:50 dalek MoarVM/reframe: in the future.
13:50 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/de11e2992e
13:52 brrt good chance i've found the (a) reframe-jit bug
13:54 jnthn \o/
13:54 jnthn Wow, we might get this merged soon, then :)
13:55 jnthn A little closer to release than I first hoped, but we've still a week...
13:56 dalek MoarVM/reframe: 08ce118 | jnthn++ | src/gc/debug.h:
13:56 dalek MoarVM/reframe: Have GC debugging off by default.
13:56 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/08ce11856a
13:58 * jnthn tries stuff with an optimized build
13:59 dalek MoarVM/reframe-jit: e6b0809 | brrt++ | src/jit/emit_x64.dasc:
13:59 dalek MoarVM/reframe-jit: Assign jit entry label prior to invoking
13:59 dalek MoarVM/reframe-jit:
13:59 dalek MoarVM/reframe-jit: sp_findmeth may be invokish if the spesh slot does not mathc the object
13:59 dalek MoarVM/reframe-jit: type. We assigned the jit entry label after getting the return value
13:59 dalek MoarVM/reframe-jit: from MVM_6model_find_spesh, which is 1 if it invoked. This used to be
13:59 dalek MoarVM/reframe-jit: no problem because FRAME (r12) used to point to the current frame at
13:59 dalek MoarVM/reframe-jit: all times, but that is no longer true now. Fixed by moving the
13:59 dalek MoarVM/reframe-jit: assignment prior to the invokish, as in other cases.
13:59 dalek MoarVM/reframe-jit: review: https://github.com/MoarVM/MoarVM/commit/e6b08090d6
14:04 brrt all is not ready yet
14:05 jnthn Ah, but closer? :)
14:06 brrt yes
14:07 brrt we finish compiling CORE.setting, but crash in the tests
14:07 brrt although... maybe i should rebase on your current branch
14:07 jnthn Yeah, quite possibly...I dunno when you branched off
14:07 jnthn I just discovered that we seem to have issues with NativeCall callbacks on Moar HEAD
14:08 jnthn nwc10: Dunno if you `make test` in your testing and also see that?
14:12 nwc10 t/04-nativecall/08-callbacks.t ........... Dubious, test returned 1 (wstat 256, 0x100)
14:12 nwc10 Failed 8/8 subtests
14:12 nwc10 oh yes, it has almost escaped the top of that window
14:13 brrt reframe-jit fails everything currently
14:15 jnthn brrt: After rebasing?
14:15 brrt everything in nativecall
14:15 brrt aye
14:15 jnthn Ah
14:15 brrt not everything outside of nativecall
14:15 brrt with segv
14:15 jnthn :)
14:15 brrt should be reasonably easy to figure out...
14:15 jnthn Yeah, I think I see what's going on with NativeCall too
14:16 domidumont joined #moarvm
14:25 jnthn grmbl, no, still bust
14:25 nwc10 ASAN says "SEGV"
14:26 jnthn Any backtrace?
14:28 nwc10 not much: http://paste.scsys.co.uk/513626
14:46 dalek MoarVM/reframe: 0fa45a1 | jnthn++ | src/core/ (3 files):
14:46 dalek MoarVM/reframe: Fix native callbacks after frame refactors.
14:46 dalek MoarVM/reframe:
14:46 dalek MoarVM/reframe: Add missing stack->heap and rooting, and make stack->heap promotion
14:46 dalek MoarVM/reframe: cope with nested runloops.
14:46 dalek MoarVM/reframe: review: https://github.com/MoarVM/MoarVM/commit/0fa45a10c3
14:48 jnthn There we go
15:08 jnthn reframe is 1,149 additions and 876 deletions over 77 files
15:15 diakopter finally github fixes its pricing
15:15 diakopter (per user instead of per repo)
15:23 nwc10 jnthn: build and test run won't be finished before I need to leave to catch the train
15:23 nwc10 it's in "stage mast"
15:26 nwc10 make test is "All tests successful.".
15:26 nwc10 Now starting spectests
15:26 * nwc10 afk
15:36 jnthn Plenty of follow-up optimizations/improvements, but will do those after the merge of reframe :)
15:44 nine jnthn: pull request updated: https://github.com/MoarVM/MoarVM/pull/364
15:47 dalek MoarVM: 222115d | niner++ | docs/gc.markdown:
15:47 dalek MoarVM: Add docs about MVMROOT courtesy of jnthn++'s excellent blog
15:47 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/222115d041
15:47 dalek MoarVM: 4115a67 | niner++ | / (8 files):
15:47 dalek MoarVM: Implement loadbytecodebuffer OP
15:47 dalek MoarVM:
15:47 dalek MoarVM: Usefull for loading mbc from an in-memory buffer.
15:47 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/4115a67c8c
15:47 dalek MoarVM: f9b7e4d | niner++ | / (10 files):
15:47 dalek MoarVM: Implement loadbytecodefh
15:47 dalek MoarVM:
15:47 dalek MoarVM: Usefull for loading mbc from an already opened file handle allowing for us
15:47 jnthn nine: Merged, danke
15:48 nine Yeah! Thanks :)
15:49 jnthn It *seems* moving ->env to be allocated on the new callstack will be pretty straightforward.
15:49 jnthn ->work will be a different matter.
15:53 jnthn Anyways, time for some rest :)
17:15 patrickz joined #moarvm
17:22 diakopter but is it faster XD
17:24 timotimo well, it gets us rid of one extra allocation per stack frame construction
17:25 domidumont joined #moarvm
17:32 nwc10 jnthn: Result: PASS
17:32 timotimo ooooh, enat
17:32 timotimo neat*
17:32 timotimo that's not a spectest, is it?
17:33 nwc10 spectest from origin/reframe
17:33 timotimo neat!
17:33 timotimo time to try a spectest with a bit of gc torture?
18:22 diakopter jnthn: one idea (for optimizing ->work): you could use 4 bits of the object header to store "custom alloc size" flag/size. Basically a way to allocate space for an object whose size will remain the same throughout its lifetime, but will vary across instances. If all four bits are 1, use the standard size from the STable (as it does now). Otherwise use the 4 bits' value as an index to a table on the STabl
18:23 diakopter e with custom size classes
19:25 dalek joined #moarvm
19:33 vendethiel joined #moarvm
20:02 geekosaur hm, google finally released the hounds^WMay update
20:03 diakopter geekosaur: not to me :( :( I guess I gotta wait "up to two weeks"
20:04 geekosaur I got it on N5 and N7 literally within the past 10 minutes
20:04 geekosaur 5X hasn't got it yet though
20:04 diakopter *cry*
20:50 TimToady joined #moarvm
21:17 timotimo now i've got code that removes the need for closures for at least core mapped ops (not hll mapped ops so far)
21:19 timotimo i'll make a sensible measurement now, hopefully
21:23 timotimo OK, so before the very first patch in that series, nqp-m -e 'nqp::say("hi")' gave me 14146.222222 maxresidentk on average
21:23 timotimo after my latest patch, it's now only 13557.538462 maxresidentk
21:24 timotimo in between it was 13809.866667 maxresidentk
21:24 timotimo m: say "percentage-wise that's { 13809.866667 / 14146.222222 } in between and { 13557.538462 / 14146.222222 } at the end"
21:24 camelia rakudo-moar a31ab3: OUTPUT«percentage-wise that's 0.976222941382 in between and 0.95838579723 at the end␤»
21:25 timotimo i'm not 100% satisfied with the code, so i'll PR it again to see what others think
21:36 timotimo https://gist.github.com/timo/0dfba0c72f06509fd7ede85f5a021688#file-03_very_latest-txt
21:36 timotimo no big closures any more! (but only in nqp)
21:39 diakopter wow, bigger difference than I would've imagined
21:39 diakopter (in memory)
21:39 timotimo oh?
21:39 timotimo well, you could have told me before i put this work in :P
21:39 timotimo hm. actually, it wasn't that much work
21:39 timotimo so i suppose the pay-off is enough compared to the work i put in
21:40 diakopter well maybe, if it carries over to rakudo I suppose
21:41 timotimo it does, but only partially, as rakudo uses the hll ops
21:41 timotimo the hll ops mechanism, that is
21:41 timotimo but also, it only has a small amount of hll ops it registers
21:41 diakopter ohhhh wait, you mean the closure mapped ops in the mast compiler
21:42 diakopter yes oh I definitely expected at least that much improvement
21:42 timotimo ah, you see, the 01 and 02 files are from rakudo, the 03 file is from nqp
21:42 timotimo that must be why it seems much better :D
21:43 * timotimo points that out more clearly in the text files
21:43 timotimo what other thing were you expecting i'd improve? :)
21:47 timotimo m: say "rakudo got an improvement of { 65352 / 65996 }"
21:47 camelia rakudo-moar bffc3a: OUTPUT«rakudo got an improvement of 0.990242␤»
21:47 timotimo much smaller win, nusurprisingly :(
21:48 diakopter nothing in particular I guess. I think it's a good patch
21:49 timotimo you haven't seen the new patch :\
21:49 diakopter er
21:49 diakopter XD where is the new patch
21:50 timotimo it's in the gist now
21:51 timotimo next thing i'll try is to hee how often moarop and op are the same thing, and maybe not put an entry into the op hash in that case
21:59 diakopter I think the lesson here is that automatic optimization has a long way to go
21:59 diakopter XD
22:00 timotimo er ... i guess?
22:00 timotimo anyway, it seems like more than 2/3rd of ops have the same $op as they have $moarop
22:00 timotimo so, i'd call that a win
22:01 timotimo i could even share decont arrays between ops, i don't think there's many different ones at all
22:02 timotimo really, those strings are all in the string heap anyway, but i figure it'll at least make the hash a bit more compact
22:03 timotimo especially since ou rhashes always have a bit of extra free storage allocated for their entries even if we could in theory shrinkwrap them for perfect size at some point during run time
22:04 timotimo 32 + 29736 bytes
22:04 timotimo that's the size of the op-to-moarop hash before that particular optimization
22:04 timotimo surprisingly big, if you ask me
22:05 timotimo then again, it has more than 500 entries
22:05 timotimo m: say 29736 / (1063 / 2)
22:05 camelia rakudo-moar bffc3a: OUTPUT«55.947319␤»
22:06 timotimo 55 bytes per entry isn't gigantic
22:06 timotimo it's big, but not end-of-the-world big
22:08 timotimo 32 + 7392 bytes
22:08 diakopter MVM_op_counts = 800
22:08 diakopter coooool
22:09 timotimo m: say "the hash shrunk to { 7392 / 29736 }x its size, by { 7392 R- 29736 } bytes"
22:09 camelia rakudo-moar bffc3a: OUTPUT«the hash shrunk to 0.248588x its size, by 22344 bytes␤»
22:09 timotimo "wow, 22 kilobytes saved" ... ...
22:10 diakopter timotimo: this is, like, worrisome: https://github.com/MoarVM/MoarVM/commit/1088538fe4408e4949b173f866b8ea7e4916f674
22:10 timotimo but it's really quite neat that the heap analyzer lets us find stuff like this
22:10 timotimo diakopter: enh. could be worse. this is only about fact discovery, and literal_facts has another switch/case that will just return: when it doesn't know an opcode
22:10 diakopter (agreed)
22:11 diakopter hm
22:11 timotimo but yeah. that it got in there is very misfortunate
22:12 timotimo guess who put it in there ;)
22:12 timotimo it was me, of course
22:12 timotimo don't let the little ones play without proper oversight
22:13 * psch better stops stumbling through nqp-j then :P
22:13 timotimo no, nqp-j will blow up if you do something wrong ... or at least it would do so much earlier than moar would
22:14 psch oh, i've had plenty of moments where things looked fine far longer than they should've... :)
22:20 timotimo ugh, even with our test suite we're basically flying blind :\
22:20 arnsholt C is notoriously bad about letting you blow entire legs off without telling you, though
22:21 timotimo :<
22:21 timotimo should have built moarvm in rust, clearly
22:22 diakopter hah, it had barely appeared
22:23 arnsholt IMO, the best thing to do when programming in C is to realize that you're going to make all kinds of stupid allocation bugs and the like and use all the tools at your disposal to root them out as fast as possible
22:23 arnsholt So things like asan and valgrind, clang static analyzer, etc. become really important
22:23 arnsholt And just plain old tests
22:23 timotimo right, we use asan and valgrind most of the time
22:29 psch yeah, i like that about "learn the c the hard way", too
22:30 psch they introduce valgrind in like chapter 2 or something
22:30 timotimo that's very wise
22:59 timotimo i wonder where exactly the nativecall-related code-gen should hang

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary