Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-08-23

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:05 dalek joined #moarvm
02:49 dalek joined #moarvm
05:38 nine afl-fuzzy++
05:39 nine timotimo++ :)
05:41 travis-ci joined #moarvm
05:41 travis-ci MoarVM build errored. Timo Paulssen 'uncuddle an else'
05:41 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/154295591 https://github.com/MoarVM/MoarVM/compare/bca85b242eb4...490666638314
05:41 travis-ci left #moarvm
05:49 lizmat joined #moarvm
06:49 domidumont joined #moarvm
06:54 domidumont joined #moarvm
07:47 zakharyas joined #moarvm
08:13 TheLemonMan joined #moarvm
08:42 TheLemonMan timotimo, wrt your param checking code, I think the code would look neater by putting the outer if into the for, this way you can avoid writing two similar for
08:43 TheLemonMan #145 can be closed as it's been fixed at the rakudo level and roast tests have been submitted
09:08 TheLemonMan also #175 seems to be fixed now
09:26 lizmat_ joined #moarvm
09:26 dalek joined #moarvm
09:31 timotimo i thought the code would be clearer if i separate it out
10:11 dalek MoarVM: bace47f | LemonBoy++ | src/profiler/heapsnapshot.c:
10:11 dalek MoarVM: snprintf returns an int, not a size_t.
10:11 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/bace47f217
10:11 dalek MoarVM: 5c7fc80 | lizmat++ | src/profiler/heapsnapshot.c:
10:11 dalek MoarVM: Merge pull request #396 from LemonBoy/tautological-compare
10:11 dalek MoarVM:
10:11 dalek MoarVM: snprintf returns an int, not a size_t.
10:11 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/5c7fc80bc3
10:47 travis-ci joined #moarvm
10:47 travis-ci MoarVM build passed. lizmat 'Merge pull request #396 from LemonBoy/tautological-compare
10:47 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/154407989 https://github.com/MoarVM/MoarVM/compare/490666638314...5c7fc80bc3c1
10:47 travis-ci left #moarvm
12:41 zakharyas joined #moarvm
12:43 zakharyas1 joined #moarvm
14:31 timotimo so, you know how python can look for modules in a zipfile that's concanetaned to the binary?
14:31 timotimo maybe we should take that route towards making moar-based "fat packs"
16:10 domidumont joined #moarvm
16:46 brrt joined #moarvm
16:47 brrt timotimo: i agree, that is probably the way to go
16:48 brrt you'd still need a reasonably clever wrapper executable
16:49 brrt and you'd need a statically linked moar (i'd think), or at least, static to our internal dependencies
16:50 brrt https://lwn.net/Articles/691070/
16:50 brrt dude says CPython guys will start making and/or using a JIT
16:50 brrt personally, i think CPython has painted itself into so many corners on that front that I think it unlikely
16:52 brrt also, the claim is that pypy and friends can't deal well with ffi calls
16:52 brrt it's a bit similar to our situation with nativecalls
16:53 brrt but... we have some hope of jitting these fast in the future
16:53 TheLemonMan that's a start!
16:54 brrt TheLemonMan: if you care, i can explain you the current state of that :-)
16:54 TheLemonMan brrt, I'm all ears :)
16:54 brrt basically, i've written the existing JIT, basically as a bolt-on to the spesh framework, about two years ago
16:55 brrt sorry, basically twice
16:56 brrt now the existing JIT is very simplistic: it literally translates the spesh graph (an annotated representation of the bytecode) to machine code using a little library called DynASM
16:56 brrt and the thing about that is: there is a literal mapping between moarvm opcodes and JIT output
16:57 brrt to such an extent that i can read from the machine code that is generated back into the bytecode
16:57 brrt that's handy for debugging, i've got to say that
16:57 brrt but it is a lousy way to generate code, and furthermore, it means that we have no easy ways to do 'lowering transformations'
16:58 brrt so if we have an array object, and we know it is an array object, and that has an integer index, then it is still kind of tricky to translate that into efficient bytecode
16:59 brrt stop me if this stops making sense
16:59 brrt so, what i've done, is split the JIT into two stages
16:59 brrt the first stage translates the moarvm bytecode into a low-level DAG structure called 'expression trees'
17:00 brrt this is a lowering that transforms from MoarVMs memory-to-memory model, to a register-to-register model
17:01 brrt all this while the existing JIT is still in place, mind you
17:01 brrt and the second stage (called 'tiling') transforms those trees (or DAGs) to a linear list of operations
17:01 brrt machine code operations
17:02 brrt well, function pointers to routines that output machine code, but you get the idea
17:02 brrt had to hack DynASM to do that, because official DynASM doesn't play nice with extended registers in x64
17:03 brrt so once i'd done that, i thought i'd just walk that list, invoke its generating functions, and call it a day, but it turns out that register allocation is both essential and tricky
17:04 brrt and that it really wants proper linear order and multiple passes and lookahead, and these things weren't there when i had to stop working on that fulltime
17:04 brrt (because i was in my final year of studying)
17:05 brrt anyway, i've more or less continued refactoring bits of the new JIT to get that nice linear order and a proper structure for the register allocator
17:05 brrt once that is done, it's basically good to be merged, and then the real fun starts
17:06 brrt because what we can do is take the specialized types and have them insert their own bit of expression code for opcodes
17:06 brrt so nativecall can be lowered at JIT time to any old function call
17:06 brrt and array indexes can be lowered to a single instruction
17:06 brrt types permitting, of course
17:08 TheLemonMan that sounds like an awesome plan!
17:09 brrt other nice bits include: to port the JIT to another arch, for example ARM, we only need to port the tiler and register allocator (or have a data-driven register allocator, which is the actual plan)
17:09 brrt and the tiles
17:09 brrt not the tiler
17:09 brrt tiler remains the same
17:09 brrt it is quite a nice plan if i do say so myself :-)
17:10 brrt (and we can do transforming modifications on the 'expression tree' like ... whatsitcalled
17:10 brrt when you remove duplicate computations of the same thing
17:12 timotimo brrt: what, pypy can't deal well with ffi calls? is that a joke?
17:12 brrt well, that's not my claim
17:12 timotimo pypy already jits ffi calls into the same code a C compiler would
17:12 timotimo where did that person get their info?
17:12 brrt really?
17:12 brrt well, then pypy is ahead of us
17:12 brrt (as i would expect, really)
17:12 brrt i dunno
17:13 brrt he's a scientist in a eh... UCB
17:13 brrt focussed on the 'big picture'
17:14 timotimo pypy is already ahead of pretty much everyone forever :)
17:15 brrt anyhow, i think the issue with pypy and numpy is, or used to be, that they had their own implementation of numpy called numpypy
17:15 timotimo i think you were looking for "common expression extraction" or something?
17:15 timotimo that's right
17:15 brrt and their vectorization just isn't as good as the C and FORTRAN compiler's
17:15 timotimo i contributed a tiny bit of code to numpypy
17:15 brrt ah, yes, that's the one
17:15 brrt common subexpression elimination
17:15 timotimo yes!
17:16 brrt CSE is cool. although not a definite optimization, because it increases register pressure
17:16 brrt can be a done with a bottom up traversal and a hash table
17:17 brrt i'll leave the rest to you :-P
17:17 brrt 'an exercise for the reader'
17:21 brrt the literal LWN quote is: "The first consequence that Smith described is that, for libraries like NumPy, there is a "catch-22". If it needs to be fast for CPython, it has to be written in C, but if it needs to be fast for a JIT, you cannot use C. He showed a simple mysum() function that totaled up the elements in an iterable. If it is passed a Python object like list(range(N)), the JIT knows what it is and can do lots of optimizations. But if it is passed a NumP
17:21 brrt y array, which is "opaque C stuff", the JIT doesn't understand it, so it will have trouble even achieving the performance of a non-NumPy version on a JIT-less CPython. "
17:22 brrt that is pretty much certainly not true, but whatever
17:22 timotimo do you know of the CSE band? :)
17:23 brrt i do not know
17:24 timotimo i bet if you use CSE for your band, you'll not be making catchy songs
17:24 brrt hahahahaha
17:24 domidumont1 joined #moarvm
17:24 brrt well, perhaps better than the deflate band
17:26 timotimo hah
17:32 brrt anyway, the latest latest bit is that i've finally started working on the register allocator, that i finally have a good theoretical basis for doing that, that, in other words, i'm an idiiot who underestimated the scope of the project
17:35 cygx joined #moarvm
17:35 cygx timotimo: perhaps they meant that pypy cannot across the FFI boundary
17:36 timotimo that'd be true, then
17:36 timotimo but neither can a C compiler, really
17:36 cygx Graal/Truffle (basically the same thing as PyPy with partial evaluation instead of tracing, cf http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-partial-evaluation/ )
17:37 cygx where did the rest of that sentence go?
17:37 cygx Graal/Truffle can do such optimizations by implementing a C as well as LLVM bitcode interpreter
17:38 timotimo then you'll have to keep the source around for things you want to ffi?
17:39 cygx if you want to do inlining/partial evaluation/... across the FFI boundary, yes
17:39 timotimo if so, i'm unwilling to keep calling that FFI :)
17:40 cygx (in practice, probably LLVM bitcode, not C source code)
17:44 brrt joined #moarvm
17:45 brrt why not bytecode interpretation
17:45 brrt it isn't that hard, it's not like it's a friggin huge interface
17:45 brrt and they have the manpower to do it
17:45 brrt however, one has to ask oneself whether any of that is the point
17:48 timotimo you mean interpreting the actual x86 machine code?
17:48 brrt aye
17:48 brrt why not
17:49 timotimo right ...
17:49 timotimo maybe we'll end up being the first ones to be crazy enough? we'll implement it for moarvm :P
17:49 brrt if you're so adamant to be faster than calling c
17:49 brrt hahaha
17:49 brrt we have a reputation for crazy to uphold
17:52 brrt ... i'm done with internet technology news, though
17:52 brrt the frequency of times that i read something that begins interesting and ends with 'oh god, not this again' is too damn high
17:52 timotimo yes, oh lord
17:52 brrt oh lord?
17:53 timotimo tech news sites
17:53 jnthn I don't think that was meant as an honorific :P
17:53 brrt no, neither did i
17:53 jnthn Though I like how it can be parsed that way :)
17:53 brrt but maybe i was coming accross as eh.. oh well
17:53 * jnthn has his weekly day of Moar / Perl 6 hackery tomorrow
17:53 brrt \o/
17:53 brrt what's on the menu this time
17:54 jnthn Same as the last few weeks, trying to robustify concurrency related things. :)
17:54 timotimo here's an amusing one
17:54 timotimo Program received signal SIGFPE, Arithmetic exception.
17:54 timotimo 0x00007ffff771e9e5 in deserialize_repr_data (tc=0x6047c0, st=0x609030, reader=<optimized out>)
17:54 timotimo at src/6model/reprs/P6opaque.c:1063
17:54 brrt amuse us
17:54 timotimo 1063             if (cur_offset % spec->align) {
17:54 brrt how
17:54 timotimo the fuzzer found that :)
17:54 timotimo :D
17:55 brrt and spec->algin can be zero how
17:55 timotimo yes, how indeed.
17:55 timotimo $2 = {inlineable = 0, bits = 0, align = 0, boxed_primitive = 0, can_box = 0, is_unsigned = 0 '\000'}
17:55 timotimo apparently: "uninitialized"
17:55 timotimo oh, apparently the fuzzer went ahead and just set the align value to 0 in the serialized blob
17:55 timotimo and that's how it asplodes
17:56 timotimo just a case of us not putting a check against 0 there to fail with "that's obviously BS."
17:56 timotimo mighty AFL finds every single flaw, and then some.
17:59 brrt hmmm
18:00 brrt yeah, arguably that is a flaw and wants a check
18:37 utat joined #moarvm
18:39 domidumont joined #moarvm
18:46 jnthn Phew, I think I might have finished writing up my grant report...
18:47 timotimo the kind that goes on your blog?
18:47 timotimo or is there a separate kind you send off to TPF directly?
18:48 timotimo #3  0x00007ffff75aef6c in MVM_interp_run (tc=tc@entry=0x6047c0, initial_invoke=0x0, invoke_data=0x1)
18:48 timotimo how does that end up with initial_invoke being null?
18:48 timotimo when called from MVM_vm_run_file
18:48 [Coke] I'm pretty sure most reports to the tpf on most grants are publically blogged
18:49 timotimo but it then goes on to call toplevel_initial_invoke anyway
18:51 timotimo anyway, it's going on to frame_force_to_heap a null pointer, but i wonder where the earliest point is that we can spot this wrongness
18:54 jnthn timotimo: The TPF kind that hopefully leads to payment :)
18:54 jnthn Yes, it will appear publicly
18:56 timotimo :D
19:00 jnthn dinner; bbl
19:25 dalek MoarVM: 9267528 | timotimo++ | src/6model/reprs/P6opaque.c:
19:25 dalek MoarVM: don't allow zero alignment in p6opaque storage spec
19:25 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/9267528871
19:25 dalek MoarVM: edd5839 | timotimo++ | src/core/bytecode.c:
19:25 dalek MoarVM: index check lexicals when reading static flags
19:25 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/edd58392e1
19:55 JimmyZ https://github.com/MoarVM/MoarVM/issues/234 # it would be nice if someone can fix it :)
19:55 JimmyZ SEGV bug
19:57 jnthn Hm, does it for me too
19:59 jnthn Will have a look in the morning :)
20:00 JimmyZ thanks
20:14 Ven joined #moarvm
21:00 TheLemonMan joined #moarvm
21:04 TheLemonMan JimmyZ, wrt #234 I think it's due a poor interaction between libuv and the exception handler, when an exception is raised you're basically longjmp'ing from a libuv callback to the opcode dispatch loop
21:05 TheLemonMan eg: https://ptpb.pw/UOuH
21:26 lizmat joined #moarvm
23:17 timotimo i suppose i can have a local patch to make that "crash" "not a crash" but a fail, so that i can get more "real" crashes when i run afl the next time
23:17 timotimo refering to the initial invoke thing where the frame is 0x0
23:27 TimToady joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary