Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-08-02

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:24 harrisi joined #moarvm
01:50 vendethiel joined #moarvm
01:52 ilbot3 joined #moarvm
01:52 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
02:00 https_GK1wmSU joined #moarvm
02:00 https_GK1wmSU left #moarvm
03:09 harrisi jnthn: You're Jonathan Worthington, I'm assuming? If so, is anything relating to your dissertation public? It sounds as though it is researching exactly a project I've been interested in working on.
03:13 harrisi jnthn: .. this is disappointing, but I found it actually. :)
05:53 brrt joined #moarvm
06:27 brrt joined #moarvm
06:29 harrisi jnthn: hope you don't me bothering you, but the link to the slides for "All Your Dynamic Language Are Belong To Us (Guest Lecture at Stockholm University)" gives a 404, from your articles page (http://www.jnthn.net/articles.shtml). Do you have that up somewhere? It looked like it might be related to bytecode translation, and I've read all your other slides on it.
06:58 brrt joined #moarvm
07:19 samcv good *
07:21 brrt good * samcv
07:21 samcv going to update Unicode Database for unicode 10 and also add in names for Hangul (Korean) codepoints
07:22 samcv since that is done algorithmically. though i still have no clue why unicode didn't just put the damn names in the unicode data file...
07:22 samcv there's literally no reason i can think of why they can't have just written them in there
07:27 brrt :-)
07:27 brrt ask them!
08:01 zakharyas joined #moarvm
08:07 harrisi jnthn: I have a lot more questions and comments, but I wanted to say that your paper now has one of my favorite quotes in any research paper I've ever seen. That quote being "As in real life, getting too intimate leads to excessive complexity." :)
08:17 vendethiel joined #moarvm
09:00 jnthn morning #moarvm o/
09:01 jnthn harrisi: Yes, I'm that Jonathan. It's been years since I worked on bytecode translation though.
09:02 jnthn I'm...not sure about those missing slides. Hmm. But I don't think that talk actually was about the bytecode translation work.
09:03 jnthn Pretty sure it was about the idea of a common AST to compile different languages to.
09:13 brrt good hi jnthn
09:13 jnthn o/ brrt
09:13 jnthn How'd the talk go yesterday?
09:13 brrt oh, my talk is next week :-)
09:14 brrt on friday
09:14 brrt sorry for the misunderstanding
09:14 brrt anyway, that's basically what the GraalVM folks are doing nowadays
09:14 jnthn Oh!
09:14 brrt (a common AST)
09:15 jnthn So at TPC in Amsterdam?
09:15 brrt uhuh
09:15 brrt i'm a bit worried that i've been 'stuck' in this subject matter for so long that what is really obvious to me isn't to others, and that i have no idea of what that is
09:20 moritz the "curse of knowledge"
09:21 jnthn brrt: Are you expecting to merge "this week"?
09:21 brrt ehm. maybe. i think i've uncovered another bug :-)
09:21 jnthn ah
09:22 brrt i'm not 100% sure it's not a template bug
09:22 brrt otherwise, yeah, this week would be good
09:22 brrt i'll do a pull-request
09:23 brrt if it causes a lot of fallout, we can always switch to an enable-on-demand model for the time being
09:29 nwc10 brrt: building even-moar-jit with ASAN. Into NQP and getting a lot of
09:29 nwc10 src/jit/linear_scan.c:433:32: runtime error: shift exponent 60 is too large for 32-bit type 'int'
09:29 nwc10 src/jit/linear_scan.c:429:26: runtime error: shift exponent 60 is too large for 32-bit type 'int'
09:29 nwc10 src/jit/linear_scan.c:437:27: runtime error: shift exponent 63 is too large for 32-bit type 'int'
09:30 dogbert17 joined #moarvm
09:30 brrt really? we shouldn't be doing that on an 32 bit int
09:30 brrt thanks!
09:32 brrt the   constant 1 must be the thing that's intepreted as an int
09:32 brrt that's silly
09:32 nwc10 that's C
09:32 nwc10 (not looked at the code, but my innacurate/incomplete understanding of C is that the constant 1 has value 1 and type int)
09:35 AlexDani` joined #moarvm
09:48 dogbert17 jnthn: is the new MoarVM, i.e. master, still supposed to be slower than e.g. 2017.07 when running multithreaded apps?
09:51 jnthn Well, the main remaining slowdown is independent of that.
09:53 jnthn (threads or not, I mean)
09:59 dogbert17 I have a script which is ~20% slower in mt mode but only 10% slower in st mode compared to 2017.07 (in 32 bit mode :) https://gist.github.com/dogbert17/fc516b360ef998c03ca63498b635dae4
10:00 buggable joined #moarvm
10:09 dogbert17 I have added callgrind results to the gist (MoarVM master)
10:18 jnthn Yeah, a lot of time in multi-dispatch which is what I'm currently repairing/improving the spesh-time resolution of
10:20 dogbert17 cool, I could do another run when those fixes show up
10:24 jnthn Yeah, was working on it, then realized I need to do a bit of work on deopt
10:42 brrt nwc10, thanks for that
10:42 brrt today i learned, C stdint has macros INT64_C(...)
10:43 brrt and UNINT64_C(..)
10:46 Geth ¦ MoarVM: 0585b8d49c | (Jonathan Worthington)++ | 10 files
10:46 Geth ¦ MoarVM: Store deopt one target directly in bytecode.
10:46 Geth ¦ MoarVM:
10:46 Geth ¦ MoarVM: Rather than needing to use the offsets to resolve them each deopt.
10:46 Geth ¦ MoarVM: This brings the interpreted approach in line with what the JIT does,
10:46 Geth ¦ MoarVM: but further to that it means that we can have a sequence of guards
10:46 Geth ¦ MoarVM: that will deopt to the same place, which will be used for upcoming
10:46 Geth ¦ MoarVM: improvements to invocation specialization.
10:46 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/0585b8d49c
10:50 brrt i think i've found my bug
10:50 brrt it's related to the 'optimistic' store insertion
10:51 Geth ¦ MoarVM: a2128f15ba | (Jonathan Worthington)++ | src/jit/graph.c
10:51 Geth ¦ MoarVM: Ensure JIT uses index same as interpreter.
10:51 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/a2128f15ba
10:55 brrt wait, you made the deopt target an instruction parameter?
10:55 Geth ¦ MoarVM: 6ce4a2d1dc | (Jonathan Worthington)++ | src/spesh/optimize.h
10:55 Geth ¦ MoarVM: Allow calls with more args to be optimized.
10:55 Geth ¦ MoarVM:
10:55 Geth ¦ MoarVM: 4 is a little low, given nameds can need a couple of slots.
10:55 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6ce4a2d1dc
10:55 jnthn brrt: Yeah
10:55 brrt jnthn++, then jnthn++, then jnth++
10:55 brrt jnthn++
10:55 jnthn brrt: So could probably simplify the JIT code further
10:56 brrt oh man
10:56 brrt i wonder why i never did that myself
10:56 brrt but anyway
10:56 brrt thanks!
10:56 brrt that helps a hole bundle
10:56 brrt *whole
10:57 jnthn The reason I want to do this is that it will let me, hopefully, have multiple guards that deopt to the same point
10:58 brrt that makes a bunch of sense
10:58 brrt it also is much simpler, i think
10:59 brrt because it avoids a lookup at guard-failure
10:59 jnthn Need to think carefully about this though
11:00 jnthn 'cus where we put the guards matters for what of the call resolution we can eliminate
11:00 jnthn I was pondering stacking guards before prepargs
11:00 brrt uhuh
11:00 brrt the careful thinking is probably why i didn't do it
11:00 brrt :-P
11:01 jnthn But if I do that and they fail then we need to keep the callee stored in a register
11:01 jnthn So the deopt can find it
11:01 jnthn We can still have optimized the resolution of it I guess
11:05 jnthn Anyway, I guess I'll stack them up ahead of the prepargs
11:05 jnthn Though deopt one points are presently always *after* the instruction
11:06 jnthn And I'd like them before in this case I guess
11:06 jnthn Maybe I should just spot this in the graph and handle it specially
11:07 jnthn I figure in most cases prepargs will have a previous instruction
11:08 brrt always, but not always within the same basic block
11:09 jnthn Yeah, hmm
11:10 brrt (i've debugged my issue though)
11:10 jnthn I guess I could introduce a :predeoptonepoint or so
11:10 jnthn That records the offset before the prepargs
11:12 brrt (retrospectively, inc_i/dec_i were the wrong way to solve an understandable problem)
11:21 harrisi jnthn: is there still source somewhere?
11:22 harrisi jnthn: I'm only about half way through your dissertation (4:22 am here - need to sleep), but there are a couple of things I'm not sure are covered in that, that I am interested in.
11:24 harrisi one, stack -> register has been a complicated thing to think through, which your paper is helping with. I planned on doing stack <-> stack (.pyc <-> .class, for example), but there are a lot of type issues I can imagine
11:25 jnthn Good question on the source. I...actually don't know. I think I open sourced it after I was done with the dissertation
11:25 jnthn It's been > 10 years by now :)
11:26 jnthn In that time the Parrot project also ceased to be developed
11:27 harrisi the above leads me to major question number two. if I'm going from python bytecode to java bytecode (.pyc -> .class), I have to handle types, since python bytecode is untyped and java bytecode is typed. I actually just got to the part in your paper about static analysis of .net instructions for handling stack types so I think looking more into the CLR in general could help. I looked a bit at the .pyc files of two python files that add integers and one that a
11:27 jnthn (got cut off at "one that ad")
11:27 harrisi jnthn: yeah, the slides say it's available in the parrot source but I think I'd have to look historically.
11:27 jnthn I was going from static -> dynamic, which is an easier problem ;)
11:28 harrisi one that adds integers and one that adds strings. identical instructions, but the metadata should provide everything I need, I hope.
11:29 harrisi yeah, static -> dynamic seems somewhat easier, but I don't know much of anything. :)
11:29 harrisi all I know is that your paper and slides have been helpful recently, so thank you.
11:29 jnthn I couldn't imagine when I did it that people would care 10 years later :-)
11:29 jnthn Glad it's of some use
11:30 harrisi one more question before I go. why did you end up devoting time to parrot and other things instead of continuing with bytecode translation?
11:30 harrisi was there something that seemed like a significant show-stopper?
11:31 harrisi I'm sure it's in the paper but it's hard to wait!
11:31 Geth ¦ MoarVM: 0ad9ce1412 | (Jonathan Worthington)++ | 5 files
11:31 Geth ¦ MoarVM: Make prepargs a pre-deopt-one point.
11:31 Geth ¦ MoarVM:
11:31 Geth ¦ MoarVM: Which is a deopt point where we record the offset before the
11:31 Geth ¦ MoarVM: instruction, so if we depot we will re-run that instruction.
11:31 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/0ad9ce1412
11:32 jnthn Well, I got into working on Parrot because I wanted to contribute something for the Perl 6 project, and that was the most practical place I could see to help with at the time.
11:34 jnthn The bytecode translation stuff was an intresting dissertation project. If there'd been people using it, I'd probably have kept going. But Parrot never really got the up-take.
11:34 harrisi I'm a little over a year into community college studying computer science (with a little bit of self teaching prior), but my dreams for various languages' bytecode to be translated between each other as needed seems.. very useful, and mostly very interesting. your work is actually the first I've seen about it which talks about this at all in detail. it's a little surprising.
11:35 jnthn There has been work on this in the .Net/JVM space I believe
11:35 jnthn ut I think
11:36 jnthn *But I think part of the practical problem is that most VMs get instruction sets designed for their target language
11:36 harrisi and there are projects like jython and ironruby but they're not bytecode translators, as far as I can tell.
11:36 jnthn And not just that, but standard library functions too
11:37 harrisi right. one of the most concerning things to me now is (thanks to your slides) the very same code behaving very different across languages.
11:38 jnthn So at some point you reach I/O, for example, or concurrency primitives. Generally, languages do some amount of this stuff in their standard library, and then have the low-level bits call into VM-provided functions
11:38 jnthn And those will differ too
11:38 harrisi in one of your examples you used incrementing strings in python and javascript, I believe. being able to translate semantics easily and effectively seems very powerful though, and I suspect dealing with it at the bytecode level would be easier than source.
11:38 jnthn Yes, in that the semantics of bytecode are far simpler
11:39 harrisi it definitely is way out of my league, but I've been interested for awhile. :)
11:39 harrisi right.
11:39 jnthn Lunch time for me; bbiab
11:40 markmont joined #moarvm
11:40 harrisi jnthn: anyway, thank you, sincerely, for the work you did. if you for some reason you know more resources about this stuff that you could point me to, I'd be forever indebted!
11:41 harrisi night, have a good lunch!
11:46 brrt harrisi: the general rule is that the lower-level the 'language', the simpler the semantics, and the easier the translation
11:46 brrt you can translate everything to everything, it is just a lot of work, and you generally lose efficiency of representation
11:56 AlexDaniel joined #moarvm
11:56 vendethiel joined #moarvm
11:57 travis-ci joined #moarvm
11:57 travis-ci MoarVM build failed. Jonathan Worthington 'Make prepargs a pre-deopt-one point.
11:57 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/260157705 https://github.com/MoarVM/MoarVM/compare/6ce4a2d1dc79...0ad9ce1412f4
11:57 travis-ci left #moarvm
12:02 harrisi brrt: do you have any resources for that kind of thing for me? :)
12:04 brrt no, that's just a general observation… :-)
12:04 brrt you might be interested in the graal project
12:04 brrt which does something similar today
12:04 brrt well, somewhat similar anyway
12:12 jnthn harrisi: One thing that occurred to me over lunch about your mention of the python add instruction earlier: on today's JVM it probably best translates into an invokedynamic than then decides what the right thing to do is.
12:16 jnthn huh, it looks successful then says at the end the build exited with 1? https://travis-ci.org/MoarVM/MoarVM/jobs/260157712
12:17 lizmat jnthn: fwiw, I've seen that a few times in travis reports of rakudo builds as well
12:17 lizmat not sure what to make of that
12:17 jnthn Me either
12:17 AlexDaniel joined #moarvm
12:41 AlexDaniel joined #moarvm
12:42 zakharyas joined #moarvm
12:42 AlexDaniel joined #moarvm
13:01 Geth_ joined #moarvm
13:08 zakharyas joined #moarvm
13:26 jnthn eek, language class time...making progress on call opts though:)
13:37 brrt joined #moarvm
13:49 brrt1 joined #moarvm
14:08 brrt joined #moarvm
14:41 zakharyas joined #moarvm
14:50 zakharyas joined #moarvm
14:58 * jnthn back
15:00 dogbert17 Vítej zp?t
15:00 jnthn Diky
15:00 lizmat proost!
15:01 jnthn Je mi horko a musim pracovat na deopt a inlining... :P
15:01 dogbert17 your language classes seems to be working
15:01 jnthn (It's hot and a have to work on deopt and inlining)
15:01 * dogbert17 had to use google translate myself :)
15:02 jnthn So it turns out that yes I can sneak extra guards in for args based on observed type tuples
15:02 jnthn And it'll net us more spesh calls and inlines (good)
15:02 jnthn But sneaking them in without an entry for them in the deopt table goes rotten when we do inlining
15:02 jnthn So I guess I can't cheat quite that hard.
15:03 jnthn The good news is that growing that table is probably completely safe
15:12 vendethiel joined #moarvm
15:18 vendethiel- joined #moarvm
15:25 markmont timotimo: A quick recap to be sure we're working on the same problem:
15:25 markmont On Fedora 26, I cloned MoarVM, updated dyncall to the latest head, added a call to strerror() to src/jit/compile.c, and then ran examples/hello_world.nqp from the NQP repo:
15:25 markmont sudo setsebool selinuxuser_execstack=off deny_execmem=on
15:25 markmont env MVM_JIT_LOG=/dev/tty nqp-m hello_world.nqp
15:25 markmont This resulted in: "Setting jit page executable failed or was denied: Permission denied. Deactivating jit." together with an SELinux denial for execmem.
15:25 markmont Is this the same problem you've been looking at?
15:26 timotimo yeah
15:26 markmont Excellent.  And please let me know if I should move discussion to a mailing list to make it easier to follow.   :)
15:26 markmont For dyncall, the mmap()/mprotect() code is only called by src/core/nativecall_dyncall.c:unmarshal_callback().  This code is not exercised in the hello_world.nqp example, so we don't know if the dyncall code works when execmem is prohibited (I suspect it does not work -- see below).
15:26 markmont I've looked at the JIT in PCRE2 and the one in Google Chrome V8.  I've found that they both require execmem.  PCRE2 does not attempt W^X at all, while V8 does use mmap()/mprotect() with the same results we're seeing in MoarVM.
15:26 markmont I've verified that the mmap()/mprotect() approach will not work with SELinux and deny_execmem.  I tested under both Fedora 26 as well as under RHEL 6 (roughly equivalent to Fedora 12).  Note that deny_execmem is off by default, so mmap()/mprotect() will work on most Linux machines, including those which disallow executable heaps and executable stacks.  I've taken a quick look at the Linux kernel source, but haven't found an obvious place with the EPERM
15:26 markmont we're getting is being generated; I can look into this more, if desired.
15:26 timotimo you should be able to "make test" in rakudo to get the nativecall tests running
15:27 AlexDaniel joined #moarvm
15:27 markmont Yes, I've done that.  But for the specific case above, I'm trying to keep things as low-level as possible.
15:27 markmont Here is a test case showing that there is no problem with MoarVM's implementation of the JIT: https://gist.github.com/markmont/dcd20d632fa753438f6fc1b3bb3711ec
15:27 markmont The temporary file approach to W^X does work in all cases, see the test case at https://gist.github.com/markmont/b5f1ee6b4c1c7a68b085bb3aae96e3b9
15:27 markmont libffi takes the temporary file approach.  It tries a number of different locations for the temporary file, searches for filesystems that are not noexec, and also attempts to create the file using O_TMPFILE so that it never appears in the filesystem and does not need to be unlinked.  See https://github.com/libffi/libffi/blob/6e2e041b6df6a3c3a5ca8a750dedbbd465e5ca97/src/closures.c#L615
15:28 markmont Based on this, I think we should decide if making MoarVM work on systems that enforce W^X in all situations is something we want to do.  I think it is worth doing and am willing to work on this, but a lot of other software projects seem to be fine with the current mmap()/mprotect() approach.  If we do want to force this, I think re-implementing the approach that libffi uses (and crediting them for it) would be a good way to proceed.  Thoughts?  (Whew,
15:28 markmont that was a lot of stuff!)
15:29 timotimo hm, we could potentially expose the more complicated W^X thing through a configure flag and have ifdefs for that and some .c files tat are only in there if that flag is set or something
15:30 timotimo then we don't have to do the whole searching filesystems song&dance on all platforms
15:30 timotimo anyway, i've gotta go AFK for a bit before i can continue discussing
15:30 markmont Sure thing!  Thanks for your time on this.
15:30 timotimo no, thank you!
15:31 markmont I'll hold off on proceeding until we've discussed it more.
15:39 Geth ¦ MoarVM: 0f94262d7a | (Jonathan Worthington)++ | 2 files
15:39 Geth ¦ MoarVM: Widen scope of deopt annotations function.
15:39 Geth ¦ MoarVM:
15:39 Geth ¦ MoarVM: And make it a tad more generic.
15:39 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/0f94262d7a
15:49 robertle joined #moarvm
16:02 Geth ¦ MoarVM/spesh-invoke: 290e2545be | (Jonathan Worthington)++ | 2 files
16:02 Geth ¦ MoarVM/spesh-invoke: Use callsite info to insert guards before prepargs
16:02 Geth ¦ MoarVM/spesh-invoke:
16:02 Geth ¦ MoarVM/spesh-invoke: These will allow us to use the type tuple to both pick a spesh
16:02 Geth ¦ MoarVM/spesh-invoke: candidate that already exists as well as do multi-dispatch, even in
16:02 Geth ¦ MoarVM/spesh-invoke: the presence of args passedin Scalar containers. This commit only adds
16:02 Geth ¦ MoarVM/spesh-invoke: the guards, and doesn't yet make use of them.
16:02 Geth ¦ MoarVM/spesh-invoke: review: https://github.com/MoarVM/MoarVM/commit/290e2545be
16:03 brrt joined #moarvm
16:04 jnthn Phew :)
16:07 zakharyas joined #moarvm
16:12 brrt \o
16:13 Zoffix |
16:13 Zoffix /\
16:14 brrt i've tracked down the errors in the JIT to the use of too-small variants for the bitmap ints
16:15 travis-ci joined #moarvm
16:15 jnthn haha, spot the bug in the above commit :)
16:15 travis-ci MoarVM build errored. Jonathan Worthington 'Widen scope of deopt annotations function.
16:15 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/260255405 https://github.com/MoarVM/MoarVM/compare/0ad9ce1412f4...0f94262d7a29
16:15 travis-ci left #moarvm
16:15 jnthn heh, not you travis :P
16:16 Zoffix :D
16:17 brrt it's not totally obvious initally
16:17 brrt markmont: if you do want to do this, please be my guest
16:18 Zoffix there's too much code for bug spotting :)
16:19 markmont brrt: I'd like to look into this more. I'm sure timotimo will have more input when he gets back.
16:19 jnthn Turns out I didn't bump the usage count for the register it deconts into, so the guard was against an empty register and so always failed
16:20 brrt my personal opinion (which is of no importance) is that W^X that can be circumvented by a filesystem hack is security theater
16:20 brrt and that people who insist on employing can enjoy the fulll benefits of doing so
16:20 brrt amongst which there are, no functional JIT
16:20 markmont Yeah, the Mozilla devs have a bug report on this from back in 2006 that says much the same thing.
16:22 brrt now, if we were to go so far as make a full ELF and DLL, then load that via dlopen, then use that, well, i'd be more appreciative, cause i can see that having some real benefits
16:22 markmont It's not clear to me why SELinux denies the mprotect() call.  The description of the deny_execmem boolean says "Deny user domains applications to map a memory region as both executable and writable", but we're not trying to do that.
16:23 zakharyas joined #moarvm
16:23 brrt hmm, i think the question is if the currrent sequence of call adequatly makes the page unwritable
16:23 brrt and, of course, if the selinux check is correct
16:25 markmont I agree that that question is still open.  I've looked at several other JITs, though, and none of them do any better.
16:25 markmont In other words, I'm not seeing anything obvious that is missing in terms of making the page unwritable.
16:27 brrt anyway. i'm personally not super-fond of 'hacks' like searching for non-noexec filesystems. that feels to me like introducing less security (because the filesystem can be overwritten), not more
16:27 brrt but like i said, you're free to try it out
16:28 brrt and i'll be interested to see what the result will be :-)
16:29 Geth ¦ MoarVM/spesh-invoke: ddb3e65596 | (Jonathan Worthington)++ | src/spesh/optimize.c
16:29 Geth ¦ MoarVM/spesh-invoke: Make sure to mark decont for guard used.
16:29 Geth ¦ MoarVM/spesh-invoke:
16:29 Geth ¦ MoarVM/spesh-invoke: Otherwise, dead instruction elimination throws it out and we guard it
16:29 Geth ¦ MoarVM/spesh-invoke: against an empty register, which will always fail.
16:29 Geth ¦ MoarVM/spesh-invoke: review: https://github.com/MoarVM/MoarVM/commit/ddb3e65596
16:29 Geth ¦ MoarVM/spesh-invoke: 87fb83fc9a | (Jonathan Worthington)++ | src/spesh/optimize.c
16:29 Geth ¦ MoarVM/spesh-invoke: Guards need STable, not type object.
16:29 Geth ¦ MoarVM/spesh-invoke: review: https://github.com/MoarVM/MoarVM/commit/87fb83fc9a
16:30 Geth ¦ MoarVM/spesh-invoke: c47281fb40 | (Jonathan Worthington)++ | src/spesh/optimize.c
16:30 Geth ¦ MoarVM/spesh-invoke: Use type tuple to find spesh candidate if present.
16:30 Geth ¦ MoarVM/spesh-invoke:
16:30 Geth ¦ MoarVM/spesh-invoke: If not, we fall back to using the facts inferred using the facts
16:30 Geth ¦ MoarVM/spesh-invoke: established thus far.
16:30 Geth ¦ MoarVM/spesh-invoke: review: https://github.com/MoarVM/MoarVM/commit/c47281fb40
16:35 markmont brrt: OK, thanks.  I agree with you about hacks leading to less security.  I don't know if there is really anything to fix here, but I'd still like to look into exactly why the mprotect() returns with EPERM since we're asking to remove PROT_WRITE and add PROT_EXEC.
16:37 Geth ¦ MoarVM/even-moar-jit: 8 commits pushed by (Bart Wiegmans)++
16:37 Geth ¦ MoarVM/even-moar-jit: cdf2b2e8ca | Update plan and todo lists
16:37 Geth ¦ MoarVM/even-moar-jit: 2dbb62f9ca | Add sp_decont op
16:37 Geth ¦ MoarVM/even-moar-jit: 73ab3c6035 | A bunch of comparison ops
16:37 Geth ¦ MoarVM/even-moar-jit: 34d181ffc9 | Make jit debug tools use blocking spesh
16:37 Geth ¦ MoarVM/even-moar-jit: fe20f04169 | Document root cause of inc_i/dec_i bugs
16:37 Geth ¦ MoarVM/even-moar-jit: 281f61ee6b | Use appropriate-sized values
16:37 Geth ¦ MoarVM/even-moar-jit: 48eb8f5098 | Add special-case logic for single operand inc_i
16:37 Geth ¦ MoarVM/even-moar-jit: 1faa30f1e8 | Add atpos_i template
16:37 Geth ¦ MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/compare/328d2e1c74...1faa30f1e8
16:38 brrt yeah, thanks for looking into it
16:38 brrt okay, afk
16:52 jnthn Good news is the above pass spectest :)
16:53 jnthn Uh, my ones :)
16:53 jnthn Hopefully brrt++'s too :)
17:23 Geth ¦ MoarVM/spesh-invoke: e7f7a18e2e | (Jonathan Worthington)++ | 3 files
17:23 Geth ¦ MoarVM/spesh-invoke: Enable type tuple use in multi-dispatch.
17:23 Geth ¦ MoarVM/spesh-invoke:
17:23 Geth ¦ MoarVM/spesh-invoke: Should get Perl 6 multi-dispatch resolution at spesh time mostly
17:23 Geth ¦ MoarVM/spesh-invoke: working again.
17:23 Geth ¦ MoarVM/spesh-invoke: review: https://github.com/MoarVM/MoarVM/commit/e7f7a18e2e
17:26 jnthn Enough!
17:26 jnthn bbl
18:36 AlexDaniel joined #moarvm
19:27 nwc10 jnthn: ASAN doesn't find your branch at all interesting
19:43 jnthn Nice :)
19:43 jnthn Hm, I think somewhere there's an RT about a spesh bug involving native args
19:44 jnthn Think I just found what's going on after running into it in some other code
19:44 jnthn In an inline, this:
19:44 jnthn arg_i           liti16(0),   r9(5)
19:44 jnthn arg_i           liti16(1),  r15(1)
19:44 jnthn Is rewritten to:
19:44 jnthn set              r30(1),   r9(5)
19:44 jnthn set              r30(2),  r15(1)
19:44 jnthn It puts two different args into the same register o.O
19:45 jnthn ooh, shop closing soon; brb
19:45 nwc10 .tell brrt ASAN finds even-moar-jit very exciting: http://paste.scsys.co.uk/564761
19:45 yoleaux nwc10: I'll pass your message to brrt.
20:17 jnthn OK, figured out what it is. Sometimes the thing we're inlining grabs the parameters and puts them into different registers
20:17 jnthn uh, into the same registe
20:17 jnthn *register
20:17 jnthn But does something (like boxing them) inbetween.
20:17 jnthn Which is fine enough
20:17 jnthn Until we inline
20:28 nwc10 .tell brrt first bad commit is 2dbb62f9cade2, "Add sp_decont op" - as the only non comment change in that is to add a template, I guess it reveals a latent bug
20:28 yoleaux nwc10: I'll pass your message to brrt.
20:38 Geth ¦ MoarVM: 3f33a8419b | (Jonathan Worthington)++ | src/spesh/inline.c
20:38 Geth ¦ MoarVM: When inlining, replace receive instruction.
20:38 Geth ¦ MoarVM:
20:38 Geth ¦ MoarVM: If two parameters are received into the same register (because, for
20:38 Geth ¦ MoarVM: example, they are then boxed and only the target of the box matters),
20:38 Geth ¦ MoarVM: then the arg passing code would be rewritten in such a way that the
20:38 Geth ¦ MoarVM: second argument overwrote the value of the first. This resolves the
20:38 Geth ¦ MoarVM: issue by flipping the rewriting to the receiving side, where this
20:38 Geth ¦ MoarVM: issue cannot happen.
20:38 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/3f33a8419b
20:58 DBeepBeep joined #moarvm
21:00 lizmat jnthn: reality check: sub a(Any:U $a) { say nqp::isconcrete($a) }     # can that ever say 1 ??   Don't think so, right ?
21:02 jnthn No
21:02 jnthn :U is defiend in terms of isconcrete
21:02 lizmat ok, figured, thanks!
21:03 https_GK1wmSU joined #moarvm
21:04 https_GK1wmSU left #moarvm
21:05 lizmat argh, I thought I could simplify code, but I can't after all...  will add comment just someone doesn't step into that trap again later
21:26 travis-ci joined #moarvm
21:26 travis-ci MoarVM build passed. Jonathan Worthington 'When inlining, replace receive instruction.
21:26 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/260362737 https://github.com/MoarVM/MoarVM/compare/0f94262d7a29...3f33a8419bfd
21:26 travis-ci left #moarvm
21:29 markmont brrt, timotimo: A W^X update:
21:30 markmont SELinux not allowing mprotect() to change from PROT_WRITE to PROT_EXEC turns out to be deliberate in order to avoid exploiting the writable memory with a timing attack in a multithreaded program as described in
21:30 markmont https://www.internetsociety.org/sites/default/files/09_2_2.pdf
21:30 markmont It turns out that the libffi implementation (trying different locations, searching for noexec filesystems, using O_TMPFILE) originated in Mozilla Firefox nanojit,
21:30 timotimo mhm mhm
21:30 markmont see https://bugzilla.mozilla.org/show_bug.cgi?id=506693 (lots of good argument about temporary files there).
21:31 markmont see https://bugzilla.mozilla.org/show_bug.cgi?id=506693 (lots of good argument about temporary files this).  However, OpenBSD 6.1 *will* allow it.
21:31 markmont Whoops, that didn't come out right, let's try again:
21:32 markmont NetBSD 8.0_BETA won't allow mprotect() to change from PROT_WRITE to PROT_EXEC by default (tested and verified).  But OpenBSD 6.1 will.
21:33 markmont Under NetBSD 8.0 it's possible to mark individual executables specially to allow them to use mprotect() this way, overriding the default behavior.
21:34 timotimo i'm going to read that paper
21:34 markmont So the bottom line is that people on systems that allow it say "use mmap() and mprotect()" while NetBSD people and Linux people who use deny_execmem say "map a temporary file twice".
21:37 MasterDuke timotimo: do you have a fix for the profiler segvs in the works? or do you need any more info?
21:40 timotimo MasterDuke: haven't looked at it yet, tbh. i was only really aware of the "can't call defined on a null method" thing
21:40 MasterDuke i don't know if there are multiple reasons
21:42 timotimo "on a null object"*
21:42 timotimo that one's a spesh-related problem (i.e. likely optimizer trouble)
21:43 MasterDuke ah, this line?: STABLE(code)->invoke(tc, code, cur_callsite, args);
21:43 MasterDuke that's what valgrind pointed to
21:45 timotimo markmont: our jit differs from what is described in that paper because we never make pages writable after making them executable
21:47 markmont indeed, but it's disallowed anyway.    There's some more information about this in the Mozilla bug above.
21:49 timotimo OK, i shall read that after i've finished with the paper
21:50 MasterDuke jnthn: seems like --profile breaking might be due to your recent spesh work, any ideas?
21:53 timotimo markmont: the drawback to what we're doing is that we're wasting at least one whole page for even the smallest pieces of jitted code
21:54 timotimo so at some point we might want to re-use memory that was once execute-only and add some more jitcode at the end
21:56 markmont Calling mprotect() again to switch back should work on platforms that support it, if we stick with the current approach.  Or if we go with the map-a-temp-file-twice approach, that can be reused without doing anything (but we'll consume a file descriptor for each one we have).
21:56 timotimo ah, yes, we'd reach the max number of open files in no time at all
21:58 markmont I imagine it might be possible to keep many pieces of jitted code in a single mapped-but-unlinked tempfile.  I'd have to try it.
21:59 markmont In other words, map per interpreter or once per thread rather than once per piece of jitted code.
22:01 timotimo if we have per-thread code pages we'll be regenerating many of the same jitted blobs on all threads
22:04 markmont right -- one per interpreter/VM then, or whatever the correct number turns out to be.  I'm very new to MoarVM and NQP and not familiar with how they work.
22:07 timotimo we don't do any of the jit-related mitigation techniques discussed in that paper
22:07 timotimo like making the jitted code slightly random and unpredictable
22:17 jnthn MasterDuke: No immediate guesses, sorry. If nobody else manages to hunt it down, I'll get there, just got quite a queue of spesh things to do.
22:19 timotimo at some point hopefully a Real Security Researcher™ will try to attack moar
22:19 timotimo i imagine if you can reach the nativecall functions, you'll be able to wreak absolute havoc
22:21 timotimo i'm already unreasonably tired ... but maybe i'll go to bed rather soon today
22:24 jsimonet joined #moarvm
22:47 https_GK1wmSU joined #moarvm
22:49 https_GK1wmSU left #moarvm
22:53 markmont joined #moarvm
23:17 jnthn Oh heck. It turns out that we don't get UV_EOF in the socket callback when the socket is closed
23:17 jnthn So never pass on the done signal
23:17 jnthn And so leak
23:23 Geth joined #moarvm
23:56 MasterDuke jnthn: the --profile problem is interp.c:973, but that's because of https://github.com/MoarVM/MoarVM/blob/master/src/core/frame.c#L1764
23:57 MasterDuke if i add a `if (!code) fprintf(stderr, "here!\n")` right before then it gets printed

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary