Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-11-21

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:44 MasterDuke joined #moarvm
01:46 timotimo oh holy wow
01:47 timotimo ===> Install [FAIL] for zef:ver<0.1.32>:auth<github:ugexe>: MoarVM panic: threads list corrupted
01:47 timotimo it's the 2017.07 rakudo, though, so don't panic
01:48 ugexe thats during precomp when installing
01:49 ugexe well im assuming it made it to precomp. its inside CURI.install(...) technically
01:51 timotimo yeah, but holy wow, that's a memory corruption or something. i've never seen this message come up before
01:52 * timotimo goes to bed
01:52 MasterDuke timotimo: what would be a good benchmark for my fsa-for-string-storage branch?
01:52 MasterDuke it now builds nqp and rakudo
01:54 MasterDuke doh, questions for tomorrow it seems
01:55 samcv MasterDuke: i wish i could answer that question
01:55 samcv but i would compare times for spectest since i have no better answer
01:59 MasterDuke hm...maybe something like `my @chars = "large_file".IO.comb; my @other-chars = @chars.map({ $_ ~ rand.Str })` ?
02:00 MasterDuke that should create and manipulate a bunch of small string, which i think would benefit the most from using the FSA
02:00 samcv i'd still like to know about how it affects things generally
02:03 MasterDuke yeah, but that's a much longer benchmark to run!
02:10 MasterDuke huh. that code takes the same amount of time on the branch as on master, but uses 1.23g ram instead of 1.21g
02:56 ilbot3 joined #moarvm
02:56 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
05:02 Geth ¦ MoarVM: 44e3cbe2a4 | (Samantha McVey)++ | docs/ChangeLog
05:02 Geth ¦ MoarVM: Better clarify and organize the ChangeLog
05:02 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/44e3cbe2a4
05:02 samcv nine: you around? two changelog entries of yours are the last ones left and i'm not sure since the messages conflict with each other what the end result was
05:02 samcv nine: here https://github.com/MoarVM/MoarVM/blob/44e3cbe2a40ff7a481cc1f10008322c5ad0b1843/docs/ChangeLog#L41-L42
05:03 samcv or if somebody else can answer that for me you can go ahead and change it if you have commit rights
05:03 samcv or just let me know
06:25 evalable6 joined #moarvm
06:26 bisectable6 joined #moarvm
06:26 unicodable6 joined #moarvm
06:26 releasable6 joined #moarvm
06:26 Geth__ joined #moarvm
07:14 domidumont joined #moarvm
07:20 brrt joined #moarvm
07:20 brrt good * #moarvm
07:20 domidumont joined #moarvm
07:31 Geth ¦ MoarVM: 13b45564ef | (Stefan Seifert)++ | docs/ChangeLog
07:31 Geth ¦ MoarVM: Unconfuse ChangeLog
07:31 Geth ¦ MoarVM:
07:31 Geth ¦ MoarVM: The first commit was a quick fix to exclude an unhandled case while the second
07:31 Geth ¦ MoarVM: commit brought code to actually handle that case correctly.
07:31 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/13b45564ef
07:32 nine samcv: fixed
07:32 samcv awesome thanks :)
07:38 brrt hey #moarvm
07:38 brrt just as a random idea
07:38 brrt maybe we can rename the JIT to the 'machine code backend' or 'mcb'
07:39 brrt that would give a much better indication of what it actually does
07:41 brrt or at least, it would communicate it better
07:41 samcv hm
07:41 samcv backend sounds kind of vague though
07:42 samcv maybe something fancier sounding
08:05 domidumont joined #moarvm
08:19 brrt joined #moarvm
08:19 * nine always thought "JIT" fits very well
08:20 brrt but, but, but
08:20 brrt it causes confusion
08:20 brrt or at least i feel like it does
08:20 nine In which way?
08:21 nwc10 well, it's "native code generator" more than anything else that's been suggested
08:21 nwc10 (I think)
08:23 nine "In computing, just-in-time (JIT) compilation, also known as dynamic translation, is a way of executing computer code that involves compilation during execution of a program – at run time – rather than prior to execution.[1] Most often this consists of source code or more commonly bytecode translation to machine code, which is then executed directly."
08:24 nine https://en.wikipedia.org/wiki/Just-in-time_compilation
08:51 samcv ok i'm going to release now
08:51 Geth ¦ MoarVM: 8eaba4ac8d | (Samantha McVey)++ | docs/ChangeLog
08:51 Geth ¦ MoarVM: Make a few minor changes to ChangeLog before release
08:51 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/8eaba4ac8d
08:51 Geth ¦ MoarVM: 8c6f97ea8e | (Samantha McVey)++ | docs/release_guide.md
08:51 Geth ¦ MoarVM: Add step to release guide to check tests with NFG_DEBUG set
08:51 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/8c6f97ea8e
08:51 Geth ¦ MoarVM: 23afb9308e | (Samantha McVey)++ | VERSION
08:51 Geth ¦ MoarVM: Bump version to 2017.11
08:51 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/23afb9308e
08:52 samcv release done
09:03 zakharyas joined #moarvm
09:33 zakharyas joined #moarvm
09:40 domidumont1 joined #moarvm
09:50 brrt samcv++
09:51 brrt nine: in the way that the MoarVM JIT does much less than what, say, a JIT for v8 does
10:08 zakharyas1 joined #moarvm
10:09 jnthn What most others call their JIT consists of our spesh and JIT subsystems together
10:11 jnthn I'm not really sure how much we should worry about this though
10:12 jnthn I think most of the world knows JIT in a fairly hand-wavey way, such as "thingy that produces machine code so stuff runs faster"
10:13 nwc10 only thing is that we have a platform-independant JIT in "spesh"
10:13 geospeck joined #moarvm
10:13 nwc10 and that might not be clear
10:13 jnthn Right
10:13 brrt so renaming it to a native code generator would make stuff more clear
10:14 nwc10 so, platform independant JIT and CPU specific JIT :-)
10:14 jnthn It would, though for marketing purposes we'd still want to say "we have a JIT" :)
10:14 nwc10 or yes "native code generator" and "platform independant JIT"
10:14 nwc10 but yes, totally agree, the magic CV keyword needs to be "JIT" else HR reject us
10:14 brrt hehe
10:21 jnthn Aww, my twitter feed has a bunch of "look, we have snow!" posts from around Europe. I ain't got any.
10:24 moritz the forecast says +10°C for today (southern Germany)
10:24 moritz far from snow
10:26 robertle joined #moarvm
10:29 nwc10 we have rain
10:32 jnthn We've had and will likely have more rain
10:56 zakharyas joined #moarvm
12:00 domidumont joined #moarvm
12:51 domidumont joined #moarvm
13:07 geospeck joined #moarvm
13:56 * dogbert17 had to wade through snow
13:57 jnthn :)
13:57 dogbert17 jnthn: interested in taking a look at a gist? Is it something I should report against MoarVM? https://gist.github.com/dogbert17/cebea4925ff5b5ca6759da1a183813fc
13:58 * dogbert17 now has 32 and 64 bit vm's at his disposal
13:59 jnthn Hm, interesting
13:59 jnthn Can file it
14:00 jnthn For now I want to try and figure out the last lastexpayload inline blocker
14:00 timotimo MasterDuke: i find it surprising that fsa-for-string-storage uses more memory than master
14:00 dogbert17 cool, will create an issue
14:01 Geth ¦ MoarVM/inline-lastexpayload: b9489ea879 | (Jonathan Worthington)++ | 2 files
14:01 Geth ¦ MoarVM/inline-lastexpayload: Remove :noinline on lastexpayload
14:01 Geth ¦ MoarVM/inline-lastexpayload: review: https://github.com/MoarVM/MoarVM/commit/b9489ea879
14:02 jnthn That's a rebase on master (since brrt++'s fix was cherry-pick'd)
14:09 jnthn Well, spesh nodelay + blocking built/tested fine
14:09 jnthn Now onto spectest
14:13 jnthn ah, yes, the same issue
14:14 jnthn (try "" ~~ /. ** {NaN}/) for ^1000; somehow stops disarming Failures after a while
14:25 jnthn The slightly good news is that it does it even without NODELAY
14:25 jnthn Which means it breaks as soon as the 65th inline, which is a whole load more tractable than the 6000th :)
14:26 jnthn Can inline Int (3309) into DYNQUANT_LIMITS (9664)
14:26 jnthn Is the one that busts things
14:29 dogbert17 https://github.com/MoarVM/MoarVM/issues/754
14:32 dogbert17 what happens if the JIT is disabled?
14:36 jnthn Same
14:36 jnthn Only disabling inlines hides it
14:40 dogbert17 so brrt is off the hook :)
14:55 markmont joined #moarvm
15:05 zakharyas joined #moarvm
15:19 timotimo hm. it's not losing a guard that it'd need?
15:22 jnthn No, it's about nqp::throwlexpayloadcaller
15:23 jnthn The inlined_and_not_lexical thingy is...very dubious
15:24 jnthn I think it was meant to be setting them onto the handlers duplicated from the inliner
15:25 jnthn Indeed, commenting out the lines that set that flag fixes the issue
15:25 jnthn Question is what it breaks
15:26 * jnthn does a stress build/test to find out
15:32 jnthn Turns out, breaks the Rakudo build right at the end
15:32 timotimo in the core dist installation things
15:32 timotimo ?
15:32 jnthn No, in create-moar-runner
15:33 timotimo wow, that's an extremely simple script, isn't it?
15:34 jnthn Yup, so that mechanism is obviously doing something important
15:36 timotimo i see the commit that added was yours
15:36 timotimo hehe.
15:36 timotimo it says it doesn't hold up any more once we can inline closure-cloned things
15:36 jnthn Commit ID?
15:37 timotimo https://github.com/MoarVM/MoarVM/commit/1185cabb
15:37 timotimo cabbage?
15:37 jnthn Also the comment on the flag...is stranger still. It matches what I'd expect a flag of that name to mean
15:37 jnthn But it wasn't being set in those cases
15:37 jnthn It was being set on an originally included one instead
15:40 timotimo so it's out of the perspective of the inlinee?
15:41 jnthn Well, the problem is that we need to make sure that when we inline, we don't accidentally find lexical handlers that we should not.
15:41 timotimo if that were the case, then i could understand it … like some things would be inlined and lexically in scope, some things would be inlined from outside the lexical scope
15:41 timotimo and in multi-inlines, it can be even weirder
15:42 jnthn Right
15:42 timotimo higher-depth-inlines or whatever we call those
15:42 jnthn But what's odd is that this fix was...seemingly wrong
15:45 jnthn So now I have the original wrong fix that fixed something and broke the case I've been debugging today, and correcting it fixes my case today but breaks something else
15:46 jnthn I wonder if...
15:46 jnthn The multi return has an inline handler
15:47 jnthn It does throwlexpayloadcaller - that is, throw it lexically, but only after walking down one frame to the caller
15:47 jnthn When we inline `return`, then we rewite that into a throwlexpayload
15:47 timotimo you mean like "multi sub return"?
15:47 jnthn Yes
15:47 timotimo ah, but we forget to do the "ignore one frame" thing
15:47 timotimo maybe
15:48 timotimo but rewriting *caller to * should do that
15:48 timotimo now i get what you're saying
15:48 jnthn Well, my incorrect fix before disabled all lexical handlers of an inlinee
15:48 timotimo if multi sub return has kept its own return handler it could then catch the one without "caller"
15:48 timotimo when previously that one would have been ignored
15:48 jnthn Right, and I think my original wrong fix made that not happen
15:49 jnthn And we relied on that broken fix.
15:49 jnthn But it does a lot too much
15:49 jnthn And too little at the same time
15:49 timotimo so we'll have a sp_throwpayloadlexignoreonereturnhandler?
15:50 jnthn Not sure yet
15:50 jnthn There are quite a few cases to consider
15:50 jnthn The one that the current thing breaks is where we inline something that has a lexical return handler
15:50 jnthn That thing calls .fail
15:50 robertle joined #moarvm
15:50 jnthn That fail does throwlexpayloadcaller
15:51 jnthn But because we have accidentally blocked out all of the lexical handlers in inlinees, then it doesn't find it
15:52 jnthn It instead finds the wrong one (that *would* have been marked with the flag in question if my original fix had been right)
15:52 jnthn So obviously correct that works, but then every case of an inline of `return` is now broken
15:53 jnthn *correcting
15:53 jnthn This has implications for moving inlines too
15:54 jnthn Because having them at the end means we can duplicate the handlers and block out the non-lexical ones
15:54 timotimo moving, as in put them in between rather than at the end?
15:54 jnthn Right
15:54 timotimo mhm
15:55 jnthn So an ideal fix for all of this will ideally be robust enough to handle that case also
15:57 jnthn Damn, this is tricky.
15:58 timotimo ... something with bit masks ...
16:06 nine Sounds to me like the simple flat handlers table may be too simple for the complex requirements.
16:07 geospeck joined #moarvm
16:08 brrt joined #moarvm
16:11 jnthn nine: When working on moving inlines, did you change it so that the handlers of inlinees were prepended rather than appended to the resultant handler table, ooc?
16:11 zakharyas joined #moarvm
16:11 jnthn (So those innermost ones of the handler would be found first)
16:13 nine jnthn: no. But my knowledge about exceptions in MoarVM is rather basic. I'd not be surprised if that would be the actual source of the remaining issues
16:14 jnthn It's just the two commits at the head of inline_in_place?
16:14 jnthn If so, I'm not seeing anything that'd account for the changes
16:14 nine yes
16:14 jnthn Yeah, that may well be it
16:14 jnthn Because the exception ahndler table is searched linearly
16:14 jnthn With the exceptation that innermost handlers come earliest
16:14 nine But why would the location of the inlined code make a difference there?
16:15 jnthn Consider:
16:15 nine Unless the table was "wrong" even before my changes and it was just luck that the code worked anyway :)
16:15 jnthn Handler Starts
16:15 jnthn some-call-here()
16:15 jnthn Handler Ends
16:16 jnthn The current "place the inlined code at the end" model results in the inlinee being placed after the HandlerEnds
16:16 jnthn Any handlers inside of the inlined call are copied into the table
16:16 jnthn Then we duplicate any handlers in effect at the point of the call
16:17 jnthn The original handlers don't count, because the code region they cover is outside of the inlined code
16:17 jnthn Since it's appended at the end
16:17 jnthn If we do the inline "in place" then the inlined code falls without the original handler's locations
16:17 jnthn So they apply
16:17 jnthn And if they're first in the table, they'll be found before any of the handlers in the code that was inlined
16:18 jnthn Meaning it could, for example, jump out of the wrong loop on a `last`
16:18 jnthn Or skip over a CATCH handler
16:18 jnthn And hit an outer one instead
16:19 jnthn uh, falls *within* the original handler's locations, I meant above
16:19 jnthn The handlers table is just a set of start/end addresses.
16:19 nine Ah, I see. So it's actually a wonder that so much works despite my ignorance of this :)
16:20 jnthn :)
16:20 squashable6 joined #moarvm
16:21 jnthn Trouble is that I think we refer to handlers by their index in a bunch of annotations
16:21 nine But it also sounds to me like inline_in_place should make finding a correct solution easier. We shouldn't actually need to duplicate handlers as the original handlers cover the inlined code quite well.
16:21 jnthn So yes, we lose the "need to duplicate" problem
16:22 jnthn Except that was an immutable solution :)
16:22 jnthn And now we need to prepend...meaning we have a load of handler indexes all over the spesh graph in annotations that will have values that become out of date upon the prepend
16:23 jnthn Of course, everything can be solved with another LoI :)
16:23 jnthn But I suspect this is much of why I did the "put them at the end" in the first place
16:23 nine Are you sure the flat table is still a good data structure for managing handlers?
16:24 timotimo handlers aren't always nested cleanly
16:25 jnthn Well, it ticks a bunch of boxes
16:26 jnthn The performance of not only every exception throw, but also every otherwise unoptimized return, next, last, take, emit, etc. depends on doing swift lookups
16:26 jnthn A set of integer comparisons in a table that's relatively compact in memory is good in that sense.
16:26 brrt joined #moarvm
16:27 jnthn (Also, the hardware likes linear reads through things, as it can predict them well with caches)
16:27 jnthn There may be alternatives with good properties too
16:28 timotimo we have all starts and ends in one contiguous array and extra info separately already?
16:28 nine Maybe multiple tables? Like switching out the handlers table on entering inlined code and switching back afterwards?
16:28 jnthn Well, the other goal is "no runtime cost unless an exception is thrown"
16:28 nine And the inlinee's handlers table contains its lexical handlers and all the dynamic ones
16:29 nine If you can make it behave correctly :)
16:30 jnthn timotimo: We don't, but the other fields in there aren't that huge either. Granted we could maybe get a small win by compacting them
16:30 nine Switching tables could be 2 very cheap ops and the tables to search would be smaller overall.
16:30 jnthn Well, an unconditional branch is arguably also a cheap op. :)
16:31 jnthn Which was kinda my reasoning for why sticking them at the end wasn't such a bad thing.
16:31 jnthn I'd be a bit sad to lose those and then gain two new things that's just as much work
16:31 nine So why do we try to move them back?
16:32 nine We'd also gain more correctness.
16:32 jnthn Largely because the inlinee might then compact into a basic block with the inliner
16:32 jnthn Which is good for the expr JIT
16:32 timotimo the thing with "entering and leaving an inline causes tables to switch" can also be done with a flat table and bitmasks
16:33 jnthn Instead of switching tables, we could just have a set of regions that pick a table
16:34 jnthn So the first scan is for an applicable table
16:34 timotimo that, too
16:34 jnthn We could potentially then mark up each of those with outer/caller indices
16:34 nine We'd also only need to switch table if the inlinee has any handlers. Is that common?
16:35 jnthn So we could traverse the tables differently in dynamic vs outer contexts
16:35 jnthn nine: Yes; because `return`
16:35 jnthn Also loop flow control
16:38 jnthn I think a set of structured tables, the first table picked by reigion, and chained for both outer and dynamic, may well do it
16:39 jnthn It'd save a bunch of lookups
16:39 jnthn In that we can skip irrelevant tables
16:40 jnthn Essentially, it's kinda what nine is suggesting but without the need for ops; we just find the table in the same way we'd find the current inline at a deopt point
16:46 jnthn And really I guess all this is really doing is preserving the callee/caller/outer structure for the sake of handler resolution post-inline
16:46 jnthn Rather than mashing it all together
16:47 Geth ¦ MoarVM: b9826e1764 | (Aleks-Daniel Jakimenko-Aleksejev)++ | docs/ChangeLog
16:47 Geth ¦ MoarVM: Point to moarvm wiki for the next changelog
16:47 Geth ¦ MoarVM:
16:47 Geth ¦ MoarVM: The contents of the wiki page will be move into the changelog file
16:47 Geth ¦ MoarVM: before the release.
16:47 Geth ¦ MoarVM:
16:47 Geth ¦ MoarVM: Sister commit: https://github.com/rakudo/rakudo/commit/47ebc4ab244e1de252201d131a1b1860226e52d0
16:47 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b9826e1764
17:04 * jnthn wrote some notes on that
17:04 zakharyas joined #moarvm
17:05 jnthn Occurs to me that the nested tables can actually hang off MVMSpeshInline
17:11 Zoffix joined #moarvm
17:18 jnthn OK, tired...but glad I've got to the bottom of what's going on and a seemingly good path forward
17:19 jnthn And maybe it'll help with nine++'s inline moving work too
17:23 zakharyas joined #moarvm
17:29 domidumont joined #moarvm
17:44 nine That would be nice :)
17:44 Zoffix left #moarvm
17:44 nine In any case, I've at least learned a lot from this discussion
18:26 Geth ¦ MoarVM: 26ad7cc55d | (Aleks-Daniel Jakimenko-Aleksejev)++ | docs/release_guide.md
18:26 Geth ¦ MoarVM: Be more specific about tar uploads (release guide)
18:26 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/26ad7cc55d
19:47 evalable6 joined #moarvm
20:11 zakharyas joined #moarvm
20:29 dogbert17 .seen timotimo
20:29 yoleaux I saw timotimo 17:44Z in #perl6: <timotimo> javascript has Object.assign which allows you to merge multiple objects i believe
20:38 dogbert17 timotimo, are there any known issues with the --valgrind option for MoarVM builds
20:40 dogbert17 on my 64 bit VM perl6-valgrind-m crashes with SEGV and, among other things the comment 'Memcheck: mc_main.c:5525 (vgMemCheck_is_valid_aligned_word): Assertion 'VG_IS_WORD_ALIGNED(a)' failed.'
20:43 dogbert17 interestingly it always point to the same place in the code, i.e. https://github.com/MoarVM/MoarVM/blob/master/src/6model/reprs/NFA.c#L239
20:51 brrt joined #moarvm
20:56 geospeck joined #moarvm
21:10 MasterDuke timotimo: this is what my branch looks like now: https://github.com/MoarVM/MoarVM/compare/master...MasterDuke17:use_fsa_for_string_storage
21:11 MasterDuke timotimo, jnthn, nine, samcv, et al.: notice anything obviously wrong ^^^ ?
21:44 xi- joined #moarvm
22:03 xi- joined #moarvm
22:31 brrt joined #moarvm
22:53 markmont joined #moarvm
23:06 xi- joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary