Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-07-13

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:11 AlexDaniel joined #moarvm
01:49 ilbot3 joined #moarvm
01:49 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
02:05 samcv ugh
02:06 samcv this is what i have so far. need to figure out why i'm getting segfaults still https://github.com/MoarVM/MoarVM/compare/master...samcv:concat-magic?expand=1
02:09 MasterDuke need moar MVMROOTS?
02:15 samcv hm
02:18 samcv ok well ifound one issue
02:53 samcv hmm yeah it seems to die when i try to get MVM_string_graphs on the returned string
02:55 samcv added an MVM root to the whole area but idk seemed to do nothing
02:57 samcv yep as soon as i try that it dies
03:02 samcv MasterDuke, so i set out to the string here https://github.com/MoarVM/MoarVM/compare/master...samcv:concat-magic?expand=1#diff-0cb6ca157d7b79b7d7e5ace869130f8cR462
03:02 samcv and then i return. and i added prints and the same pointer is in both places but it causes a fault as soon as i access the graphs
03:03 samcv and i wrapped the whole area in MVMROOT(tc, renormalize_rtrn, { hmm
03:03 samcv maybe because it's not set to anything when i call MVMROOT... yeah that could do it maybe
03:03 samcv it only gets set to the right value when the function is called
03:05 samcv err god i'm dumb
03:05 samcv i gave the integer to MVMROOT not the pointer
03:05 * samcv facepalms hard
03:06 MasterDuke heh
03:08 samcv well no segfault now. now it just hangs. so that's in improvement
03:08 samcv and all my printf's get triggered
03:11 samcv it returns and then who knows what occurs
03:12 samcv oh it seems to loop in MVM_string_gi_get_grapheme somewhere someplace
03:12 MasterDuke what's the 'gi' mean?
03:13 MasterDuke oh, grapheme iterator, right?
03:17 MasterDuke well i'm outa here, good luck
03:17 samcv yep
03:19 samcv well it's somehow not composing it properly or something
03:53 vendethiel joined #moarvm
04:34 vendethiel- joined #moarvm
04:44 vendethiel joined #moarvm
04:49 samcv well ok i got it not GCing me all the time. made it a MVMString ** instead
04:49 samcv now need to fix it trying to iterate past the end of the graphemeiterator
04:50 samcv now that i'm not getting corrupted information
05:42 samcv okay making some progress. not segfaulting and working in certain cases ;0
05:42 samcv :)
05:42 samcv though semi limited
06:11 brrt joined #moarvm
06:35 brrt good *
06:40 samcv good *
06:41 domidumont joined #moarvm
06:48 * brrt is seeing your progress
06:48 domidumont joined #moarvm
06:51 * brrt doesn't know any better
07:01 domidumont joined #moarvm
07:09 samcv i'll get more to work on it tomorrow
07:10 samcv issue right now is handling when due to the string concatenation, some of the strands turn out to be set to the same start and same end codepoint
07:11 samcv in which case it panics because it's not meant to handle that case so doesn't go to the next strand
07:11 samcv but i am pretty close i think
07:13 samcv it's at least working in certain cases so that's a whole lot better than failing horribly in 110% of cases :)
07:13 brrt joined #moarvm
07:18 domidumont joined #moarvm
07:36 zakharyas joined #moarvm
07:44 domidumont joined #moarvm
07:46 lizmat joined #moarvm
08:08 brrt so, meta-thing i learned yesterday
08:08 brrt i've become somewhat of a heavy user of org-mode over the last few years
08:09 brrt for me it's an awesome way to manage and organize attention over an increasing set of jobs
08:09 brrt but yesterday i realized
08:10 brrt that's not an improvement per se
08:10 brrt it's an adaptation
08:10 brrt i used to be able to manage all these things in my head, and now i cannot
08:31 samcv brrt, what's org-mode?
08:41 brrt why, it is awesome
08:42 brrt it is an emacs 'major mode' that allows you to manage all sorts of stuff in a text file
08:42 brrt calendars, to do lists, papers, dependency chains, even has a spreadsheet
08:42 brrt exports to HTML, ODT, LaTeX
08:43 brrt http://orgmode.org/
08:53 jnthn morning o/
08:55 nwc10 good *, *
08:55 jnthn samcv: MAX_STRANDS is a soft limit so far as I recall; the idea is that if we end up with a huge number of strands then we'll collapse them
08:58 brrt joined #moarvm
08:58 brrt moarning jnthn, nwc10, samcv
09:55 * jnthn is looking at the callsite thing and thinking, surely this can be done in a simpler/more robust way
10:05 robertle joined #moarvm
10:05 brrt joined #moarvm
10:14 samcv brrt, is it worth using emacs if i don't use it jsut for that?
10:15 jnthn Grr, after trying some changes I now see more of why it's the way it is :S
10:21 brrt well, that is useful, isn't it?
10:22 brrt samcv, i can't say that for you. using emacs makes sense if you can get your brain to agree with it
10:22 brrt if you're a hardcore vim user, maybe not
10:22 jnthn Sorta, just means I can't get the simplificiation I expected. Now trying a different way.
10:22 samcv we i know how to use vim. but i mostly use atom
10:23 samcv not a hardcore anything
10:23 brrt right. if you use atom, then at least emacs isn't going to feel slow for you
10:24 brrt emacs has all the convenience you might expect, if you're willing to write bits of lisp :-P
10:36 nwc10 and you can run vim inside it./
10:36 nwc10 oh wait, that's not the point. :-)
10:37 timotimo for vim there's vim-journal or something. i've had it open in a tab for multiple months now, never actually tried it >_<
10:46 brrt oh, you can have a vim-mode for emacs, yeah
10:46 brrt i sometimes try to use vim for serious editing...
10:47 brrt just doens't agree with my brain as well as emacs does
10:48 jnthn Hm, it seems that the only way spesh may be involved in the bug in question is that due to its logging, it keeps some MVMCallsite objects alive for longer than they otherwise would be
10:48 jnthn And so they actually live long enough to be marked
10:51 brrt left #moarvm
10:51 brrt joined #moarvm
10:54 dogbert17 jnthn: are you looking at the destructuring issue?
10:55 jnthn dogbert17: Yeah
10:56 dogbert17 looks like you managed to do some code cleanup yesterday
11:00 * dogbert17 wonders where all the 'one line patch bugs' has gone
11:01 nwc10 welcome to a mature project
11:02 jnthn OK, interesting, it seems the extended lifetime is not, after all, an artefact of spesh logging, in that if I make sure we never log a call capture we still end up marking them
11:11 jnthn huh, turns out that I can golf the issue a good bit further
11:11 jnthn I think when I did it before my criteria was "does it SEGV"
11:12 jnthn But if it's "does it make valgrind unhappy" then a bunch of stuff goes away
11:12 jnthn What notably does not go away is that it seems to need to be a multi-dispatch *and* to use destructuring
11:13 jnthn My suspicion is starting to fall upon invokewithcapture
11:13 jnthn And the effective callsite mechanism
11:13 nwc10 you've giving it a long hard stare? :-)
11:13 brrt n
11:14 nwc10 (don't forget the marmalade sandwich)
11:14 brrt jnthn: have you checked if it's not JIT-specific
11:14 jnthn brrt: Yeah, it shows up just the same with MVM_JIT_DISABLE=1
11:14 jnthn But goes away with MVM_SPESH_DISABLE=1
11:14 brrt bummer...
11:15 jnthn Which must be part of the puzzle too
11:15 nwc10 what does your golf case look like currently?
11:16 jnthn I'm running it as MVM_SPESH_INLINE_DISABLE=1 MVM_SPESH_OSR_DISABLE=1 MVM_JIT_DISABLE=1 fwiw
11:16 jnthn https://gist.github.com/jnthn/5e7a5d8ebac1dd9e1b3607d415b46efe
11:18 jnthn Also it's possible to remove the @a and just flatten in the list
11:18 jnthn And to remove the logging
11:18 jnthn uh, the $i
11:18 nwc10 ASAN finds this code worthy of excitement: http://paste.scsys.co.uk/564605
11:18 nwc10 s/this/that/
11:18 jnthn Yeah, that's exactly the same finding valgrind has
11:19 jnthn Like, precisely the same 3 stack traces
11:19 nwc10 OK. Good, in as much as we've quickly established that there wasn't anything missed.
11:19 nwc10 missed by valgrind but seen by ASAN
11:19 nwc10 so it's purely a heap error
11:21 jnthn Yeah
11:21 jnthn So the ingredients seem to be all of spesh, multi-dispatch, and destructuring
11:22 jnthn Multi-dispatch and destructuring being notable in both using MVMCallCapture
11:22 nwc10 ponder it over lunch?
11:22 jnthn Yeah, think it's time to eat :)
11:23 jnthn bbiab
11:27 timotimo when you have it open in rr, you can step forwards and backwards between where it gets created, where it gets freed, and where it gets overwritten later, and where the gc stumbles upon the bogus data in it
11:27 timotimo i didn't know what the next step in actual debugging should have been, though
11:29 timotimo it might be interesting to have a debug mode where we allocate everything in gen2 (so that nothing moves) but we still gc every now and then (as if we were running normally)
11:29 timotimo since this bug relies on having things gc marked in order to asplode at all
11:30 timotimo gotta go
12:11 * jnthn back
12:13 brrt \o
12:20 jnthn Turns out that commenting out MVM_spesh_log_add_logging hides the issue
12:20 jnthn Commenting out instead any of the differnet optimization phases spesh does has no effect on hiding the bug
12:21 jnthn So it seems that it *is* that spesh logging elongates lifetimes, and so shows up something that'd be an issue anyway
12:21 jnthn It just isn't that it's a call capture itself that is being logged, but presumably something that references it
12:22 jnthn That also fits with the output in that we get an error from valgrind but then can continue without errors
12:27 jnthn Yeah, my hunch that it might be about invokewithcallcapture seems to be it
12:31 jnthn Got a working fix
12:32 jnthn Downside being that it makes frames a pointer larger for a relatively uncommon case
12:51 Catbert17 joined #moarvm
12:58 Geth_ ¦ MoarVM: 7c5aaa3546 | (Jonathan Worthington)++ | src/6model/reprs/MVMCallCapture.c
12:58 Geth_ ¦ MoarVM: Remove out-dated comment.
12:58 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/7c5aaa3546
12:58 Geth_ ¦ MoarVM: 6aa01090e7 | (Jonathan Worthington)++ | 3 files
12:58 Geth_ ¦ MoarVM: invokewithcapture must keep the used capture alive
12:58 Geth_ ¦ MoarVM:
12:58 Geth_ ¦ MoarVM: The frame's argument processing context will come to reference the
12:58 Geth_ ¦ MoarVM: effective callsite. If it were to be collected before the frame was
12:58 Geth_ ¦ MoarVM: executed, then the frame would get a dangling callsite pointer. This
12:58 Geth_ ¦ MoarVM: was most noticable because a further MVMCallCapture could then come to
12:58 Geth_ ¦ MoarVM: refer to this same callsite, be marked, and the mark would cause a use
12:58 Geth_ ¦ MoarVM: after free.
12:58 Geth_ ¦ MoarVM:
12:58 Geth_ ¦ MoarVM: Making sure that it stays alive costs an extra pointer in MVMFrame,
12:58 Geth_ ¦ MoarVM: which is a bit of a pity, but less of one than SIGSEGV.
12:58 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6aa01090e7
12:59 Geth_ ¦ MoarVM: 3d59fac35e | (Jonathan Worthington)++ | src/6model/reprs/MVMCallCapture.c
12:59 Geth_ ¦ MoarVM: Remove more leftovers from capture use-mode.
12:59 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/3d59fac35e
13:03 lizmat jnthn: apart from using more memory, do you see any other downsides ?
13:06 jnthn Not really, but I think this only kicks the can further down the road
13:06 jnthn In that there's still a way to get a dangling pointer
13:07 lizmat :-(
13:07 lizmat .oO( damn danglers )
13:07 jnthn Which is fixable by just making sure we *always* copy the callsite, never reference it
13:08 jnthn Perhaps with an exception for those that are interned
13:08 jnthn It's plausible that we can also use a copying approach to get rid of the extra pointer in frame too, thinking about it
13:11 deep-book-gk_ joined #moarvm
13:13 deep-book-gk_ left #moarvm
13:18 jnthn Always copying the callsite seems to work out
13:18 jnthn (for fixing the remaining potential dangling pointer issue)
13:18 jnthn It also makes MVMCallCapture 4 or maybe 8 bytes smaller (depends on alignment)
13:19 jnthn *And* gets rid of a bunch of logic
13:21 jnthn grr, some spectest issue
13:22 * jnthn does a pull on Rakudo just in case, 'cus it may have been a hang in a set test
13:22 jnthn Though gotta go for langauge lesson now
13:22 jnthn bbl
13:26 brrt (is the callsite a large thing?)
13:28 nwc10 That (unconditionally) doesn't matter, does it? (for a delta value) The question is "are there lots of them?"
13:28 nwc10 (I think)
13:33 domidumont joined #moarvm
13:39 ggoebel joined #moarvm
14:35 AlexDaniel joined #moarvm
14:45 brrt joined #moarvm
14:54 jnthn Callsites in common use are interned. And no, they ain't so large.
14:54 jnthn Spectset was clean with HEAD Rakudo
14:54 jnthn So guess I was missing a fix
14:55 jnthn Hm, only question now is "why was that sepctest run so slow"
14:56 timotimo 90% time spent in copy_callsite? :o
14:58 Geth_ ¦ MoarVM: 3dd74960ac | (Jonathan Worthington)++ | 4 files
14:58 Geth_ ¦ MoarVM: Always copy callsite into an MVMCallCapture.
14:58 Geth_ ¦ MoarVM:
14:58 Geth_ ¦ MoarVM: Rather than trying to reference it. That turns out to be reliably
14:58 Geth_ ¦ MoarVM: fragile, and it's not a big thing to copy. The reward is that we get
14:58 Geth_ ¦ MoarVM: to eliminate a bunch of logic, making for clearer code, and also lose
14:58 Geth_ ¦ MoarVM: a flag that - after alignment considerations - will save 4 or 8 bytes
14:58 Geth_ ¦ MoarVM: per call capture.
14:58 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/3dd74960ac
14:58 jnthn Probably a debug build or some GC flag turned on :)
14:58 timotimo i tend to have optimize in gcc turned off, gives about 2x difference
15:05 * jnthn does a non-debug build and fires off another run just in case
15:06 Zoffix hm, I too had a 40%-slower-than normal spectest run :/
15:07 timotimo hmm, cosmic rays?
15:09 jnthn huh, no, it was super slow again
15:09 brrt joined #moarvm
15:10 jnthn Hm, startup maybe got a bunch slower?
15:10 Zoffix parse was slower for me. 87s vs ~67s in the past
15:10 jnthn 40% is what I@m seeing too
15:16 geekosaur joined #moarvm
15:17 Geth_ ¦ MoarVM: 67202501f4 | (Jonathan Worthington)++ | src/core/args.c
15:17 Geth_ ¦ MoarVM: Clean up some duplicated code.
15:17 Geth_ ¦ MoarVM:
15:17 Geth_ ¦ MoarVM: The bind error handling can just call the code to save the arg capture
15:17 Geth_ ¦ MoarVM: instead of repeating it.
15:17 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/67202501f4
15:24 timotimo i pulled latest moar and my time went up from 53 to 61 (stage parse)
15:25 brrt correctness is costly...
15:26 Zoffix brrt: is that what is is?
15:26 jnthn I...can't imagine it being *that* costly o.O
15:27 timotimo did we accidentally throw out spesh logging?
15:28 jnthn If we did, I can't see wehre
15:28 timotimo right
15:28 jnthn I mean, can't see any evidence I comitted that by accident
15:28 jnthn I'll take a look shortly
15:28 timotimo yeah, my theory was you perhaps acidentally left in a line or something
15:29 timotimo didn't glance the diffs yet
15:29 timotimo anyway, a perf report doesn't show something massive
15:30 jnthn Turns out I can shave another 4 or 8 bytes off MVMCallCapture too
15:33 Geth_ ¦ MoarVM: 321eee7cb4 | (Jonathan Worthington)++ | 6 files
15:33 Geth_ ¦ MoarVM: Remove now-unrequired effective_callsite.
15:33 Geth_ ¦ MoarVM:
15:33 Geth_ ¦ MoarVM: It's always the same as the callsite pointer inside of the argument
15:33 Geth_ ¦ MoarVM: processing context, so don't duplicately store that.
15:33 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/321eee7cb4
15:34 jnthn Anybody else working on hunting the slowdown?
15:35 timotimo not really
15:36 * jnthn tries 2f9be082 which is before his work today
15:37 jnthn Ah, that's fast
15:38 jnthn Anyone want the SEGV back? :P
15:39 * jnthn tries 7c5aaa3
15:41 jnthn Also fine
15:43 * dogbert17 doesn't notice any slowdown but that might be due to running on 32 bit
15:44 jnthn Testing 6aa01090 now
15:45 jnthn Minor slowdown perhaps due to larger MVMFrame or perhaps noise, certainly not the huge one we're hutning
15:45 jnthn *hunting
15:46 jnthn That only leaves 4 more commits
15:46 jnthn Trying 3d59fac
15:47 jnthn Which would be an odd one to cause it
15:47 jnthn I suspect it may turn out to be 3dd7496
15:48 yoleaux joined #moarvm
15:48 * dogbert17 notices that the number of tests being run have increased by several thousand in the last few days
15:48 jnthn Nope, not 3d59fac
15:50 jnthn Seems it is that one
15:52 jnthn Ohhh
15:52 jnthn "oops"
15:53 jnthn So it turns out if we always copy the callsites...we never end up with an interned callsite being seen on various code paths
15:53 timotimo oh, *always*
15:53 timotimo i thought we never copy interned ones
15:53 timotimo still after today's work i mean
15:53 jnthn No, I figured that was an optimization for when I'd got things working
15:53 jnthn Turns out that I was right. :P
15:53 brrt hehe
15:54 jnthn Didn't realize it'd regress so much
15:54 timotimo :D
15:54 jnthn But it turns out that it means we will never insert into the multi-cache
15:54 timotimo right, ouch :)
15:54 timotimo multi dispatch can be expensive, yeah
16:00 jnthn Having done all the previous cleanups, it was easy to put the don't-copy-on-intern change it.
16:02 jnthn Yeah, that fixes the perf regression :)
16:06 Geth_ ¦ MoarVM: 2e683f95f7 | (Jonathan Worthington)++ | 3 files
16:06 Geth_ ¦ MoarVM: Don't copy interned callsites.
16:06 Geth_ ¦ MoarVM:
16:06 Geth_ ¦ MoarVM: Not only is it not required because their memory management is simple
16:06 Geth_ ¦ MoarVM: (they live so long as the VM does), but also a callsite that is not
16:06 Geth_ ¦ MoarVM: interned will not be entered into a multi-dispatch cache - meaning the
16:06 Geth_ ¦ MoarVM: earlier change to copy always introduced a huge performance regression
16:06 Geth_ ¦ MoarVM: that this commit addresses.
16:06 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/2e683f95f7
16:07 jnthn And the thing I set out to fix in the morning is still fine :)
16:10 dogbert17 jnthn++, that means that you've at the very least fixed MoarVM issues #612 and #608
16:10 jnthn Yup :)
16:10 dogbert17 and perhaps more considering you found a missing MVMroot yesterday
16:14 jnthn Yeah, maybe so
16:16 jnthn Closed those two
16:19 jnthn Probably also https://github.com/MoarVM/MoarVM/issues/412
16:21 jnthn And https://github.com/MoarVM/MoarVM/issues/552 wow
16:22 jnthn So, 4 issues down :)
16:22 dogbert17 and perhaps RT #128553
16:22 synopsebot6 Link:  https://rt.perl.org/rt3/Public/Bug/Display.html?id=128553
16:26 jnthn looks like, yes :)
16:27 jnthn Maybe the gist linked in there can be turned into a spectest :)
16:27 jnthn Not by me right now though, I think it's time to rest and dinner :)
16:28 jnthn I think I'll also work on putting MVMFrame on a bit of a diet later in the month also
16:28 jnthn Given I added one more thing into it today :)
16:31 jnthn afk for a while
18:01 domidumont joined #moarvm
19:14 zakharyas joined #moarvm
19:38 agentzh joined #moarvm
19:39 agentzh joined #moarvm
19:47 AlexDaniel joined #moarvm
20:10 robertle joined #moarvm
20:25 agentzh joined #moarvm
21:38 colomon_ joined #moarvm
22:02 samcv it's alivvvveeee!
22:02 jnthn o.O
22:02 jnthn :)
22:04 samcv heh. it's working when concatenating things if it copies or doesn't copy
22:04 samcv yesterday i finally got it working but it only worked when there were no copies
22:04 samcv of strands
22:04 jnthn :)
22:05 jnthn Sounds good
22:06 jnthn Rest time, 'night
22:17 samcv doesn't pass all tests though hm
22:19 samcv gonna run spectest and hopefully will find something i can easily test and replicate
22:37 samcv \o/ yay it passes my concat test now :)
22:37 samcv exciting
22:38 MasterDuke and it's faster?
22:38 samcv as long as both things are CCC 0 then we can do the shortcut and not re_nfg
22:38 samcv well it's gotta be faster
22:38 samcv it doesn't have to run re_nfg every single time renormalization is needed
22:39 samcv for now it only works when it joins into two codepoints, it won't trigger if the CCC isn't 0 for both codpoints. but it still catches most cases
22:39 MasterDuke CCC?
22:39 samcv i will time it once i make sure that spectests all pass
22:39 samcv canonical combining class
22:42 samcv though i might be able to only trigger it if the ccc on one side is higher than the other but this is pretty decent for now
22:43 MasterDuke so you created a fast(er) path for concatenating strings where the CCCs are the same and renormalization is required?
22:44 samcv yep. it will combine the last codepoint of string a and the first codepoint of string b
22:44 samcv and leave the rest alone
22:44 samcv so can save loads of renormalization depending on the length of the strings
22:47 MasterDuke but that is a slightly special case, right? i.e., `my $a = 'a' ~ 'b'` won't be any faster?
22:48 MasterDuke but will `my $a = 'a' ~ "\n"` be faster?
22:50 samcv hmm it seems to make something fail and i'm not sure why. but it occurs if it's \r + \n
22:50 samcv and i just confirmed that it does generate the right synthetic grapheme
22:50 samcv but tests in t/spec/S17-supply/lines.t fail
22:50 samcv the test name is: "handle chunked lines"
23:01 samcv MasterDuke, looks like it will be probably faster for concetting two codepoints which concat into 1 as well
23:01 samcv like "\c[BOY]" ~ "\c[ZWJ]" for example
23:01 samcv if it consumes string a and string b then it returns early
23:01 samcv well since they stick together
23:08 samcv gonna update the whole toolchain and then try rerunning tests
23:10 samcv MasterDuke, but yeah simple cases aren't going to be faster. it's only renormalization gets triggered
23:10 samcv and i already made it trigger much less often, and now trying to make it so it doesn't have to renormalize *the entire of both strings* when you concat when it's triggered
23:40 MasterDuke cool
23:46 samcv oh no MoarVM panic: Internal error: invalid thread ID 10787 in GC work pass
23:47 timotimo hooray :\
23:49 samcv happening on a lot of roast files
23:49 samcv not sure if my changes have anything to do with it
23:50 samcv ok i don't get it with master. odd
23:50 samcv why would my changes do that...
23:51 samcv it's always the same thread id. maybe i'm passing some wrong value as a thread id somewhere?
23:52 samcv ugh

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary