Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-07-20

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:49 ilbot3 joined #moarvm
01:49 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:03 Geth_ ¦ MoarVM: 916403fd76 | (Samantha McVey)++ | src/strings/ops.c
03:03 Geth_ ¦ MoarVM: Start off collapsing strands as 8bit instead of 32bit
03:03 Geth_ ¦ MoarVM:
03:03 Geth_ ¦ MoarVM: Here instead of starting as 32-bit and then at the end converting
03:03 Geth_ ¦ MoarVM: the already iterated 32-bit blob to 8bit, we instead start off
03:03 Geth_ ¦ MoarVM: converting as 8-bit and as soon as we see a character that won't
03:03 Geth_ ¦ MoarVM: fit, we do the following:
03:03 Geth_ ¦ MoarVM:
03:03 Geth_ ¦ MoarVM: <…commit message has 12 more lines…>
03:03 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/916403fd76
03:03 Geth_ ¦ MoarVM: 42f7fb2d18 | (Samantha McVey)++ (committed using GitHub Web editor) | src/strings/ops.c
03:04 Geth_ ¦ MoarVM: Merge pull request #616 from samcv/collapse-strands-bitwork
03:04 Geth_ ¦ MoarVM:
03:04 Geth_ ¦ MoarVM: Start off collapsing strands as 8bit instead of 32bit
03:04 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/42f7fb2d18
04:59 Geth_ ¦ MoarVM: a739fc829f | (Samantha McVey)++ | src/strings/ops.c
04:59 Geth_ ¦ MoarVM: Don't trigger re_nfg on concat for base character + combiner
04:59 Geth_ ¦ MoarVM:
04:59 Geth_ ¦ MoarVM: This loosens the requirements so that only last_a needs to have a
04:59 Geth_ ¦ MoarVM: Canonical Combining Class of 0.
04:59 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/a739fc829f
05:48 brrt joined #moarvm
06:32 domidumont joined #moarvm
06:38 domidumont joined #moarvm
06:43 brrt joined #moarvm
06:51 lizmat joined #moarvm
07:11 Geth_ ¦ MoarVM/even-moar-jit: 374beb9ade | (Bart Wiegmans)++ | docs/jit/todo.org
07:11 Geth_ ¦ MoarVM/even-moar-jit: Document win64 presumed call arg failure
07:11 Geth_ ¦ MoarVM/even-moar-jit:
07:11 Geth_ ¦ MoarVM/even-moar-jit: Not entirely obvious why it happens, but there is certainly
07:11 Geth_ ¦ MoarVM/even-moar-jit: something strange going on.
07:11 Geth_ ¦ MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/374beb9ade
07:58 zakharyas joined #moarvm
08:01 domidumont joined #moarvm
08:48 timotimo https://github.com/MoarVM/MoarVM/compare/refuse_dangerous_inlines?expand=1 - why didn't we actually need this?
08:56 brrt joined #moarvm
08:57 jnthn morning o/
08:57 brrt ohai, jnthn, timotimo
08:58 timotimo yo
08:58 timotimo okay so we have a bug now where someone is running a long-running script and updates rakudo, then moarvm explodes when new code is touched
08:59 timotimo clearly it's accidentally lazy-reading new bytecode to do some code it hadn't run before
08:59 timotimo we might want to have rakudo put the core setting .moarvm file in a versioned place
09:00 jnthn Yeah, that sounds like "mmap'd file being changed under us" or some such
09:00 timotimo alternatively, is there a way to open the file such that any modifications will be invisible to us?
09:00 timotimo yup
09:00 jnthn Maybe
09:01 * timotimo reads man page
09:01 timotimo MAP_PRIVATE
09:01 timotimo Create a private copy-on-write mapping.
09:01 nwc10 MAP_PRIVATE - ... It is unspecified whether changes made to the file after the  mmap() call are visible in the mapped region.
09:01 nwc10 yay POSIX.
09:01 jnthn Or can we install in a smarter way?
09:01 timotimo It
09:01 nwc10 no, we can't be sure.
09:01 timotimo is unspecified whether changes made to the file after the
09:01 timotimo mmap() call are visible in the mapped region.
09:01 timotimo ;(
09:01 jnthn Such that the filename points to a different inode or whatever?
09:02 nwc10 that plan seems better
09:02 jnthn I kinda thought that we'd already have that...
09:02 jnthn But if we're seeing this issue it must be actually opening the existing file, truncating, and writing to it
09:03 timotimo oh, you mean it should mv the file away, then create a new one
09:03 timotimo that ought to work
09:04 timotimo or maybe even just unlink
09:04 timotimo that'll still get us into a spot of bother if an already-running script suddenly decides to "use" some new code that's installed in the same way
09:04 timotimo clearly the solution should be to prepend new versions to the file and when you see an unsupported moar version being required you skip ahead to the next bit
09:05 timotimo unbounded growth for our .moarvm files \o/
09:05 jnthn :P
09:05 jnthn "clearly" :P
09:06 jnthn As to string scanning/flattening, I think "only the regex engine flattens things out" is an easily learnable guideline
09:10 timotimo i'll send a part of our findings off to that bug report
09:11 timotimo or maybe write a fix for the installation part of the makefile
09:13 timotimo so we're currently installing it via cp
09:13 timotimo it has a --remove-destination flag - at least on my system
09:17 timotimo maybe i'll just put an RM_F in front
09:18 jnthn Hmm, curious
09:18 jnthn For
09:18 jnthn CORE.setting compilation, the amount of time the spesh worker spends is:
09:19 jnthn Total statistics time:      63.479ms
09:19 jnthn Total planning time:        4.184ms
09:19 jnthn Total specialization time:  3182.48ms
09:20 Geth_ ¦ MoarVM/spesh-worker: 76f127995a | (Jonathan Worthington)++ | tools/spesh-time.p6
09:20 Geth_ ¦ MoarVM/spesh-worker: Add a script to sum up logged spesh times.
09:20 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/76f127995a
09:21 jnthn So there goes the theory that the compilation is so slow 'cus spesh is doing an insane amount of work...
09:22 jnthn Maybe it really is just that we're missing incorporating various bits of the logging data still :)
09:22 jnthn Useful new script to have, anyway ;)
09:24 timotimo so gnu make offers a function called addprefix which would let me generate the to-delete paths from an array like M_PERL6_LANG_OUTPUT
09:24 timotimo i can't find anything about portability questions
09:28 timotimo i'll write a perl5 snippet instead i guess?
09:28 timotimo actually i can already run perl6 at this point
09:29 jnthn Yeah
09:30 nine jnthn: btw what do you think about the suggestion to ship a make.exe with MoarVM sources, so we'd no longer be held back by nmake?
09:30 Geth_ ¦ MoarVM/spesh-worker: 2f6f6868df | (Jonathan Worthington)++ | 3 files
09:30 Geth_ ¦ MoarVM/spesh-worker: Pass spesh plan into optimize phase.
09:30 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/2f6f6868df
09:31 jnthn nine: Not really too convinced by it
09:31 timotimo should i use rm-f on the output of the script or should i .IO.unlink inside perl6?
09:32 nwc10 Likely then any tarballs would get impounded by (some) virus scanners
09:32 jnthn I suspect there are other make variants out there that are also not smart
09:32 nwc10 certainly HP-UX make
09:33 nwc10 er, HP/UX
09:33 jnthn And yeah, shipping binary blobs in general is kinda icky
09:34 timotimo is HP/UX the ibm thing that's still getting development for it funded?
09:34 timotimo no, HP isn't ibm
09:34 timotimo what was the one i'm thinking about ..
09:35 jnthn I also suspsect that anything really clever you might try to do with makefiles would be easier to understand if it were done by the makefile calling a script
09:37 timotimo ok so how do i put a $_ in between '' inside a makefile without make replacing it with ""
09:37 timotimo just introduce a $_ variable that has the contents $_?
09:39 brrt joined #moarvm
09:40 nine jnthn: well "clever" is a very subjective thing with dmake being even too stupid for simple backticks - which makes even calling the clever script impossible
09:40 timotimo ha, i can just .Str instead of $_
09:42 jnthn We could just write boring, simple, makefiles. :)
09:43 jnthn It's apparently got us this far, so... :)
09:46 robertle joined #moarvm
10:12 Geth_ ¦ MoarVM/spesh-worker: a7e6d46595 | (Jonathan Worthington)++ | src/spesh/optimize.c
10:12 Geth_ ¦ MoarVM/spesh-worker: Re-instate lexical per invocant type optimization.
10:12 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/a7e6d46595
10:13 jnthn Putting that back shaved a good chunk off CORE.setting compilation
10:16 jnthn Remaining to re-instate are logged facts, size-based thresholds, and OSR
10:16 jnthn After that I'll probably do a round of tuning and bug-fixing to try and get this mergeable, before going on to the more clever things we should be able to do with it
10:22 nine exciting times :)
10:32 jnthn Putting the logged facts back is...interesting :)
10:33 jnthn (We do the logging in a totally different way now)
10:44 Geth_ ¦ MoarVM/spesh-worker: 33c8635db4 | (Jonathan Worthington)++ | 2 files
10:44 Geth_ ¦ MoarVM/spesh-worker: Mark further ops with :logged.
10:44 Geth_ ¦ MoarVM/spesh-worker:
10:44 Geth_ ¦ MoarVM/spesh-worker: So we have the index under which the logged values can be located.
10:44 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/33c8635db4
10:48 timotimo more opportunities for a cross-reference tool ...
10:51 brrt joined #moarvm
10:58 brrt joined #moarvm
10:59 vendethiel joined #moarvm
11:06 nwc10 Stage parse      : 612.736
11:07 nwc10 jnthn: master is just under 500 with ASAN
11:08 jnthn Seems you get the same size of improvement since yesterday as I got, then :)
11:09 jnthn Well, ratio of :)
11:10 jnthn lunch; making good progress on adding back the other logged facts though
12:19 zakharyas joined #moarvm
12:21 brrt joined #moarvm
12:22 lucasb joined #moarvm
13:04 Geth_ ¦ MoarVM/spesh-worker: 24f3f2fb86 | (Jonathan Worthington)++ | 8 files
13:04 Geth_ ¦ MoarVM/spesh-worker: Rework guard ops.
13:04 Geth_ ¦ MoarVM/spesh-worker:
13:04 Geth_ ¦ MoarVM/spesh-worker: The container-related guards are now gone; we'll guard at the point of
13:04 Geth_ ¦ MoarVM/spesh-worker: deopt instead. There's a new guard op which just guards on STable, not
13:04 Geth_ ¦ MoarVM/spesh-worker: on concrete vs. type object.
13:04 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/24f3f2fb86
13:04 jnthn bah, typo
13:04 Geth_ ¦ MoarVM/spesh-worker: 8a50ac77a8 | (Jonathan Worthington)++ | 8 files
13:04 Geth_ ¦ MoarVM/spesh-worker: Rework guard ops.
13:04 Geth_ ¦ MoarVM/spesh-worker:
13:04 Geth_ ¦ MoarVM/spesh-worker: The container-related guards are now gone; we'll guard at the point of
13:04 Geth_ ¦ MoarVM/spesh-worker: decont instead. There's a new guard op which just guards on STable,
13:04 Geth_ ¦ MoarVM/spesh-worker: not on concrete vs. type object.
13:04 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/8a50ac77a8
13:04 jnthn Better :)
13:11 jnthn Now getting a pretty obscure crash after putting the log-based facts/guards back in
13:30 jnthn Of course, it would just have to happen in a 200 basic block frame :/
13:47 jnthn bah, found it...mis-inserted the guard
13:47 jnthn Now NQP build/test is happy again
13:52 jnthn Hm, Rakudo one trips over a real NULL sneaking into an argument list
13:55 Geth_ ¦ MoarVM/spesh-worker: 5ca9e628f8 | (Jonathan Worthington)++ | 2 files
13:55 Geth_ ¦ MoarVM/spesh-worker: Add facts/guards based on logs.
13:55 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/5ca9e628f8
14:03 jnthn Latest problem is another issue in another giant frame
14:04 jnthn This time `unwanted` from Perl6::Actions
14:13 nwc10 ASAN goes Boom on the NULL pointer
14:14 nwc10 just in red. no technicolour barfage :-(
14:26 Geth_ ¦ MoarVM/spesh-worker: 5068ef91dd | (Jonathan Worthington)++ | 5 files
14:26 Geth_ ¦ MoarVM/spesh-worker: Re-introduce thesholds based on bytecode size.
14:26 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/5068ef91dd
14:26 nwc10 does that fix it for real, or just work around a problem?
14:28 jnthn Doesn't fix it for real, but does re-apply the same kind of hiding that we had in master :)
14:28 jnthn It's entirely possible (will need to check) that the thresholding was all that kept us from stumbling on this in master
14:28 domidumont1 joined #moarvm
14:31 jnthn The spesh "torture test" that should show these up is MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1
14:32 jnthn And that does show up some problems
14:52 timotimo very thunderstorm right now
14:53 jnthn Same here :D
14:53 jnthn Most welcome
14:54 timotimo aye
15:07 brrt joined #moarvm
15:09 Geth_ ¦ MoarVM/spesh-worker: aaafe12f73 | (Jonathan Worthington)++ | 3 files
15:09 Geth_ ¦ MoarVM/spesh-worker: Clean up spesh stats no longer updated.
15:09 Geth_ ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/aaafe12f73
15:09 jnthn That should bring memory down some :)
15:09 geekosaur joined #moarvm
15:10 jnthn Darn, spectset looks awful :/
15:10 jnthn *spectest
15:23 jnthn But at least I managed to find it doing something wrong in a frame wiht just a handful of basic blocks
15:29 domidumont joined #moarvm
15:31 AlexDaniel joined #moarvm
15:42 hoelzro joined #moarvm
16:01 jnthn Well, don't think I'll find it today; too tired now
16:01 jnthn Hopefully tomorrow
16:08 zakharyas joined #moarvm
16:15 domidumont joined #moarvm
16:33 lizmat joined #moarvm
17:06 robertle joined #moarvm
17:22 Util joined #moarvm
19:16 Geth_ ¦ MoarVM: 499059dccf | (Jonathan Worthington)++ | 3 files
19:16 Geth_ ¦ MoarVM: Enable native callbacks on different threads.
19:16 Geth_ ¦ MoarVM:
19:16 Geth_ ¦ MoarVM: Prior to this change, the callback was tied to the thread that passed
19:16 Geth_ ¦ MoarVM: it to the native library, so any attempt to call it on another thread
19:16 Geth_ ¦ MoarVM: would crash and burn. This issue can be resolved by conveying the
19:16 Geth_ ¦ MoarVM: instance, and resolving it to a thread context by using the native
19:16 Geth_ ¦ MoarVM: thread ID.
19:16 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/499059dccf
19:17 jnthn That could be a nasty one
19:17 jnthn Ended up with two threads trying to use the same thread context. You get some very, very interesting failure modes :P
19:17 timotimo oh, huh. interesting.
19:21 * timotimo BBL, groceries and such
19:41 nwc10 jnthn: $ ./nqp t/nqp/066-pararole.t
19:41 nwc10 MoarVM panic: Spesh arg guard: trying to add duplicate result for same guard
19:41 nwc10 nqp master, MoarVM origin/spesh-worker
20:06 nwc10 and ./perl6-m t/04-nativecall/21-callback-other-thread.t goes SEGV
20:09 jnthn On spesh-worker?
20:09 jnthn Yeah, that's what happened before the fix I put in above :)
20:10 jnthn The spesh arg guard dupe is odd, though you can get it to happen in NQP build with setting MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1
20:10 jnthn I'm guessing we must somehow end up with a duplicate in the plan, but I dunno how that could happen
20:13 nwc10 yes
20:13 nwc10 yes on spesh-worker
20:15 jnthn Guess I'll be doing some more bug hunting tomorrow :)
21:05 samcv jnthn, i'm going to make a grapheme codepoint iterator function ok
21:05 samcv MVM_grapheme_ci_init MVM_grapheme_ci_has_more MVM_grapheme_ci_get_grapheme
21:06 samcv can deduplicate some code that does it and make it easier to iterate on a synthetic
21:30 jnthn samcv: Sounds sensible enough
21:30 jnthn Slightly breaks the MVM_subsystemname_foo naming scheme I guess :)
21:34 samcv hmm
21:34 samcv well
21:34 samcv MVM_string_grapheme_foo
21:35 samcv whatever you want to call it i guess
21:36 samcv well my code seems to work. pretty useful
21:37 samcv https://gist.github.com/94faeab2ea8a1a3e1b5ec0d2189d3909
21:38 samcv i may optimize it a little more and have MVM_grapheme_ci_has_more check ci->visited_synth_codes == ci->total_synth_codes instead hm
21:38 samcv anyway it works
21:44 jnthn "Can only be used on synthetics"
21:44 jnthn It looks like it should work fine on non-synthetics too? :)
21:44 samcv well it can heh
21:44 samcv i wrote that before i added a non-synthetic case
21:45 samcv these functions make me very happy
21:45 jnthn Yeah, it's good that it handles the non-synthetic case
21:45 jnthn Avoids accidents :)
21:46 jnthn +1 to adding them, maybe with a name tweak
21:47 samcv though i might tweak it to not return the base character and just return the total number of cp i nthe synthetic
21:47 samcv hm
21:47 samcv not sure how i'd do that but hm
21:47 jnthn It'd match the APIs of the others if init didn't return anything I guess
21:48 jnthn But maybe the discrepancy is convenient?
21:50 lizmat joined #moarvm
21:51 samcv i could set visited synth codes to 0 when i init it maybe
21:51 samcv and then return the base character that way
21:52 samcv oh i see why. the base char isn't stored in the codepoint iterator
21:52 samcv that's why i return it
21:52 jnthn Guess it hinges on if things using the codepoint interator will generally want the base char separately or not
21:53 jnthn If they all just want the codepoints one at a time then they'll all have to work around the first one being delivered differently.
21:53 samcv so i'd need to store the base char in the ci
21:54 samcv hm
21:54 samcv so yeah i could add a basechar storage point to the ci
21:55 samcv and add that to the ci struct
21:55 samcv would that be ok jnthn?
21:56 jnthn Sounds fine to me.
21:56 samcv ok cool
21:56 jnthn Provided the use cases on the whole prefer it that way :)
21:56 jnthn Plus the consistency with the other iterators is nice
21:56 samcv yeah
21:56 samcv i think the consistency might be worth doing it that way
21:57 samcv jnthn, can i have the MVM_string_graphem_ci_init return the number of codepoints in the iterator though?
21:57 samcv the other one is void. but getting the total length would be quite useful
22:00 jnthn samcv: Yeah, I think that'd be OK
22:01 samcv unlike iterating on a string, we can know right then how long it is
22:03 jnthn Aye
22:54 samcv almost done getting synthetics to work in the concat thing :)
23:13 samcv jnthn, MVMROOT doesn't like me. not sure what i'm doing wrong. but it's giving me errors about suplynig it four arguments
23:15 jnthn Oh, the joy of C macros :)
23:16 jnthn The C preprocessor can get confused if you do a variable declaration
23:16 jnthn Like
23:16 jnthn MVMint32 i, n;
23:16 jnthn It somehow thinks the , in there is a macro argument separator?!
23:16 samcv yeah i have a var declaration above the MVMROOT
23:16 jnthn No, this is just inside of it
23:17 samcv https://gist.github.com/5e39280fe1639c0258787f2a5a5547e2
23:17 jnthn MVMCodepointIter last_a_ci, first_b_ci;
23:17 jnthn Try breaking that into two separate decls
23:17 jnthn (Yes, this is horrible. :/)
23:20 samcv yay :)
23:32 timotimo wow, that's pretty incredible
23:39 samcv commas not allowed
23:39 samcv or something. hah.
23:40 timotimo no comma allowed
23:40 timotimo like the topic of the python channel: NO ,
23:46 AlexDaniel joined #moarvm
23:51 Geth_ ¦ MoarVM/adjust_nursery_stringops: bb7e757265 | (Timo Paulssen)++ | src/strings/ops.c
23:51 Geth_ ¦ MoarVM/adjust_nursery_stringops: shrink remaining nursery when huge strings get made
23:51 Geth_ ¦ MoarVM/adjust_nursery_stringops: review: https://github.com/MoarVM/MoarVM/commit/bb7e757265
23:51 timotimo i thought i had already pushed tha
23:55 Geth_ ¦ MoarVM: cb98db0fdd | (Samantha McVey)++ | src/strings/iter.h
23:55 Geth_ ¦ MoarVM: Add functions for iterating the codepoints of a grapheme
23:55 Geth_ ¦ MoarVM:
23:55 Geth_ ¦ MoarVM: These functions act similarly to MVM_string_ci functions except
23:55 Geth_ ¦ MoarVM: MVM_string_grapheme_ci_init returns the number of codepoints in the
23:55 Geth_ ¦ MoarVM: grapheme instead of void. If using these you should exclusively use
23:55 Geth_ ¦ MoarVM: the associated MVM_string_grapheme_ci functions.
23:55 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/cb98db0fdd
23:55 Geth_ ¦ MoarVM: 092cc0d9f2 | (Samantha McVey)++ | src/strings/ops.c
23:55 samcv yay
23:55 Geth_ ¦ MoarVM: Don't re_nfg when last_a or first_b are synthetics in concat
23:55 Geth_ ¦ MoarVM:
23:55 Geth_ ¦ MoarVM: This also removes the canonical combining class restriction as well.
23:55 Geth_ ¦ MoarVM: If both codepoints aren't synthetic it will construct the renormalized_section
23:55 Geth_ ¦ MoarVM: from that. Otherwise it will use the new MVM_string_grapheme_ci functions to
23:55 Geth_ ¦ MoarVM: extract the codepoints from the graphemes.
23:55 Geth_ ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/092cc0d9f2
23:59 jnthn samcv++ timotimo++
23:59 jnthn Righty, cool enough to sleep. 'night o/

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary