Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-07-25

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:25 Geth ¦ MoarVM/collation-arrays: 9 commits pushed by (Samantha McVey)++
00:25 Geth ¦ MoarVM/collation-arrays: 9afa4fed0b | Don't dereference things already pointers. Make a macro to decrease code
00:25 Geth ¦ MoarVM/collation-arrays: 17a5752551 | Remove some dead code
00:25 Geth ¦ MoarVM/collation-arrays: 3d55e0c153 | Fix a bug that wouldn't let it iterate to the end
00:25 Geth ¦ MoarVM/collation-arrays: dd7f9e696e | Remove now unneeded specialcasing of æ and Æ
00:25 Geth ¦ MoarVM/collation-arrays: 5a81339607 | Have push_MVM_collation_values() return the number of keys it pushed
00:25 Geth ¦ MoarVM/collation-arrays: 9c18b6ae92 | Make more debug messages use hex instead of decimal for codepoints
00:25 Geth ¦ MoarVM/collation-arrays: 29e1f3edab | No need to initialize stack_b for no reason
00:25 Geth ¦ MoarVM/collation-arrays: dc6f355896 | Remove some code and make the code much more elegant
00:25 Geth ¦ MoarVM/collation-arrays: 317a15e11f | Make sure codepoints are sent back through again when not processed
00:25 Geth ¦ MoarVM/collation-arrays: review: https://github.com/MoarVM/MoarVM/compare/201720a973...317a15e11f
01:52 ilbot3 joined #moarvm
01:52 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
01:54 samcv was confused at why this particular test was failing. i guess we have incorrect property value for 2B81E.
01:54 samcv m: 0x2B81E.uniprop('Unified_Ideograph').say
01:54 camelia rakudo-moar 2fb8c7: OUTPUT: «True?»
01:54 samcv at least the unicode site says that should be False
01:57 samcv ahh
01:57 samcv 2B740..2B81D  ; Unified_Ideograph
01:57 samcv m: 0x2B81F.uniprop('Unified_Ideograph').say
01:57 camelia rakudo-moar 2fb8c7: OUTPUT: «True?»
01:57 samcv hmm i wonder how many we have that incorrect
03:46 samcv hmm not sure what i did with the C macro. turning it into a function for now just so no errors
03:50 Geth ¦ MoarVM/collation-arrays: 8dd36c2ad5 | (Samantha McVey)++ | src/strings/unicode_ops.c
03:50 Geth ¦ MoarVM/collation-arrays: Create is_unified_ideograph() and use func for gen. coll values
03:50 Geth ¦ MoarVM/collation-arrays:
03:50 Geth ¦ MoarVM/collation-arrays: Use functions to simplify things. I was going to make them macros but
03:50 Geth ¦ MoarVM/collation-arrays: the C preprocessor did not seem to like it.
03:50 Geth ¦ MoarVM/collation-arrays:
03:50 Geth ¦ MoarVM/collation-arrays: Add is_unified_ideograph() because there are some incorrect values for
03:50 Geth ¦ MoarVM/collation-arrays: this value in the MVM data.
03:50 Geth ¦ MoarVM/collation-arrays: review: https://github.com/MoarVM/MoarVM/commit/8dd36c2ad5
05:36 AlexDaniel joined #moarvm
06:17 domidumont joined #moarvm
06:47 domidumont joined #moarvm
06:48 brrt joined #moarvm
06:48 brrt good * #moarvm
06:50 domidumont joined #moarvm
06:54 brrt lizmat++ for p6weekly
07:02 samcv good * brrt
07:02 brrt ohai samcv
07:02 brrt ehm, have you seen the perl conference schedule?
07:03 brrt stuff looks weird
07:03 brrt talks splitted
07:03 samcv i haven't yet
07:05 samcv i couldn't find my 2nd talk on Thursdays page hmm. but it says so if i look on my page
07:07 brrt yeah, it's on thursday
07:32 Geth ¦ MoarVM: a052a3338b | (Jimmy Zhuo)++ | 3rdparty/tinymt/tinymt64.c
07:32 Geth ¦ MoarVM: Update TinyMT to version 1.1
07:32 Geth ¦ MoarVM:
07:32 Geth ¦ MoarVM: BUG fix of floating point conversion. In version 1.0.3 tinymt64_generate_double() may return 1.0
07:32 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/a052a3338b
07:40 zakharyas joined #moarvm
07:53 FROGGS joined #moarvm
08:38 robertle joined #moarvm
08:52 jnthn morning o/
09:04 brrt moarning jnthn
09:05 jnthn hi brrt
09:06 jnthn You'll be glad to know that the JIT Just Worked after being moved to a background thread :)
09:06 jnthn Hopefully it'll be the same story for the expr JIT
09:06 brrt i am glad to know that
09:06 jnthn Also hope the merge won't be hell... :)
09:07 brrt for the expr jit, or for the spesh worker?
09:07 jnthn Good news is that some of the more gnarly guards are gone
09:07 brrt \o/
09:07 jnthn Oh, for spesh worker into master it'll be easy
09:07 jnthn For master with spesh-worker merged then into expr jit is what I meant :)
09:07 brrt yeah, makes sense
09:07 jnthn If it's really ugly I can look at it, but hopefully... :)
09:08 brrt well, i'm not super worried as the branch is reasonably up to date
09:08 brrt the only big mistake i made was to refactor some parts of the jit graph builder in expr jit but not in master
09:08 brrt which means i need to patch that up when i merge
09:09 jnthn ah, OK
09:09 jnthn I didn't do any significant changes to the JIT, just minor bug fixes
09:09 jnthn Most of which weren't spesh-worker related anyway
09:09 jnthn Just uncovered by it
09:10 brrt alright. when are you planning to merge?
09:11 brrt wrt the expr jit, i was going to say, 'lets merge this month', but i'm a little less confident now that i've found bugs in the win64 version
09:12 jnthn brrt: I'm happy with the branch now in terms of it not having bugs (I suspect it's got less spesh bugs than master)
09:12 brrt hehe
09:13 jnthn brrt: I'm less happy that CORE.setting compilation time is 15% longer on the branch :/
09:13 samcv m: say 31/190377 * 100
09:13 camelia rakudo-moar acf9f9: OUTPUT: «0.0162835?»
09:13 samcv so i've achieved 0.016% failing tests with my collation-arrays branch :)
09:13 jnthn samcv++
09:13 brrt progress, yay
09:14 brrt jnthn, any idea why? synchronization overhead?
09:14 samcv got proper recursion implemented as well so it passes back to the same function nonmatching codeponits
09:15 jnthn brrt: I don't think it'd be that
09:15 samcv so right now i'm working on rewriting the code that generates the extra collation data. interestingly there were lots of errors in it. things like i'd watch the debug printing
09:15 samcv would see it go from A->B in the nodes. then check the unicode raw data. then notice that doesn't exist
09:15 samcv but it must have pushed the collation data for A+B onto it even though it wasn't a sequence.
09:16 jnthn brrt: Or at least, nothing suggests that in profile data
09:16 samcv i still can't figure out how the hell that happened. but rewriting that code and making decent progress on that as well
09:17 samcv i mean it's complicated enough that i had a not perfectly tidy piece of code and got it only failing 120/190377. then threw out a ton of code
09:17 samcv since i knew even though it passed pretty well it didn't have actual recursion. it would just "fake it" and get the naive values of single codepointns and not do actual recursion
09:18 samcv and got it generating data for many Tangut, Unified Ideograph, Unified Ideograph and in in one of two particular blocks. plus an unassigned one for unassigned codepoints
09:19 samcv so we have wrong data for Unified_Ideograph property. i'm not sure how but it extends too far
09:20 samcv and i checked the data. so there's something wrong in the current UCD. may not get to a whole rewrite of the unicode database for this grant. but i may do it after the grant is complete in my own time. it's something i don't really want to rush anyway
09:20 samcv and replacing it won't have as big a user facing impact as these other things
09:20 nine jnthn: correctness trumps speed, doesn't it?
09:21 brrt correctness++
09:21 jnthn nine: Yes, which is why I spent the last days getting it correct before looking into the speed thing :)
09:22 brrt (also, when I read samcv++s unicode work, i kind of feel how some of you must feel  when i'm rambling about the JIT :-))
09:22 samcv that's how i feel reading about JIT heh
09:22 samcv though i'd love to learn how the JIT works sometime :-)
09:22 samcv after my current project at least
09:23 samcv also. i will have collation working for a demo at the conference. which is neat. i had been trying to make sure i had a working full UCA implementation
09:23 samcv so feel pretty great about that
09:24 samcv hopefully other people will think it's amazing that you can sort æ with ae and ? with ffi
09:27 samcv <fri a ?  dri z ffi>.collate.say # (a dri ffi ?  fri z)
09:27 samcv amazing
09:27 nine samcv: it is :)
09:30 samcv jnthn, so compiling the CORE.setting is slower with JIT in a background thread — but it's faster when you actually run things? y/n
09:30 brrt (well, drop by my talk, and you'll know everything ;-))
09:30 samcv yay
09:30 jnthn samcv: Didn't do enough measurements to say conclusively yet, tbh
09:31 jnthn Though certainly I've seen cases where, because it's done on a background thread, the foreground one just completes the work before the opt is ready, which is likely a win :)
09:36 jnthn Well, seems the issue isn't that the JIT inserts the SC write barrier checks at least, tossing those out to try it doesn't help
09:44 * dogbert17 ran a stresstest with latest spesh-worker, looks good, only t/spec/S17-promise/nonblocking-await.t causes problems (MoarVM panic: Adding item to past fromspace to GC worklist).
09:45 jnthn Yeah, that one is flappy in master too
09:45 dogbert17 and there's already an issue for it. I guess it has nothing to do with spesh
09:45 jnthn No
09:46 jnthn heh, decided to callgrind the CORE.setting compilation to see if I get more useful data than perf
09:46 jnthn This is taking a little time :)
09:48 * jnthn notes that with the spesh worker on a background thread we now always have user threads
09:49 jnthn So could save the checks on that in a few places since the answer is always "yes" now
09:56 jnthn Hm, if valgrind's right then the long-standing GC sync-up issue is part of it
09:57 jnthn Though it's hard to tell how much of that is/isn't its due to how valgrind serializes things onto one thread
09:58 jnthn Also a lot of time in the (slow) get_attribute which makes we wonder if we're somehow doing a less good job of lowering those
10:03 brrt joined #moarvm
10:04 Geth ¦ MoarVM/spesh-worker: 0930889d10 | (Jonathan Worthington)++ | 3 files
10:04 Geth ¦ MoarVM/spesh-worker: We always have threads now; skip checks.
10:04 Geth ¦ MoarVM/spesh-worker:
10:04 Geth ¦ MoarVM/spesh-worker: The spesh worker thread counts as a thread, since it uses the fixed
10:04 Geth ¦ MoarVM/spesh-worker: size allocator. So, the paths for "not threaded" are never taken any
10:04 Geth ¦ MoarVM/spesh-worker: more. Toss those and the checks that go with them, which at least gets
10:04 Geth ¦ MoarVM/spesh-worker: rid of the branching cost.
10:04 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/0930889d10
10:04 Geth ¦ MoarVM/spesh-worker: 564b2fd593 | (Jonathan Worthington)++ | src/spesh/worker.c
10:04 Geth ¦ MoarVM/spesh-worker: Log time waiting for logs.
10:05 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/564b2fd593
10:59 brrt jnthn: since we currently still need perl for a fresh build anyway, and since strawberry perl ships a competent gcc environment anyway....
10:59 brrt we couild consider dropping support for msvc
11:01 brrt without loss of windows support
11:01 brrt not saying we should
11:29 JimmyZ joined #moarvm
11:30 JimmyZ brrt: not everyone uses strawberry perl :)
11:31 dogbert17 tested a multi threaded program against current MoarVM and spesh-worker, there's a 10+ percent performance loss
11:31 JimmyZ brrt: maybe use activeperl
11:31 * dogbert17 is still on 32 bit ...
11:33 brrt1 joined #moarvm
11:33 brrt1 JimmyZ, dogbert17: well, you're just wrong :-P
11:35 Geth ¦ MoarVM/spesh-worker: 9bdff8d4e0 | (Jonathan Worthington)++ | 10 files
11:35 Geth ¦ MoarVM/spesh-worker: Add non-logging variants of ops and use them.
11:35 Geth ¦ MoarVM/spesh-worker:
11:35 Geth ¦ MoarVM/spesh-worker: For the case where we can't JIT the frame. This keeps us from needing
11:35 Geth ¦ MoarVM/spesh-worker: to check if we need to log every time.
11:35 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/9bdff8d4e0
11:35 Geth ¦ MoarVM/spesh-worker: f4cce975a9 | (Jonathan Worthington)++ | src/core/frame.c
11:35 Geth ¦ MoarVM/spesh-worker: Don't take entry logging path when no spesh log.
11:35 Geth ¦ MoarVM/spesh-worker:
11:35 Geth ¦ MoarVM/spesh-worker: This could impede data collection for some frames by bumping the
11:35 Geth ¦ MoarVM/spesh-worker: logged entries counter up when not actually logging data.
11:35 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/f4cce975a9
11:35 Geth ¦ MoarVM/spesh-worker: 0c7b13239e | (Jonathan Worthington)++ | src/spesh/log.h
11:35 Geth ¦ MoarVM/spesh-worker: From measurement, fewer but larger logs are better
11:35 Geth ¦ MoarVM/spesh-worker:
11:35 Geth ¦ MoarVM/spesh-worker: 8192 is a notable improvement for 4096, and 16384 is a little better
11:35 Geth ¦ MoarVM/spesh-worker: still. At that size, 2 logs outstanding seems to be better than 1,
11:35 Geth ¦ MoarVM/spesh-worker: and about tied with 3, so may as well take the better memory use and
11:35 Geth ¦ MoarVM/spesh-worker: drop it down to 2.
11:35 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/0c7b13239e
11:35 brrt1 i mean, in seriousness
11:36 brrt1 we continue to invest a lot of effort into working arround msvc, which basically doesn't care about the language we're using
11:37 moritz doesn't jnthn develop with msvc?
11:37 nine brrt: would it be possible to compile the code in C++ mode? AFAIK that supports modern C much better
11:38 brrt that would still leave us with a makefile that has to work arround nmake
11:38 brrt but yes, we could try that
11:38 brrt and istr that jnthn does his work in a linux vm these days
12:00 stmuk joined #moarvm
12:06 timotimo nine: if by "in c++ mode" you mean "use msvc++ instead", then ... maybe
12:14 jnthn When I do work on Windows, though, it's with MSVC
12:14 jnthn Largely 'cus I know the tools well, and like the debugger
12:15 jnthn It's not likey I'll do windows dev at all if we drop MSVC support.
12:16 timotimo any particular reason to stay with msvc instead of msvcpp?
12:16 JimmyZ MSVC does have a good debugger UI :)
12:16 timotimo microsoft keeps telling people who complain about msvc to use msvcpp instead, i believe?
12:18 JimmyZ but still have nmake
12:18 timotimo at least it lets us put variable declarations in for loop headers
12:18 JimmyZ even you use msvcpp
12:19 JimmyZ so someone hates nmake and someone hates variable declaration?
12:24 jnthn timotimo: msvcpp
12:24 jnthn oops
12:24 jnthn timotimo: msvcpp could be worth a try though if it makes us wrap everything in extern C or something...
12:25 timotimo i don't think it will
12:25 timotimo i'd expect you can give it .c files and it'll behave itself
12:26 jnthn Well, worth a try if somebody fancies :)
12:26 timotimo i don't have a msvcpp :)
12:27 JimmyZ I did have but nmake does'nt support -j
12:27 timotimo of course it doesn't
12:28 timotimo but you can "set CL=/MP" so it "will use all the CPU cores"
12:28 domidumont joined #moarvm
12:28 jnthn win 23
12:28 buggable jnthn, Thank you for entering Accidental /win Lottery! The next draw will happen in 6 days, 11 hours, 31 minutes, and 7 seconds
12:29 timotimo but that seems to require that you put many .c files into the same commandline
12:29 timotimo whereas we have one invocation of the compiler for .o file we want to create
12:30 domidumont1 joined #moarvm
12:42 dogbert17 spesh-worker and --profile seems to be a less than optimal combination atm
12:53 brrt oh, what is worse, is linking though
12:57 jnthn dogbert17: How so?
12:57 jnthn The profiler is certaily gonna be out of date with regard to some things though, I susepct
12:57 jnthn For one, there's no spesh time any more since it's done on a background thread.
13:03 dogbert17 jnthn: immediate SEGV
13:04 dogbert17 setting MVM_SPESH_DISABLE=1 restores functionality
13:05 dogbert17 e.g. the following SEGV's, perl6 --profile -e 'say "Hello World!"'
13:13 dogbert17 here's the gist of the matter, https://gist.github.com/dogbert17/13246b9241bc492112737256ad910d68
13:40 Geth ¦ MoarVM/spesh-worker: 6ff2b3c525 | (Jonathan Worthington)++ | 4 files
13:40 Geth ¦ MoarVM/spesh-worker: Simplify our dealings with the thread list.
13:40 Geth ¦ MoarVM/spesh-worker:
13:40 Geth ¦ MoarVM/spesh-worker: Just take a lock for the cases where we need to update or snapshot it.
13:40 Geth ¦ MoarVM/spesh-worker: This eliminates a low-benefit, and diffult to reason about, lock-free
13:40 Geth ¦ MoarVM/spesh-worker: appraoch in the GC orchestration, making for far simpler code there.
13:40 Geth ¦ MoarVM/spesh-worker: It also disentangles it a bit from the GC start flag handling, which
13:40 Geth ¦ MoarVM/spesh-worker: will be getting a refactor shortly to try to improve its resource
13:40 Geth ¦ MoarVM/spesh-worker: usage.
13:40 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/6ff2b3c525
13:41 jnthn It's apparently difficult to type difficult...
13:42 dogbert17 appraoch :)
13:42 jnthn bah
13:43 Geth ¦ MoarVM/spesh-worker: 5e2ac66028 | (Jonathan Worthington)++ | 4 files
13:43 Geth ¦ MoarVM/spesh-worker: Simplify our dealings with the thread list.
13:43 Geth ¦ MoarVM/spesh-worker:
13:43 Geth ¦ MoarVM/spesh-worker: Just take a lock for the cases where we need to update or snapshot it.
13:43 Geth ¦ MoarVM/spesh-worker: This eliminates a low-benefit, and difficult to reason about,
13:43 Geth ¦ MoarVM/spesh-worker: lock-free approach in the GC orchestration, making for far simpler
13:43 Geth ¦ MoarVM/spesh-worker: code there. It also disentangles it a bit from the GC start flag
13:43 Geth ¦ MoarVM/spesh-worker: handling, which will be getting a refactor shortly to try to improve
13:43 Geth ¦ MoarVM/spesh-worker: its resource usage.
13:43 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/commit/5e2ac66028
13:43 jnthn It's my branch and I can force if I want to... :P
13:44 jnthn I've figured I'll take on the GC orchestration changes now
13:44 jnthn Because every time I try to callgrind something concurrent it distorts it a good bit
13:45 jnthn Plus it really does cause performance issues
13:45 jnthn Possibly accounting for some of the slowdown we're seeing
13:53 zakharyas joined #moarvm
13:59 zakharyas joined #moarvm
14:38 FROGGS joined #moarvm
14:52 Geth ¦ MoarVM/spesh-worker: 4 commits pushed by (Jonathan Worthington)++
14:52 Geth ¦ MoarVM/spesh-worker: fd1ef61a81 | Introduce GC orchestration mutex/condvars.
14:52 Geth ¦ MoarVM/spesh-worker: db5ba00ee3 | Switch gc_start to using condvar.
14:52 Geth ¦ MoarVM/spesh-worker: 4d0a9d1463 | Switch gc_finish to using condvar.
14:52 Geth ¦ MoarVM/spesh-worker: 54fb63c4e9 | Switch gc_intrays_clearing to use condvar.
14:52 Geth ¦ MoarVM/spesh-worker: review: https://github.com/MoarVM/MoarVM/compare/5e2ac66028...54fb63c4e9
14:52 zakharyas joined #moarvm
14:53 jnthn There we go
14:53 yoleaux 13:51Z <zengargoyle> jnthn: is that concurrency talk/slides supposed to be not shared around?
14:55 jnthn Unfortunately, not much difference to see in CORE.setting build time, though wins back a bit on make spectest. Will surely win something off various more real-world threading code that I've seen end up paying a lot in the GC orchestration.
14:55 jnthn Hopefully the callgrind output will be more sensible also
14:55 jnthn Recording some now
15:06 AlexDani` joined #moarvm
15:07 jnthn Yeah, looks rather better
15:08 brrt joined #moarvm
15:11 [Coke] jnthn++
15:14 zakharyas joined #moarvm
15:46 jnthn Some intesting hotspots, though none really shining a light on the spesh changes
15:46 jnthn Leaving a callgrind run going, anyways, so I'll have a complete one to look at in the morning
15:47 jnthn Also one local change to try and make a things a bit better, but inconclusive so far
16:16 AlexDani` joined #moarvm
16:35 timotimo here's something i find a bit puzzling
16:35 timotimo the statistics for lower_signature
16:35 timotimo almost every offset has two different things it might have logged:
16:35 timotimo a) some kind of QAST:: node
16:36 timotimo b) a BOOTCode
16:36 timotimo apparently always in exact same amounts each
16:37 timotimo is something wonky with regards to logging?
16:48 timotimo something very similar happens in install_lexical_symbol. this time it's observable across a whole bunch of Type Tuples
16:49 timotimo https://gist.github.com/timo/374ae300d4abdd593b24d51be5e4131a
16:50 timotimo the more i look the more often i see these bootcode pieces sprinkled around
17:18 zakharyas joined #moarvm
17:46 zakharyas joined #moarvm
18:38 domidumont joined #moarvm
18:50 jnthn timotimo: Is it a BOOTCode value and a QAST::Node type ooc?
18:51 jnthn iirc invoke logs the return type and the invoked value
19:11 brrt joined #moarvm
19:13 timotimo ooooh
19:13 timotimo well, then that's it, and not as surprising as i thought
20:24 zakharyas joined #moarvm
20:32 camelia joined #moarvm
20:32 eater joined #moarvm
20:32 samcv good *
20:32 timotimo good
20:38 samcv *
20:42 robertle joined #moarvm
20:42 jnthn o/ samcv
21:18 jpf4 joined #moarvm
22:37 TimToady joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary