Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-01-11

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:14 btyler joined #moarvm
00:15 pyrimidine joined #moarvm
00:25 jnthn joined #moarvm
01:11 pyrimidine joined #moarvm
02:11 pyrimidine joined #moarvm
02:48 ilbot3 joined #moarvm
02:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:17 pyrimidine joined #moarvm
03:45 dalek joined #moarvm
03:47 pyrimidine joined #moarvm
04:56 pyrimidine joined #moarvm
05:16 pyrimidine joined #moarvm
05:42 pyrimidine joined #moarvm
05:52 pyrimidine joined #moarvm
07:01 domidumont joined #moarvm
07:07 domidumont joined #moarvm
07:09 pyrimidine joined #moarvm
07:33 brrt joined #moarvm
07:35 brrt good * #moarvm
07:37 brrt joined #moarvm
07:51 pyrimidine joined #moarvm
08:03 pyrimidine joined #moarvm
08:31 domidumont joined #moarvm
08:35 zakharyas joined #moarvm
08:43 samcv hi brrt
08:44 brrt hi samcv
08:44 samcv brrt, what's the best way to add getstrfromname to jvm and js?
08:44 samcv i mean i'm going to make an op that just basically calls getcodepointfromname and runs char on it for backends that don't have getstrfromname
08:44 samcv where's the best place to add this?
08:45 brrt i'd expect NQP, to be honest
08:45 brrt not moarvm clearly
08:47 samcv yeah but what file any clue
08:48 arnsholt samcv: The main file for nqp::ops in the JVM backend is src/vm/jvm/runtime/org/perl6/nqp/runtime/Ops.java
08:48 samcv well it needs to be for jvm and js
08:48 samcv i know that file
08:49 samcv but moarvm is going to implement that natively and all other backends will have a sub
08:52 domidumont joined #moarvm
09:09 pyrimidine joined #moarvm
09:14 samcv arnsholt, thoughts?
09:18 brrt moarvm leads jvm, js, and that's mostly ok
09:19 arnsholt Yeah, just not implementing it for JVM/JS is probably not that big a deal either
09:19 domidumont joined #moarvm
09:24 pyrimidine joined #moarvm
09:26 samcv i'd like to not have to do #?if though in Actions.nqp
09:27 samcv what timezone is jnthn in btw
09:29 arnsholt European
09:33 lizmat CET
09:33 lizmat $ date
09:33 lizmat Wed Jan 11 10:33:18 CET 2017
09:33 samcv cool
10:01 pyrimidine joined #moarvm
10:13 jnthn samcv: Still need help on the new op stuffs?
10:13 samcv uhm. arnsholt helped me and wrote a java one for me
10:16 jnthn OK
10:16 jnthn You figured out the rebootstrap thingy?
10:18 samcv nope
10:19 pyrimidine joined #moarvm
10:25 jnthn Ah
10:25 jnthn There may be a Makefile target
10:26 jnthn But I tend to
10:26 jnthn 1) Add the new op that I'd like to use inside of NQP
10:26 jnthn 2) cp *.moarvm src/vm/stage0/
10:26 jnthn And then you've updated the stage0 compiler with the new build and the op will be available.
10:27 samcv there is a makefile target for genning the stage0
10:27 samcv yeah i've done that and it works
10:31 jnthn ah, OK :)
10:50 samcv jnthn, going to sleep soon if you can review that PR if you aren't busy
10:53 dogbert17 joined #moarvm
10:54 jnthn samcv: Have been a bit busy with $other-task but free in 2 mins
10:54 samcv kk np
11:05 jnthn 2 unused variables, otherwise it's fine :)
11:07 samcv ok i will remove those vars jnthn
11:07 samcv extra vars removed now jnthn
11:08 jnthn oh, sorry :(
11:08 jnthn I just spotted a memory leak oto
11:08 jnthn *too
11:08 jnthn Shoulda caught that earlier
11:09 jnthn Commented where
11:10 samcv good catch
11:10 jnthn Righty, what should I work on next...
11:13 samcv memleak fixed
11:14 nwc10 jnthn: lunch!
11:15 nwc10 jnthn: there remains a concurrency issue that you say you know the cause of, that can cause spectest6 under ASAN to explode with barfage
11:15 nwc10 is that one hard to fix?
11:20 jnthn Yes :)
11:20 jnthn But probably also worthwhile :)
11:20 nwc10 bother
11:20 nwc10 it might be the last real blocker before making spectest6 the default spectest
11:20 nwc10 dogfood!
11:21 jnthn Yeah
11:21 jnthn Well, it may not be too difficult, just need to think about it carefully and chase down whether a simple solution can work without totally screwing things up :)
11:22 samcv i am going to try and sleep a bit. i will be up in 2.5 hours though jnthn. please merge my PR if everything seems fine
11:22 samcv and i'll stage the next one when i wake up
11:23 jnthn samcv: Approved the PR; will merge after Travis gets done :)
11:24 samcv kk :)
11:24 jnthn Rest well
11:24 samcv o/
11:24 jnthn o/
11:25 dogbert17 jnthn: are you doing Perl6 work today?
11:29 jnthn dogbert17: At least some of the day, yes :)
11:29 jnthn I think I'll try and sort out the spectest6-affecting inlining/threads/compunit/locks crash
11:29 dogbert17 start with something simple (if there is such a thing) and then bring out the big guns, i.e. cheese :-)
11:29 jnthn heh, add gc on the lsit too :)
11:29 jnthn *list
11:41 jnthn Pro tip: do kill all gdb instances, remember to | grep gdb before you | xargs kill -9
11:41 jnthn *to
11:41 nwc10 just kill init - it's much faster
11:42 jnthn :P
11:42 jnthn The reason I ended up with a dozen GDB instances is because if you run the Perl 6 spectest harness using perl6-gdb-m, that executable name is inherited
11:42 jnthn And used to run all the spectests
11:43 jnthn So it's like gdbception
11:43 jnthn And this doesn't go very well
11:46 brrt joined #moarvm
11:54 jnthn lunch &
11:55 pyrimidine joined #moarvm
12:03 mst joined #moarvm
12:07 * ilmari too
12:18 pyrimidine joined #moarvm
13:19 pyrimidine joined #moarvm
14:22 pyrimidine joined #moarvm
14:25 jnthn OK, so having analyzed things a bit
14:25 jnthn inlining can try to acquire the compilation unit update_mutex in 3 cases
14:26 jnthn None of those cases ever inside of them trigger deserialization nor GC nor acquire any further mutexes
14:27 jnthn And the data they touch is disjoint from MVM_bytecode_finish_frame and prepare_and_verify_static_frame which also use update_mutex
14:27 jnthn Which are more involved and really could do with the mutex in question
14:27 jnthn Uh, reentrant
14:28 jnthn Thus it's possible to split this up into a pair of mutexen, one GC-aware fancy one, one boring uv_mutex
14:29 jnthn Thus solving the GC in spesh issue
14:29 jnthn So, that part seems relatively easy to solve
14:29 jnthn There is however a second problem.
14:29 jnthn In that in fixing up a compunit for the sake of an inline, we realloc
14:30 jnthn But other threads could easily still hang on to the old pointer and use it
14:30 jnthn This is likely why ASAN triggered
14:30 jnthn I believe this can be fixed using the fixed size allocator and its free_at_safepoint functionality.
14:46 pyrimidine joined #moarvm
15:10 Geth MoarVM/fix-inline-and-threads: 1f20dc77bb | (Jonathan Worthington)++ | 6 files
15:10 Geth MoarVM/fix-inline-and-threads: Fix GC in spesh triggered by managed mutex use.
15:10 Geth MoarVM/fix-inline-and-threads:
15:10 Geth MoarVM/fix-inline-and-threads: When inlining, the update_mutex of a MVMCompUnit was acquired. This
15:10 Geth MoarVM/fix-inline-and-threads: for some time has been a high-level mutex, made so since these are
15:10 Geth MoarVM/fix-inline-and-threads: reentrant and also cannot lead to GC deadlocks. Unfortunately, spesh
15:10 Geth MoarVM/fix-inline-and-threads: has a "no GC during specialization" constraint, which use of this
15:10 Geth MoarVM/fix-inline-and-threads: mutex violated. Thankfully, the things it used the mutex for were
15:10 Geth MoarVM/fix-inline-and-threads: disjoint from the set of things that need a high-level mutex. So,
15:10 Geth MoarVM/fix-inline-and-threads: split it into two mutexes protecting the different fix, and make the
15:10 Geth MoarVM/fix-inline-and-threads: one that inlining needs to acquire be a low-level mutex which will
15:10 Geth MoarVM/fix-inline-and-threads: never cause entry to the GC.
15:10 Geth MoarVM/fix-inline-and-threads: review: https://github.com/MoarVM/MoarVM/commit/1f20dc77bb
15:10 jnthn That deals with the first half of the issue.
15:11 * brrt looks
15:11 jnthn This covers https://github.com/MoarVM/MoarVM/issues/478 and 2 related cases
15:15 FROGGS joined #moarvm
15:16 jnthn No such method 'made' for invocant of type 'TAP::Runner::State'
15:16 jnthn ...well, I didn't fix that issue...
15:17 pyrimidine joined #moarvm
15:19 timotimo the free_at_safepoint is definitely the right way to deal with that one
15:20 pyrimidi_ joined #moarvm
15:21 Geth MoarVM: 388cbb69e4 | (Samantha McVey)++ | 14 files
15:21 Geth MoarVM: Implement getstrfromname/MVM_unicode_string_from_name
15:21 Geth MoarVM:
15:21 Geth MoarVM: Returns a string from its Unicode name. Unlike MVM_unicode_lookup_by_name, this works
15:21 Geth MoarVM: for multiple codepoints. This allows us to support Emoji sequences and other potential
15:21 Geth MoarVM: names which are sequences of codepoints.
15:21 Geth MoarVM: review: https://github.com/MoarVM/MoarVM/commit/388cbb69e4
15:21 Geth MoarVM: c23bb45cd4 | (Jonathan Worthington)++ | 14 files
15:21 Geth MoarVM: Merge pull request #492 from samcv/getstrbyname
15:21 Geth MoarVM:
15:21 Geth MoarVM: Implement getstrbyname/MVM_unicode_string_from_name to retrieve grapheme clusters by name
15:21 Geth MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c23bb45cd4
15:40 Geth MoarVM/fix-inline-and-threads: a98812b14a | (Jonathan Worthington)++ | 3 files
15:40 Geth MoarVM/fix-inline-and-threads: Fix data race in inlining extop fixup.
15:40 Geth MoarVM/fix-inline-and-threads:
15:40 Geth MoarVM/fix-inline-and-threads: This switches to use the fixed size allocator for this memory, which
15:40 Geth MoarVM/fix-inline-and-threads: provides a free-at-safepoint function. Previously, the realloc could
15:40 Geth MoarVM/fix-inline-and-threads: read to a thread reading memory that was freed by the realloc.
15:40 Geth MoarVM/fix-inline-and-threads: review: https://github.com/MoarVM/MoarVM/commit/a98812b14a
15:40 timotimo lead* :)
15:41 * [Coke] wonders if it would be worth elminating the branch name after the first line.
15:41 Geth MoarVM/fix-inline-and-threads: 151cb272db | (Jonathan Worthington)++ | 3 files
15:41 Geth MoarVM/fix-inline-and-threads: Fix data race in inlining extop fixup.
15:41 Geth MoarVM/fix-inline-and-threads:
15:41 Geth MoarVM/fix-inline-and-threads: This switches to use the fixed size allocator for this memory, which
15:41 Geth MoarVM/fix-inline-and-threads: provides a free-at-safepoint function. Previously, the realloc could
15:41 Geth MoarVM/fix-inline-and-threads: lead to a thread reading memory that was freed by the realloc.
15:41 Geth MoarVM/fix-inline-and-threads: review: https://github.com/MoarVM/MoarVM/commit/151cb272db
15:42 jnthn Nice thing about working in a branch: I can get away with rebase :)
15:42 timotimo hah, i didn't expect you to actually fix that typo :D
15:43 dogbert17 dogbert@dogbert-VirtualBox ~/repos/doc $ perl6 htmlify.p6
15:43 dogbert17 Initializing ...
15:43 dogbert17 ===SORRY!=== Error while compiling /home/dogbert/repos/doc/htmlify.p6
15:43 dogbert17 Variable '$no-highlight' is not declared
15:43 dogbert17 at /home/dogbert/repos/doc/htmlify.p6:1002
15:43 dogbert17 ------>     if !⏏$no-highlight and $proc.started {
15:43 dogbert17 samcv ^^
15:44 brrt so, i've pondered a bit more with regards to the issue i was discussing yesterday; the issue is really that there can be more-than-one live range starting at a single 'point'
15:47 [Coke] dogbert17: thanks for the heads up. (moved to #perl6)
15:49 brrt so in a way, the 'only expire once per point' is pretty good, but you'd have to do it at the *last* point :-(
15:52 brrt huge headache altogether :-(
16:00 timotimo hopefully not actual pain in your head area
16:01 Geth MoarVM/fix-inline-and-threads: fc771134f2 | (Jonathan Worthington)++ | 3 files
16:01 Geth MoarVM/fix-inline-and-threads: Fix data race in inlining extop fixup.
16:01 Geth MoarVM/fix-inline-and-threads:
16:01 Geth MoarVM/fix-inline-and-threads: This switches to use the fixed size allocator for this memory, which
16:01 Geth MoarVM/fix-inline-and-threads: provides a free-at-safepoint function. Previously, the realloc could
16:01 Geth MoarVM/fix-inline-and-threads: lead to a thread reading memory that was freed by the realloc.
16:01 Geth MoarVM/fix-inline-and-threads: review: https://github.com/MoarVM/MoarVM/commit/fc771134f2
16:01 Geth MoarVM/fix-inline-and-threads: 5389d6080b | (Jonathan Worthington)++ | 3 files
16:01 Geth MoarVM/fix-inline-and-threads: Fix data race in callsite fixup during inlining.
16:01 Geth MoarVM/fix-inline-and-threads:
16:01 Geth MoarVM/fix-inline-and-threads: Switches to using the fixed size allocator, so we can free the array
16:01 Geth MoarVM/fix-inline-and-threads: of callsites at a safepoint, thus preventing reads of freed memory.
16:01 Geth MoarVM/fix-inline-and-threads: review: https://github.com/MoarVM/MoarVM/commit/5389d6080b
16:01 jnthn 2 down, 1 to go
16:02 brrt hmmm
16:02 dogbert17 oO
16:02 timotimo do we have a reason to turn free_at_safepoint into an immediatel free if we have only one thread active? (or are we perhaps already doing that?)
16:03 pyrimidine joined #moarvm
16:04 jnthn Already doing it
16:04 timotimo OK
16:06 dogbert17 so the "Must not GC when in the specializer/JIT" message will be a thing of the past, nice
16:09 jnthn Well, for all the reasons it can occur that we've identified so far, yes :)
16:09 jnthn The bad news is that, unless this final fix to the strings heap actually nails it, the realloc bug is not the one that could cause the "method made not found" or whatever failure mode.
16:09 jnthn But it does remove a way to SEGV in spectest6
16:10 jnthn And maybe in htmlify --parallel too
16:16 Geth MoarVM/fix-inline-and-threads: ef4d6a767a | (Jonathan Worthington)++ | 3 files
16:16 Geth MoarVM/fix-inline-and-threads: Fix data race in string fixup during inlining.
16:16 Geth MoarVM/fix-inline-and-threads:
16:16 Geth MoarVM/fix-inline-and-threads: Switches to using the fixed size allocator, so we can free the array
16:16 Geth MoarVM/fix-inline-and-threads: of strings at a safepoint, thus preventing reads of freed memory.
16:16 Geth MoarVM/fix-inline-and-threads: review: https://github.com/MoarVM/MoarVM/commit/ef4d6a767a
16:17 pyrimidine joined #moarvm
16:22 Geth MoarVM: jnthn++ created pull request #495: Fix bugs in interaction between inlining, GC, and threads
16:22 Geth MoarVM: review: https://github.com/MoarVM/MoarVM/pull/495
16:23 jnthn There we go :)
16:23 jnthn nwc10: This should quiet the ASAN noise in spectest6
16:28 * timotimo is going over the diff
16:29 timotimo do we want to build a realloc for FSA at some point?
16:30 timotimo it could allow us to sometimes not do anything because we still have a bit of extra space in the same location
16:34 jnthn I doubt we ever will withthe FSA
16:34 jnthn Since it buckets stuff
16:34 timotimo we tend to have 8 bytes at the end of a bucket if we had to round up, don't we?
16:34 jnthn Hm, though there could be some I guess
16:34 jnthn Yeah
16:35 jnthn I guess we could consider it
16:35 timotimo if we have lists of pointers, that'd cut our alloc/free amount by half if we have workloads that always add a single pointer
16:35 jnthn It may be worth factoring the pattern out if nothing else.
16:35 timotimo aye, and that
16:35 timotimo but i took that as a given ;)
16:41 timotimo looke dover the code, looks good so far
16:41 timotimo it'll get a bit cleaner when we have realloc :)
16:42 timotimo "looke dover"
16:51 mst small village near the coast
16:51 mst lovely cafe
17:02 domidumont joined #moarvm
17:03 dogbert17 noticed an oddity wrt specttesting
17:04 dogbert17 if I run 'make spectest' it will immediatly log the message 'Inline::Perl5 not installed: not running Perl 5 integration tests', naturally no perl5 integration tests are run
17:05 dogbert17 but if I write 'make spectest HARNESS_TYPE=6' it start running stuff like 't/spec/S01-perl-5-integration/array.rakudo.moar ................... ok'
17:06 dogbert17 how is that possible, anyone knows?
17:09 * brokenchicken didn't know you could pass env vars AFTER the command...
17:10 * dogbert17 seems to do the right thing though: './perl6-m -Ilib t/harness6 --fudge --tests-from-file=t/spectest.data'
17:12 pyrimidine joined #moarvm
17:20 pyrimidine joined #moarvm
17:28 timotimo that's just something make allows you to do
17:28 brokenchicken ah
17:29 timotimo i.e. it'll parse its arguments for something that looks like an env var assignment
17:37 pyrimidine joined #moarvm
18:01 Geth MoarVM: samcv++ created pull request #496: Fix RT #117683 \c[LINE FEED] \c[CARRIAGE RETURN]
18:01 Geth MoarVM: review: https://github.com/MoarVM/MoarVM/pull/496
18:01 samcv oh well i didn't do it 4 hours ago but just pulled my next PR. which is pretty minor but fixes an RT
18:03 samcv once this is merged I can update roast to use these names instead of the unicode 1 names as per TimToady's blessing
18:07 jnthn samcv: Where are these names that we're adding from?
18:07 samcv unicode alias names
18:07 samcv they are in roast as well
18:07 samcv (already in roast)
18:09 samcv eventually we're going to programmaticlly add all the alias names when we construct the hash, but for now fixing this will allow me to correct roast
18:09 samcv since i already fixed it on jvm
18:10 jnthn Right, that was going to be my next question, why aren't we adding all of them from the UCD source :)
18:10 samcv we will soon :)
18:10 jnthn OK. I'm fine with this in the meantime, then :)
18:10 samcv sweet
18:10 diakopter originally we overrode some of those control chars because.. I can't remember
18:11 samcv because otherwise \c[NULL] wouldn't work and others etc
18:11 samcv so once I programmatically add all the alias names we can stop storing unicode 1 names in the normal name fields
18:13 samcv and then can add the unicode 1 names back in
18:13 samcv as step 3
18:14 samcv though as part of step 3 we will need to temporarily whitelist the 2 unicode 1 names which don't cooraspond to the alias names, like LINE FEED (LF) is the unicode 1 name, but the alias name is LINE FEED or LF
18:14 samcv and just warn the user for a while if they use one of the unicode 1 names which are not alias names
18:39 pyrimidine joined #moarvm
18:58 ggoebel joined #moarvm
19:10 pyrimidine joined #moarvm
19:19 samcv jnthn, gonna get a merge on that before you go to bed tonight? sucks we're in totally different timezones :P
19:20 pyrimidine joined #moarvm
19:21 samcv though i guess i'm awake hours earlier than I usually am so that is good
20:30 domidumont joined #moarvm
20:48 Geth MoarVM: ef734fc8d7 | (Samantha McVey)++ | 2 files
20:48 Geth MoarVM: Fix RT #117683 \c[LINE FEED] \c[CARRIAGE RETURN]
20:48 Geth MoarVM:
20:48 Geth MoarVM: Also fixes \c[NEXT LINE] and \c[FORM FEED] as well.
20:48 Geth MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ef734fc8d7
20:48 Geth MoarVM: 64e2d938df | (Jonathan Worthington)++ | 2 files
20:48 Geth MoarVM: Merge pull request #496 from samcv/RT#117683
20:48 Geth MoarVM:
20:48 Geth MoarVM: Fix RT #117683 \c[LINE FEED] \c[CARRIAGE RETURN]
20:48 samcv yey
20:48 Geth MoarVM: review: https://github.com/MoarVM/MoarVM/commit/64e2d938df
20:48 jnthn samcv: Figured I'd wait on the Travis build...they're a bad slow these days
20:48 samcv np. yeah they are slow
20:48 jnthn Well, to be fair, seems the OSX ones are to blame
20:49 timotimo oh yeah, clang is really bad at our interp.c
20:50 samcv also jnthn how to silence 3rdparty/uthash.h:151:10: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
20:50 samcv head = (add);
20:50 samcv i mean everything is fine. but the type checking is causing it to warn
20:51 samcv probably because i added a new typedef for the new hash to store the sequence names
20:52 jnthn samcv: Urgh, I suspect the answer lies in a nest of macros...
20:52 timotimo most likely
20:52 timotimo isn't it supposed to show like 20 lines of explanatory text?
20:54 pyrimidine joined #moarvm
21:00 jnthn Any feedback on https://github.com/MoarVM/MoarVM/pull/495 welcome
21:01 jnthn I wouldn't mind checking that the SEGVs I see now in spectest6 also happen on master
21:01 jnthn Will do that tomorrow or so
21:01 timotimo that, or if?
21:02 jnthn Well, if they're not on master and they are on this branch, it's a tad suspect :)
21:02 timotimo sure
21:08 timotimo i don't actually have a spectest6 target, so i actually have to HARNESS_TYPE=6 make spectest?
21:08 lizmat timotimo: yes
21:08 timotimo 'k
21:09 timotimo when i run into that segfault, the whole thing stops, right? not just one single test file?
21:09 jnthn Right, it's in the harness
21:09 lizmat yup
21:09 timotimo good
21:09 timotimo have we tested the harness with valgrind/asan?
21:09 jnthn Not yet :)
21:09 lizmat afaik a loooong time ago
21:09 jnthn Oh
21:09 timotimo i can try that, then
21:10 jnthn nwc10++ did
21:10 jnthn But I fixed the bug he found
21:10 jnthn That's what my branch is about :)
21:10 jnthn So "not since my branch" :)
21:12 timotimo should i investigate what happens if i cherry-pick the "make gen2 collection happen more often" commit? or is that already landed in master?
21:13 jnthn It's in master
21:13 timotimo there it is, yes
21:13 jnthn My branch is the PR you reviewed
21:13 timotimo right
21:13 jnthn Off for a little walk in the snow :)
21:14 timotimo gdb be like
21:14 timotimo detached after fork. detached after fork. detached after fork. detached after fork. detached after fork.
21:22 pyrimidine joined #moarvm
21:31 pyrimidine joined #moarvm
21:33 timotimo interesting
21:33 timotimo moar: 3rdparty/libuv/src/unix/stream.c:1499: uv_read_start: Assertion `((stream)->io_watcher.fd) >= 0' failed.
21:38 timotimo i wonder how that happens?
21:39 timotimo i mean, this is from syncstream, so we literally just started the thing up
22:04 timotimo two times in a row i got that assertion
22:05 geekosaur either something is not catching an error return (likely inside of libuv since it should fail if explicitly given a negative fd), or there is memory corruption
22:32 pyrimidine joined #moarvm
22:38 pyrimidi_ joined #moarvm
22:44 timotimo annoyingly i can't print the moar-level stacktrace, because it gets a signal while trying to dump the backtrace
22:44 geekosaur I think you just selected the "or" there...
22:45 timotimo :)

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary