Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-11-02

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:25 FROGGS_ joined #moarvm
04:01 synopsebot6 joined #moarvm
04:01 synopsebot6 joined #moarvm
04:01 synopsebot6 joined #moarvm
06:51 synopsebot6 joined #moarvm
06:55 FROGGS[webchat] joined #moarvm
06:56 FROGGS[webchat] all lights are green for debian! \o/ https://buildd.debian.org/status/package.php?p=moarvm \o/ https://buildd.debian.org/status/package.php?p=rakudo \o/
06:59 moritz \o/
07:17 synopsebot6 joined #moarvm
07:18 domidumont joined #moarvm
07:23 domidumont joined #moarvm
07:51 domidumont joined #moarvm
08:33 zakharyas joined #moarvm
08:34 FROGGS[webchat] left #moarvm
10:16 jnthn moarning! :)
10:17 nwc10 good *, jnthn
10:18 jnthn Great work on the Debian builds. Nice to see the list of places we run :)
10:19 nwc10 it's not easy being green. (well done)
10:29 domidumont thanks ! :-)
11:27 dalek MoarVM: 5c65917 | jnthn++ | src/core/nativecall_dyncall.c:
11:27 dalek MoarVM: Mark a thread GC-blocked while its in native code.
11:27 dalek MoarVM:
11:27 dalek MoarVM: This means that setting off some long-running C code on one thread
11:27 dalek MoarVM: will not cause other threads to be unable to perform GC.
11:27 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/5c659178a0
11:27 nwc10 jnthn: is there TFM I should read on what "GH blocked" means?
11:28 nwc10 I'm assuming that when a thread is GC blocked, the others don't wait for it when doing GC
11:28 nwc10 but who checks what objects owned by it are pointing to?
11:29 jnthn When a thread is "GC blocked", if GC happens, another thread will steal its collection work.
11:30 jnthn And if the thread unblocks while that is happening, it will wait until the GC is over before continuing onwards.
11:30 nwc10 basically it's sort of an anti-Local-Interpreter-Lock
11:30 nwc10 ie "this MoarVM's state isn't going to be touched"
11:31 nwc10 by default, every thread "locks" its state and only it gets to read it
11:31 jnthn Yes, when you enter blocked state you are promising that you're not going to do stuff like allocate
11:31 nwc10 but when going AWOL, a thread releases that lock
11:31 jnthn *allocation
11:31 nwc10 (and re-acquires it when it returns from elsehwere)
11:31 nwc10 s/read/write/ # earlier
11:31 jnthn Right. We already use this technique for things like I/O
11:32 jnthn And sleep
11:32 nwc10 :-) in that
11:32 jnthn So that one thread sleeping or blocking on I/O won't prevent others from being able to collect and get on with stuff
11:32 nwc10 with green threads there is only one OS thread, and you release your lock for the duration of doing something "slow"
11:32 domidumont joined #moarvm
11:33 nwc10 here effectively every MoarVM thread then has a green threading-alike on it
11:33 nwc10 which is held either by the thread itself (when running MoarVM code)
11:33 nwc10 or release, so that someone else can do its housework
11:33 nwc10 released, so that someoen else can hold it as needed, when housework is needed
11:34 jnthn Yup
11:34 nwc10 Good.
11:34 jnthn That's a good way of seeing it
11:34 * jnthn just did a build with libffi so he can apply the same change there
11:34 nwc10 I'd never thought of this before. The design seems elegant and simple
11:35 jnthn Might also be interseting to know that the "am I blocked" flag is also unified with the GC interupt
11:36 jnthn https://github.com/MoarVM/MoarVM/blob/master/src/core/threadcontext.h#L1 has some documentation around this
11:36 nwc10 thanks - will read after lunch
11:46 dalek MoarVM: f769569 | jnthn++ | src/core/nativecall_libffi.c:
11:46 dalek MoarVM: Add nativecall GC block/unblock for libffi too.
11:46 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f769569b78
12:05 diakopter timotimo: here's a new fuzzing tool for you: https://blog.trailofbits.com/2016/11/02/shin-grr-make-fuzzing-fast-again/
12:21 stmuk joined #moarvm
12:48 timotimo jnthn: does that properly unblock a thread when entering a callback? and re-block when returning from a callback into the C code "in between" interpreters?
12:48 timotimo i'm not sure how it interacts with the sub-interpreter stuff
12:51 timotimo ah, i see the handling of callbacks exists
12:51 timotimo it's mentioned explicitly in the bump commit
13:23 jnthn Yes :)
14:24 pyrimidine joined #moarvm
14:40 dalek MoarVM: 2699370 | jnthn++ | src/ (3 files):
14:40 dalek MoarVM: Remove dead code.
14:40 dalek MoarVM:
14:40 dalek MoarVM: Firstly, we never modified num_user_threads, so it was always zero.
14:40 dalek MoarVM: Secondly, the one thing that tried to use it would just hang in a
14:40 dalek MoarVM: place it should not if we actually did it that way.
14:40 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/2699370e8e
14:40 dalek MoarVM: 4b9e3a8 | jnthn++ | src/6model/reprs/ReentrantMutex.c:
14:40 dalek MoarVM: Fix spello; diakopter++.
14:40 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/4b9e3a8aa9
14:41 diakopter num_user_threads did something someday ago
14:41 diakopter hard to recall what
14:42 jnthn Yeah
14:43 jnthn Fixing global destruction up proper is better done without it anyways
16:38 synopsebot6 joined #moarvm
16:39 ggoebel joined #moarvm
16:40 timotimo jnthn: do you have an idea where about ten thousand strings that are all stringified numbers may come from in the string heap? :)
16:44 jnthn cuids
16:44 jnthn They used to be much longer :)
16:47 timotimo oh, duh
16:47 timotimo i wonder if it'd be very hard to turn them into ints now that they are actually just ints anyway?
16:48 jnthn Well, it's an interface change
16:48 jnthn And bytecode format chance
16:48 jnthn *change
16:48 jnthn Well, not if we atoi them I guess :)
16:48 timotimo they take about 8 byte each :)
16:48 jnthn Uh, more than that surely
16:49 timotimo at least the 4-digit ones
16:49 jnthn And MVMString is bigger than 8 bytes before the storage?
16:49 jnthn *An
16:49 timotimo just in the string heap, i mean
16:50 timotimo i wonder why each string in the string heap is accompanied by four bytes of size info rather than a varint
16:50 jnthn OH, right
16:51 nwc10 $because ?
16:52 timotimo if it's just $because and not @because, maybe a work-around can be found ;)
16:52 nwc10 be careful that you don't break the look-ahead code in the object lazy deserialisation if you change stuff
16:52 jnthn Since this is not in the serialization blob, the risk of that should be quite low
16:52 nwc10 ah OK
16:53 nwc10 superstituous bugs might have figured out quantum tunneling :-)
16:57 timotimo BBIAB, and then maybe i'll turn the string lengths into varints
17:32 FROGGS joined #moarvm
17:35 FROGGS o/
17:36 nwc10 \p
17:36 nwc10 er, ooops
17:36 nwc10 \o
17:36 nwc10 that's better
17:36 FROGGS it is :o)
20:12 timotimo dinner time \o/
20:41 FROGGS nebuchadnezzar: this bug is closable now, right? https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=841344
20:42 nwc10 jnthn: http://paste.scsys.co.uk/538942 -- t/04-nativecall/20-concurrent.t can fail - known?
20:42 nwc10 ASAN does not have an opinion about this
20:44 FROGGS ewww
20:45 FROGGS I bet this does not happen on windows...
20:46 * FROGGS 's experience with concurrency is that it is usually rock solid on windows, less so on linux, and even more less so on bsd
21:10 timotimo MVMuint32 bytes = read_uint32(cur_pos) >> 1;
21:10 timotimo cur_pos += 4 + bytes + (bytes & 3 ? 4 - (bytes & 3) : 0);
21:10 timotimo ^- is this something like "string lengths are always multiples of 2" or something?
21:12 timotimo or something for the fast table somehow?
21:23 jnthn timotimo: I think we pad so that the read of the uint32 is aligned
21:24 jnthn nwc10: I couldn't make it fail at all locally
21:24 timotimo ah so with a varint that's no longer an issue
21:24 jnthn FROGGS: I actually devloped that test and the stuff it fixes on Windows :)
21:24 jnthn oops
21:24 jnthn *on Linux
21:24 timotimo i didn't yet see the code that *writes* the string heap :\
21:25 jnthn nwc10: Thanks for the example of how it fails though. Travis didn't offer up so much info.
21:26 jnthn I'll have to hammer on the test a bit more tomorrow
21:40 timotimo oh! we have a bit of the size be "decode utf8 or not"
21:51 timotimo seriously, where is that code hiding o_O
21:55 jnthn To assemble the string heap?
21:55 jnthn src/mast/compiler.c or so
21:56 timotimo ah, i see it now
21:56 timotimo you saved me from madness
22:08 jnthn :)
23:25 timotimo i have to fake up MVMSerializationWriters left, right, and center to make all this work >_>
23:43 timotimo having the fake SerializationReader on the stack won't work when reading fails, because then it'll try to MVM_free that address when something goes wrong

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary