Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-10-11

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:11 benchable6 joined #moarvm
00:18 rba joined #moarvm
00:23 travis-ci joined #moarvm
00:23 travis-ci MoarVM build failed. Jonathan Worthington 'Ensure ->caller/->static_info can't get outdated
00:23 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/286136655 https://github.com/MoarVM/MoarVM/compare/29250d60f146...dc9a338b2369
00:23 travis-ci left #moarvm
00:31 Zoffix just the bump syncage issue
00:31 rba_ joined #moarvm
01:55 ilbot3 joined #moarvm
01:55 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
02:08 travis-ci joined #moarvm
02:08 travis-ci MoarVM build failed. Zoffix Znet 'Merge pull request #726 from jsimonet/patch-1
02:08 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/286204762 https://github.com/MoarVM/MoarVM/compare/dc9a338b2369...599eb46642e9
02:08 travis-ci left #moarvm
03:52 evalable6 joined #moarvm
04:31 ugexe has async io been left out because of tuits or design? the libuv plumbing seems easy enough, but api wise doesn't seem like it would be as simple as syncsocket/asyncsocket split
05:26 Geth ¦ MoarVM: jsimonet++ created pull request #727: typo
05:26 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/727
05:52 timotimo well, it was certainly night …
05:52 yoleaux 02:09Z <MasterDuke> timotimo: i've got a question about multis and find_best_dispatchee here https://irclog.perlgeek.de/perl6-dev/2017-10-10#i_15286003, with some relevant info here https://irclog.perlgeek.de/perl6-dev/2017-10-11#i_15286163
05:54 timotimo MasterDuke: i have no intuition about find_best_dispatchee :(
05:55 domidumont joined #moarvm
06:00 domidumont joined #moarvm
06:04 domidumont joined #moarvm
06:15 brrt joined #moarvm
06:15 brrt good * #moarvm
06:15 timotimo hey brrt
06:16 timotimo top of the morning to you
06:16 brrt yes, you too
06:17 timotimo i just investigated a fun bug, where the Proxy returned by a SetHash's assignment gets returned from a lock.protect block and sunk outside of it, causing multiple threads to concurrently look at a given key and segfault
06:17 timotimo concurrently with the given key being deleted
06:18 brrt that's
06:18 brrt nice
06:18 brrt for the record, i still blame the Container abstraction for 90% of what's wrong with perl6
06:19 timotimo really? containers are really powerful, though
06:19 timotimo or would you rather we had implemented explicit references instead?
06:20 brrt i dunno. maybe
06:21 brrt yes
06:21 brrt something being powerful is never by itself a proper reason to support it
06:21 brrt random gotos to memory locations are also powerfull
06:21 brrt *powerful
06:23 patrickz joined #moarvm
06:23 brrt oh, timotimo++ on the performance analysis tooling grang
06:23 brrt *grant
06:23 timotimo thanks
06:36 timotimo how cool is it that uthash already has a size parameter ready for all its uthash_free calls
06:37 brrt i don't know how cool that is
06:41 timotimo it's rather cool if you want to replace its use of malloc with fixed size allocator calls
06:41 brrt i see
06:42 brrt is the FSA multi-threaded
06:43 timotimo yep
06:43 timotimo that's not the important part, though
06:44 timotimo we use the "free at safepoint" mechanism to make that if other threads were looking at the hash at the same time they still have memory they're allowed to do whatever they want with
06:44 timotimo the fsa takes "size, pointer", uthash has it "pointer, size" %)
06:47 timotimo i have no good reason to work on this, i should get started on my grant instead %)
06:49 brrt i should be working on so many things
06:51 timotimo hey look every perl6 run now gives a big "free: invalid pointer" right away
06:54 brrt progress?
06:58 timotimo no more crashes
06:59 timotimo the crash in the example program is now no longer reliable
06:59 timotimo oh, no, it totally still is
07:00 timotimo well, all of the work i've done right now will be required when somebody wants to fix this for real
07:00 timotimo might as well push it upstream
07:01 timotimo er, into the git repo i mean
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: 0908c9a4c7 | (Timo Paulssen)++ | 8 files
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: use the FSA for MVMHash and friends
07:07 Geth ¦ MoarVM/mvmhash_use_fsa:
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: it's supposed to make concurrent hash accesses not as crashy
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: by using the "free at safepoint" mechanism, but more changes
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: are needed to make it any safer than it was before.
07:07 Geth ¦ MoarVM/mvmhash_use_fsa:
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: This commit creates a bunch of versions of the HASH_ functions
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: that end in _FSA and take a tc argument and then use the
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: fixed sized versions of alloc and free.
07:07 Geth ¦ MoarVM/mvmhash_use_fsa: review: https://github.com/MoarVM/MoarVM/commit/0908c9a4c7
07:26 brrt joined #moarvm
07:29 brrt sounds like progress yes
07:30 timotimo oh, only now do i notice that in my grant proposal there were … that got turned into …
07:30 timotimo good job, movable type
07:35 brrt i don't think i've read your grant proposal yet
07:35 timotimo http://news.perlfoundation.org/2017/09/grant-proposal-rakudo-perl-6-p.html - that's the url
07:36 timotimo "â—‹ GC explorer" - the hell is this?
07:36 timotimo dang, the indentation also got messed up a little
07:43 jsimonet joined #moarvm
07:53 timotimo wow, perl6 -e 'say "hi"' is really consistent when run under callgrind
07:53 timotimo that's nice to see
07:53 timotimo (only with spesh disabled, of course)
07:54 timotimo 445,654,044 Ir with hashes using malloc/free, 445,301,169 Ir with hashes using the FSA
07:55 timotimo doesn't blow me away, but could be a nice saving perhaps?
07:57 timotimo 69097 average maxresidentk (range 68832..69420) with fsa
07:58 timotimo 69152 average maxresidentk (range 68904..69364) with malloc/free
07:58 timotimo ran it 9 times each
08:09 timotimo looking at the file i sent over to [Coke] for the grant application i can see 1) it's my fault the … got messed up, 2) it's not my fault the indentation levels got messed up
08:10 zakharyas joined #moarvm
09:00 robertle joined #moarvm
09:45 timotimo i can hardly believe our ref-indexes and ref-tos arrays need to hold 64bit ints, since they are indices to other arrays and if we have more than can be addressed with 32bit we're potentially screwed :)
10:00 Geth ¦ MoarVM: ed7831dee7 | (Julien Simonet)++ (committed using GitHub Web editor) | docs/jit/overview.org
10:00 Geth ¦ MoarVM: typo
10:00 Geth ¦ MoarVM:
10:00 Geth ¦ MoarVM: This -> ""
10:00 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ed7831dee7
10:00 Geth ¦ MoarVM: b2ea17d7d8 | (Zoffix Znet)++ (committed using GitHub Web editor) | docs/jit/overview.org
10:00 Geth ¦ MoarVM: Merge pull request #727 from jsimonet/patch-2
10:00 Geth ¦ MoarVM:
10:00 Geth ¦ MoarVM: typo
10:00 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b2ea17d7d8
10:03 rba joined #moarvm
10:27 brrt joined #moarvm
10:28 timotimo i think the heap analyzer could really benefit from splice on same-typed native arrays being faster
10:29 leont joined #moarvm
10:32 timotimo or i should forget about splitting the work there and then not have to splice data
10:33 timotimo and for some reason memory usage keeps growing even though it *should* only be working inside super fast C code
10:34 harrow joined #moarvm
10:39 timotimo oh!
10:40 timotimo it's not using nqp::splice, it's doing a straight-up loop with push and pop
10:42 timotimo i thought .splice was a kick-ass optimization, but it turns out .append is much directer
10:43 timotimo about 800 megs maxresidentk now
10:49 timotimo the before-measurement is not going so well ... 8.5 gigs already
10:49 timotimo so yeah, i know who the winner is
10:50 timotimo huh, i wonder if the *@ in the signature for splice is problematic here
10:52 timotimo 12 gigs, wow
11:15 timotimo MFW OO::Monitors dies with an "invalid BUILDALL plan"
11:16 timotimo installing it, that is
11:16 timotimo oh, no, it was installed - perhaps with an older version - and testing a cro component broke
11:34 timotimo when i'm splicing a few million elements from one array into another ... maybe it should do a gc sync point every now and then %)
11:50 timotimo dang, it's not using the impl i wrote for my native int arrays ..
11:58 jnthn We should try and memcpy in splice when we determine that we can do so
11:58 yoleaux 02:09Z <MasterDuke> jnthn: i've got a question about multis and find_best_dispatchee here https://irclog.perlgeek.de/perl6-dev/2017-10-10#i_15286003, with some relevant info here https://irclog.perlgeek.de/perl6-dev/2017-10-11#i_15286163
11:58 timotimo yup, i just implemented that
11:58 jnthn ah, nice
11:59 timotimo what worries me is that this doesn't even go into splice at all:
11:59 timotimo m: my int @a = 1, 2, 3; my int @b = 9, 9, 9; @a.append(@b)
11:59 camelia rakudo-moar a72214: ( no output )
11:59 timotimo m: my int @a = 1, 2, 3; my int @b = 9, 9, 9; @a.append(@b); say @a
11:59 camelia rakudo-moar a72214: OUTPUT: «1 2 3 9 9 9␤»
11:59 jnthn Maybe .append isn't yet implemented in terms of splice?
11:59 timotimo there's a candidate for intarray that would accept a intarray @b in there
11:59 timotimo that would use splice
11:59 timotimo it just doesn't get taken
11:59 jnthn intarray @b would be an array of intarry
12:00 timotimo oh, yes
12:00 timotimo it's intarray:D: int @other
12:00 timotimo i blame the *@ and **@ candidates i guess?
12:00 jnthn Maybe but slurpy should be less specific
12:02 timotimo s: (my int @), "append", \(my int @)
12:02 SourceBaby timotimo, Sauce is at https://github.com/rakudo/rakudo/blob/a72214c4f/src/core/native_array.pm#L427
12:02 timotimo that is indeed the one that runs nqp::splice ... ?!
12:04 timotimo hom. now it does use it. i wonder how i measured it wrong before?
12:09 timotimo ah, of course
12:09 timotimo m: multi doit(int @foo) { say "native int array" }; multi doit(@other) { say "other" }; doit(my int @); doit(my int32 @)
12:09 camelia rakudo-moar a72214: OUTPUT: «native int array␤other␤»
12:10 timotimo it's because it's not an int64 one but a int32 one
12:12 timotimo hm, so ... i could also implement a memcpy-based version for object arrays. i'll just have to go through the array another time to make sure everything's barriered correctly
12:12 timotimo but if the array itself is in the nursery still, i don't have to barrier, right?
12:13 timotimo Ambiguous call to 'append'; these signatures all match:
12:13 timotimo :(array::intarray:D $: array::intarray:D $values, *%_)
12:13 timotimo :(array::intarray:D $: @values, *%_)
12:13 timotimo that's not a fix :|
12:15 * timotimo tries out "is default"
12:16 timotimo hm, not yet available at that point, it seems
12:17 timotimo or there's some other reason it explodes with "Cannot invoke this object (REPR: Null; VMNull)"
12:19 Zoffix timotimo: if the file you're editing above this line, then yes, that'd be the reason. https://github.com/rakudo/rakudo/blob/nom/tools/build/moar_core_sources#L147
12:20 Zoffix (`is default` uses `does` op from that file)
12:20 timotimo ooooh!
12:20 timotimo yes, that would explain it
12:22 timotimo adding one candidate for each int kind we have ... not so great, but works
12:22 Zoffix m: multi x(Int) { say "here" }; multi x(Int) { say "there" }; BEGIN &x.candidates[1].^mixin: role { method default(--> True) { } }; x 42
12:22 camelia rakudo-moar a72214: OUTPUT: «there␤»
12:23 Zoffix m: multi x(Int) { say "here" }; multi x(Int) { say "there" }; BEGIN &x.candidates[0].^mixin: role { method default(--> True) { } }; x 42
12:23 camelia rakudo-moar a72214: OUTPUT: «here␤»
12:23 Zoffix :)
12:27 timotimo cool. doing a whole bunch of memcpys now where the arrays are object arrays
12:29 timotimo perl6 -e '' does it 49 times for a total of 215 elements
12:29 timotimo ah, it's not only about generational barriers, it's also about SC write barriers
12:29 timotimo so this code would crash and burn if that has to happen but doesn't
12:34 timotimo ah, ASSIGN_REF is really only about gen2 stuff, and if the root object is not in gen2 it's completely bypassed
12:44 zakharyas joined #moarvm
12:53 timotimo 403,273,321 vs 403,285,477 is what difference memcpy for object vmarrays brings for perl6 -e ''
12:54 timotimo hardly something to write home about, but it's nice to know it impacts it at least a tiny bit
13:05 Geth ¦ MoarVM: b9d3f6da34 | (Timo Paulssen)++ | src/6model/reprs/VMArray.c
13:05 Geth ¦ MoarVM: a fast path for VMArray splice using memcpy
13:05 Geth ¦ MoarVM:
13:05 Geth ¦ MoarVM: happens when both vmarrays are of the same slot type.
13:05 Geth ¦ MoarVM: also works with objects, but only if the array isn't
13:05 Geth ¦ MoarVM: in the old generation, because otherwise we'd have to
13:05 Geth ¦ MoarVM: go through all pointers and write-barrier them.
13:05 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b9d3f6da34
13:06 Geth joined #moarvm
13:06 timotimo bbl
13:26 stmuk_ joined #moarvm
13:44 rba joined #moarvm
13:56 timotimo mhh, had some noms
14:17 dogbert2 joined #moarvm
14:48 jnthn No time to read this at the moment but looks owrth a look: http://soft-dev.org/pubs/html/barrett_bolz-tereick_killick_mount_tratt__virtual_machine_warmup_blows_hot_and_cold_v6/
15:02 brrt joined #moarvm
15:20 leont joined #moarvm
15:50 ugexe jnthn: is async filesystem io not in moarvm because its not a high priority, or because it wouldn't be useful enough to warrant the complexity?
16:04 jnthn ugexe: A bit of both and also a question of whether there's a win in doing it in the VM (more)
16:04 zakharyas joined #moarvm
16:06 jnthn ugexe: In terms of priority: async sockets are really useful 'cus you tend to be juggling dozens of sockets and latencies are quite high, and async procs because you can't sanely collect stdout and stderr without some kind of async. File access has a bunch less latency.
16:06 moritz ... unless you deal with slow NFS shares :/
16:07 jnthn Well, there is that
16:07 jnthn Beyond that, I recall reading that libuv actually uses a thread pool behind the scenes to manage outstanding file I/O requests
16:08 jnthn I don't know if that means that it just has threads doing "normal" blocking I/O, but if that's all it's doing, then we have a perfectly fine threadpool at Perl 6 level
16:08 jnthn So stick a `start` before your I/O code and all's well
16:11 jnthn I guess the other piece in this is that sockets and procs more naturally lend themselves to async handling, in that you want to react when something arrives. Files are usually processed as fast as we can grab the data and do stuff with it.
16:11 ggoebel joined #moarvm
16:13 jnthn Yeah, here's the quote I was thinking of, it's top of the doc: "The libuv filesystem operations are different from socket operations. Socket operations use the non-blocking operations provided by the operating system. Filesystem operations use blocking functions internally, but invoke these functions in a thread pool and notify watchers registered with the event loop when application interaction is required."
16:13 jnthn from https://nikhilm.github.io/uvbook/filesystem.html
16:14 jnthn So that suggests there's no win from more complexity in the VM
16:14 jnthn When we can do the same just fine at Perl 6 level
16:14 ugexe i wonder if that is due to difficulty in providing it cross platform, since I thought windows has decent async file system IO
16:17 jnthn Maybe, dunno
16:17 jnthn Maybe it's also a case of "possible but nobody wanted it enough"
16:18 jnthn Are you asking out of curoisity, or out of having a use case? :)
16:20 ugexe curiosity mostly. having been digging around libuv IO stuff I started wondering how useful async mkdir could possibly be
16:21 timotimo jnthn: cro is *really* unhappy with me using start it looks like
16:22 jnthn ugexe: Probably more useful when you're in Node.js and have a single thread
16:22 jnthn timotimo: Huh?
16:22 jnthn timotimo: It already runs every request handler inside of a start
16:23 timotimo should i be seeing output if i put a "start { say 'hi' }" after the $http.start (in the single page app example)
16:23 jnthn I'd expect so
16:24 timotimo it's unreliable at best
16:24 jnthn Are you running it under `cro run`?
16:24 timotimo i am
16:24 jnthn 'cus we don't have a way to allocate a ptty yet
16:24 jnthn So output gets held back by buffering
16:24 jnthn 'cus the Proc::Async that runs it has an output pipe
16:24 timotimo oh, that's a good hint
16:25 jnthn I've been wondering what to do about that
16:25 timotimo got an idea yet for "cro trace is nice, but not if you have bundled javascript to serve"? :)
16:25 [Coke] (ptty) hey, did you hear about the weird terminal state I got in using proc::async to run ssh -t? :)
16:27 jnthn No
16:27 jnthn timotimo: I was just going to truncate it after a certain number of bytes :)
16:27 timotimo that should work well
16:27 jnthn timotimo: It's an easy patch, I'd think
16:27 jnthn [Coke]: fwiw, there's an SSH::LibSSH module :)
16:28 zakharyas joined #moarvm
16:36 rba_ joined #moarvm
16:39 robertle joined #moarvm
16:46 rba joined #moarvm
17:03 zakharyas joined #moarvm
17:17 domidumont joined #moarvm
18:35 evalable6 joined #moarvm
18:37 Ven joined #moarvm
19:15 evalable6 joined #moarvm
20:22 stmuk joined #moarvm
20:57 stmuk_ joined #moarvm
22:11 domidumont joined #moarvm
22:36 leont joined #moarvm
22:54 leont joined #moarvm
23:25 leont joined #moarvm
23:56 evalable6 joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary