Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2015-12-02

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 ilmari src/platform/threads.h:5:35: warning: implicit declaration of function ‘pthread_yield’
00:17 ShimmerFairy joined #moarvm
02:24 FROGGS_ joined #moarvm
03:03 konobi https://blog.8thlight.com/colin-jones/2015/12/01/ask-dtrace-why-are-my-tests-so-slow.html
03:03 konobi "Ask DTrace: Why are my tests so slow?"
03:37 vendethiel joined #moarvm
07:13 kjs_ joined #moarvm
07:34 domidumont joined #moarvm
07:40 domidumont joined #moarvm
07:57 FROGGS joined #moarvm
07:57 kjs_ joined #moarvm
08:02 nwc10 good *, #moarvm
08:02 JimmyZ \o/ nwc10
08:22 FROGGS o/
08:30 jnthn ilmari: Feel free to send a PR for it, if you have time.
08:35 zakharyas joined #moarvm
08:46 jnthn teaching &
09:18 * nwc10 spots in https://blog.8thlight.com/colin-jones/2015/12/01/ask-dtrace-why-are-my-tests-so-slow.html -- And our $LOAD_PATH, like those in many other Rails projects, is pretty big: ... So every time whatever-it-is looks for these shared library files, it might have to look in 236 * 6 = 1416 places before finding the file it's looking for.
09:23 lizmat joined #moarvm
09:25 kjs_ joined #moarvm
10:07 domidumont joined #moarvm
10:27 japhb nwc10: Yeah, there are a fair number of lessons in there.  For one, "Brute force looks good enough until you multiply it by a large number."  For another "Locking your patch levels is not a panacea -- now you have to manually decide which future patches you need to pull."
10:38 kjs_ joined #moarvm
10:41 zakharyas joined #moarvm
12:40 rjbs Why don't Ruby projects that use this strategy build merged link trees so avoid combinatorial explosions?
12:40 nwc10 because they don't think like sysadmins?
13:05 kjs_ joined #moarvm
13:07 kjs_ joined #moarvm
13:20 llfourn .tell timotimo you are probably asleep by now. Just poking you for benchmarks :D. No rush.
13:21 llfourn sorry wrong chan :(
13:24 rjbs nwc10: So if I write the code but give it a punchy name, maybe I can make my fame and fortune? :)
13:25 nwc10 you'd need a logo too, and a pastel coloured website
13:33 domidumont1 joined #moarvm
15:09 domidumont joined #moarvm
16:12 zakharyas joined #moarvm
16:12 harrow joined #moarvm
16:12 JimmyZ joined #moarvm
16:12 flussence joined #moarvm
16:12 BinGOs joined #moarvm
16:12 dalek joined #moarvm
16:12 timotimo joined #moarvm
16:12 avar joined #moarvm
16:12 leedo joined #moarvm
16:12 tadzik joined #moarvm
16:12 nwc10 joined #moarvm
16:12 ShimmerFairy joined #moarvm
16:12 colomon joined #moarvm
16:12 llfourn joined #moarvm
16:12 flaviusb joined #moarvm
16:12 ggoebel7 joined #moarvm
16:12 synbot6 joined #moarvm
16:12 diakopter joined #moarvm
16:12 pyrimidine joined #moarvm
16:12 nebuchadnezzar joined #moarvm
16:12 Util_ joined #moarvm
16:12 ashleyde1 joined #moarvm
16:12 moritz joined #moarvm
16:12 arnsholt joined #moarvm
16:12 ilmari joined #moarvm
16:12 domidumont joined #moarvm
16:28 rjbs joined #moarvm
16:28 hoelzro joined #moarvm
16:28 japhb joined #moarvm
16:28 [Coke] joined #moarvm
16:28 mtj_ joined #moarvm
16:28 bonsaikitten joined #moarvm
16:28 TimToady joined #moarvm
16:28 camelia joined #moarvm
16:28 btyler joined #moarvm
16:28 psch joined #moarvm
16:28 konobi joined #moarvm
16:47 zakharyas joined #moarvm
16:47 harrow joined #moarvm
16:47 JimmyZ joined #moarvm
16:47 flussence joined #moarvm
16:47 BinGOs joined #moarvm
16:47 dalek joined #moarvm
16:47 timotimo joined #moarvm
16:47 avar joined #moarvm
16:47 leedo joined #moarvm
16:47 tadzik joined #moarvm
16:47 nwc10 joined #moarvm
16:47 domidumont joined #moarvm
16:47 ilmari joined #moarvm
16:47 arnsholt joined #moarvm
16:47 moritz joined #moarvm
16:47 ashleyde1 joined #moarvm
16:47 Util_ joined #moarvm
16:47 nebuchadnezzar joined #moarvm
16:47 pyrimidine joined #moarvm
16:47 diakopter joined #moarvm
16:47 synbot6 joined #moarvm
16:47 ggoebel7 joined #moarvm
16:47 flaviusb joined #moarvm
16:47 llfourn joined #moarvm
16:47 colomon joined #moarvm
16:47 ShimmerFairy joined #moarvm
16:50 camelia joined #moarvm
16:51 ilbot3 joined #moarvm
16:51 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
16:57 sivoais joined #moarvm
16:58 lizmat joined #moarvm
17:04 ChanServ joined #moarvm
17:05 hoelzro o/ #moarvm
17:31 timotimo yo ro
17:32 dalek MoarVM/spesh_nativeref_assign_decont: a83b3cd | timotimo++ | src/spesh/facts.c:
17:32 dalek MoarVM/spesh_nativeref_assign_decont: teach our guard op generator about NativeRef
17:32 dalek MoarVM/spesh_nativeref_assign_decont: review: https://github.com/MoarVM/MoarVM/commit/a83b3cdfd1
17:32 timotimo jnthn: ^- does that look okay-ish, semantically?
17:48 timotimo m: say "with this series of patches, doing 500M +1's on an rw lexical takes { 5.4 / 10.1 }x as much time as without the patches"
17:48 camelia rakudo-moar 704c58: OUTPUT«with this series of patches, doing 500M +1's on an rw lexical takes 0.534653x as much time as without the patches␤»
17:48 timotimo how does that sound?
17:50 jnthn timotimo: I think the patch is correct
17:52 timotimo sweet
17:53 timotimo i'm doing another measurement now to see the impact it has on ++
17:53 timotimo if any
17:53 diakopter timotimo++
17:54 TimToady well, we'd like ++ to be 100x faster or so...
17:54 timotimo yes, ideally
17:54 jnthn That'll need some rather involved patching of the inliner :)
17:54 japhb TimToady: prefix, postfix, or "yes"?
17:54 timotimo that'll get faster via in...yes
17:54 timotimo japhb: yes
17:54 TimToady prefix always, postfix in sink
17:55 jnthn But it's fraught enough to do right I'm kinda actively discouraging trying this side of 6.c
17:55 TimToady postfix in non-sink can be a bit slower
17:55 TimToady in that case we should AOT optimize ++a to a = a + 1
17:55 jnthn (Also 'cus I think doing it well probably has architectural implications for how spesh works overall)
17:56 TimToady after optimizing a++ to ++a in sink
17:56 japhb Does postfix:<++> convert to prefix:<++> in sink only for the CORE version?
17:59 timotimo turning $a++ into ++$a might not be worth as much as you think
17:59 timotimo as long as we don't inline, that's drowned out so strongly by the invocation overhead
17:59 TimToady it's more valuable currently for natives, not much difference for non-natives
18:00 timotimo aye
18:00 TimToady maybe it's already doing the opt for non-natives
18:00 TimToady haven't checked
18:00 timotimo it probably does inlining there
18:00 jnthn Yeah, it can do
18:01 jnthn Native refs are designed to often be optimized out totally, but it's just not done yet.
18:01 timotimo there's even more win to be had from turning lexical $n into local_n in sub do-loop(int $n is rw) { while $n < 100_000_000 { $n = $n + 1 } }
18:01 timotimo because right now we do two getlex in a row for that and both times do a guard against its type
18:02 timotimo that's pure overhead, as we bind the value of that lexical from argument 0 and never change it ever and it doesn't escape, either
18:02 timotimo but spesh doesn't know that, and the static optimizer in rakudo isn't smart enough yet
18:02 TimToady CSE is such a modern notion :P
18:02 timotimo :)
18:03 timotimo well, lexicals could change out of nowhere in some cases
18:03 domidumont1 joined #moarvm
18:03 timotimo so lexicals are still "HOT HOT DON'T TOUCH" in spesh's eyes
18:03 timotimo mostly, anyway
18:04 dalek MoarVM: c7706c4 | timotimo++ | / (6 files):
18:04 dalek MoarVM: A spesh op like sp_bind/get_i64 with an extra pointer deref
18:04 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c7706c4a6c
18:04 dalek MoarVM: de68134 | timotimo++ | src/ (2 files):
18:04 dalek MoarVM: Allow NativeRef to spesh assign_i and decont_i
18:04 dalek MoarVM:
18:04 dalek MoarVM: this uses the op the previous commit introduced.
18:04 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/de68134b2c
18:04 dalek MoarVM: 6a95b60 | timotimo++ | src/jit/ (2 files):
18:04 dalek MoarVM: implement sp_deref_bind/get_i64/n in the JIT
18:04 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6a95b605ea
18:04 dalek MoarVM: a83b3cd | timotimo++ | src/spesh/facts.c:
18:04 dalek MoarVM: teach our guard op generator about NativeRef
18:04 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/a83b3cdfd1
18:04 timotimo spec tests are quite noisy, but they already were without my patches
18:05 jnthn Wsa gonna say, with lexicals we have to be really careful on memory model issues
18:05 TimToady we defined parameters to be readonly by default partly with the intention that we could optimize low-level stuff to is-copy like C does
18:05 jnthn anyway, bbiab :)
18:05 timotimo this isn't about parameters, though
18:06 timotimo in this case the parameter comes in "via a local" that then gets bound into the lexical
18:08 timotimo have another itty bitty local patch that cuts down on the number of spesh slots by re-using spesh slots for these guards
18:08 diakopter surely someone could write an ircbot in p6 by Christmas to replace the functionality of (p5) dalek & (p5) camelia & (ruby) yoleaux
18:08 timotimo but that's for a bit later.
18:08 timotimo sauna time! \o/
18:11 TimToady diakopter: maybe you should do that for your advent post :)
18:23 diakopter TimToady: the current IRC module (Net::IRC::Bot) says this, which doesn't bode well for its current usefulness:
18:23 diakopter It's currently in active development, but it is mostly stable until perl 6 implements either threads or async I/O so it can work correctly as a bot (timed events and things that block the runtime are not possible yet.)
18:24 diakopter that's from the README which hasn't been updated in 3 years
18:27 [Coke] Yes, there are yaks to shave, which is why no one has written that bot yet.
18:31 TimToady .oO(Shave a yak, go to sheol.)
18:34 diakopter seems an advent slot opened for ... tomorrow
18:44 sivoais joined #moarvm
18:45 Peter_R joined #moarvm
19:03 vendethiel joined #moarvm
19:14 FROGGS joined #moarvm
19:20 sivoais joined #moarvm
19:35 domidumont joined #moarvm
19:52 brrt joined #moarvm
19:53 brrt good * #moarvm
19:53 timotimo hey brrt! :)
19:54 brrt hey timotimo
19:54 timotimo did you see i was able to get $foo = $foo + 1 to become faster! <3
19:54 timotimo 184519   camelia │ rakudo-moar 704c58: OUTPUT«with this series of patches, doing 500M +1's on an rw lexical takes
19:54 timotimo │ 0.534653x as much time as without the patches␤»
20:00 brrt oh, very good!
20:00 brrt i did not see
20:01 timotimo \o/
20:01 brrt i'd complain about your barbaric indentation, but that would serve noone :-)
20:02 timotimo in my patch or in the thing i pasted?
20:03 brrt patch, but it doesn't matter
20:03 brrt cool
20:03 brrt very cool
20:03 brrt timotimo++
20:03 timotimo oh no what did i do
20:03 timotimo oh, perhaps the multiline if?
20:04 timotimo i haven't found a good solution to how to do these :(
20:04 brrt hmm, i think emacs indents them directly to the parentheses of the if
20:04 brrt but like i said, doesn't matter :-)
20:06 timotimo yeah, but then won't it be the exact same position as the code in the ifs code block?
20:09 brrt hmm, yes
20:09 brrt good point
20:09 timotimo that's why i put the extra 4 spaces
20:14 zakharyas joined #moarvm
20:21 dalek MoarVM/even-moar-jit: b15b90a | brrt++ | / (7 files):
20:21 dalek MoarVM/even-moar-jit: Allow tiler table generator to specify args
20:21 dalek MoarVM/even-moar-jit:
20:21 dalek MoarVM/even-moar-jit: This is actually a complete refactor of the tiler table generator
20:21 dalek MoarVM/even-moar-jit: to support arguments in tiles, to not use a plethora of global
20:21 dalek MoarVM/even-moar-jit: variables, and to compute the optimum rule given a ruleset
20:21 dalek MoarVM/even-moar-jit: more completely. (Which is unfinished, but I think I know how to
20:21 dalek MoarVM/even-moar-jit: complete it). As a result, small changes throughout the tiler and
20:21 dalek MoarVM/even-moar-jit: tile implementations were necessary. eq/lt/gt and friends have also
20:21 dalek MoarVM/even-moar-jit: been reduced to a single cmp tile implementation.
20:21 dalek MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/b15b90a7d1
20:22 brrt anyway, as far as i know that commit unblocks the work on the tiler linearisation
20:22 timotimo neato!
20:22 timotimo glad to hear you're coming along :)
20:22 brrt and, importantly, that commit still works with nqp :-)
20:22 jnthn brrt++ # perserverance :)
20:22 timotimo nice!
20:23 jnthn uh, I'm sure I spelt that wrong :)
20:23 brrt :-)
20:23 timotimo it doesn't look wrong to me
20:23 timotimo better to have perseverance than to have preserverance
20:23 timotimo or having to preserve rants
20:23 brrt or to have to increase your font because these words all look too much alike
20:23 brrt i'll be off now :-)
20:23 * moritz has got to preserve france
20:24 jnthn Have a nice evening o/
20:24 moritz or so :-)
20:24 brrt you too o/
20:44 dalek joined #moarvm
20:45 * [Coke] perseverates.
21:05 leont joined #moarvm
21:30 timotimo jnthn: do you think you can spot something wrong with the no_atomic_if_single_threaded branch? it's just a single commit and it makes multi-threaded tests in the spec test suite livelock
21:34 colomon joined #moarvm
21:40 timotimo using  "my int $n = 0; while $n < 50_000_000 { $n++ }"  for a benchmark, it goes from 21.1s to 19.4s with the patch
21:41 timotimo m: say "not using atomics for frame refcounts gets us to { 19.4 / 21.1 }x as much time spent"
21:41 camelia rakudo-moar 704c58: OUTPUT«not using atomics for frame refcounts gets us to 0.919431x as much time spent␤»
21:41 timotimo 8% time improvement; not the biggest win, but can surely be felt
21:43 timotimo i wonder if the improvement gets bigger if deeper call stacks are involved
21:44 timotimo as dereferencing a frame derefs the caller pointer, too. and that can recurse
21:45 domidumont joined #moarvm
21:48 jnthn timotimo: There's already a function for checking if there are threads, fwiw
21:48 jnthn It'd be better to use that
21:48 timotimo oh, can do!
21:48 jnthn fixedsizealloc.c calls it in the free at safepoint function, for example
21:50 Peter_R joined #moarvm
21:53 timotimo ah, i see it
22:03 timotimo i'm not sure why, but that seems to indeed fix the problem
22:03 timotimo damn, that's sweet
22:04 jnthn :)
22:04 timotimo have permission to merge? and someone a bit cleverer can look at MVM_frame_acquire_ref getting a no-atomic version, too
22:05 jnthn Can you push an updated patch for me to glance over?
22:05 timotimo mind if i force-push, or just add on top of it?
22:06 lizmat joined #moarvm
22:06 jnthn force is fine
22:08 timotimo urgh, i have a merge situation going on locally because too many changes in working copy %)
22:10 dalek MoarVM/no_atomic_if_single_threaded: c65ea1b | timotimo++ | src/core/frame.c:
22:10 dalek MoarVM/no_atomic_if_single_threaded: cheaper access to ref counts if only 1 thread exists.
22:10 dalek MoarVM/no_atomic_if_single_threaded:
22:10 dalek MoarVM/no_atomic_if_single_threaded: MVM_frame_acquire_ref also ought to be enhanced like this
22:10 dalek MoarVM/no_atomic_if_single_threaded: review: https://github.com/MoarVM/MoarVM/commit/c65ea1b0d2
22:11 timotimo that's rebased onto master and melded the old commit and the MVM_has_user_thread function change into one commit
22:11 jnthn timotimo: Holy code dupe...
22:11 jnthn I think that
22:12 jnthn -    while (MVM_decr(&frame->ref_count) == 1) {
22:12 jnthn Wants to be replaced with while (some_static_fnction(tc, frame) == 1) {
22:13 timotimo oh, that'd be kind of better
22:13 jnthn Where that static function does the if
22:13 timotimo MVM_STATIC_INLINE?
22:13 jnthn If you mark it MVM_INLINE_STATIC then it'll get inlined
22:13 timotimo that's the one
22:13 timotimo actually, mine's the one
22:14 timotimo hmm. freeing something at a safe point ... isn't that only relevant if we have multiple threads?
22:15 jnthn yes, thus why if we're in single threaded mode we free it right away
22:16 timotimo ah, indeed!
22:16 timotimo very good
22:17 * leont spend much of the past week dealing with refcounting issues (in a different project). Ran into some fun issues where the reference was thread safe, but the referee wasn't :-/.
22:17 timotimo ugh
22:33 diakopter well you need the MVM_decr in there *somewhere* (in the slow path)
22:33 colomon joined #moarvm
22:33 diakopter I don't know why you'd take it out entirely
22:34 diakopter oh, I see what you're saying, put the branch in a function
22:34 diakopter gotha
22:34 jnthn Right :)
22:54 timotimo why was i confident about the branch before? it seems to hang some multi-threaded tests again
22:56 leont Sounds like my entire week!
22:57 diakopter timotimo: push the branch and I'll look at the diff?
22:57 timotimo sure
22:58 dalek MoarVM/no_atomic_if_single_threaded: ca20e3c | timotimo++ | src/core/frame.c:
22:58 dalek MoarVM/no_atomic_if_single_threaded: cheaper access to ref counts if only 1 thread exists.
22:58 dalek MoarVM/no_atomic_if_single_threaded:
22:58 dalek MoarVM/no_atomic_if_single_threaded: MVM_frame_acquire_ref also ought to be enhanced like this
22:58 dalek MoarVM/no_atomic_if_single_threaded: review: https://github.com/MoarVM/MoarVM/commit/ca20e3c7c0
22:58 timotimo i git push -f'd over it again, because i'm naughty like that
22:58 diakopter lol ok
22:59 timotimo detecting force-pushes could be a thing that dalek does ... it'd probably be a bit of work, though
23:00 diakopter you need a ! in the function's check
23:00 diakopter if (!MVM_instance_have_user_threads(tc)) {
23:00 diakopter or swap the branches, of course
23:00 timotimo ha!
23:01 timotimo thank you! that's excellent!
23:01 diakopter you're eggsellent
23:01 timotimo man, you have such a fresh pair of eyes
23:01 diakopter the better to eat you with
23:03 timotimo you want to eat your fresh pair of eyes as a side dish to go with tasty timo flesh?
23:03 diakopter that too
23:04 pyrimidine joined #moarvm
23:05 timotimo if we're single-threaded, do it slow. if we're multi-threaded, do it unsafe! what a flawless plan!
23:06 diakopter actually wait
23:06 * timotimo actually waits
23:07 timotimo do you want to say we should totally multi-thread dereferencing of frames?
23:08 diakopter frames are also released during GC cleanup, which happens in another thread
23:09 timotimo wait. it does?
23:09 timotimo i didn't know we actually do that?
23:09 timotimo i know we wanted to have a second thread that handles things that require "free" to be called
23:12 diakopter find all the places that call MVM_frame_dec_ref
23:13 timotimo sure will
23:13 timotimo nothing in the gc, but things in the reprs
23:14 diakopter anything from gc_free ?
23:16 timotimo oddly enough, no
23:16 timotimo where did you find something?
23:19 diakopter well is your branch working now?
23:20 timotimo yeah
23:20 diakopter so push it :D
23:20 diakopter er, merge it
23:20 timotimo there's a few things that explode in the spec tests but tthat doesn't seem like my fault
23:20 timotimo like, t/spec/S17-scheduler/at.rakudo.moar dies with Method 'cancel' not found for invocant of class 'Any'
23:20 diakopter hm well do they explode using master
23:21 timotimo not ok 24 - multiple occurence for named array param
23:21 timotimo for t/spec/S06-other/main-usage.rakudo.moar
23:21 timotimo t/spec/S32-array/adverbs.t fails test 21, but only if run in the test harness
23:22 diakopter sigh
23:22 timotimo t/spec/S32-io/IO-Socket-Async.t only fails in the harness, too
23:22 timotimo t/spec/S32-hash/adverbs.t is the same as the array one
23:22 timotimo t/spec/S32-io/socket-recv-vs-read.t also only fails in the harness
23:22 timotimo crashes in the harness, that is.
23:32 lizmat timotimo: so how do you not it actually crashes?
23:35 timotimo lizmat: perl6 filename; exit code's 0 :)
23:35 lizmat timotimo: but then it doesn't run under the harness ??
23:38 timotimo yeah. if i run it under the harness, i.e. make spectest, it fails
23:38 timotimo i.e. exitcode != 0
23:38 diakopter nine was seeing tests silently fail under the harness when TEST_JOBS > 1... I wonder if it's related
23:39 timotimo huh
23:39 timotimo silently fail? what does silently in this case mean?
23:39 timotimo i'll run a spec test with 1 test job
23:39 diakopter they "passed" when run with TEST_JOBS > 1, but failed otherwise
23:40 timotimo um ... what?
23:40 timotimo that's ... different :o
23:41 timotimo lizmat: when you run spec tests, are they clean? they aren't for me. neither on moar's master nor on moar's no_atomic_if_single_threaded ...
23:42 lizmat timotimo: they haven't been clean for over a week for me
23:42 timotimo OK, good to know
23:42 lizmat I'll have an up-to-date list for you shortly
23:42 timotimo i would ticket + fudge something, but then the tests run to completion without failure when i run them individually ...
23:44 diakopter sounds like it's using a different lib or binary
23:44 timotimo i made extra sure to "make install" before
23:45 timotimo but i can double-check once this spec test run is finished
23:45 diakopter hrm
23:45 diakopter but are you running from the install location?
23:45 timotimo nah, from inside rakudo/
23:45 timotimo i think that's the same thing "make spectest" does
23:46 lizmat timotimo: https://gist.github.com/lizmat/b1be7dce17c5e639625b   # spectest status
23:46 diakopter well, the perl6-m script in rakudo/ runs all the things from the putative install location
23:48 diakopter exec /home/mwilson/src/rakudo/install/bin/moar  --execname="$0" --libpath="/home/mwilson/src/rakudo/install/share/nqp/lib" --libpath="." /home/mwilson/src/rakudo/perl6.moarvm "$@"
23:48 diakopter well, the nqp things
23:48 timotimo lizmat: that looks just a bit different from mine :|
23:48 lizmat timotimo: I'm not getting the same every time... lemme run one again
23:49 timotimo yeah. me neither :(
23:49 diakopter me nyether
23:53 timotimo https://github.com/coke/perl6-roast-data/commit/4119832d80335bd64e0e149272e99d324f8e0c56#diff-b122697401aaba9a55a4cc52f524ebf5L4785
23:53 timotimo cokes roast data also has something to say
23:54 lizmat hmmm.. second one identical to first
23:56 timotimo oh hey!
23:57 timotimo this time only at.rakudo.moar, main-usage.rakudo.moar, and both adverbs.t failed!
23:57 timotimo that's fewer than all my last runs!

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary