Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-09-04

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 samcv yeah if you build with clang i'm guessing you may get the same thing though
00:03 MasterDuke clang version 4.0.0-1ubuntu1 (tags/RELEASE_400/rc1)
00:03 MasterDuke same with or without _nocheck
00:03 samcv i get 1.6894568 with clang and 1.8 on gcc. but if i change to nocheck on clang i get 2.3 seconds
00:03 MasterDuke slightly slower in either case than gcc
00:04 samcv i have clang 4.0.1
00:04 samcv you seem to have rc1?
00:04 samcv mine says clang version 4.0.1 (tags/RELEASE_401/final)
00:04 samcv hm
00:04 MasterDuke afk for a bit, but might be able to check on my arch linux machine afterward (using kubuntu 17.04 on this laptop)
00:04 samcv ok cool
00:17 samcv i'm using:
00:17 samcv my $a = slurp "tolstoy.txt"; use nqp; my $t1 = now; nqp::index($a, "abdiicate") for ^400; say now - $t1
00:17 samcv where that txt is found here http://www.gutenberg.org/files/2600/2600-0.txt
00:17 samcv doing it 400 times i get ~5 vs ~7. when you get back run that test and let me know
00:31 lizmat joined #moarvm
00:32 MasterDuke on the laptop, with clang, that exact test gives ~10.2s for regular, 7.0s for _nocheck
00:35 MasterDuke on the laptop, with gcc, that exact test gives ~8.9s for regular, 7.4s for _nocheck
00:35 MasterDuke now trying on the desktop
00:40 MasterDuke on the desktop, with gcc, that exact test gives ~9.6s for regular, 9.5s for _nocheck
00:43 MasterDuke on the desktop, with clang, that exact test gives ~10.6s for regular, 10.7s for _nocheck
00:43 MasterDuke clang version 4.0.1 (tags/RELEASE_401/final)
00:44 MasterDuke gcc version 7.1.1 20170630 (GCC)
00:45 MasterDuke samcv: so it looks like _nocheck ranges from no speedup to 30% faster, depending on hardware and compiler
00:45 samcv yeah
00:45 samcv that's still insane how much slower it is at times :\
00:48 MasterDuke it was never really slower for me
00:49 samcv yeah maybe i got it backwards my bad
00:49 samcv but regardless it shouldn't make *that* much of a differenec for a 3+MB file
00:49 samcv :\
00:49 samcv oh well
00:49 samcv maybe it's not always inlining properly or something?
00:54 MasterDuke heh, it's pretty small, i'd hope there's no problem inlining it
00:59 samcv not sure why else it would have such a difference in speed
01:02 MasterDuke no idea here
01:05 samcv clang says it is a mild hint to the compiler
01:05 samcv i mean that's the same in gcc. they just make different choices i guess hm
01:05 samcv that's what i'm guessing at least
01:08 samcv well in configure.pl it detects clang as gcc. not sure how relevant that is
01:31 lizmat joined #moarvm
01:52 ilbot3 joined #moarvm
01:52 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
02:24 samcv on this test i'm getting 7.9163688 for clang and 11 seconds for gcc
02:24 samcv only difference is --compiler flag
02:27 samcv it's 13.75 before my grapheme caching change though. but still annoying i don't see as much improvement
02:27 samcv and most people use gcc
03:51 sivoais joined #moarvm
06:02 stmuk joined #moarvm
06:13 unicodable6 joined #moarvm
06:17 Geth ¦ MoarVM/master: 8 commits pushed by (Samantha McVey)++
06:17 Geth ¦ MoarVM/master: c449c87f13 | Add new cached grapheme iterator
06:17 Geth ¦ MoarVM/master: 6a9b5b5071 | Always cache even for flat haystacks with KMP
06:17 Geth ¦ MoarVM/master: 22ba6bb5a1 | Move gi_cached to iter.h
06:17 Geth ¦ MoarVM/master: 75d2b289ef | Remove some debugging code
06:17 Geth ¦ MoarVM/master: fec81bb6ff | Merge branch 'master' into grapheme_iterator_cached
06:17 Geth ¦ MoarVM/master: ce76c994b1 | Only use MVMGraphemeIter_cached for strands in KMP index
06:17 Geth ¦ MoarVM/master: 1a7dafc881 | Use 16bit to store KMP jump table. Set max needle to 8192
06:17 Geth ¦ MoarVM/master: 4a998d5896 | Merge pull request #661 from samcv/grapheme_iterator_cached
06:17 Geth ¦ MoarVM/master: review: https://github.com/MoarVM/MoarVM/compare/d65be80cff...4a998d5896
06:33 travis-ci joined #moarvm
06:33 travis-ci MoarVM build errored. Samantha McVey 'Merge pull request #661 from samcv/grapheme_iterator_cached
06:33 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/271568505 https://github.com/MoarVM/MoarVM/compare/d65be80cff1c...4a998d5896b5
06:33 travis-ci left #moarvm
06:59 brrt joined #moarvm
07:11 brrt .tell nine not sure what is your intention with the SAVE_RV node, but i think | mov TMP6, R12 is suspscious, as iirc R!2 == TMP5
07:11 yoleaux brrt: I'll pass your message to nine.
07:11 brrt also, WORK[arg.v.i64], why?
07:12 brrt i should be saying though, very cool work :-)
07:13 brrt oh, i see
07:13 brrt there's two more sane strategies you could us
07:13 brrt *use
07:14 brrt one, allocate an extra register (functionality for that exist in guess-which-branch); two; use the 'args' space for your frame to allocate the temporary
07:14 nine brrt: the intent of SAVE_RV is to move the return value into a callee saved register, so it will survive the call to MVM_gc_mark_thread_unblocked that I have to do between calling the function and boxing the return value
07:14 yoleaux 07:11Z <brrt> nine: not sure what is your intention with the SAVE_RV node, but i think | mov TMP6, R12 is suspscious, as iirc R!2 == TMP5
07:14 brrt (e.g. see what unless_i do)
07:14 brrt TMP12 is not callee-saved
07:15 brrt (or is it?)
07:15 brrt r12
07:15 brrt let me think before i say something silly
07:15 nine In the best case MVM_gc_mark_thread_unblocked doesn't even use r12 and thus we don't have to access memory at all.
07:15 brrt no, i think you're right
07:15 nine It is according to the comments :)
07:15 nine line 25
07:16 nine Written by a Bart Wiegmans. No idea how much he knows about JITs ;)
07:16 brrt not that much
07:16 brrt depending on when it was written :-P
07:16 brrt okay, right
07:17 brrt r12 is callee saved, i renember now, that's conveniently uniform between windows and posix
07:17 brrt giving that, you have a minor issue in your code :-P
07:18 brrt you forgot the other part of callee-saved
07:18 nine which is?
07:18 brrt you have to save it for your caller
07:18 brrt so in the function prologue/epilogue, your're going to need to store it
07:18 nine oooh
07:19 nine That kinda makes sense
07:19 brrt hmm…. now that i know what you want it for, if it is really just that short storage, you might be better of using the stack
07:19 nine Which means that I cannot save memory access at all by that
07:19 brrt we have ample stack space left for a pointer
07:20 brrt and you can use a .define to define a symbolic represetnation of the stack position
07:20 nine Can I just use good old fashioned push/pop for that? Haven't seen that in JIT code so far
07:21 brrt you can but apparently on some (windows) platforms the official ABI is to have stack frames 16-byte-aligned
07:21 brrt don't ask me why, and in fact i don't think it is practically required
07:21 brrt as in, everything just works if you disregard it
07:22 brrt other than that minor nitpick, do whatever feels natural :-)
07:22 brrt nine++ for digging into this
07:26 nine Cool :) Thanks for those hints!
07:38 brrt np
07:39 samcv brrt, is there a way to compare code for a specific function
07:39 samcv because i want to debug why gcc is like 20% slower on the KMP function
07:39 brrt it can be done, yes
07:39 brrt i suppose you'd instruct objdump
07:40 samcv do you know how to do that?
07:40 brrt not offhand, no
07:45 brrt you can basically just say gobjdump -d your_file.o
07:45 brrt and it will prefix the symbols with the correct function names
07:45 brrt it's generally useful
08:02 leont joined #moarvm
08:08 zakharyas joined #moarvm
08:17 robertle joined #moarvm
08:21 samcv hm
08:37 brrt yeah, that's less useful than you might want
08:37 jnthn morning o/
08:47 brrt good hi jnthn
08:48 brrt jnthn, re: COPY_ARRAY; a MVM_STATIC_INLINE couldn't take over the type implicitly
08:49 brrt i could introduce a MVM_VECTOR_COPY, but i'd have to convert the array pointers to 'full' MVM_VECTOR structures, which means that we are storing the allocated size uselessly
08:49 brrt (maybe a macro MVM_VECTOR_SIZE would be handy)
08:52 brrt other than that, i agree with pretty much all your comments :-)
08:53 jnthn brrt: Yeah, just commented; I missed that detail
08:58 lizmat jnthn: something from perl6-users: can we rely on the main thread always having thread ID 1 ?
09:08 Skarsnik joined #moarvm
09:10 jnthn lizmat: Umm...so far as VM-internal IDs go, presumably. For the native IDs, that'd be OS-specific and probably "no"
09:10 jnthn Also I'm not sure you can portably assume it across backends
09:11 brrt what about a method 'is-starting-thread'
09:11 jnthn Just store it somewhere and === it I guess :)
09:11 jnthn my $main-thread = $*THREAD; at startup
09:11 lizmat ok
09:11 lizmat thanks!
09:12 jnthn That'd be more robust/portable.
09:12 * jnthn wonders why somebody wishes to do this :)
09:12 lizmat yeah, trying to figure that out as well  :-)
09:13 brrt i'm going to be presumptive, and say, probably to do something in a react block
09:13 brrt conditionally on the main thread
09:21 brrt btw, i recently got my hands on 'Learning Perl'
09:21 brrt and read the preface
09:21 brrt it said 'Perl is an easy language to use, and a hard language to learn'
09:21 brrt and i thought to myself
09:22 brrt that pretty much explained the fate of perl in the last 20 years
09:23 Geth ¦ MoarVM/jit-legacy-cleanup: 19 commits pushed by (Bart Wiegmans)++
09:23 Geth ¦ MoarVM/jit-legacy-cleanup: review: https://github.com/MoarVM/MoarVM/compare/2aa92db4d2...b150b63d57
09:53 Geth joined #moarvm
10:00 Geth ¦ MoarVM/jit-legacy-cleanup: 84f70712b9 | (Bart Wiegmans)++ | src/jit/compile.c
10:00 Geth ¦ MoarVM/jit-legacy-cleanup: Move size check into COPY_ARRAY
10:00 Geth ¦ MoarVM/jit-legacy-cleanup:
10:00 Geth ¦ MoarVM/jit-legacy-cleanup: Is duplicated, so why not embed it, and make the code simpler.
10:00 Geth ¦ MoarVM/jit-legacy-cleanup: review: https://github.com/MoarVM/MoarVM/commit/84f70712b9
10:17 brrt jnthn, if no objections, i'm going to merge jit-legacy-cleanup :-)
10:17 jnthn brrt: All my comments are dealt with? :)
10:17 brrt i think so
10:18 jnthn Looks like it to me
10:18 jnthn +1
10:18 brrt (all systems go!)
10:19 brrt re: the move of the sequence number from jit code to spesh, i'm not 100% positive, but should we be adding it to the spesh candidate, or to the spesh graph
10:20 brrt i'm thinking candidate, but am i correct in thinking that the candidate is produced before the graph?
10:25 Geth ¦ MoarVM/master: 29 commits pushed by (Bart Wiegmans)++
10:25 Geth ¦ MoarVM/master: review: https://github.com/MoarVM/MoarVM/compare/4a998d5896...19caf7da3b
10:25 jnthn Not any more
10:26 jnthn The candidate is now produced after code-gen time
10:26 jnthn But before JIT
10:35 brrt hmmmm
10:35 brrt what would be the right place to assign a sequence number?
10:36 jnthn brrt: What's the purpose of the sequence number?
10:36 brrt to uniquely identify specific frames by time-of-specifalization (i.e. only when blocking)
10:36 brrt so that you can have a spesh-bisect similar to jit-bisect
10:37 jnthn But...we already have that feature?
10:37 jnthn MVM_SPESH_LIMIT
10:40 brrt do we?
10:40 brrt how do you do that without sequence numbers?
10:40 brrt or well, i see how you would
10:40 jnthn There's a global count of how many frames we spesh'd
10:41 jnthn Well, actually, how many specializations we produced
10:42 brrt hmmm
10:42 brrt so to build a bisect based on that would actually be really simple
10:44 travis-ci joined #moarvm
10:44 travis-ci MoarVM build passed. Bart Wiegmans 'Merge pull request #669 from MoarVM/jit-legacy-cleanup
10:44 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/271630723 https://github.com/MoarVM/MoarVM/compare/4a998d5896b5...19caf7da3b32
10:44 travis-ci left #moarvm
11:00 jnthn samcv: Dunno if this warning means anything to you:
11:00 jnthn src/strings/unicode.c:73800:78: warning: unknown escape sequence: '\X' {"KEYCAP: 6",342},{"KEYCAP: 7",343},{"KEYCAP: 8",344},{"KEYCAP: 9",345},{"KEYCAP: \X{23}",346},
11:00 samcv not really i can fix it temporarily. i mean that is KEYCAP: #
11:00 samcv which never worked anyway
11:02 brrt joined #moarvm
11:04 jnthn Think I've figured out https://rt.perl.org/Ticket/Display.html?id=131961
11:04 samcv oh nice
11:04 samcv also did you see the issue in moarvm/moarvm i opened about it requesting the same bytes forever
11:04 jnthn No
11:04 samcv well it requests 0 bytes for infinity for certain utf8 files
11:05 samcv that are missing bytes at the end of the file
11:05 samcv we hypothesized it kept asking for more since it expected more, but there was no more in th efile
11:05 samcv so it just repeated forever
11:06 jnthn Hmm
11:06 jnthn Probably not a bug at MoarVM level
11:06 jnthn But rather in the Perl 6 I/O code, which is presumably mis-handling zero-length buffers
11:06 jnthn Or EOF or similar
11:07 dogbert17 oh, 131961 is it the Malformed utf8 problem
11:07 jnthn Yeah. I dunno how we'll get a test for it though :P
11:07 jnthn I mean, I don't really want to commit the huge file :)
11:07 jnthn I guess a smaller test is possible
11:07 dogbert17 I had a smaller golf
11:07 jnthn Ah, cool
11:07 dogbert17 sec
11:07 jnthn Maybe you'd like to do the test? :)
11:08 dogbert17 jnthn: https://irclog.perlgeek.de/moarvm/2017-09-02#i_15107306
11:09 jnthn grrr, DNS resolution failure on irclog
11:09 dogbert17 here's the gist of it:
11:09 dogbert17 create a file with this specific code:    perl6 -e 'print "a" x (2**20 - 1) ~ "«"' > golf
11:09 dogbert17 then run: perl6 -ne '' golf
11:09 jnthn ah, cool
11:10 jnthn Presumably just .lines of the file works too
11:10 dogbert17 probably
11:10 dogbert17 [Coke] will be very happy :)
11:11 brrt joined #moarvm
11:14 jnthn yay, got a test that fails before my fix and passes after
11:14 jnthn dogbert17++
11:15 dogbert17 jnthn++
11:15 dogbert17 nice SPW talks btw
11:17 jnthn :)
11:17 Geth ¦ MoarVM: 1dd4486899 | (Jonathan Worthington)++ | src/strings/utf8.c
11:17 Geth ¦ MoarVM: Correct buffer handling in UTF-8 fast->slow path
11:17 Geth ¦ MoarVM:
11:17 Geth ¦ MoarVM: When transitioning between the two paths, we could incorrectly mark
11:17 Geth ¦ MoarVM: some bytes that made up part of a codepoint as having been consumed,
11:17 Geth ¦ MoarVM: when in reality we should have left them around for the next buffer.
11:17 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/1dd4486899
11:19 timotimo fantastic!
11:24 Geth ¦ MoarVM: ee06f56f88 | (Samantha McVey)++ | 2 files
11:24 Geth ¦ MoarVM: UCD: For sequences, convert from \x{ } to the Unicode codepoints
11:24 Geth ¦ MoarVM:
11:24 Geth ¦ MoarVM: This doesn't affect using "\c[ ]" since names with # were not accessible
11:24 Geth ¦ MoarVM: but it *does* work with .parse-names or with the nqp call directly.
11:24 Geth ¦ MoarVM:
11:24 Geth ¦ MoarVM: Currently the only thing that uses this is `keycap: #` which uses the \x
11:24 Geth ¦ MoarVM: syntax so that it parses correctly (though previous versions of the file
11:24 Geth ¦ MoarVM: had an actual `#` in there, which is why we didn't have this problem before).
11:24 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ee06f56f88
11:24 samcv fixed the unicode thing
11:28 ilmari why not just $name =~ s/ \\x \{ ([[:xdigit:]]) \}/chr hex $1/gex;?
11:28 jnthn lunch &
11:28 ilmari 1) \d doesn't match a-f, 2) do we really want to decode _multiple_ layers of \x escapes?
11:32 brrt (if we don't, will Damian be able to do cool things?)
11:35 ilmari this is parsing Unicode data files, isn't it?
11:35 ilmari not damiancrack
12:02 brrt Geth didn't show my push
12:04 ilmari which one? 11:25 < Geth> ¦ MoarVM/master: 29 commits pushed by (Bart Wiegmans)++
12:04 ilmari (timestamp in UTC+1)
12:05 ilmari isn't it supposed to detect merges better?
12:07 Skarsnik do I need to patch the jvm backend for https://github.com/MoarVM/MoarVM/pull/659 ?
12:14 leont joined #moarvm
12:39 zakharyas joined #moarvm
12:49 brrt joined #moarvm
12:58 Geth ¦ MoarVM: e0ece7f828 | (Jonathan Worthington)++ | 4 files
12:58 Geth ¦ MoarVM: Flush standard handles at exit.
12:58 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e0ece7f828
13:01 brrt joined #moarvm
13:02 brrt theres still a bunch of small changes in even-moar-jit to the legacy JIT
13:02 brrt that i somehow missed
13:08 timotimo looks like i found the problem that causes different outputs for new and old heap snapshot formats
13:08 dogbert17 jnthn: did you see https://github.com/MoarVM/MoarVM/issues/653
13:09 jnthn Yeah, not yet sure what's going on there
13:10 dogbert17 I can't replicate MasterDuke's findings on my 32 bit box
13:13 travis-ci joined #moarvm
13:13 travis-ci MoarVM build failed. Jonathan Worthington 'Flush standard handles at exit.'
13:13 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/271676756 https://github.com/MoarVM/MoarVM/compare/ee06f56f885a...e0ece7f8288a
13:13 travis-ci left #moarvm
13:14 timotimo 100.61user 0.45system 1:08.87elapsed 146%CPU (0avgtext+0avgdata 722380maxresident)k
13:14 timotimo > 49.54user 0.51system 0:17.51elapsed 285%CPU (0avgtext+0avgdata 733140maxresident)k
13:15 timotimo i'm a bit surprised it doesn't change memory usage. that's quite good.
13:20 timotimo oh, hold on
13:20 timotimo that still has the change that prints all data to a text file
13:20 timotimo oh, no it doesn't
13:21 timotimo oh and i guess the first one had precompilation in it
13:24 dogbert17 I often get this when running spectests:
13:24 dogbert17 t/spec/S17-lowlevel/atomic-ops.t .................................. Dubious, test returned 255 (wstat 65280, 0xff00)
13:24 dogbert17 Failed 18/28 subtests
13:25 timotimo that doesn't sound good
13:25 timotimo atomic ops not implemented on 32bit?
13:25 dogbert17 dunno, when running the file by itself it always passes
13:26 dogbert17 according to spectest something goes wrong with test 11
13:26 timotimo try having some load on your cpu
13:27 dogbert17 which is what I have when running the suite
13:30 timotimo time to work on variable-sized entries for collectables and references
13:31 dogbert17 timotimo: got it while under load
13:31 dogbert17 ok 10 - Updated value seen by non-atomic too
13:31 dogbert17 A IntLexRef container does not know how to do an atomic load
13:31 dogbert17 in block <unit> at repos/rakudo/t/spec/S17-lowlevel/atomic-ops.t line 45
13:32 dogbert17 IntLexRefContainer?
13:32 timotimo oh, huh, that's not supposed to happen
13:32 dogbert17 :(
13:35 MasterDuke joined #moarvm
13:35 jnthn huh
13:35 jnthn Yes it does!
13:37 timotimo oh?
13:37 timotimo oh, it does know how to do that
13:37 timotimo not it does supposed to happen
13:37 timotimo (because grammar)
13:38 dogbert17 ah, that's the same error that MasterDuke got in https://github.com/MoarVM/MoarVM/issues/653
13:41 Geth ¦ MoarVM: 4e7821be4b | (Jonathan Worthington)++ | src/io/syncfile.c
13:41 Geth ¦ MoarVM: Flush output buffer where needed.
13:41 Geth ¦ MoarVM:
13:41 Geth ¦ MoarVM: This makes :rw and similar modes work out properly, along with
13:41 Geth ¦ MoarVM: ensuring tell gives the right output.
13:41 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/4e7821be4b
13:55 Skarsnik_ joined #moarvm
13:56 Geth ¦ MoarVM: 6260e6fceb | (Jonathan Worthington)++ | src/io/syncfile.c
13:56 Geth ¦ MoarVM: Ensure write to unwritable handle fails right off.
13:56 Geth ¦ MoarVM:
13:56 Geth ¦ MoarVM: This is relied on in Rakudo, since it doesn't try to track modes to
13:56 Geth ¦ MoarVM: the point of doing I/O (which is OK, so long as we behave consistently
13:56 Geth ¦ MoarVM: even when buffering is turned on).
13:56 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6260e6fceb
13:57 Skarsnik__ joined #moarvm
14:01 Skarsnik_ joined #moarvm
14:04 timotimo hmpf. simplest version of variable-length encoding for collectables compresses it down to 75% size :|
14:06 Geth ¦ MoarVM: 41e1b7cef5 | (Jonathan Worthington)++ | 5 files
14:06 Geth ¦ MoarVM: Don't try and sync when flushing is enough.
14:06 Geth ¦ MoarVM:
14:06 Geth ¦ MoarVM: At exit we only want to flush our own buffering, not try and fsync or
14:06 Geth ¦ MoarVM: whatever the platform equivalent is.
14:06 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/41e1b7cef5
14:07 brrt joined #moarvm
14:09 jdv79 joined #moarvm
14:11 timotimo the references on the other hand benefit vastly
14:12 jdv79 so I was looking into RT#127772.  i don't this:  https://github.com/MoarVM/MoarVM/blob/master/src/io/dirops.c#L51
14:12 synopsebot6 Link:  https://rt.perl.org/rt3/Public/Bug/Display.html?id=127772
14:13 jdv79 looks like a useless stat call to me
14:15 jnthn To me also
14:16 jnthn Presuming that uv_fs_stat just passes along whatever stat does
14:16 jdv79 the docs suggest so, yes
14:16 jdv79 ok, i'll continue under the assumption that i'm not missing something
14:16 jdv79 thanks
14:22 AlexDaniel joined #moarvm
14:37 timotimo 5637523 24M -rw-r--r--. 1 timo timo 24M Sep  4 16:36 new_format_after_compression
14:37 timotimo 5637511 45M -rw-r--r--. 1 timo timo 45M Sep  4 16:34 new_format_before_compression
14:40 dogbert17 timotimo++
14:42 timotimo original format is 23 megabytes btw
14:43 jnthn So the new format is larger?
14:43 timotimo a little bit, yeah
14:45 brrt joined #moarvm
14:46 timotimo i can make the collectables chunks shrink down to about 75%, too
14:47 timotimo the reason i'm doing all this is to make parsing faster, though
14:51 timotimo and it is significantly faster :)
14:51 timotimo more than 4x
14:53 timotimo m: say 36_980_688 / 1_540_862
14:53 camelia rakudo-moar 4b02b8: OUTPUT: «24?»
14:54 timotimo 24 bits per reference on average
14:57 timotimo huh
14:58 timotimo m: 15_292_008 / 1_540_862
14:58 camelia rakudo-moar 4b02b8: OUTPUT: «WARNINGS for <tmp>:?Useless use of "/" in expression "15_292_008 / 1_540_862" in sink context (line 1)?»
14:58 timotimo m: say 15_292_008 / 1_540_862
14:58 camelia rakudo-moar 4b02b8: OUTPUT: «9.9243203?»
14:58 timotimo the old format beats the new format significantly with regards to size here
14:58 timotimo the collectables part is also noticably bigger
14:58 timotimo what gives o_O
15:03 timotimo ahahaha
15:04 timotimo yeah, the $size-per-reference in the file is the original size pre-compression
15:04 brrt joined #moarvm
15:08 timotimo 7.6 bytes per entry
15:10 * brrt is planning to do some documentation fixes and reopen the PR for even-moar-jit
15:13 timotimo the collectables in the old format have 19.5 bytes per entry
15:13 timotimo i think it's 28 in the new format
15:15 timotimo 9.7 chars per entry in the references one
15:15 timotimo (vs references in the new format having only 7.6 bytes per entry on average)
15:16 timotimo jnthn: do you think the size hit is acceptable for the performance gain, or should i put the 75% compression into the collectables section, too?
15:17 timotimo the new format seems to compress better. 3.5mb vs 3.2mb
15:17 * timotimo BBL
15:37 travis-ci joined #moarvm
15:37 travis-ci MoarVM build errored. Jonathan Worthington 'Don't try and sync when flushing is enough.
15:37 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/271704634 https://github.com/MoarVM/MoarVM/compare/6260e6fceb24...41e1b7cef56e
15:37 travis-ci left #moarvm
15:54 Geth ¦ MoarVM: 9fdd7b37c9 | (Jonathan Worthington)++ | src/spesh/optimize.c
15:54 Geth ¦ MoarVM: Remove some dead returns.
15:54 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/9fdd7b37c9
15:54 Geth ¦ MoarVM: ab8032ebc0 | (Jonathan Worthington)++ | src/spesh/optimize.c
15:54 Geth ¦ MoarVM: Better if_o/unless_o optimization in some cases.
15:54 Geth ¦ MoarVM:
15:54 Geth ¦ MoarVM: When the test is for a type object, we would turn it into isconcrete.
15:54 Geth ¦ MoarVM: However, we may in some cases know the answer to that already. Thus,
15:54 Geth ¦ MoarVM: specialize the added isconcrete, and then re-specialize the branch in
15:54 Geth ¦ MoarVM: case we can now eliminate it.
15:54 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ab8032ebc0
15:54 timotimo nice
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: fdccbe3381 | (Timo Paulssen)++ | src/profiler/heapsnapshot.c
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: introduce a binary format for heapsnapshot
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format:
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: it starts with a version identifier to differentiate between
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: the old format and this one. all records except the string heap
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: are fixed sizes and the file ends in a table outlining the
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: sizes of the individual segments.
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format:
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: <…commit message has 7 more lines…>
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: review: https://github.com/MoarVM/MoarVM/commit/fdccbe3381
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: cac6b6d492 | (Timo Paulssen)++ | 7 files
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: WIP gc_describe functions for new spesh datastructures
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: review: https://github.com/MoarVM/MoarVM/commit/cac6b6d492
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: 15fac4b2ed | (Timo Paulssen)++ | src/profiler/heapsnapshot.c
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: make references variable-length, add halfway to index
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format:
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: this reduces the average size of references entries from
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: 24 bytes to something more like 7.6 bytes on average in
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: one example dataset. The old format took 9.7 chars on
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: average in the same dataset.
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format:
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: the halfway point in the index serves to allow a second
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: thread to start parsing half-way into the blob, which
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: is what allows the new heap snapshot parser to be so fast.
16:00 Geth ¦ MoarVM/heapsnapshot_binary_format: review: https://github.com/MoarVM/MoarVM/commit/15fac4b2ed
16:00 timotimo (rebased on latest master)
16:01 timotimo not so happy about the commit in the middle, but without parts of that the heap snapshot profiler would crash immediately
16:01 Geth ¦ MoarVM: bdw++ created pull request #674: Merge 'expression' JIT backend
16:01 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/674
16:05 jnthn Hm, it would appear that doing a BB elimiantion pass right after we have deleted a conditional can be quite a win
16:05 jnthn As it marks writers dead
16:05 jnthn And we pick up on that in PHI merges sooner
16:07 jnthn m: say 1.11 / 1.29
16:07 camelia rakudo-moar 4b02b8: OUTPUT: «0.860465?»
16:07 timotimo i'd like to have a quick way to read just a single char from a file handle in binary mode; i don't really want to allocate a buf_8 for every one-byte read, so would i have to add a new op?
16:08 jnthn Um
16:08 jnthn Just read a buf and pick things off it in Perl 6
16:08 timotimo that's what i'm doing right now; i created a MyLittleBuffer class that reads in big chunks and then i can .shift off of the buf i get from it
16:08 jnthn The buf allocation will probably be swamped by the fact that every call down to the I/O handle acquires a mutex
16:09 timotimo hm, that seems fair
16:09 timotimo are you +1 on making getc on a binary-mode handle work like .read(1)[0] ?
16:09 jnthn Not really...
16:09 timotimo OK
16:09 jnthn c means char
16:09 jnthn I'd rather we don't confuse those APIs
16:10 jnthn Of course you could put the thing in latin-1 mode and ord the char that comes back... :)
16:10 jnthn And then enjoy the buffering
16:10 timotimo then i'll have to keep a counter around rather than being able to just shift
16:11 timotimo so i suppose i'll keep my buf and buffer approach
16:11 jnthn About the calc above, somehow I've managed to get 14% off the write a million lines benchmark with the immediate BB cleanup thing I described
16:11 timotimo wow
16:11 jnthn Well, in combination with what I pushed a moment ago
16:11 timotimo is it because we don't iterate to fixpoint?
16:11 jnthn I think it may have saved a late-bound can
16:11 timotimo cool
16:12 jnthn But what that kind of size drop I also start to wonder if we might have pulled say below the inline limit also
16:12 jnthn *time drop
16:12 timotimo right
16:12 jnthn These kinds of things add up
16:13 timotimo they sure do
16:14 jnthn Currently doing spesh stressing
16:14 jnthn Before I commit this
16:14 jnthn I wonder if this'll cause wider improvemnets though
16:19 jnthn Yeah, 70% inlined to 80% inlined
16:19 jnthn uh, 85% sorry
16:20 timotimo *nice*
16:20 jnthn Now everything inside of say is inlined
16:20 jnthn In fact, say is the only thing that ain't inlined
16:24 Geth ¦ MoarVM: dd04dd8094 | (Jonathan Worthington)++ | src/spesh/optimize.c
16:24 Geth ¦ MoarVM: Do dead BB elimination pass after branch deletion
16:24 Geth ¦ MoarVM:
16:24 Geth ¦ MoarVM: This means that writers in eliminated basic blocks will be marked dead
16:24 Geth ¦ MoarVM: right away, which in turn means that we'll be able to be more certain
16:24 Geth ¦ MoarVM: in a bunch of PHI merges. The more precise type information can then
16:24 Geth ¦ MoarVM: lead to better downstream optimizations.
16:24 Geth ¦ MoarVM:
16:24 Geth ¦ MoarVM: In a Perl 6 write a million lines benchmark, as well as the immediate
16:24 Geth ¦ MoarVM: gains this leads to a further inlining, giving 10%-15% time reduction
16:24 Geth ¦ MoarVM: and bringing the benchmark to within 1.2x of Perl 5.
16:24 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/dd04dd8094
16:30 jnthn m: say 8306128726 / 9554038194
16:30 camelia rakudo-moar 4b02b8: OUTPUT: «0.86938408214?»
16:30 jnthn Similar sized reduction according to callgrind too
16:31 dogbert2 joined #moarvm
16:37 timotimo i'll need a change in API for the heap snapshot profiler now
16:38 timotimo i currently write to a file handle i open right before i write the heap snapshot, but the hll code will want to specify the filename, except the only way to do that would require the filename to be decidde at the beginning of profiling rather than at the end
16:39 jnthn Ah, hmm
16:39 jnthn I think maybe we pass a hash of profiling options though?
16:39 timotimo right
16:39 jnthn So we could send the filename in that way
16:40 timotimo any worries about the filename being stolen in between start and finish of the profile run?
16:47 jnthn Not really, I mean it could in theory happen today
16:48 timotimo right
16:49 timotimo should i immediately kick out the old format from moarvm and just keep it around in the heapanalyzer?
16:50 jnthn I don't know there's much point having both, if you're confident the new one is working well
16:52 timotimo i should definitely test with more than one snapshot before i do the merge *cough*
16:54 robertle joined #moarvm
16:54 timotimo hmm, if we have the filename up front already, it would perhaps be an interesting idea to stream out individual heap snapshots and later add the string heap, static frames, types, ...
16:55 jnthn Hmm, of the time that we spend doing the UTF-8 encoding, about 75% of it is in the codepoint iterator
16:55 jnthn timotimo: Yeah, that probably makes sense
16:55 timotimo in that case i'll have to juggle the sections around
16:55 timotimo but we might be able to save quite a bit of RSS over long-running programs
16:56 jnthn For sure
17:01 lizmat joined #moarvm
17:03 jnthn dinner &
17:03 timotimo oops, 1.5 GB of heap snapshot
17:06 timotimo oops, hard drive filled up all the way
17:06 Skarsnik_ lol
17:18 timotimo throwing 9 gigs of backup stuff from the laptop to the fileserver
17:22 travis-ci joined #moarvm
17:22 travis-ci MoarVM build failed. Jonathan Worthington 'Ensure write to unwritable handle fails right off.
17:22 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/271700536 https://github.com/MoarVM/MoarVM/compare/4e7821be4b47...6260e6fceb24
17:22 travis-ci left #moarvm
17:24 timotimo jnthn: would it ever be interesting to see what owner each individual collectable has?
17:55 leont joined #moarvm
18:00 travis-ci joined #moarvm
18:00 travis-ci MoarVM build passed. Jonathan Worthington 'Better if_o/unless_o optimization in some cases.
18:00 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/271744461 https://github.com/MoarVM/MoarVM/compare/41e1b7cef56e...ab8032ebc055
18:00 travis-ci left #moarvm
18:12 domidumont joined #moarvm
18:30 timotimo there's still lots of bugs in handling multiple snapshots %)
18:32 solarbun1y joined #moarvm
18:35 committable6_ joined #moarvm
18:43 eater joined #moarvm
18:48 ilmari[m] joined #moarvm
18:56 timotimo just a copy-pasto, a change to some calculation not needing to factor in a header's size any more, and an off-by-one caught by my little ascii chunk headers
18:59 MasterDuke nice, about done?
19:00 Skarsnik_ should I restart the bots?
19:02 jnthn timotimo: Not really, I don't think
19:04 timotimo thought so
19:08 timotimo MasterDuke: nope, still got problems :D
19:17 MasterDuke heh, don't we all...
19:25 lizmat joined #moarvm
19:26 timotimo while transporting my laptop to the dining room, i hit its edge against the door frame and it rebooted
19:27 MasterDuke hope stuff was backed up!
19:27 timotimo nothing was in peril
19:54 travis-ci joined #moarvm
19:54 travis-ci MoarVM build passed. Jonathan Worthington 'Do dead BB elimination pass after branch deletion
19:54 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/271755550 https://github.com/MoarVM/MoarVM/compare/ab8032ebc055...dd04dd80945c
19:54 travis-ci left #moarvm
20:40 Geth ¦ MoarVM/jit_nativecall: 16 commits pushed by (Stefan Seifert)++
20:40 Geth ¦ MoarVM/jit_nativecall: review: https://github.com/MoarVM/MoarVM/compare/92198c815f...520d47068f
20:40 nine rebased it on master and worked in brrt++'s changes to the JIT
20:59 lizmat and another Perl 6 Weekly hits the Net: https://p6weekly.wordpress.com/2017/09/04/2017-36-blogs-a-reading/
21:17 dogbert2 joined #moarvm
21:46 timotimo oh wow, that was a really bad miss
21:46 timotimo i've been grabbing the data from references into a uint8 even though only the first field will fit into that
21:46 timotimo in the old version
21:48 timotimo i've been looking for the error in all the wrong places
22:02 timotimo now the complete refs table as read by version 1 and version 2 match exactly
22:03 MasterDuke good deal
22:03 * timotimo turns splitting back on
22:04 timotimo let's see if it properly calculates the sizes for both halves to match up
22:04 timotimo cool, they still match
22:15 timotimo the new heap snapshot format just came up smaller than the old one in an example test case
22:16 timotimo (i no longer write the kind of reference in a bigger integer since it's at most 2 bits anyway!
22:24 MasterDuke smaller and faster! does it also do julienne fries?
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format: af10d049e6 | (Timo Paulssen)++ | src/profiler/heapsnapshot.c
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format: fix accidental truncation of references values
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format: review: https://github.com/MoarVM/MoarVM/commit/af10d049e6
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format: 7274060bc3 | (Timo Paulssen)++ | src/profiler/heapsnapshot.c
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format: never store ref kind in more than 8 bit
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format:
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format: currently there's only 2 bits dedicated to this.
22:24 Geth ¦ MoarVM/heapsnapshot_binary_format: review: https://github.com/MoarVM/MoarVM/commit/7274060bc3
22:24 Ven joined #moarvm
22:46 timotimo 25% time spent in the gc when parsing those snapshots ... not such a great look :)
23:03 AlexDaniel joined #moarvm
23:04 robertle square
23:04 timotimo MasterDuke: if you want, you can test the latest stuff i pushed to moarvm and the heap analyzer
23:06 * timotimo goes to bed
23:13 samcv jnthn, how would you feel if for strands we cached a grapheme iterator? that way you could make subsequent calls to the same strand and get much faster access to sequential sections of the strand
23:42 jnthn Everything in MVMString and hanging from it must be immutable
23:42 jnthn An iterator is mutable
23:42 jnthn I'm not sure where else you'd be thinking of caching it
23:43 jnthn But if it's on MVMString or on a strand within it, then I don't see how that could be done without violating the immutability of strings
23:44 bisectable6 joined #moarvm
23:44 committable6 joined #moarvm
23:44 nativecallable6 joined #moarvm
23:44 benchable6 joined #moarvm
23:44 coverable6 joined #moarvm
23:44 quotable6 joined #moarvm
23:44 bloatable6 joined #moarvm
23:44 greppable6 joined #moarvm
23:44 unicodable6 joined #moarvm
23:44 releasable6 joined #moarvm
23:44 evalable6 joined #moarvm
23:44 squashable6 joined #moarvm
23:44 statisfiable6 joined #moarvm
23:46 jnthn sleep time here; bbiab :)

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary