Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2015-05-02

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
03:57 vendethiel joined #moarvm
07:56 rurban joined #moarvm
08:19 Ven joined #moarvm
08:42 dalek MoarVM: 15908a2 | jnthn++ | src/gc/roots.c:
08:42 dalek MoarVM: Mark the cached backend config hash.
08:42 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/15908a2433
08:46 nwc10 jnthn: much rakduo SEGVing on linux. at this line of moar.c:
08:46 nwc10 MVM_interp_run(tc, &toplevel_initial_invoke, start_frame);
08:46 nwc10 seems to be that the address of start_frame has the lowest bit set
08:47 jnthn nwc10: Did you have 15908a2?
08:47 Ven joined #moarvm
08:48 nwc10 I think I was on 15908a2^
08:48 jnthn OK, but...weird. I've a build on Linux here too and all's well with latest of everything
08:49 nwc10 I'm trying to figure out what caused it
09:36 jnthn m: say 9611866 - 9552454
09:36 camelia rakudo-moar f3ecf8: OUTPUT«59412␤»
09:41 jnthn Another 60KB off CORE.setting
09:42 jnthn That's 60KB less *bytecode* to verify and interpret
09:43 nwc10 with the cached backend config?
09:44 jnthn No
09:44 jnthn That was a smaller win (about 1.5 million less CPU instructions)
09:44 jnthn Well, a different kind of win, rather.
09:45 nwc10 spectest seems to be happy with Moar at 15908a2
09:45 nwc10 (Rather than 15908a2^)
09:47 jnthn Oh, I get a nice CPU win too
09:48 jnthn m: say 541034601 / 552393040
09:48 camelia rakudo-moar f3ecf8: OUTPUT«0.9794377587␤»
09:48 jnthn Another 2% off startup. I'll take that.
09:51 dalek MoarVM: 0acd74b | jnthn++ | src/ (6 files):
09:51 dalek MoarVM: Dual-purpose comp unit string heap as SC one.
09:51 dalek MoarVM:
09:51 dalek MoarVM: This means we don't have to construct an array of strings in the order
09:51 dalek MoarVM: the deserialization blob wants them, and can instead just point the
09:51 dalek MoarVM: deserialization at the string heap of the compilation unit. When the
09:51 dalek MoarVM: NQP change to take advantage of this is applied, we knock 60KB off the
09:51 dalek MoarVM: size of CORE.setting, and save 2% off startup time.
09:51 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/0acd74b92c
09:54 jnthn m: say 541034601 / 644090520
09:54 camelia rakudo-moar f3ecf8: OUTPUT«0.8399977708␤»
09:54 jnthn Now running only 84% of the instructions at startup we used to compare to yesterday.
10:00 nwc10 cool.
10:06 dalek MoarVM: 5179453 | jnthn++ | src/core/bytecode.c:
10:06 dalek MoarVM: Re-order bytecode read struct to be smaller.
10:06 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/5179453056
10:06 dalek MoarVM: 48171d9 | jnthn++ | src/core/bytecode.c:
10:06 dalek MoarVM: Don't repeatedly calculate limit in bytecode read.
10:06 dalek MoarVM:
10:06 dalek MoarVM: Also mark it MVM_INLINE_STATIC and remove a bogus NULL check.
10:06 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/48171d93ca
10:10 jnthn m: say 540112560 / 541034601 # a tiny but easy win
10:10 camelia rakudo-moar f3ecf8: OUTPUT«0.9982957818␤»
10:18 Ven joined #moarvm
11:43 Ven joined #moarvm
12:14 Ven joined #moarvm
12:20 rurban joined #moarvm
12:57 timotimo cool, i like these changes
13:17 vendethiel joined #moarvm
14:02 vendethiel joined #moarvm
14:39 mj41 joined #moarvm
14:56 zakharyas joined #moarvm
15:18 dalek MoarVM: 6b7a61d | jnthn++ | 3rdparty/uthash.h:
15:18 dalek MoarVM: Toss some bits of ut_hash we don't use.
15:18 dalek MoarVM:
15:18 dalek MoarVM: As part of preparations for more extensively modifying it.
15:18 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6b7a61d76b
15:26 dalek MoarVM: cf4ef03 | jnthn++ | 3rdparty/uthash.h:
15:26 dalek MoarVM: Toss hash bloom test bits we don't use.
15:26 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/cf4ef03ad5
15:27 timotimo ooooh
15:27 timotimo if we heavily modify it, maybe we'll end up having MVMString objects that actually store latin1 if they can
15:27 timotimo er, MVMString isn't an "object"
15:27 timotimo but you know what i me
15:27 timotimo mean*
15:28 jnthn It is an object as far as the GC cares :P
15:28 jnthn But yeah, that's kinda where I'm going :)
15:30 timotimo yus!
15:30 timotimo memory usage could go down a whole lot from this
15:31 timotimo on the weekly, someone commented having NFG strings for everything could lead to problems because it balloons up the memory usage of a single character even if it could be stored in latin1
15:31 timotimo like if you analyze log files or something and the log file happens to be a gig in size
15:32 jnthn If you're reading it all into memory you may already have issues ;)
15:32 jnthn But yeah :)
15:32 timotimo fair enough
15:34 jnthn I think if you explicitly decode the thing as latin-1 then we can probably get it stored byte size.
15:35 timotimo mhm
15:35 timotimo since we have ropes, we could also decode until we find something that latin1 can't handle and switch over and have a two-part string
15:35 timotimo except if the latin-1-codable prefix is too short
15:36 dalek MoarVM: 315aac7 | jnthn++ | 3rdparty/uthash.h:
15:36 dalek MoarVM: Toss unused hash signature.
15:36 dalek MoarVM:
15:36 dalek MoarVM: We don't make any use of this, so it's just 32 wasted bits.
15:36 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/315aac719d
15:36 jnthn timotimo: Yeah, I'd pondered that too :)
15:37 timotimo maybe i ought to freshen up the heap analyzer plugin to gdb and have a look at what's on the heap these days
15:38 timotimo last time i looked we had multiple copies of the same MVMString in the nursery for a big amount of strings
15:38 timotimo i wonder where that came from and if it's still there
15:45 Ven joined #moarvm
15:46 dalek MoarVM: 77e3fbc | jnthn++ | Configure.pl:
15:46 dalek MoarVM: Optimize at level 3 by default.
15:46 dalek MoarVM:
15:46 dalek MoarVM: We've been at 2 for a good while without issues; let's bump it a tad
15:46 dalek MoarVM: further. Doesn't make a big difference, but every little helps.
15:46 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/77e3fbc112
15:58 dalek MoarVM: 42d7dd8 | jnthn++ | 3rdparty/uthash.h:
15:58 dalek MoarVM: Toss even more hash bits we don't use.
15:58 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/42d7dd8ec4
16:03 Ven joined #moarvm
16:14 timotimo i've been on -O3 for a long time now
16:14 timotimo (and i even -march=native)
16:26 jnthn This is the best error I've got out of a C compiler in ages!
16:26 jnthn C2064: term does not evaluate to a function taking -1000 arguments
16:27 arnsholt \o/
16:30 timotimo m)
16:32 * jnthn is further hacking up the code once known as UT hash
16:32 jnthn I'm currently working on getting rid of the linked list for the order stuff was added in.
16:32 jnthn Which'll save us 16 bytes (on 32-bit machines) per item put into a hash.
16:33 jnthn I *think* it should be do-able without a total re-write of the thing. I hope. :)
16:57 rurban joined #moarvm
17:25 nwc10 sort of a too-late question now - you started on the version we had, not the most recent (and now abandoned) version upstream?
17:27 jnthn nwc10: No 'cus we already have some more minor modifications, though looking at the change log I think pretty much all that we'd have missed was in the code I tossed anyway
17:29 jnthn Seems I've managed to re-work the HASH_ITER macro to go through the buckets rather than needing the ordered linked list.
17:29 * jnthn spectests that piece
17:30 jnthn I think there's another iteration path I'll need to change.
17:30 jnthn And then can see if tossing the addition order linked list is as easy as I hpe.
17:30 jnthn *hoe
17:30 jnthn *hope
17:30 jnthn ...yeah, apparently it's dinner time too :)
18:10 vendethiel joined #moarvm
18:44 vendethiel joined #moarvm
18:49 dalek MoarVM: 7c1aaf7 | jnthn++ | / (9 files):
18:49 dalek MoarVM: Re-implement HASH_ITER in terms of buckets.
18:49 dalek MoarVM:
18:49 dalek MoarVM: This is the first step in removing the use of the double-linked list
18:49 dalek MoarVM: that maintains insertion order, which from a NQP/Perl 6 perspective is
18:49 dalek MoarVM: simply time/memory overhead that we don't need.
18:49 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/7c1aaf75e2
18:52 lizmat which means we finally get randomly ordered hashes ?
19:03 jnthn lizmat: I don't think MVMIter uses the macro, so that's next in line
19:03 jnthn But yeah, we'll lose insertion order
19:03 lizmat yes!  :-)
19:03 jnthn "randomly" is maybe a bit strong :)
19:03 lizmat wonder how much this will break in the ecosystem  :-)
19:04 jnthn I hope not much given that JVM hasn't given the ordering
19:06 lizmat weill, it'll allow some .pick(*) in some .perl's, to be removed
19:13 timotimo keys will be placed pseudorandomly depending on their values as well as insertion order
19:13 timotimo OSLT
19:16 timotimo we'll have to do a bit of extra work to make the order different on every execution of the same script
19:17 Ven joined #moarvm
19:17 lizmat I don't think that's necessary just yet
19:17 jnthn timotimo: We'll likely have to do that to avoid security issues
19:18 lizmat well, before 6.0 sure  :)
19:18 jnthn Thankfully there's some good knowledge on these issues in the Perl community
19:19 lizmat true, but it's also a source of discussion, and sometimes feuds  :-(   let's not go there  :-)
19:19 timotimo yes
19:25 jnthn It's fine, I only have to listen to the people who are right. :P
19:25 zakharyas joined #moarvm
19:25 lizmat just like picking the right standard  :-)
19:26 lizmat hmmm... I just realized this may break typed hashes
19:26 lizmat as it uses 2 hashes: one for the keys and one for the values
19:30 jnthn Should be fine, I think...
19:31 Ven joined #moarvm
19:35 lizmat yeah, HashIter is doing a lookup
19:36 lizmat jnthn: do you have any idea why it was done that way, rather than having a single hash with Pairs (original object => value) ??
19:36 Ven joined #moarvm
19:37 lizmat generation of .pair/.kv and friends would be comparable, without the need of keeping a whole 2nd hash with identical keys
19:37 lizmat fwiw, that's how I implemented bags/mixes
19:39 jnthn lizmat: Because so far we've only got (at the lower level) support for hashes with string keys
19:39 lizmat yes, I get that
19:40 lizmat but why not the approach of StringKey => ( Key => Value )
19:40 lizmat instead of (StringKey => Key), (StringKey => Value)
19:40 lizmat internally, I mean
19:41 Ven joined #moarvm
19:41 lizmat the natural form of a Hash is a list of pairs, which is easily generated from the former setup  :-)
19:41 jnthn Ah, I see your point
19:41 jnthn Not sure; what you're suggesting could be better
19:42 lizmat and is proven, as bags/mixes are implemented that way  :-)
19:42 jnthn *nod*
19:43 nwc10 jnthn: ASAN! http://paste.scsys.co.uk/475870
19:46 vendethiel joined #moarvm
19:47 jnthn urgh
19:56 cygx joined #moarvm
19:56 cygx jnthn: while you're working in the general vincinity - I just got a ===SORRY!=== Missing serialize REPR function for REPR VMIter
19:57 jnthn cygx: That's probably quite orthogonal.
19:57 cygx jnthn: just saw an MVMIter in the logs
19:58 cygx in related news: hello world from the Tiny C Compiler at https://github.com/cygx/p6-tinycc/blob/master/example.p6
19:58 cygx precompiling that is where the SORRY got triggered
19:58 nwc10 jnthn: I've looked at the source code and I can't work out what went wrong from code inspection
19:59 jnthn nwc10: I've an idea.
19:59 nwc10 I'm guessing that the hash manages to get freed before the iterator
19:59 Ven joined #moarvm
20:00 nwc10 I might be about to go to bed, so may not be around if you have a fix to be tested.
20:00 jnthn nwc10: Yeah, it's somethin glike that. Looks like we get to keep a bonus bit of state...
20:01 jnthn lizmat: Hm, seems we may have some spectests sensitive to hash ordering...
20:02 lizmat yeah, that was bound to happen
20:02 lizmat many ?
20:02 dalek MoarVM: ea6fe32 | jnthn++ | src/6model/reprs/MVMIter. (2 files):
20:02 dalek MoarVM: Re-implement MVMIter on hashes using buckets.
20:02 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ea6fe3242d
20:02 jnthn 3
20:02 jnthn S32-hash/delete-adverb.t
20:02 jnthn S32-list/roll.t
20:02 lizmat ah, just ignore them for now, I will take care of them
20:02 jnthn S05-capture/caps.rakudo.moar
20:03 jnthn Sure, well, may not bump MOAR_REVISION/NQP_REVISION tonight anyways.
20:03 jnthn I need to fix the ASAN barf nwc10++ reported
20:03 lizmat those tests shouldn't be a reason not to bump, the ASAN barf is  :-)
20:03 jnthn *nod*
20:03 jnthn Well, more important is "can I remove the whole linked list thingy"
20:04 jnthn (Which is the goal of all this)
20:06 lizmat jnthn: I'll look at the tests after you bump NQP, is that ok?
20:06 lizmat or if you could gist the output, that would be great
20:06 jnthn lizmat: Sure, though I may get to them anyway to make sure there's nothing more sinister than hash order going on
20:09 Ven joined #moarvm
20:14 dalek MoarVM: ad17517 | jnthn++ | 3rdparty/uthash.h:
20:14 dalek MoarVM: Toss HASH_FSCK addition order check.
20:14 dalek MoarVM:
20:14 dalek MoarVM: This gets rid of the easiest thing depending on ->prev and ->next.
20:14 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ad17517301
20:14 dalek MoarVM: 282861f | jnthn++ | 3rdparty/uthash.h:
20:14 dalek MoarVM: Use a simpler "deleting the last item" check.
20:14 dalek MoarVM:
20:14 dalek MoarVM: Of note, one that doesn't depend on ->next and ->prev.
20:14 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/282861ff6a
20:17 Ven joined #moarvm
20:34 vendethiel joined #moarvm
20:43 dalek MoarVM: 6774020 | jnthn++ | 3rdparty/uthash.h:
20:43 dalek MoarVM: Eliminate double-linked-list in hashes.
20:43 dalek MoarVM:
20:43 dalek MoarVM: With this, we save 16 bytes per hash entry (on 64-bit platforms) and
20:43 dalek MoarVM: a further 8 bytes per hash. Halve those for 32-bit.
20:43 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6774020fe2
20:49 jnthn No particular change in instructions executed, but a nice memory saving of course. :)
20:50 lizmat 16 bytes / hash entry: how many is it now?
20:51 lizmat (just prepping info for timotimo's P6W :-)
20:53 jnthn I see 4 pointers and 2 32-bit ints still
20:53 jnthn Which isn't great, but this is a step in the right direction.
20:57 dalek MoarVM: 0e6e0b6 | jnthn++ | 3rdparty/uthash.h:
20:57 dalek MoarVM: Toss an outdated comment.
20:57 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/0e6e0b6761
21:00 jnthn Getting tired now, so I think I'll fix the ASAN thingy in the morning.
21:09 cygx left #moarvm
22:17 Ven joined #moarvm
22:35 timotimo i see about 1.5 megabytes less maxresidentk for a newer rakudo vs an older rakudo
22:36 timotimo to be exact: f4ff0a8..56cae10  nom    547d08a..18ece1f  master (nqp)    f04b1d8..0e6e0b6  master (moarvm)
22:36 Ven joined #moarvm
22:39 timotimo that contains both the "re-use string heap from comp-unit" and the "remove doubly-linked list from hashes" commits
23:06 vendethiel joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary