Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2014-01-26

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:13 FROGGS[mobile] joined #moarvm
00:50 jnap joined #moarvm
01:25 ozmq joined #moarvm
01:51 jnap joined #moarvm
02:05 krunen joined #moarvm
02:09 ozmq joined #moarvm
02:20 ozmq joined #moarvm
03:53 jnap joined #moarvm
04:28 d4l3k_ joined #moarvm
04:43 japhb_ joined #moarvm
04:47 crab2313 joined #moarvm
04:54 jnap joined #moarvm
05:54 jnap joined #moarvm
06:09 krunen joined #moarvm
06:55 jnap joined #moarvm
07:56 jnap joined #moarvm
08:57 jnap joined #moarvm
09:18 FROGGS o/
09:24 timotimo o/
09:38 dalek MoarVM/varint: 1eb0650 | (Timo Paulssen)++ | src/6model/serialization.c:
09:38 dalek MoarVM/varint: an implementation of varint reading/writing
09:38 dalek MoarVM/varint: review: https://github.com/MoarVM/MoarVM/commit/1eb0650ce2
09:57 jnap joined #moarvm
10:44 nwc10 japhb_: first thought - "don't know"
10:44 nwc10 second thought, if "don't know" isn't good - write benchmarks in Moose. But make sure that everyone is happy that it's ideomatic efficient Moose
10:44 nwc10 because Moose vs Rakudo is the first comparison to be made
10:45 nwc10 (and perl6-m is getting close to Moose on *startup* time)
10:45 nwc10 third thought: once Moose is done, or if someone else wants to do it, also writing the code as "native" Perl 5 OO would be interesting too. (Not ugly hack micro-optimised code)
10:46 nwc10 but that's as much for a benchmark comparison of Moose to "native" as from Moose to Perl 6
10:46 nwc10 but longer term, it's going to be interesting to see how perl6-m converges first on Moose startup performance, then on Moose runtime performance, and finally on "native" Perl 5
10:46 nwc10 it may never get there, but I hope that that isn't the case
10:47 FROGGS I would be happy (very happy) if it would get close
10:47 nwc10 but to make sure that perl6-m wins, ensure enough benchmarks use Unicode code points > 256 :-)
10:48 FROGGS because I am more efficient in P6 than P5 nowadays
10:49 timotimo more efficient? as in lines of working code written or something?
10:51 FROGGS timotimo: no, there are less P6 lines in the end usually :o)
10:51 FROGGS but time span is shorter to solve a problem in P6 compared to P5
10:51 FROGGS (when I don't need non-existent modules, that is)
10:55 nwc10 jnthn: slow thing update (Should have been yesterday)
10:55 nwc10 Stage optimize   : 201539.280
10:55 nwc10 been at least a day in the next stage
10:58 jnap joined #moarvm
11:06 nwc10 also, will Moose and Perl 6 code look reasonably simlar?
11:06 nwc10 er, similar
11:12 woolfy joined #moarvm
11:14 tadzik a bit; mop code is a lot closer
11:14 tadzik (stevan's p5mop)
11:16 timotimo yo tadzik :)
11:16 timotimo is the pod table replacement code still on the agenda? :)
11:18 jnthn nwc10: slow indeed!
11:18 jnthn nwc10: But no crash yet :)
11:19 tadzik timotimo: I guess :) I'll look through it today
11:20 timotimo since it garbage collects so often, the memory usage is probably really low! :P
11:20 timotimo okay gotta run catch a train
11:22 * jnthn also goes to the station to go to the airport to go to the land of the fjords...
11:22 jnthn &
11:37 tadzik _another_ land of the fjords? :P
11:37 nwc10 well, I guess that southern Sweden considers northern Sweden to be the same country
11:38 tgt joined #moarvm
11:48 vmspb joined #moarvm
11:59 jnap joined #moarvm
12:27 * timotimo is working on varint support in the serializer
12:27 * FROGGS is fixing a segfault in v5@moar
12:31 cognominal joined #moarvm
12:40 timotimo oh? have you gotten past the repo conflict code already?
12:41 FROGGS timotimo: yes, was pretty easy actually
12:41 FROGGS was just about six or seven lines of nqp-p code that wanted translating
12:42 FROGGS and the currect segfault is just about moar being moar strict about stringifying than parrot
12:42 timotimo cool :)
12:43 FROGGS and moar segfaults when calling existskey on a null object, where parrot does not
12:43 timotimo i need to find more spots to put varint instead of int64/int32
12:43 FROGGS so, whatever I do to make it work under moar will alos be needed for jvm
12:43 nwc10 A further thought I had over lunch - I suspect that a "native" Perl 5 OO version of benchmarks would be very useful if it also runs under v5
12:43 FROGGS what does that varint actually do?
12:44 timotimo i wonder if i need to consider aligning for serialization? probably not.
12:44 FROGGS nwc10: to spot unreasonable slowdowns?
12:44 arnsholt timotimo: Yeah, serialization shouldn't need alignment
12:44 timotimo great
12:45 nwc10 FROGGS: not what I was thinking. Just would be curious to benchmark the exact same code running under v5 on MoarVM and the JVM as on the classic hairball.
12:45 timotimo unfortunately i don't know enough about the data structures to have a good idea where a varint is better than an int32 or regular int
12:46 timotimo like, the static sc id?
13:00 jnap joined #moarvm
13:00 timotimo meh. so many things are calculating nonvariable entry sizes :/
13:02 krunen joined #moarvm
13:07 timotimo a few times i get "using varints saved us 0 bytes!" and then a segfault %)
13:08 FROGGS hehe
13:08 FROGGS nice :o)
13:08 JimmyZ :-)
13:10 timotimo it asplodes in assert_can_read_varint
13:10 timotimo r: print 0x80.base(2)
13:10 camelia rakudo-parrot e51b6c, rakudo-jvm e51b6c, rakudo-moar e51b6c: OUTPUT«10000000»
13:16 timotimo aha. i never get "writing a varint array", but it does try to read a varint array
13:16 timotimo that doesn't suqite make sense
13:25 timotimo Attempt to read past end of string heap (index 5632)
13:25 timotimo :(
14:09 jnap joined #moarvm
14:15 odc joined #moarvm
14:31 woolfy left #moarvm
14:33 woolfy joined #moarvm
14:33 timotimo success!
14:34 woolfy left #moarvm
14:34 timotimo after spending the whole train ride fixing several off by ones and zeros
14:34 timotimo one file during the rakudo compilation said: using varints saved us 1317935 bytes in serialization
14:34 timotimo none of them give me a negative savings :))
14:34 timotimo using varints saved us 493557 bytes in serialization
14:35 timotimo that's the biggest winning file from nqp
14:35 woolfy joined #moarvm
14:36 tadzik that's a full megabyte, nice :)
14:37 tadzik timotimo++
14:37 dalek MoarVM/varint: e51cf28 | (Timo Paulssen)++ | src/6model/serialization. (2 files):
14:37 dalek MoarVM/varint: first step towards using varint in serialization.
14:37 dalek MoarVM/varint:
14:37 dalek MoarVM/varint: serialize arrays of integers with varints
14:37 dalek MoarVM/varint: as well as regular integers
14:37 dalek MoarVM/varint: review: https://github.com/MoarVM/MoarVM/commit/e51cf28b64
14:37 timotimo i could already merge this, actually. but i'd prefer other people testing it a little bit first
14:39 timotimo total 4.8M ← bootstrap files before rebuilding
14:40 timotimo total 4.2M ← after
14:42 timotimo not absolutely stunning, but i wouldn't want to miss it :)
14:44 tadzik 13%-ish better, I'll take it :)
14:49 timotimo aye
14:49 moritz probably doesn't warrant a rebootstrap yet
14:53 timotimo 98680maxresident ← say 1 right now
14:53 timotimo that's a tiny bit less i think
14:54 timotimo AFK for now
15:26 tgt joined #moarvm
16:07 tgt joined #moarvm
16:13 * jnthn waves from Oslo
16:13 jnthn First weather this winter that's been cold enough to justify wearing my winter hat \o/
16:14 FROGGS \o/
16:14 JimmyZ timotimo: there may be some s/int/varint/ in repr serialization/de-serialization too?
16:15 timotimo JimmyZ: maybe, yes. i'll have to make extra sure not to change them in places where the whole system expects a fixed length
16:16 timotimo or maybe change that together with the serialization/deserialization
16:16 jnthn timotimo: As a rule, anything in a segment described as data is a good candidate to apply varint to, and anything in tables is not.
16:17 jnthn timotimo: And I think all writes of data are done through the writer->function mechanism.
16:17 raiph joined #moarvm
16:18 nwc10 jnthn: it's -8 here and snowing
16:18 jnthn nwc10: ooh
16:18 nwc10 and I think that this is the first day this winter that it's got that cold
16:18 jnthn nwc10: That'd qualify
16:18 jnthn nwc10: Exactly the same here, fwiw :)
16:18 jnthn It's not been this cold down in south Sweden...
16:18 jnap joined #moarvm
16:19 FROGGS -11 and sunshine (in theory)
16:23 FROGGS STable conflict detected during deserialization.
16:23 FROGGS aww
16:26 jnthn FROGGS: If you're getting that on one backend but not another, then it's something wrong elsewhere
16:26 FROGGS jnthn: yeah, like twenty lines below the other one :o)
16:27 FROGGS jnthn: the other bit is working, but I won't commit unless you had the chance to review the patch
16:27 FROGGS until*
16:27 jnthn FROGGS: gist it, I had a good idea of what I expected to write...
16:28 jnthn FROGGS: It's one of those things where if things work it's probably right, and if it's wrong things probably SEGV...
16:28 FROGGS https://gist.github.com/FROGGS/cc7e4db40bd18c1aec66
16:29 JimmyZ good night
16:29 FROGGS gnight JimmyZ
16:30 jnthn FROGGS: copy_to is allowed to allocate, so you'll likely want to root backup too
16:30 jnthn FROGGS: And the type object thing matters.
16:30 FROGGS jnthn: yeah, I was not sure if it would allocate or not
16:31 FROGGS ahh, k
16:31 jnthn FROGGS: I'd handle it by doing the current allocate/copy_to thing you do right now if it's not a type object
16:31 timotimo jnthn: i already added a write_varint function (as to not mess up tabular structures)
16:31 FROGGS jnthn: you can commit it if you want to btw
16:31 timotimo so i can 1) use it in reprs directly and 2) just set write_varint to write_int if the version number is too low
16:32 FROGGS (Justin Case)
16:32 jnthn FROGGS: If it is one then MVM_gc_allocate_type_object can be used
16:34 jnthn FROGGS: Hm, and should we be allocating the reposession conflict list...
16:34 jnthn I had thought that got passed in...
16:35 FROGGS jnthn: the last chunk of the patch allocates the list
16:36 jnthn yes, I know, I'm asking if it should
16:36 jnthn FROGGS: no, it souldn't
16:36 jnthn void MVM_serialization_deserialize(MVMThreadContext *tc, MVMSerializationContext *sc, MVMObject *string_heap, MVMObject *codes_static, MVMObject *repo_conflicts, MVMString *data) {
16:36 jnthn It should use the conflicts list that is passed in there
16:37 jnthn Otherwise it just drops the conflicts on the floor, rather than actually passing them off for resolution...
16:39 FROGGS ohh
16:41 timotimo ah. i just switched the NFAs over to varints
16:41 timotimo that ought to make a very big difference.
16:43 jnthn timotimo: oh, yes...very much so
16:47 timotimo i made many changes :)
16:47 timotimo also many in p6opaque
16:47 FROGGS jnthn: gist updated
16:47 timotimo things like storing a "0 or 1" in an int16 will now save yet another byte.
16:49 timotimo ah, i need fallback pointers for 32 and 16 bit integers that are to be read from a previous version
16:52 FROGGS damn, v5 forces me to implement openpipe
16:53 timotimo heh :)
17:14 tgt joined #moarvm
17:15 krunen joined #moarvm
17:18 FROGGS I am unable to provide a code without v5 that show the STable conflict
17:18 FROGGS :o(
17:37 jnthn FROGGS: Well, the only way it should ever happen is if you do something like trying to load two modules that both augment a class in different ways
17:39 timotimo i've added varints throughout the reprs, but i seem to have missed something; i'm getting a segfault :\
17:39 arnsholt jnthn: I had a thought about the way P6opaque lays out things in memory
17:40 arnsholt D'you think it would make sense to order attributes so that they're in order of descending ALIGNOF?
17:40 arnsholt IIRC we just do them in order of declaration ATM
17:41 arnsholt It doesn't look like NQP or Rakudo use much in the way of int{8,16,32} or num32 ATM, but that'd cut some bytes off each instance in classes where they do get used
17:42 timotimo i think i found the issue and fixed it
17:42 timotimo we'll see :)
17:44 timotimo oh no :(
17:58 jnthn arnsholt: Well, at the moment we depend in some places on it laying things out in order
17:59 arnsholt Ah, right. What places depend on it?
18:01 jnthn arnsholt: There's two cases really. One is that in some C / Java code we hard-code some structs or hints to get at stuff.
18:01 jnthn arnsholt: Even if we fixed that, the hint mechanism means you would have to do it within a given class, not over all the fields of the object, otherwise the hints won't be single-inheritance stable...
18:02 jnthn arnsholt: One other consideration, mind, is that sometimes people who are thinking of performance will lay things out in a certain way deliberately, to put commonly accessed things together.
18:02 jnthn arnsholt: Many MoarVM structs are this way.
18:18 arnsholt jnthn: Right, so probably not worth it, at least not at the moment
18:21 V_S_C joined #moarvm
18:22 V_S_C https://gist.github.com/VSChawathe/7c0c7f9f69e4b50e8b79
18:23 V_S_C @jnthn this is with updated rakudo
18:23 tgt joined #moarvm
18:23 V_S_C left #moarvm
18:31 V_S_C joined #moarvm
18:32 V_S_C The errors same as parrot
18:34 V_S_C Only difference is with parrot backend it traced caller
18:36 FROGGS jnthn: I only augment classes in one module, this one indeed gets loaded, maybe even twice
18:49 timotimo i'm not exactly sure how to look for this error :/
18:50 timotimo https://gist.github.com/timo/a711c8450a80683c94e6 ← does this look busted?
18:50 FROGGS jnthn: the problem seems to be a custom postcircumfix declaration O.o
18:52 tgt joined #moarvm
18:53 timotimo i think i'll push my changes to the branch and note that it segfaults
18:57 dalek MoarVM/varint: ce705b0 | (Timo Paulssen)++ | src/6model/reprs/ (5 files):
18:57 dalek MoarVM/varint: teach the 6model reprs about varints
18:57 dalek MoarVM/varint:
18:57 dalek MoarVM/varint: this currently segfaults upon
18:57 dalek MoarVM/varint: deserialization while building nqp.
18:57 dalek MoarVM/varint: review: https://github.com/MoarVM/MoarVM/commit/ce705b098f
18:58 timotimo it seemed like i have replaced every write_int* with a write_varint and every read_int through a read_varint and every read_int* with an if/else to do the proper sized read or a varint read
18:59 timotimo but i must have missed something
19:00 timotimo maybe i wrote something variable-sized into a table somehow?
19:05 timotimo jnthn: maybe you'll see the probably obvious mistake i made when you come back from dinner :)
19:07 diakopter hi
19:07 * diakopter updated vs2013
19:08 diakopter Microsoft (R) C/C++ Optimizing Compiler Version 18.00.21005.1 for x64
19:09 jnap joined #moarvm
19:16 timotimo diakopter: would you like to look at the varint branch?
19:16 diakopter well I finally realized how to do the profiler efficiently so let me finish that first
19:17 diakopter (yes I know I'm slow at design/problem solving)
19:17 timotimo cool :)
19:17 timotimo no, it's fine
19:23 tgt joined #moarvm
19:50 timotimo i see nothing obvious, but maybe i'll find out what's wrong by putting lots and lots of ints through the serialization and back
19:54 diakopter :)
19:54 diakopter .oO( if only we had a way to deserialize stuff )
19:54 timotimo i wanted to do it inside gdb, but it doesn't seem to work properly
19:55 timotimo d'oh
19:55 timotimo i can just put a read directly after the write
19:59 timotimo oh, yeah
19:59 timotimo i did mess up
19:59 timotimo a whole lot.
20:03 jnthn Where does it segv?
20:04 timotimo it seems like i fail at writing out -1
20:07 diakopter jnthn: trying to decide where to put the profiling timestamp hooks - into MVM_args_setup_thunk or every ->invoke repr function
20:07 timotimo i might be messing up my negation mask
20:07 diakopter can't really put it in the frame creation/allocation
20:08 diakopter (I was assuming anyway)
20:09 jnthn diakopter: Why can't it go in MVM_frame_invoke?
20:09 timotimo wait a minute
20:09 timotimo where did my negation mask code go?
20:09 diakopter jnthn: well, sometimes the determination of which frame...
20:09 diakopter oh, I guess we'll just have to imagine that always as included in the parent.
20:09 jnthn timotimo: It met with negation unmask code and was anhiliated
20:10 diakopter or mask mask
20:10 moritz negation masak?
20:13 timotimo hehe
20:13 timotimo no, the negation mask code was on the read side
20:13 timotimo i was looking at the write side
20:13 timotimo turns out the write side wasn't the right side
20:14 diakopter you leftist
20:15 timotimo it's not so easy to get a bitwise print-out of a piece of memory it seems >_<
20:15 diakopter keep shifting by one and anding with 1
20:16 moritz timotimo: (s)printf %b ?
20:16 timotimo what, that exists!?!
20:16 timotimo okay. it turns out i had been looking at the *address* of the value i read, not the value itself
20:16 diakopter :)
20:16 timotimo m)
20:17 timotimo but 127 is still not -1
20:17 moritz timotimo: uhm, at least perl's sprintf understands %b :-)
20:18 timotimo yeah, it turns out the line of code i put in there at one point and then commented out was correct in the end
20:22 diakopter timotimo: of course, the fastest way is a lookup table of 65536 string representations of 16 bits
20:22 timotimo :D
20:22 timotimo sounds like a good idea
20:23 diakopter well, maybe
20:23 * timotimo learns about git reset --patch and is quite happy
20:26 timotimo okay, the bootstrap files are now down to 4.0 mb instead of 4.8 with the varint stuff
20:26 timotimo and now i'll have a look at annotations
20:29 timotimo the stuff that gets read in dissect_bytecode, is it suspected to be of fixed length? doesn't seem so to me. it's probably one-per-file at the very start
20:31 timotimo huh, where does that stuff get written?
20:34 timotimo ah, i see, in serialization.c of course
20:35 timotimo er ...
20:35 timotimo "concatenate_outputs", eh?
20:38 jnthn timotimo: What are you looking for?
20:38 jnthn The stuff read in bytecode.c is written by mast/compiler.c
20:39 timotimo ... oh?
20:39 jnthn It's nothing to do with object serialization.
20:39 timotimo well, i was looking at concatenate_outputs and the dissect_bytecode function
20:39 timotimo which seem to be the two parts of the same piece of data
20:40 jnthn Which concatenate_outputs?
20:40 arnsholt joined #moarvm
20:40 timotimo serialization.c
20:40 diakopter jnthn: should the continuations do anything to th etimings?
20:40 jnthn I suspect the code for emitting bytecode and serializing look superficially quite similar, probably 'cus they were primarily (at least originally) written by the same person.
20:40 timotimo but now i understand why it is the way it is
20:41 jnthn diakopter: Hmm... :)
20:41 jnthn diakopter: Distort them terribly, I should think :P
20:41 timotimo well, it's probably not too helpful to have varints there
20:41 timotimo at least compared to the amount of trouble it is :P
20:42 jnthn timotimo: No, I think the serialization stuff is most helpful
20:43 diakopter jnthn: well, I can make those pause
20:43 jnthn diakopter: That may be best.
20:44 timotimo jnthn: and annotations, yeah?
20:44 jnthn timotimo: Well, maybe annotations too...
20:44 jnthn timotimo: Thing is that if we make them more compact, we make them slower to access
20:44 jnthn I dunno what that means for diakopter++'s current work on profiling.
20:44 timotimo oh, annotations don't get deserialized?
20:44 timotimo they get mmapped, too?
20:44 jnthn timotimo: Yeah, mmap'd and only read on demand.
20:44 timotimo in that case i won't touch them
20:45 timotimo does a full rakudo compile suffice as a test if my serialization stuff is right?
20:45 timotimo does a spectest add anything of value at all?
20:45 diakopter yeah the annotations are, like highly optimally architected
20:45 jnthn timotimo: Please spectest too
20:45 timotimo will do
20:46 timotimo should i leave diagnostic code ifdef'd in the code?
20:46 jnthn timotimo: If you think we're likely to need it.
20:47 jnthn timotimo: Could leave if ifdef'd there for a while and leave a ticket to remove it in a month or two once we don't run into any issues with it :)
20:52 timotimo right
20:52 timotimo first i'm fixing my statistics
20:58 timotimo ah, haha
20:58 timotimo buckets 0 to 9, that's 9 bockets, no? :P
20:58 diakopter jnthn: I'm asserting that microsecond resolution is plenty :)
20:59 * jnthn demands nanos!
20:59 jnthn :P
20:59 diakopter will there be invocations less than a microsecond?
20:59 jnthn At the moment? I rather doubt it :)
21:00 jnthn Was kidding about the nanos, in case it wasn't clear :)
21:00 diakopter hmm well I'll leave it nanos for now, even though many unices and windows don't even have nanos resolution
21:00 diakopter we'll see how fast the gigs grow
21:01 timotimo the vast majority of varints are 2 bytes or 1, a whole bunch are 3, some are 4
21:02 timotimo i've not seen any 8 or 9 byte sized varints during nqp compilation
21:02 diakopter 2 **32 nanoseconds should be enough for any invocation. oh wait.
21:02 diakopter well certainly 2**36 anyway
21:03 diakopter bah.
21:03 moritz 2**64-1 should be enough :-)
21:04 timotimo a few 5 byte big varints showed up just now during stage optimize
21:07 jnthn timotimo: I guess you mean the stage after optimize, when it actually serializes... :)
21:15 diakopter jnthn: what do I do with callsites that don't actually have callsites (the thunk things)
21:15 diakopter [where to mark that callsite as from]
21:15 jnthn diakopter: There's still always a callsite object...
21:15 diakopter yeah but
21:15 diakopter doesn't help the output data trace it back to anything
21:16 jnthn diakopter: Whenever we do one of those we set the return_address in the current frame before doing so.
21:16 jnthn Or am I missing the point?
21:16 jnthn The thunks don't really work too differently from any other invoke, though.
21:16 diakopter yes I'm just wondering how to represent it in the cachegrind visualization
21:17 jnthn Does it add value to, especially?
21:17 timotimo jnthn: er, of course
21:17 diakopter well I just need something other than [someplace on line 44 of xxx.nqp]
21:18 timotimo so, how do i interpret the output of rakudo's spectest now? >_>
21:18 jnthn timotimo: "is it different to before"? :) Or "is it different to daily roast" is a good approximation
21:18 timotimo i should try the latter
21:18 jnthn timotimo: If something fails that didn't in daily roast, you most likley busted something
21:19 diakopter jnthn: I just need some way to describe the callsite
21:19 jnthn diakopter: Would the kind of op that did it be any use?
21:19 jnthn diakopter: There's only so many ops that cause thunks...
21:20 diakopter bah, I'll just use current annotation
21:20 timotimo the output of daily roast has a different format from mine, so it isn't a straight diff
21:20 jnthn "From smrt_strngfy thunk" or so
21:20 jnthn timotimo: true
21:20 timotimo i'll just do a second run with master moar
21:20 diakopter yeah maybe
21:20 diakopter but we don't preserve the current op
21:21 diakopter well it could be re-derived. boo.
21:22 jnthn True
21:26 timotimo a full spectest run in 400 seconds is soooo nice
21:26 timotimo even on my laptop
21:28 diakopter when it gets to 100 seconds it'll be even nicer
21:29 timotimo not a single new failure in S01-S10
21:31 jnthn timotimo: Any chance you could time startup with and without the changes too?
21:32 timotimo well, spectests took 9 seconds less
21:32 timotimo but i listened to music during the second run
21:32 jnthn timotimo: with them?
21:33 timotimo one second.
21:33 dalek MoarVM/varint: eb8dc90 | (Timo Paulssen)++ | src/6model/serialization.c:
21:33 dalek MoarVM/varint: turns out this is actually needed
21:33 dalek MoarVM/varint: review: https://github.com/MoarVM/MoarVM/commit/eb8dc90083
21:34 timotimo without varints, it's about 0.20 seconds, so it's hardly measurable :)
21:34 timotimo well, almost exactly 0.23s elapsed
21:41 timotimo jnthn: there is no measurable difference in startup time, but the maxresidentk for -e "say 1" went from always above 100_000k to never above 100_000k
21:41 jnthn \o/
21:41 jnthn Win, then.
21:42 timotimo more precisely: from never below 100_200k to never above 98_000k
21:42 jnthn A 2.2MB saving. Nice.
21:42 timotimo do i have the captain's permission to perform the merging maneuver?
21:45 timotimo except i called them varint128, which is wrong
21:45 timotimo hm, actually
21:46 jnthn timotimo: Are you confident there's no way to feed crafted input to the varint decoder that will cause a buffer overrun?>
21:46 timotimo p: say 7 * 9
21:46 camelia rakudo-parrot c2982d: OUTPUT«63␤»
21:46 timotimo i think i need to give it special treatment for "there have already been 8 bytes"
21:46 timotimo because then the 8th bit of the last byte must be used as number data rather than a flag wether or not to continue
21:46 timotimo p: say 7 * 8 + 8
21:46 camelia rakudo-parrot c2982d: OUTPUT«64␤»
21:46 timotimo then it checks out again
21:48 tadzik timotimo: where is the tables patch?
21:48 timotimo tadzik: you did look at my pod stuff!
21:48 timotimo i gave it to you and deleted all my copies forever!!!!!k
21:48 tadzik timotimo: I did
21:48 tadzik noooo
21:48 timotimo nah, it's still somewhere
21:48 tadzik I hope you're kidding :P
21:48 timotimo let me dig.
21:49 tadzik gah, rong channel again :)
22:00 timotimo tadzik: https://gist.github.com/timo/6132249
22:02 tadzik *whew*, saved :)
22:03 diakopter jnthn: gah, all the various ways of leaving a frame
22:04 timotimo jnthn: i don't think we've got a sufficiently big number to test the serialization "in the wild", so i'll write a module with big numbers in it :)
22:06 krunen joined #moarvm
22:11 jnap joined #moarvm
22:12 diakopter jnthn: what to do with multi-shot
22:13 dalek MoarVM/varint: 6bb4d26 | (Timo Paulssen)++ | src/6model/serialization.c:
22:13 dalek MoarVM/varint: special treatment for the last byte
22:13 dalek MoarVM/varint:
22:13 dalek MoarVM/varint: we need that to reach full 64 bits, otherwise
22:13 dalek MoarVM/varint: we are short exactly one bit.
22:13 dalek MoarVM/varint:
22:13 dalek MoarVM/varint: and in the 9th bit we already know we're going
22:13 dalek MoarVM/varint: to stop reading anyway.
22:13 dalek MoarVM/varint: review: https://github.com/MoarVM/MoarVM/commit/6bb4d262e0
22:15 jnthn diakopter: All the ways of leaving a frame converge on remove_one_frame though :)
22:15 jnthn diakopter: We don't use multi-shot anywhere in Rakudo
22:15 timotimo dang, i segfault'd again :|
22:15 diakopter except continuation
22:15 jnthn diakopter: Our continuations are one-shot
22:16 jnthn diakopter: uh, sorry
22:16 diakopter doesn't go through remove_one_frame
22:16 timotimo i'll finish this up after sleep &
22:16 diakopter timotimo: 'n
22:16 jnthn diakopter: Our continuations in theory support multi-shot, but they are only used as one-shot
22:16 jnthn diakopter: Right, we don't leave the frame in any sense
22:16 jnthn it's just no longer in the current caller chain.
22:16 diakopter yeah but I need to pause the timers
22:17 diakopter ish
22:17 diakopter hm, can time it in the continuation object itselr
22:17 diakopter itself
22:17 jnthn We walk over all the frames we're suspending in continuations.c at some point, I think
22:17 jnthn So could stop them there maybe
22:21 timotimo for the record
22:22 timotimo writing a normal script to a .moarvm file with --target=mbc --output=foobar.moarvm and then replacing the last piece of the perl6-m runner with that foobar.moarvm should work, yeah?
22:22 timotimo huh. that doesn't even write a single varint128
22:23 jnthn timotimo: should
22:23 timotimo it also doesn't write out the summary of bytes saved and usages per bucket in serialization
22:23 timotimo ... oh wait ...
22:23 timotimo i'm confused
22:24 timotimo huh
22:24 diakopter jnthn: but.. cloning frames...
22:24 jnthn diakopter: the continuationclone op?
22:24 jnthn diakopter: Pretend it ain't there. Again, never actually comes up in Rakudo.
22:24 diakopter why wouldn't cloning frames imply multi-shot
22:24 diakopter oh
22:25 jnthn diakopter: When sorear++ did the continuations stuff for JVM, he made rather more work that we actually needed for gather/take in Rakudo.
22:25 diakopter oh
22:25 timotimo i still kind of wish we could serialize away continuations :|
22:25 jnthn diakopter: I ported the stuff we really need carefully, and the rest is... <handwave>
22:25 jnthn :)
22:26 jnthn timotimo: In theory we could implement that in Moar. In practice, arrrghhh my brane. :)
22:27 timotimo jnthn: did i tell you how much i enjoyed using nagare in pythonland? they use stackless python's serializable continuations to freeze and thaw user sessions to an external database if they've been unused for a bit
22:27 jnthn timotimo: How does that interact with deploying an updated version of the application?
22:27 timotimo you restart it and wipe the sessions
22:28 jnthn Figured as much... :)
22:28 timotimo it's still a cool mechanism for that kind of idea
22:28 timotimo because now you can write your logic as a coroutine and have it persist between sessions without having to be kept in the process' ram
22:29 timotimo displaying two forms that ask for integers and showing one more page to display the result of the number's sum is just like show_result(ask_integer() + ask_integer())
22:32 timotimo A-HA!
22:32 timotimo it segfaults even on moarvm master
22:32 timotimo so my code is not at fault!
22:33 timotimo jnthn: i think my branch is now ready to be merged.
22:33 jnthn timotimo: How did you segfault it,ooc?
22:34 jnthn timotimo: Can you file an issue on it?
22:34 jnthn timotimo: +1 to merge, it seems like an improvement without any real drawbacks
22:34 timotimo echo say "hello world"; >> othertest.pm
22:35 timotimo perl6-m --target=mbc --stagestats --output testlib.moarvm  othertest.pm
22:35 timotimo /home/timo/perl6/rakudo/../install/bin/moar --libpath="/home/timo/perl6/rakudo/../install/languages/nqp/lib" --libpath="/home/timo/perl6/rakudo/../install/languages/perl6/lib" --libpath="/home/timo/perl6/rakudo/../install/languages/perl6/runtime" testlib.moarvm
22:35 jnthn ah, ok
22:35 jnthn Could be a null thing
22:35 timotimo https://gist.github.com/timo/b7ee0a68eb7d819e57e8
22:36 jnthn Loads of test failures seem to be to do with char lookup by name
22:39 FROGGS jnthn: yes, at a given point the name lookup start spitting out wrong codepoints (with a small but increasing offset)
22:40 dalek MoarVM/varint: cc57bd4 | (Timo Paulssen)++ | src/6model/serialization.c:
22:40 dalek MoarVM/varint: varint128 is a bogus name, since it's 9bytes max.
22:40 dalek MoarVM/varint: review: https://github.com/MoarVM/MoarVM/commit/cc57bd4093
22:42 dalek MoarVM: 1eb0650 | (Timo Paulssen)++ | src/6model/serialization.c:
22:42 dalek MoarVM: an implementation of varint reading/writing
22:42 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/1eb0650ce2
22:42 dalek MoarVM: e51cf28 | (Timo Paulssen)++ | src/6model/serialization. (2 files):
22:42 dalek MoarVM: first step towards using varint in serialization.
22:43 dalek joined #moarvm
22:49 diakopter FROGGS: the problem is not just the name lookup; it's also property lookup of those same codepoints
22:52 jnthn OK. Fixing it will cut our failures about in half, anyways... :)
22:52 jnthn Time for some sleep here...teaching tomorrow
22:52 jnthn o/

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary