Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2014-01-27

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:13 jnap joined #moarvm
01:52 jnap joined #moarvm
02:00 dalek MoarVM/patch-1: 14c623e | jimmy++ | src/6model/ (2 files):
02:00 dalek MoarVM/patch-1: removed REFVAR_VM_HASH_STR_VAR, and fix MSVC build
02:00 dalek MoarVM/patch-1: review: https://github.com/MoarVM/MoarVM/commit/14c623ee82
02:00 JimmyZ timotimo: ^^
02:14 dalek MoarVM/patch-1: d5a6b8c | jimmy++ | src/6model/ (2 files):
02:14 dalek MoarVM/patch-1: removed REFVAR_VM_HASH_STR_VAR, and fix MSVC build
02:14 dalek MoarVM/patch-1: review: https://github.com/MoarVM/MoarVM/commit/d5a6b8ca11
02:47 ilbot3 joined #moarvm
02:47 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
04:08 lue joined #moarvm
04:59 krunen joined #moarvm
05:02 flussence joined #moarvm
05:04 flussence joined #moarvm
06:57 krunen joined #moarvm
07:35 FROGGS joined #moarvm
07:41 timotimo JimmyZ: msvc doesn't have 0b... literals? :o
07:42 FROGGS timotimo: I learned recently: is has not :o)
07:43 timotimo thank you, JimmyZ
07:43 JimmyZ timotimo: I removed REFVAR_VM_HASH_STR_VAR too, it's not needed in MoarVM after bump version :P
07:44 timotimo i didn't know that. seems good, feel free to merge
07:44 timotimo can i force my gcc to enforce the "define vars only at the beginning" stuff?
07:46 JimmyZ timotimo: feel free to merge if you compile nqp ok, I got problems on windows, not sure if it's because of my patch or not.
07:47 JimmyZ it is: Invalid dependencies table index encountered (index -2139095040)
07:49 timotimo oh
07:49 timotimo i'll have a look
07:49 FROGGS timotimo: maybe try this? -std=c89 -pedantic-errors
07:49 timotimo it could very well be a mistake i made. where does it explode?
07:49 JimmyZ Thanks
07:50 JimmyZ compile NQPCORE.setting
07:50 FROGGS (or -std=c89 -pedantic)
07:50 FROGGS see http://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc/Standards.html
07:51 timotimo thank you
07:51 timotimo 3.4 is a tad old, though :)
07:51 FROGGS I guess the flags are still there :o)
07:51 timotimo fair enough
07:52 timotimo JimmyZ: if you want i'll push my "varint_stats" branch that will verify every varint it writes to make sure it's the same before as after
07:52 timotimo i'm building rakudo-moar right now, so i can build nqp fine :\
07:54 dalek MoarVM/varint_diagnostics: 25f59e1 | (Timo Paulssen)++ | src/6model/ (2 files):
07:54 dalek MoarVM/varint_diagnostics: statistics for varint serialization
07:54 dalek MoarVM/varint_diagnostics: review: https://github.com/MoarVM/MoarVM/commit/25f59e1cee
07:54 dalek MoarVM/varint_diagnostics: 45dedae | (Timo Paulssen)++ | src/6model/serialization.c:
07:54 dalek MoarVM/varint_diagnostics: Merge branch 'varint' into varint_diagnostics
07:54 dalek MoarVM/varint_diagnostics:
07:54 dalek MoarVM/varint_diagnostics: Conflicts:
07:54 dalek MoarVM/varint_diagnostics: src/6model/serialization.c
07:54 dalek MoarVM/varint_diagnostics: review: https://github.com/MoarVM/MoarVM/commit/45dedae07e
07:54 timotimo JimmyZ: ^^ try that
07:54 dalek MoarVM/varint_diagnostics: a2f48d3 | (Timo Paulssen)++ | src/6model/serialization.c:
07:54 dalek MoarVM/varint_diagnostics: more verbosity
07:54 dalek MoarVM/varint_diagnostics: review: https://github.com/MoarVM/MoarVM/commit/a2f48d35af
07:54 dalek MoarVM/varint_diagnostics: af02b68 | (Timo Paulssen)++ | src/6model/ (2 files):
08:04 JimmyZ timotimo: still the same error
08:12 dalek MoarVM: f40b466 | jimmy++ | src/6model/ (2 files):
08:12 dalek MoarVM: removed REFVAR_VM_HASH_STR_VAR, and fix MSVC build
08:12 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f40b466739
08:30 odc joined #moarvm
08:43 arnsholt_ joined #moarvm
08:43 timotimo joined #moarvm
09:07 * JimmyZ doesn't know how to debug into moar.dll by using msvc2012
09:08 JimmyZ Is there a easy way to do it?
09:08 diakopter did you --debug ?
09:08 JimmyZ yeah
09:09 JimmyZ moar.exe is fine. just can't debug  into moar.dll
09:09 JimmyZ Microsoft Visual Studio 2012 Express
09:09 diakopter hm, don't know..
09:12 timotimo i better get rid of my changes until we can figure out what's wrong?
09:12 timotimo but maybe jnthn wants to try building it before we do that
09:12 * diakopter wants to see the branch diff
09:15 timotimo https://github.com/MoarVM/MoarVM/compare/dd7ddb4...cc57bd4 ← diakopter
09:16 JimmyZ timotimo: I think it read int32 as int64 somewhere
09:18 timotimo oof.
09:18 JimmyZ timotimo: I tried to debug into moar.dll and msvc doesn't allow me to do it and I don't know how to ...
09:19 timotimo :o
09:19 timotimo i looked through the diff to make sure i never replaced an int\d\d with a varint directly
09:20 timotimo and at least in reprs/ there are no read_int\d\d left to transform
09:20 timotimo maybe you can find it like that?
09:20 timotimo i'm AFK for a bit now
09:32 * JimmyZ can't find it
09:47 krunen joined #moarvm
09:54 timotimo JimmyZ: just a safety-ask, did you try a complete clean rebuild yet?
09:54 JimmyZ timotimo: yeah
09:54 timotimo i have no idea what's wrong :(
09:55 JimmyZ me either.
09:58 crab2313 joined #moarvm
10:00 JimmyZ timotimo: I think it's a easy fix if someone get a `bt ful` output on msvc :P
10:01 * JimmyZ decommutes
10:24 timotimo maybe i'll tackle storing a 32bit integer for our big ints if we can today
11:05 timotimo now i'll have to lay out p6smallbigintbody so that it matches up its "these must all be 1" field with the mp_digit *dp
11:05 timotimo that should be fairly easy
11:07 timotimo oh, huh, but i need to make extra sure i handle 32bit architectures right
11:15 moritz and big endian :(
11:16 timotimo in this case, i don't think i'll have to
11:16 timotimo oh, wait.
11:16 timotimo i think i know what you mean
11:17 timotimo but in this case i don't care if the int32s end up switched, i'm thinking i should first approximate bigint serializing for "compressed" bigints by turning them into a bigint before serializing
11:17 timotimo since there'll have to be a bounds check on creation anyway
11:17 timotimo hm. and since i don't have to serialize the padding anyway, i think it'd Just Work
11:20 timotimo is there some sane place i can put an assertion like "assert sizeof(MVMP6smallbigintbody) == sizeof(mp_int)?
11:20 timotimo "
11:21 timotimo hm. i could use a value of 1 instead of 0b1111....1111 for the flag, because pointer
11:21 timotimo and then i don't have to #define a constant that'll have the proper amount of ones on each platform
11:23 colomon joined #moarvm
13:32 * timotimo is some ways into the thing and will soon be able to try it out
13:32 timotimo where soon is like ... half an hour to an hour :P
13:35 timotimo src/6model/reprs/P6bigint.h:18:9: error: unknown type name ‘MVMP6smallbigintBody’
13:35 timotimo ... but i declared it just above that?!
13:35 timotimo i even copypasted the name and it's still complaining :o
13:36 FROGGS timotimo: gist?
13:36 timotimo one sec.
13:36 dalek MoarVM/small_big_ints: e8654bb | (Timo Paulssen)++ | src/6model/reprs/P6bigint. (2 files):
13:36 dalek MoarVM/small_big_ints: first progress into smallbigints.
13:36 dalek MoarVM/small_big_ints: review: https://github.com/MoarVM/MoarVM/commit/e8654bb448
13:37 timotimo https://github.com/MoarVM/MoarVM/commit/e8654bb448#diff-6e3ed7dacdfde2d3ae35dabea91aeebaR18
13:38 FROGGS timotimo: maybe you need to add it to: MoarVM/src/types.h:76:typedef struct MVMP6bigintBody MVMP6bigintBody;
13:38 timotimo oh!
13:38 timotimo who would have known >_<
13:38 FROGGS
13:38 FROGGS :P
13:39 timotimo yeah.
13:39 timotimo i was sure i'd have to put a typedef in there for C, but i didn't see one in that file and the only other file that file included was libtommatht
13:40 timotimo how do i manually set the address of a pointer to a specific value?
13:40 timotimo &thepointer = thevalue says i need an lvalue on the left side
13:40 timotimo is it just thepointer = thevalue?
13:40 hoelzro timotimo: yes
13:40 hoelzro assuming thevalue is a pointer
13:41 hoelzro if you want to point to where thevalue is stored, thepointer = &thevalue;
13:41 timotimo no, thevalue is an integer
13:41 timotimo i want to put the integer into the raw storage of the pointer
13:41 timotimo so if thevalue is 0, i get a null pointer
13:42 FROGGS thepointer = &thevalue ?
13:42 FROGGS err, hoelzro mentioned it already
13:42 timotimo
13:42 FROGGS ahh
13:42 FROGGS *thepointer = thevalue
13:42 timotimo ... er
13:42 timotimo doesn't that look where thepointer points and changes the data that it points at?
13:42 FROGGS yes
13:43 timotimo how can i express myself any more clearly?
13:43 FROGGS you want to cast the integer value to a pointer address instead?
13:43 timotimo yes
13:43 FROGGS cast it then, no?
13:43 timotimo that will work?
13:43 timotimo fine, then
13:43 FROGGS should work
13:44 [Coke] (moar had better numbers) yes, but that was when the JVM was failing.
13:44 [Coke] if you check the # of passing tests, I bet it's steadily climbing.
13:46 nwc10 how are the numbers? Is there a web site with a simple graph for the lazy and curious?
13:47 timotimo we have a csv file
13:47 FROGGS nwc10: https://github.com/coke/perl6-roast-data/blob/master/perl6_pass_rates.csv
13:47 FROGGS it is a pure textual graph
13:49 dalek MoarVM/small_big_ints: 80e7764 | (Timo Paulssen)++ | src/ (3 files):
13:49 dalek MoarVM/small_big_ints: first progress into smallbigints.
13:49 dalek MoarVM/small_big_ints: review: https://github.com/MoarVM/MoarVM/commit/80e776480a
13:50 arnsholt Small bigints ^_^
13:50 timotimo :)
13:50 timotimo i expect this to explode horribly when i try to compile stuff
13:51 timotimo but i'll set up some debug output to see what's happening
13:51 [Coke] you can search for "moar" at that URL, which helps a bit.
13:53 arnsholt timotimo: Making bigints for small numbers less memory-hungry, or somesuch I assume?
13:55 timotimo yes
13:55 timotimo if the number fits into a 32bit integer, we store that and then we don't have to mp_init which mallocs
13:55 moritz and moar storage efficient
13:55 timotimo well, at the moment a smallbigint will force itself to become an actual bigint before serialization
13:56 timotimo at some point i can implement that, though
13:56 timotimo how do i factor out the force_bigint and make_smallbigint functions so that i can use it inside the math ops?
13:56 timotimo oh, wait
13:57 timotimo the bigintops act on mp_int rather than P6bigint
13:57 timotimo i'd have to move the code one layer up, that'd probably be in interp.c
13:58 timotimo alternatively i can just introduce the P6smallbigintBody to the ops and do the logic that way with just a cast
13:59 timotimo ah, i was wrong
13:59 timotimo it *does* handle MVMObjects, just further down in the code file
14:05 timotimo arnsholt: the program "say 1" in rakudo takes 98 megs of ram on my machine now; the same with parrot takes about 240
14:10 arnsholt Nice! What's the win relative to master Moar?
14:11 JimmyZ timotimo: union needs a name, solaris doesn't support anonymous union
14:11 timotimo JimmyZ: thank you, i can do that a bit later
14:11 timotimo arnsholt: this is about a change already in master
14:12 timotimo i'm not at the point where smallbigints do anything but segfault in compilation
14:12 timotimo but it used to be about 100.5 mb before the serialization with varints stuff
14:12 timotimo unfortunately there seems to be a bug that makes nqp not compile on JimmyZ's windows
14:12 timotimo :(
14:12 arnsholt Oh, that's annoying
14:13 arnsholt But 2.5 megs shaved off sounds like a pretty good win to me
14:13 JimmyZ well, I think I cant fix it if I know how to debug into maor.dll
14:13 JimmyZ s/cant/can/
14:25 jnap joined #moarvm
14:28 cognominal joined #moarvm
14:43 cognominal joined #moarvm
14:46 timotimo what type do i cast a pointer to if i want it to be an integer value?
14:47 arnsholt long, IIRC
14:47 arnsholt But there be dragons
14:47 diakopter uint_ptr
14:47 timotimo what diakopter said sounds nice.
14:48 timotimo what do i have to include to get that?
14:48 arnsholt Looks like uint_ptr might be a Windows-only type
14:48 diakopter wtypes
14:48 timotimo OK
14:49 timotimo i'm having some difficulty with IS_SBI and SET_SBI >_>
14:49 diakopter er
14:49 arnsholt Why do you need to convert a pointer to an int, though? That's really, really evil
14:50 dalek MoarVM/small_big_ints: 8fd1d5e | (Timo Paulssen)++ | / (6 files):
14:50 dalek MoarVM/small_big_ints: towards a p6smallbigint that does nothing
14:50 dalek MoarVM/small_big_ints:
14:50 dalek MoarVM/small_big_ints: especially not crash.
14:50 dalek MoarVM/small_big_ints: which it currently does.
14:50 dalek MoarVM/small_big_ints: review: https://github.com/MoarVM/MoarVM/commit/8fd1d5eac5
14:50 timotimo i am using the same data region as a pointer to store a flag
14:50 timotimo in the spot where normally there'd be the mp_digit *  i want to have a flag that i can set to 1
14:51 diakopter usually one uses the lowest 2-3 bits
14:51 arnsholt Oh
14:51 arnsholt You just want to set the pointer to some flag value?
14:51 timotimo yes
14:52 timotimo i should probably just use long instead of mp_digit *
14:52 arnsholt In that case you can just set it to a literal, I think
14:53 timotimo i don't understand what i'm doing wrong. i ought to check more stuff i'm doing
14:54 diakopter well in moarvm source, AO_t is already an alias of pointer-sized integer due to libatomicops
14:55 diakopter timotimo: you can also use a union
14:56 timotimo i was using an union :)
14:56 timotimo but apparently too far outside
14:59 timotimo https://gist.github.com/timo/5804138fc21801a3498e
14:59 timotimo the flag is 0x1 and it steps into the else-branch of that ... ?!?
15:07 dalek MoarVM/small_big_ints: 346850c | (Timo Paulssen)++ | src/6model/reprs/P6bigint.h:
15:07 dalek MoarVM/small_big_ints: maybe this is correcter?
15:07 dalek MoarVM/small_big_ints: review: https://github.com/MoarVM/MoarVM/commit/346850cbf4
15:22 timotimo errand&
15:22 timotimo i figured out what's wrong with the code at least.
15:47 jnthn Shouldn't do force_bigint modifying the thing in place. P6bigint must the immutable or we're in for a world of pain when we thread...
15:49 jnthn um, when I try and build NQP on Moar HEAD, I get "Invalid dependencies table index encountered (index -2139095040)"
15:50 diakopter eww
15:51 jnthn It's NQP HEAD also
15:56 jnthn I wish folks would make atomic commits
15:56 jnthn f40b466739 does two things, meaning I can't revert half of it easily to see if it's to blame :/
16:04 timotimo git reset --patch :)
16:05 jnthn Well, reverting the hash_str_ref thing still leaves it broken...
16:06 timotimo JimmyZ mentioned it breaks on nqp for him, but he can't debug into moar.dll
16:06 timotimo it builds cleanly for me on linux, so i can't help debug :(
16:06 jnthn :(
16:07 jnthn Well, going back to before the varint merge works...
16:13 timotimo some of my bit operations may be unsafe on MSVC?
16:13 timotimo i have a branch that reads a varint after it writes it and makes sure the values match up
16:14 timotimo it didn't give any extra output for JimmyZ, though
16:16 jnthn I don't immediately see what's wrong...
16:16 jnthn Tried one thing, but no help
16:16 timotimo i looked the diff over twice :(
16:17 jnthn int read_on = !!(buffer[offset] & 0x80) + 1;
16:17 jnthn huh...
16:17 jnthn What's the !!
16:17 timotimo ensure it's either 0 or 1
16:17 tadzik boolification
16:18 jnthn oh...
16:18 timotimo could have used > 1, too, but that has looser precedence
16:19 timotimo oof. in order to convert the smallbigints, i'll have to carry around the threadcontext and probably steal the type of the MVMObject, too
16:22 jnthn Convert?
16:23 timotimo i mean create new big integers that are actually big integers
16:23 timotimo like you said; they must remain immutable
16:23 jnthn Yeah, but that only needs creating an mp_int...
16:24 timotimo er ... it does?
16:24 timotimo i think i misunderstood you in that case and all is going to be well :)
16:25 timotimo but they are never shared, so all i need to do is have an atomical swap from one pointer to the other and mp_clear the other one?
16:26 jnthn huh
16:26 jnthn There should be no atomic ops needed in this.
16:27 jnthn If the operation needs a bigint the logic should be just like
16:27 timotimo okay. please rephrase this sentence: Shouldn't do force_bigint modifying the thing in place. P6bigint must the immutable or  we're in for a world of pain when we thread...
16:27 jnthn if (it's already big)
16:27 jnthn the_mp_int_to_work_with = directly_from_P6bigint;
16:27 jnthn else
16:28 jnthn the_mp_int_to_work_with = make_mp_int_from(the_32_bit_value);
16:28 jnthn And some tracking so we know the free the mp_int at the end of the operation in the latter caes.
16:28 jnthn *case
16:29 timotimo huh. but then using the same value over and over will cause a conversion from 32bit int to mp_int every time we use it
16:29 jnthn Yes, but that's going to be relatively rare, given most operations (e.g., not is_prime) can be done directly on the 32-bit integers...
16:29 timotimo fair enough.
16:30 timotimo i'll try that then
16:30 jnthn Oh, and you don't need to free track...
16:30 timotimo not exactly sure how to do the tracking properly yet
16:30 jnthn Just keep the mp_int we box to on the stack
16:30 jnthn And the free tracking is just checking if the address matches or so...
16:31 timotimo you mean if the address of the mp_int i got from the force_bigint is the same as TheMVMObject->body.i
16:31 jnthn Yeah
16:31 timotimo that sounds fair
16:31 timotimo thank you for your insight :)
16:31 jnthn Though I think obtain_bigint is maybe a better name.
16:32 jnthn Hopefully most of the time we don't need that, though...
16:32 timotimo yes, that'll be a better name now that it's had its functionality changed
16:32 timotimo that's my hope as well.
16:32 timotimo i just wanted to get to a state where i do have small bigints that are just always forced to become bigints immediately again
16:32 jnthn I mean, I know we will for primality testing and a few other thing, but the common operations (add, etc) don't...
16:32 timotimo so that i can make sure the fallbacks already work and don't segfault :)
16:32 jnthn That's a reasonable way to go.
16:33 timotimo do you see a problem with force_smallbigint?
16:33 jnthn I'm having no luck working out what's wrong with varint :(
16:33 timotimo okay then; revert it and we'll get back to it some time in the future?
16:33 timotimo i could also rewrite the serialization code that it can emit either version 8 or 9
16:34 timotimo and have the current version ifdef decide what it does
16:34 timotimo that way we could get the old, working, but fixed-sized ints on windows and the new, shiny, also working varints on linux
16:34 jnthn I'm sure it's going to work out to something small/weird...
16:34 timotimo that's my fear :(
16:34 jnthn Not immediately sure of the best way to debug it, though...
16:36 jnthn What's the goal of force_smallbigint?
16:42 jnthn timotimo: I added to the end of read_varint9 this:
16:42 jnthn printf("read %d bytes; got %d\n", (int)inner_offset, (int)*value);
16:43 jnthn And see things like
16:43 jnthn read 8 bytes; got 0
16:43 jnthn read 8 bytes; got 8
16:43 jnthn I'm guessing for numbers that size it should...not have read 8 bytes?
16:57 krunen joined #moarvm
16:59 timotimo that's right
17:06 timotimo i'll try it now, too
17:07 timotimo i'm filtering out every output with inner_offset == 1
17:08 timotimo the values here seem correct
17:08 timotimo you should check if the bytes really do have their 8th bit set
17:08 timotimo or if it keeps going regardless
17:09 timotimo assuming we're writing them correctly and the read part is wrong in reading 8 bytes instead of 1, the following bytes must 1) have their 8th bit set and 2) must otherwise be null, otherwise you'd get much different values
17:12 jnthn timotimo: Yeah, it claims it's 128 each time...
17:13 timotimo that doesn't seem right
17:13 timotimo does the writer write properly at all?
17:13 timotimo i do value & 0x7f, so the uppermost bit shouldn't leak through
17:14 timotimo and the other line sets the uppermost bit for all except the needed_bytes-1th bit
17:14 timotimo does needed_bytes give you the expected values?
17:14 jnthn Well, now, the 128 each time is the flag bit
17:15 timotimo yeah
17:15 timotimo does the writer write the value and then a bunch fo 128?!
17:15 timotimo well, the value | 0xf0
17:16 jnthn Writing 0 in 11 bytes
17:16 jnthn oops :)
17:16 jnthn That's from
17:16 jnthn printf("Writing %d in %d bytes\n", (int)value, (int)needed_bytes);
17:17 timotimo wow, yowch!
17:17 timotimo yeah, that's not right at all
17:17 jnthn What's log2 of 0? :)
17:17 timotimo oh ... minus infinity?
17:18 diakopter infinity over zero
17:18 timotimo fortunately i nudge it +1
17:18 timotimo if the value is 0
17:18 timotimo before calculating the log2
17:18 jnthn oh...
17:18 timotimo at least now we know the failure is inside varintsize
17:20 jnthn If I do:
17:20 jnthn printf("%d, %d, %d\n", (int)sign_nudge, (int)varlog, (int)needed_bytes);
17:20 jnthn I get
17:20 jnthn 1, 8100, 1158
17:20 jnthn 1, 128, 19
17:20 jnthn Writing 0 in 19 bytes
17:20 jnap joined #moarvm
17:21 timotimo er .. huh?
17:21 jnthn indeed
17:22 jnthn given it was meant to be being called with the same value each time...
17:22 timotimo stack corruption?
17:23 timotimo undefined behaviour?!
17:23 timotimo ... the ... what ?
17:24 diakopter teh wut
17:27 jnthn Seems it's the log2 to blame...
17:29 jnthn What's odd is that I look it up and it claims it's not supported by MSVC...
17:29 diakopter oh, assumes positive?
17:29 jnthn We're only giving it positive...
17:30 jnthn question is, if math.h with MSVC is supposedly missing log2...where're we getting it from...
17:30 diakopter url to the algorithm?
17:30 diakopter oh
17:32 timotimo it's a function pointer into a random place? :)
17:32 timotimo and that place changes itself in between invocations?
17:33 jnthn no clue
17:33 jnthn size_t varlog = ceil(log(abs(value) + sign_nudge) / log(2));
17:33 jnthn seems to work
17:33 timotimo very well
17:33 timotimo gcc will be able to turn that into a log2 easily, i believe.
17:33 timotimo or even into a "what's the highest bit set?" operation if available
17:34 jnthn tossed my debug code, doing make test...
17:35 jnthn pass
17:35 timotimo thank you kindly
17:36 jnthn I still don't quite know how/where log2 is getting defined to...something.
17:36 timotimo yes, agreed. it's very wacky
17:37 jnthn ah well, at least we don't need to revert it now...
17:39 jnthn Happy Rakudo make test too
17:39 dalek MoarVM: 6558605 | jonathan++ | src/6model/serialization.c:
17:39 dalek MoarVM: Make varint serialization work on MSVC.
17:39 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/6558605326
17:40 jnthn There we go.
17:41 jnthn And yeah, makes a good difference to size of various things
17:45 diakopter :)
17:45 diakopter timotimo++ jnthn++
17:45 diakopter (likely adds negligible unmeasurable cpu time)
17:46 jnthn What we add in CPU instructions we win back in cache locality, I suspect.
17:46 timotimo we only ever run through that thing once, don't we?
17:46 timotimo oh, but we don't trash as many cache lines, i guess
17:47 jnthn Well, we just have to go through less memory
17:48 timotimo how does instruction caching work? is the instruction cache usually separate from the data cache?
17:49 diakopter likely depends on the OS
17:49 jnthn Probably varies...I would imagine it typically is.
17:49 diakopter and config
17:49 jnthn I'd have guessed it's hardware dependent more than OS dependent...
17:50 jnthn I would imagine if it's separate then it's only separate instruction cache and L1 data cache, though, and they both source stuff through L2. Could be wrong...
17:50 timotimo well, anyway. using less memory for the source data before making a round trip on the same instructions again might indeed help us
17:51 timotimo and for things like integer arrays, we get a bunch of small varints in a row
17:53 timotimo i bet someone proficient in C could make that code a whole bunch prettier, too
17:54 timotimo the different parts of the data i used are all over the place
17:57 FROGGS joined #moarvm
17:59 tadzik which MoarVM level structs are used the most when running rakudo-moar?
17:59 tadzik apart from 6model
18:00 tadzik by used the most I mean: allocated most often
18:05 jnthn Probably MVMFrame 'cus we don't hit the frame cache as much as we should
18:06 jnthn (improvable, though)
18:06 timotimo another thing i like hearing :3
18:07 timotimo that would also cause the memory usage to be too big, eh?
18:07 tgt joined #moarvm
18:07 jnthn Well, also malloc/free more than needed
18:07 timotimo i wonder if the february release gets down to 50 MB of ram usage for "say 1" in perl6
18:07 timotimo that would be amazing
18:07 timotimo well, everything else is amazing, too :P
18:09 diakopter jnthn: why missing the frame cache?
18:10 jnthn diakopter: 'cus it's full
18:10 jnthn diakopter: Many frames live longer than they really should.
18:10 jnthn The easy case to fix is lexotic.
18:10 jnthn The tricker one is immediate
18:10 jnthn bbl
18:22 tadzik heh. I tried reordering the struct members in src/6model by size, in order to improve their caching. The performance win, as expected, is zero :P
18:22 tadzik it was worth a try though
18:32 timotimo it may make more sense to group by which attributes are often accessed together
18:39 moritz tadzik: what did you go for? alignment?
18:40 * nwc10 suggests taking a look at http://linux.die.net/man/1/pahole
18:40 nwc10 in debian (at least) it's in package dwaves
18:41 nwc10 coo, MoarVM is fast at building the setting when you're not doing anything "stupid" :-)
18:41 krunen joined #moarvm
18:43 tadzik moritz: yup
18:43 tadzik well, mostly space-saving in the end
18:44 moritz and thus cache saving
18:44 tadzik so if it has MVMint8 and then a pointer, it doesn't waste 56 bytes
18:44 moritz and thus performance, and world saving
18:44 tadzik yeah
18:44 tadzik that was the general idea :)
18:44 nwc10 just use pahole
18:45 tadzik interesting, thanks
18:48 tgt joined #moarvm
19:03 FROGGS[mobile] joined #moarvm
19:19 tgt joined #moarvm
19:37 tgt joined #moarvm
19:54 nwc10 aarrrrrrrrrrgh. zillions of bloody errors under valgrind. Not Good.
19:54 timotimo :(
19:55 nwc10 including, but certainly not limited to, varintsize
19:56 timotimo varintsize? oops :)
19:56 timotimo that'd be me
19:56 nwc10 not totally - it's being fed bogus data
19:57 nwc10 writer->write_varint(tc, writer, repr_data->unbox_slots[i].repr_id);
19:57 nwc10 I think that repr_data->unbox_slots[i].repr_id is the culprit
20:03 nwc10 yaks :-(
20:06 nwc10 jnthn: when it comes to serialize_repr_data, are all of repr_data->unbox_slots supposed to be initialised?
20:08 diakopter if they're not currently, sounds like they should be
20:08 diakopter <- captain obvious, sorry :)
20:12 FROGGS jnthn: am I allowed to commit that? https://gist.github.com/FROGGS/cc7e4db40bd18c1aec66
20:14 nwc10 jnthn: specifically, static void compose in P6opaque.c does repr_data->unbox_slots = (MVMP6opaqueBoxedTypeMap *)malloc(total_attrs * sizeof(MVMP6opaqueBoxedTypeMap));
20:14 nwc10 jnthn: and in this case, total_attrs is 3, but  only the first is used, so the rest are garbage memroy
20:14 nwc10 memroy
20:14 nwc10 gah
20:15 jnthn nwc10: I'm not sure if we look at the invalid ones to use them...
20:15 nwc10 serialise (attempts to) serialise garbage memory
20:15 jnthn Yeah. It's wasteful now we've varint.
20:15 nwc10 which gives a ton of crap from valgrind, which gets in the way of working out if something else is unhappy.
20:15 jnthn Since a 0 would encode much smaller
20:16 nwc10 I think I can make it a 0 with calloc()
20:16 jnthn That'll do too
20:16 nwc10 also not sure why the code likes malloc() and memset()
20:16 nwc10 as best I can tell from the source of jemalloc() (used at least by FreeBSD() calloc() is no real advantage over malloc() [other than maybe less code size]
20:17 nwc10 but I thought that one some OSes, calloc() would track if memory was already zeroed, and avoid work
20:17 diakopter nwc10: jnthn explained he preferred malloc/memset because later we might need to change what it memset to - ie for pointers to the null pmc
20:17 nwc10 also, serialising crap means that I can't binary compare the setting to see if the build is unchanged
20:17 nwc10 aha
20:17 jnthn Yes, that holds true in various cases
20:18 nwc10 but I think you can searcn & destroy^Wreplace on calloc() just as easily as memset()
20:18 diakopter hm, true
20:18 diakopter afk&
20:19 jnthn FROGGS: Looks good to me
20:19 FROGGS cool, thanks
20:21 dalek MoarVM: f22a386 | (Tobias Leich)++ | src/6model/serialization.c:
20:21 dalek MoarVM: port reposession conflict resolution for objects from nqp-p
20:21 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f22a386691
20:23 jnthn FROGGS: How's v5 now?
20:23 FROGGS jnthn: stumbling about the STable still :/
20:23 jnthn FROGGS: I guess it doesn't do that on Parrot?
20:24 FROGGS I am bisecting Perl5::Terms atm, but it looks like it explodes when a critical number of mixins is reached
20:25 FROGGS jnthn: correct
20:29 FROGGS m: multi infix:«P5<=>»(\a, \b) is export { nqp::p6box_i(nqp::cmp_I(nqp::decont(&prefix:<P5+>(a)), nqp::decont(&prefix:<P5+>(b)))) }
20:29 camelia rakudo-moar acf7bd: OUTPUT«[31m===[0mSORRY![31m===[0m Error while compiling /tmp/JkcPUJdup2␤Undeclared name:␤    &prefix:<P5+> used at line 1␤␤»
20:30 FROGGS m: sub prefix:<P5+>(\a) { }; multi infix:«P5<=>»(\a, \b) is export { nqp::p6box_i(nqp::cmp_I(nqp::decont(&prefix:<P5+>(a)), nqp::decont(&prefix:<P5+>(b)))) }
20:30 camelia rakudo-moar acf7bd: ( no output )
20:37 FROGGS yeah, it *is* about some sort of critical mass
20:38 FROGGS it does not matter what infix/prefix I uncomment, either breaks the code
20:38 FROGGS I can precompile Perl5::Terms (the one with the infixes), and I can precompile Perl5::Config, which uses it
20:38 FROGGS but when I load Config, it asplodes
20:41 FROGGS funny thing is that I can uncomment subs but not multis
20:41 FROGGS jnthn: where do I need to bump MAX_MULTI_COUNT_PER_CU ?
20:43 FROGGS btw, I do not declare any proto
20:44 jnthn Max...multi count? Huh?
20:44 * jnthn didn't know that existed...
20:45 FROGGS well, it does not... (officially)
20:45 FROGGS but... why does it explode?
20:45 FROGGS can you make sense of it whatsoever?
20:46 jnthn No, though one thing you may wish to do is to add something into the STable write barrier in sc.c and see when it gets hit
20:47 jnthn When I was tracking donw the others I just shoved an MVM_exception_throw_adhoc into there
20:48 FROGGS yeah, printing a backtrace (without dying) is also helpful
20:48 FROGGS I remember I did that in interp.c once or twice
20:48 jnthn also that, yeah
20:50 FROGGS hmmm, it already hits, even for the working-code-case
20:51 FROGGS when loading the setting or so
20:53 jnthn yeah, the problem tends to show up downstream though
20:53 jnthn Where's it hit?
20:53 jnthn And which setting? The Rakudo one?
20:55 FROGGS https://gist.github.com/FROGGS/75012929138cb207b35e
20:55 FROGGS rakudo's yeah
20:56 FROGGS that is a backtrace for every time MVM_sc_wb_hit_obj is called... that was right, right?
20:57 jnthn oh, I thought you were going for the _st one, since it's an STable conflict?
20:58 FROGGS ahh :o)
20:59 FROGGS that looks different, yes: https://gist.github.com/FROGGS/75012929138cb207b35e
21:01 FROGGS this is the offending line btw: multi infix:<P5cmp>(int $a, int $b) is export { nqp::p6box_i(nqp::cmp_i($a, $b)) }
21:01 FROGGS and the last bt shows that pretty well
21:04 jnthn hmm, i wonder...
21:13 jnthn FROGGS: Does v5 work under JVM, ooc?
21:15 FROGGS dunno
21:15 FROGGS need to create a Makefile.in for it first
21:15 FROGGS but I can do that (tomorrow)
21:19 timotimo what, it'll take you more than 1.75 hours? :)
21:21 FROGGS no, it will take less, but I am too sleepy to do that right today :o)
21:27 * FROGGS does it now
21:27 FROGGS dah!
21:27 FROGGS gah*
21:36 jnap joined #moarvm
21:46 dalek MoarVM: ae04515 | jonathan++ | src/io/fileops.c:
21:46 dalek MoarVM: Don't try to sync things that can't.
21:46 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ae04515100
21:48 FROGGS cool
21:49 FROGGS hmmm, v5-j explodes because it can't find Perl6::Pod
21:53 FROGGS jnthn: tomorrow by that time I will have v5 on jvm I guess...
21:53 FROGGS but now: sleep
21:53 FROGGS gnight all
21:56 timotimo jnthn: i just got some $workload for $dayjob, so we'll see if i'll get to something semi-working for smallbigint tomorrow
21:57 jnthn timotimo: OK, no hurry. I'm quite distracted with $dayjob these next days also.
21:57 timotimo d'aaw
21:58 timotimo well, i don.t think i can keep my fingers off of rakudo and moarvm for long :3
21:58 jnthn :)
22:37 eternaleye joined #moarvm
23:24 dalek joined #moarvm
23:47 colomon_ joined #moarvm
23:55 krunen joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary