Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2014-08-05

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:15 ren1us joined #moarvm
01:25 cognome joined #moarvm
01:40 FROGGS_ joined #moarvm
02:01 colomon joined #moarvm
02:19 ventica joined #moarvm
02:50 ventica2 joined #moarvm
02:56 ggoebel111119 joined #moarvm
06:14 brrt joined #moarvm
06:15 brrt timotimo: in the dynasm file they're obligatory (the // comments), because dynasm doesn't understand the other type of comments very well
06:15 brrt in the other file they're temporary :-)
06:37 ventica2 joined #moarvm
06:43 brrt optional params are ... funky
06:44 brrt neither true accessors or branches
07:11 jnthn They either store the provided value and jump, or do nothing at all and fall through to the next instruction, which does the default handling
07:13 sergot o/
07:15 jnthn o/ sergot
07:31 brrt yes, that's funky, isn't it?
07:32 jnthn Yes, fairly. Though also encapsulates an existence check and a retrieval
07:32 brrt i'm pondering adding a special node for it
07:33 brrt can't do much harm i think
07:37 nwc10 brrt: what fixed the big boom bug from yesterday?
07:37 brrt uncommenting the offending lines :-(
07:37 brrt eh, commenting them
07:37 nwc10 oh
07:45 brrt it's become remarkably difficult to actually get a findmeth line
07:46 brrt that's isn't removed by spesh
07:46 nwc10 can you hack spesh to stop remving them?
07:47 brrt probably, yes
07:54 zakharyas joined #moarvm
08:16 ventica2 joined #moarvm
08:32 cognome_ joined #moarvm
08:37 zakharyas joined #moarvm
09:17 brrt joined #moarvm
09:31 zakharyas joined #moarvm
09:32 brrt jnthn: how do frame handlers work, exactly?
09:34 jnthn brrt: The handlers list is an extent list, sorted by most deeply nested first.
09:34 brrt extent list?
09:38 * timotimo gets a jit log for parsing a big json document
09:38 timotimo perf claims it spends 25% in the interpreter loop
09:38 brrt o
09:38 timotimo but the per-instruction heat listing doesn't seem helpful
09:39 brrt 25% is not really that much
09:39 brrt where's the rest of the time spent?
09:40 timotimo process worklist, then frame_invoke, then frame_dec_ref, gc_mark, at_key, gc_collect_free_gen2_unmarked
09:40 timotimo the first gc_mark is from P6opaque
09:40 brrt i'm not so surprised with frame_invoke there
09:41 jnthn Yeha, the VS profiler reckons around 28%
09:41 timotimo the second one from MVMArray
09:41 jnthn We hvae more P6opaques than anything else, so not so surprising that it's the hottest gc_mark function.
09:41 timotimo i'd like to put a little statistics collection thingie in that tells us more about how the program behaves in relation to the GC
09:42 jnthn at_key is also very hot here too
09:42 jnthn I see 5%-6% there
09:42 timotimo as in: after each minor collection, how far - percentage wise - did the pointer move back in relation to the nursery's size
09:43 jnthn Could be interesting to know.
09:44 timotimo and also something about gen2. maybe how many objects of what size got killed each time?
09:57 brrt tl;dr; semantic use of cur_interp_op is not so good for JIT
09:59 colomon joined #moarvm
10:00 jnthn I guess one approach for handlers in the JIT is to use the labels to build an extent list and then look at the return address
10:00 jnthn Given you know where all the handlers are (thanks to annotations)
10:01 brrt yeah, that would basically be the core of it
10:01 brrt but that will still require quite extensive modification in exception handling
10:04 jnthn Yes; I'm not sure I see an easy way out of teaching the exception handling code about the JIT.
10:32 * brrt isn't sure if the following is cruft or much awesome
10:33 brrt http://en.wikipedia.org/wiki/Direction_flag
10:39 * lizmat remembers using that in her "write 8086 assembly code by hand" days
10:46 brrt madness
10:48 lizmat yup, but if you need to work with max 640KB of RAM. you start to do crazy things
11:23 brrt joined #moarvm
11:38 jnthn heh, wow :)
11:38 * jnthn didn't know about that :)
11:39 jnthn Mmm...lunch curry.
11:39 * jnthn realizes he's now eaten curry 3 days running.
11:40 moritz running curry? /me only knows running sushi
11:48 brrt :-o
11:48 brrt 3 days of curry
12:12 dalek MoarVM/moar-jit: dc1b22a | (Bart Wiegmans)++ | / (5 files):
12:12 dalek MoarVM/moar-jit: Add numeric comparison operators
12:12 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/dc1b22a09c
12:18 brrt should an object WHO ever be zero?
12:18 brrt also
12:19 brrt i'm not sure "\340\003a" is really a good name for a repr
12:21 brrt :-p
12:23 jnthn WHO may legit be zero
12:23 jnthn No, tht does not look like a valid RER name :P
12:26 brrt could it be we're really looking at something other than MVMObject?
12:26 jnthn Maybe
12:27 jnthn You could cast it to an MVMStable
12:27 brrt hmm let's try that
12:27 jnthn Did you get this out of a spesh slot, by any chance?
12:27 jnthn Those are MVMCollectable *, meaning you might get an MVMObject * or an MVMSTable *.
12:27 jnthn In the case of th emethod cache I *think* we use two slots
12:27 jnthn One for STable of the type we've cached the result for
12:27 jnthn And one for the method result
12:28 jnthn I mean sp_findmethod here
12:31 brrt could be, difficulty is that it seems to be a returned object
12:31 brrt nah, doesn't look like a STable
12:31 jnthn OK
12:32 brrt basically the whole thing is 0
12:32 jnthn oh
12:32 brrt difficulty is that it doesn't actually crash in the JITted frame
12:32 jnthn What makes the cache entries ooc?
12:32 brrt cache entreis?
12:32 jnthn Yeah
12:32 jnthn What installs the method? That's not JITted, right?
12:33 jnthn I mena, the whole body of sp_findmeth is in interp.c in master, iirc.
12:34 brrt yeah, i'm trying without sp_findmeth first
12:35 brrt just findmeth and findmeth_s
12:35 jnthn ah
12:35 jnthn Are yo uhandle the invokey case of these two correctly?
12:36 jnthn Well, these 3 including sp_findmeth I guess...
12:36 brrt well, they are noted invokish, yes
12:36 brrt and the log shows a invokish guard is compiled, too
12:50 woolfy joined #moarvm
12:51 lizmat_ joined #moarvm
12:59 brrt why doesn't my test script just ... break
13:03 brrt :-(
13:15 avuserow joined #moarvm
13:15 TimToady joined #moarvm
13:15 japhb joined #moarvm
13:17 brrt $?FOO variables, what kind of variables are these?
13:18 [Coke] $?foo       compiler hint variable
13:19 jnthn Just compile as lexicals iirc
13:24 dalek MoarVM/moar-jit: 109b117 | (Bart Wiegmans)++ | / (6 files):
13:24 dalek MoarVM/moar-jit: Move repr id checks to primitives
13:24 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/109b11706e
13:26 brrt hmmm
13:26 brrt weird
14:05 * brrt afk for a bit
14:57 btyler joined #moarvm
15:30 ventica2 joined #moarvm
15:32 ventica3 joined #moarvm
16:06 ren1us joined #moarvm
16:55 klaas-janstol joined #moarvm
17:08 dalek MoarVM: a9906fa | jnthn++ | src/mast/compiler.c:
17:08 dalek MoarVM: Don't hunt for outer frame index if it's cached.
17:08 dalek MoarVM:
17:08 dalek MoarVM: This was, curiously enough, one of the biggest time sinks in the
17:08 dalek MoarVM: entire assembly of large things like CORE.setting.
17:08 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/a9906fa86a
17:17 timotimo oh, interesting
18:04 lizmat joined #moarvm
18:25 dalek MoarVM: a4b0381 | jnthn++ | src/ (5 files):
18:25 dalek MoarVM: Add an int -> str cache for 0..^64.
18:25 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/a4b03818c9
18:28 timotimo does now seem like a good time to look at what b0rks my branch with the lazy wval sc lexical initialization thinie?
18:28 timotimo (yes, that is the scientific term)
18:28 jnthn Hm
18:29 jnthn Well, seems to me like it's a good time for dinner, given I'm hungry
18:29 jnthn But yeah, I can take a look at that after dinner :)
18:29 jnthn Thanks for the reminder.
18:30 nwc10 m: say 6.068e+01/6.1583e+01; say 6.068e+01/6.597e+01
18:30 camelia rakudo-moar 3ad15f: OUTPUT«0.985336862445805␤0.919812035773837␤»
18:30 nwc10 so, ho hum, that's another 1.5% since the last time
18:30 nwc10 and about 8% since a bit ago
18:30 nwc10 which is about as useful as http://xkcd.com/363/
18:30 nwc10 jnthn++ # making things faster
18:31 nwc10 and that's the commit before the one you just pushed
18:36 * TimToady wonders if the JIT is the right place to pick up a divmod optimization when we $x/$y, $x%$y with the same $x and $y
18:36 TimToady since some CPUs can do both of those at once
18:37 nwc10 does Perl 6 code end up doing it often?
18:37 nwc10 I really have no good feel for this.
18:38 TimToady not very, but it's usually time critical when it happens in an algorithm that's more ambitious than decoding time
18:39 TimToady that is, in brute-force factoring algorithms
18:39 nwc10 even if the CPU can't do both at once, is it usually faster to do one of the two, and figure out the other using multiplication?
18:39 nwc10 I fear that that's a bit of a stupid question.
18:40 TimToady could well be
18:40 nwc10 but I'm not sure how to make something that generalises well
18:40 nwc10 both in terms of recognising what the programmer wanted to acchieve
18:40 nwc10 and in terms of propagating that information down in a form that the CPU understands
18:41 nwc10 and I am not jnthn - I have not been working on VMs
18:41 nwc10 (I've been working on "legacy green field code", as someone on #london.pm termed it, and a spec that only appeared about 3 months after the project started)
18:42 nwc10 [we] thank Big Brother for raising the chocolate ration to twenty grammes a week.
18:42 TimToady well, I'm just thinking of something like http://rosettacode.org/wiki/Count_in_factors#Perl_6 compared to the Python solution
18:42 TimToady since they just supply a divmod function outright, which I can argue both ways
18:43 TimToady of course, a JIT would probably handle only native ints
18:44 nwc10 I wonder if there's a python equivalent of grep.cpan.me to see how often the divmod function is used
18:44 nwc10 google-- for canning codesearch
18:45 nwc10 clearly that fits right in with "Google’s mission is to organize the world’s information and make it universally accessible and useful. "
18:45 nwc10 sorry, that's an off topic grumble
18:46 nwc10 although the on-topic part is that grep.cpan.me is built using the open source technology originally built for codesearch
18:46 nwc10 I should go and investigate the sunset
18:53 brrt joined #moarvm
18:53 TimToady well, tommath's mp_div() returns both quotient and remainder, so we should take advantage of that somehow
18:55 brrt TimToady: in fact, div and mod are one and the same op on x86
18:55 brrt iirc, you stuff RAX with the number you want to divide
18:56 brrt you extend rax to an octoword byc combining with rdx
18:57 brrt and division stores the quotient in rax and the remainder in rdx
18:58 TimToady so we can JIT a divmod for natives, anyway
18:58 brrt yes
18:58 TimToady doesn't look like we currently take advantage of tommath's divmod semantics
18:58 brrt it could be something for spesh to do
18:58 brrt div followed by mod of the same operands could be merged into divmod
18:59 brrt which can have a naive implementation for the interpreter and a fast implementation for the JIT
18:59 brrt in fact, i'd personaly recommend that to do on a whole lot of stuff
18:59 TimToady well, in the RC entry I showed, the mod happens first
18:59 brrt doesn't matter semantically :-)
19:00 TimToady with a take in between :)
19:00 nwc10 agree, hence my comment about "recognising what the programmer wanted to acchieve"
19:00 brrt in the original jit tree graph structure that i'd proposed, such situations could also be eliminated using common subexpression elimination
19:00 * jnthn back
19:00 nwc10 you're still tormenting language implementors :-)
19:01 * TimToady tormented myself for years, so I'm used to doing that :)
19:01 TimToady oops, viewpoint shift
19:01 brrt \o jnthn
19:01 brrt curry today?
19:01 TimToady *himself, *he's
19:02 TimToady but I'd like to point out that such torment is why P5 still beats the pants off of P6 most of the time :)
19:03 * brrt isn't sure why TimToady would be glad by that :-)
19:03 TimToady well, except for the bits about redesigning the language to be nearly unimplementable :)
19:03 jnthn brrt: Yes, at lunch time... But dinner was salad with falafel and fetaost
19:03 jnthn uh, feta cheese
19:03 brrt much approve
19:03 nwc10 are your fingers going native?
19:04 jnthn No, that's just what it said on the container it came out of... :)
19:06 nwc10 t/spec/S17-lowlevel/thread.rakudo.moar has SEGVed on me, but as it was an optimised build I have very little to go on
19:06 nwc10 #0  0x00007f0f6c5009b6 in MVM_bytecode_finish_frame () from /home/nicholas/Sandpit/moar/lib/libmoar.so
19:06 nwc10 ...
19:06 nwc10 #8  0x00007f0f6c0f4b6d in clone () from /lib64/libc.so.6
19:07 brrt jnthn, can i pick your brain a bit
19:07 jnthn nwc10: Yeah, I know about a race around there...so I've a good guess what that one is.
19:07 jnthn brrt: sure
19:08 brrt ok, i'm kind of stuck with the findmeth situation, as in, findmeth itself isn't obviously wrong - it's just an invokish c call at my point, and the calls to it seem to be quite correct
19:08 jnthn OK
19:10 jnthn How can I reproduce the issue here?
19:10 brrt so my suspicion is, this has to be somewhere else, but somewhere else is a rather large space
19:10 brrt ehm... let's see what i've committed last
19:11 jnthn srcjitemit_x64.dasc(1426) : warning C4129: 'j' : unrecognized character escape sequence
19:11 jnthn srcjitemit_x64.dasc(1426) : warning C4129: 'e' : unrecognized character escape sequence
19:11 jnthn ffs :)
19:11 jnthn It doesn't escape the filename :)
19:11 brrt basically, open src/jit/graph.c, find the lines in jgb_consume_ins that deal with findmeth and findmeth_s
19:11 brrt don't worry about that, that doesn't do anything
19:11 jnthn I know :)
19:11 jnthn Just warnings.
19:12 brrt you can re-run make dasm_all to have windows-compatible filenames and no warnings
19:12 brrt and uncomment that
19:12 brrt then, build core.setting - which should crash - or nqp - which should throw
19:12 brrt i say should because it's been erratic
19:12 brrt although it has also never been OK
19:15 jnthn BOOM SEGV
19:16 brrt \o/
19:16 brrt but why?
19:16 jnthn You'll have to give me a moment to get an unoptimized debug verison... :)
19:18 brrt :-)
19:18 nwc10 jnthn: OK, cool. Let me know if you want help when you're going to start trying to chase it down. I can't promise that I'll be available, but I'll try
19:18 jnthn OK, it's running under my fave debugger
19:18 jnthn nwc10: Yeah; I'll probably look tomorrow. Planning to look at some other threading/async things then
19:19 jnthn brrt: Well. We explode in decont...
19:19 brrt in decont?
19:19 brrt not mine
19:19 brrt (what's your favorite debugger then?)
19:19 jnthn In the CORE.setting explosion
19:19 jnthn The VS one
19:20 brrt hmm.. haven't figured out how to use VS in a reasonable way for moar, so i'm using windbg directly
19:20 jnthn bah, this memory is total bs
19:21 brrt well, that i see too
19:23 jnthn 0x0000000000a60258 0x0000000000bb0040 0x00000000007a0040
19:23 jnthn oops
19:23 brrt ehm?
19:25 jnthn m: say (0x0000000000a60258 - 0x00000000007a0040) < 4194304
19:25 camelia rakudo-moar 3ad15f: OUTPUT«True␤»
19:25 jnthn hmmm
19:25 jnthn The pointer is into tospace.
19:26 brrt hmmm
19:26 jnthn Which is where we allocate
19:26 brrt so that'd be good
19:26 jnthn Yes.
19:26 jnthn Well
19:26 jnthn If it's wrong then it's basically an extremely outdated pointer.
19:28 jnthn If I'm seeing this right, it does especially suck in so far as it means we screwed more at least 2 GC runs ago.
19:28 jnthn s/more//
19:28 brrt that
19:28 brrt is possible
19:28 brrt but hopefully not very likely
19:28 jnthn I'm afraid it's the best hypothesis I have so far.
19:29 jnthn If we were one GC run out of date the pointer would be into fromspace
19:29 jnthn And invalid pointer in tospace likely means we flipped again
19:29 jnthn Also, it matches in another way: the error. This happens when we're lucky enough to land up on a valid object. However, it's a completely different one than expected.
19:30 brrt uhuh
19:30 brrt ok, that's pretty bad
19:30 * brrt bbiab
19:31 jnthn It is in a sense; otoh, we've a relatively small search space...
19:31 jnthn Provided MVM_JIT_DISABLE makes it work.
19:32 jnthn My first guess - I'll go audit the code now - is that at some point we're putting a pointer on the stack or into a CPU register.
19:32 jnthn And then doing something that causes an allocation.
19:32 jnthn And then we've a bad pointer.
19:32 jnthn Yes, disabling JIT seems to make stuff work.
19:35 nwc10 m: say 5.8165e+01/6.068e+01/6.1583e+01; say 5.8165e+01/6.597e+01
19:35 camelia rakudo-moar 3ad15f: OUTPUT«0.0155652219810724␤0.881688646354403␤»
19:35 nwc10 m: say 5.8165e+01/6.068e+01; say 5.8165e+01/6.597e+01
19:35 camelia rakudo-moar 3ad15f: OUTPUT«0.958553065260382␤0.881688646354403␤»
19:35 nwc10 OK, that's 4% less time from just the last MoarVM commit. If I got that right
19:38 jnthn Wow :)
19:38 nwc10 that's actually what I thought.
19:39 jnthn if (op == MVM_OP_gethow) {
19:39 jnthn | mov TMP1, STABLE:TMP1->HOW;
19:39 jnthn This one isn't The Problem, but it's A Problem now that HOW is lazily deserialized. But that'd just lead us to getting a NULL
19:40 timotimo i saw a segfault above involving bytecode_finish_frame
19:40 timotimo could that need a lock?
19:41 timotimo i mean ... we're updating something very close to the sc
19:41 timotimo rather than something that hangs off of the tc
19:41 timotimo shouldn't be surprising that that can race, eh?
19:41 jnthn Yes, it needs a lock
19:41 timotimo suggest a name and i'll implement it
19:41 timotimo sc->lazy_vivify_lock?
19:42 jnthn That one isn't about the SC; it's about frames
19:42 jnthn I'd hang a lock off the comp_unit
19:42 timotimo OK
19:42 jnthn comp_unit already has a lock for rare things
19:42 jnthn I think
19:42 timotimo i'll look
19:42 jnthn I'd just use that one and generalize its name
19:43 timotimo update_pools_mutex
19:43 timotimo so maybe just "update_mutex"?
19:43 jnthn yeah, that'll do
19:46 jnthn MVM_OP_sp_p6ogetvc_o is vulnerable
19:49 jnthn /* TMP1 now contains address of item */ # I think this should say TMP2, looking at the code
19:50 jnthn But the heart of the issue is
19:50 jnthn | callp &MVM_repr_clone;
19:50 jnthn This can allocate
19:51 jnthn And an allocation at this point would invalidate TMP1 and TMP2
19:51 jnthn Which are both on the stack
19:51 timotimo simply putting the lock before and unlock after the loop over the static lexicals when finishing the frame leads to a deadlock ... oh well
19:52 timotimo valgrind has some tool that helps with deadlocks, odesn't it?
19:52 jnthn It wants to be around the entire finish deserialize of the frame
19:52 timotimo OK
19:52 jnthn With a re-check immediately after taking the lock to make sure we don't re-do work another thread did before we did the check outside of lock
19:53 timotimo OK
19:55 timotimo it seems like we want a re-entrant lock instead or something like that?
19:55 timotimo or a second lock
19:55 timotimo finishing the frame could end up adding strings to the compunit or something, which wants to acquire the lock
19:56 jnthn I don't think that can happen
19:56 jnthn At least, I'd like to see the codepath where it does if you think so... :)
19:56 jnthn 'cus I can't remember one
19:56 jnthn The only thing that adds strings etc is inline.c
19:56 jnthn and we can't be - in a given thread - inlining and deserializing a frame :)
19:57 colomon joined #moarvm
19:57 timotimo OK
19:57 jnthn cvtsi2sd # such op name
19:58 timotimo https://gist.github.com/timo/87a3c075c59a2ff917fc - could this mean the lock got created in a "locked" state?
19:59 timotimo oooooh
19:59 jnthn ohhhh
19:59 jnthn Wow.
19:59 timotimo we're recursively forcing the vivification of things
19:59 jnthn Yes.
19:59 jnthn Wow.
19:59 timotimo due to dependencies
19:59 jnthn Right. I didn't realize that was possible :)
19:59 timotimo (this is in nqp, when building QRegex.nqp)
19:59 jnthn *nod*
19:59 jnthn Well, darn.
20:00 jnthn Well, let's replace it with an MoarVM reentrant mutex object then :)
20:01 jnthn Apart from I dunno if we actually have a type for that yet
20:01 jnthn So may need to do that in bootstrap.c
20:02 timotimo welp, this task just got a whole lot bigger, didn't it?
20:02 jnthn A little ;)
20:03 jnthn But hey, you can look for GC bugs in assembly code instead if you want to swap ;-)
20:03 timotimo hah
20:06 brrt joined #moarvm
20:09 timotimo i don't really know where i could cargo-cult stuff from
20:10 jnthn timotimo: I think it sets up a conc queue type in bootstrap.c
20:10 brrt jnthn ; you're quite a hero right now :-)
20:11 lizmat brrt: fancy giving a talk about your work at the next Amsterdam.PM meeting ?
20:11 timotimo oke, i can look at that
20:11 brrt ehm... when will that be? but i wouldn't mind per se :-)
20:12 lizmat first Tuesday of the month
20:12 lizmat 2 Sep
20:12 timotimo um .. is it really just adding a REPRId, create_stub_boot_type and meta_objectifier?
20:13 timotimo there's a reprid already
20:13 jnthn There's already a repr ID sure
20:13 jnthn The repr exists
20:13 jnthn Just not the type
20:13 jnthn And yes, it's those two plus of course a field to hold it
20:14 brrt jnthn: why would cloning invalidate the pointer to the object?
20:15 jnthn brrt: clone allocates a new object
20:15 jnthn brrt: That could trigger a GC
20:15 jnthn brrt: And the GC moves objects
20:15 brrt lizmat: i'd like to, but i'll have use public transport and get back the same evening
20:15 jnthn Note that in interp.c it actually takes evasive action to prevent against this.
20:16 brrt .... /me checks that out
20:16 brrt differences between interp.c and JIT are the most common source of bugs
20:16 jnthn (basically, a re-fetch)
20:16 brrt nice instruction names, btw, huh :-)
20:16 jnthn Sorry if the comment there was too opaque :)
20:16 brrt especially as SSE is involved
20:17 brrt uh... i have very probably read over that
20:18 * brrt facepalms
20:18 timotimo :)
20:19 brrt no matter, i can do this
20:20 jnthn I *hope* this will be it.
20:21 brrt it's a bug for sure
20:21 brrt and just the sheer quantity of objects allocated by the grammar makes it plausible that this could cause corruption
20:22 klaas-janstol joined #moarvm
20:23 brrt i see that i can probably simplify sp_p6oget_ with conditional moves
20:23 brrt (ARM has conditional everything, x86 has conditional moves, imho these are sufficient for 90% of the benefit)
20:24 brrt but let's focus on bugfixing first
20:24 lizmat brrt: back to Groningen ?
20:24 brrt lizmat: yes :-), i'm not sure but it's likely i'll be studying again
20:24 lizmat :-)
20:26 lizmat seems the last train from Amsterdam CS to Groningen leaves at 23:37
20:27 lizmat but the most comfortable last would be 23:07
20:27 lizmat (arrival 01:23)
20:27 itz_ joined #moarvm
20:27 lizmat so that should work out, I would think!
20:28 brrt yeah, seems fun :-)
20:28 jnthn brrt: Don't suppose you'll make YAPC::EU? :-)
20:29 brrt i haven't arranged anything yet, and my funding is a bit low for a hotel + flight, so no, i'm sorry
20:29 jnthn np, was just wondering.
20:30 jnthn I sadly won't be able to make your talk in Amsterdam.
20:31 jnthn (Got a $dayjob assignment far far away...)
20:32 brrt how far is far?
20:33 jnthn Wrong continent far.
20:33 brrt anyway, i think i'll be arround, though
20:33 brrt oh
20:33 brrt that's far :-)
20:38 brrt pls don't blow up :-o
20:38 [Coke] jnthn: you ever come to the states for $dayjob?
20:39 brrt nope, much blowup
20:39 jnthn [Coke]: Rarely. That's happened once ever...
20:39 jnthn Other times were confs or vacation
20:40 jnthn bbi15
20:40 brrt see you in 15 then :-)
20:41 timotimo brrt: oh well :\
20:41 brrt might be a bug in my fix though
20:42 brrt :-)
20:45 brrt it's a bug, in my fix
20:47 brrt or, the fix made another bug apparant
20:52 timotimo at least we know how to move forward
20:53 brrt hmmmm
20:54 * timotimo has a moarvm that doesn't immediately crash
20:56 brrt \o/
20:56 brrt how do you get that
20:56 timotimo er
20:56 timotimo that's with changing the uv_mutex to a MVMReentrantMutex
20:56 timotimo in the compunit
20:57 timotimo the segv nwc10 mentioned earlier is either not easily reproducible or fixed.
20:58 brrt that.. doesn't seem like it'll be the same as my bug
20:58 dalek MoarVM: ef00cc5 | (Timo Paulssen)++ | src/ (7 files):
20:58 dalek MoarVM: compunit->update_mutex is now a MVMReentrantMutex.
20:58 dalek MoarVM:
20:58 dalek MoarVM: There is also a BOOTReentrantMutex now.
20:58 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ef00cc5900
20:58 timotimo no, definitely not.
20:59 timotimo ^- nwc10, this ought to fix the segfault you saw earlier
21:01 brrt no nqp blowup so far.... much tension
21:02 FROGGS_ and again an NFA debug session takes place \o/ https://gist.github.com/FROGGS/e991e1b3bf8796d91b82
21:04 brrt nqp works, rakudo not yet
21:06 brrt oh, small bug
21:07 timotimo only a small one? :3
21:08 brrt well... lets say i'm sure this bug is a small bug
21:08 brrt the one i just fixed
21:08 brrt that doesn't quite fix the rakudo crash yet, though
21:09 brrt although maybe ASAN now shows us something
21:09 jnthn Does it crash in a different place in Rakudo now?
21:09 timotimo no push yet?
21:11 colomon joined #moarvm
21:11 dalek MoarVM/moar-jit: c06f75e | (Bart Wiegmans)++ | src/jit/ (4 files):
21:11 dalek MoarVM/moar-jit: Reload object and address in case of clone
21:11 dalek MoarVM/moar-jit:
21:11 dalek MoarVM/moar-jit: jnthn++ for pointing out that cloning might
21:11 dalek MoarVM/moar-jit: allocate, which might invalidate our pointer.
21:11 dalek MoarVM/moar-jit: This seems to fix the NQP build, however the
21:11 dalek MoarVM/moar-jit: rakduo build of CORE.setting still crashes.
21:11 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/c06f75edf6
21:17 brrt somethings wrong with my asan
21:28 brrt obj is garbage, again
21:32 brrt hmm
21:32 brrt maybe checkout what create does and if i've implemented that already
21:34 jnthn Only sp_fastcreate so far
21:34 jnthn | callp &MVM_gc_allocate_zeroed;
21:34 jnthn That's called before anything else
21:34 jnthn So it looks correct
21:34 brrt hmmm
21:34 jnthn As in, you don't grab any GC-able objects before that.
21:35 brrt no
21:37 jnthn The patch for the other thing looks correct at first glance too. Hmm
21:38 cxreg anything already in, or thoughts towards, mvm similar to ruby 1.9+'s cow-friendly gc?
21:39 brrt what's a cow-friendly gc
21:39 timotimo it doesn't try to overwrite data in its "old" generations
21:39 timotimo or something like that
21:39 cxreg http://patshaughnessy.net/2012/3/23/why-you-should-be-excited-about-garbage-collection-in-ruby-2-0
21:41 brrt despite my inherent interest in GC algorithm's, tl;dr :-)
21:41 cxreg it uses a bitmap for mark and sweep vs touching the object and triggering a copy-on-write
21:42 cxreg pretty simple; very effective
21:42 timotimo well, we would want to update pointers inside the objects
21:42 brrt i see
21:42 brrt hmmm
21:42 timotimo so we'd have to replace them into our newer pages i guess
21:43 cxreg update the pointers of in-use objects during gc?
21:43 timotimo we have a semispace copying gc
21:43 timotimo that means young objects move back and forth between from- and tospace and then the two of those get switched
21:44 cxreg gotcha
21:44 jnthn Well, for the young generation
21:44 jnthn Second generation doesn't move
21:45 cxreg without something like it, forking is a lot more expensive
21:46 timotimo well, we have a concept for multithreading
21:46 timotimo we don't need to do the "prefork" thing
21:46 timotimo :P
21:46 cxreg so "don't fork, ever" is your genuine feedback?
21:47 timotimo nope
21:47 cxreg i just wanted to point smart people at a neat idea, hopefully it worked
21:47 jnthn cxreg: Read it
21:48 timotimo there's more to having the old objects in pages that we won't ever write over again and then having a bitmap tell us which pieces are still of interest to us
21:48 jnthn cxreg: Objects in the young generation moving is fairly irrelevant for the optimization, in that it's the long-lived ones you're interested in being able to share
21:48 timotimo since whenever an old object has a reference to a young object, that old object would have to be changed
21:49 cxreg jnthn: very likely true
21:49 jnthn cxreg: These days gen2 marks are handled by a bit in the object header. It should be feasible to move it out in the future to a bitmap.
21:49 jnthn cxreg: The math is easyish for a bitmap table since gen2 is arranged into fixed-size pages.
21:50 cxreg timotimo: perfection isn't really the goal, that might be alright.  the odds of an old thing pointing to a new thing are maybe low-ish?
21:50 timotimo not sure
21:50 cxreg i'm sure it's profileable
21:50 jnthn (old -> new) the generational hypothesis assumes this is the rare case
21:50 timotimo fair enough
21:51 jnthn If it wasn't then generational GC wouldn't be worthwhile :)
21:51 jnthn What's curious is it started out as an observation about how most programs behave
21:51 jnthn And now is a performance rule for programmers writing in garbage collected languages. :)
21:51 timotimo that's true %)
21:52 cxreg generational gc you mean?  heh.  where did that originate?
21:52 brrt ok, so decont is actually necessary for the bug to show up
21:52 brrt that's not totally surprising
21:52 timotimo don't we do decont all over the place?
21:53 jnthn cxreg: Hmm, good question. I don't know who wrote the first paper in generational GC. But I'd not be surprised if it was done before I was born. :)
21:54 jnthn Let's see if I can find out from my gc handbook...
21:56 jnthn Seems, perhaps unsurprisingly, it coulda originated with LISP folks :)
21:56 jnthn aha
21:57 * brrt sleep &
21:57 jnthn I think http://www.cs.utexas.edu/users/mckinley/395Tmm-2003/talks/LH83.pdf may be it
21:57 brrt i'll.. go and debug further tomorrow
21:57 jnthn brrt: Yeah, I'll have a look again tomorrow too :)
21:57 timotimo i think i'll quick-and-dirty hack a bit of logging into moarvm's gc to see how much of the nursery is free after each minor collection
21:58 brrt thanks for your help :-)
21:58 brrt i'm planning to change graph loging, btw
21:58 jnthn Or http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=95B522BCDC3C67CBE2B53A5ACE5718FD?doi=10.1.1.123.5055&amp;rep=rep1&amp;type=pdf
21:58 timotimo graph logging?
21:58 brrt yes, the contents of the jit log
21:59 brrt baiscally, i can't really tell the difference easily between compilation with and without certain ops
21:59 jnthn cxreg: Anyway, long story short - no, we don't do the thing Ruby 2 does yet. However, bitmaps can have significant advantages for other reasons too.
21:59 brrt and what happens really often is that some old code path is triggered / misused in new ways.. and i need to find out where
22:00 * brrt nods with regards to the bitmaps argument
22:00 jnthn cxreg: For example, we can avoid even *reading* objects that don't point to other objects.
22:00 jnthn Which could have significant benefits
22:00 brrt although bitfields in objects have their advantages too; i'm not sure how bitmaps and concurrent collection would go together
22:00 jnthn We could even go so far as segregating on both size and touchiness
22:01 jnthn brrt: Yeah, but concurrent collection is...well...argh. :)
22:01 brrt such fun, much amaze
22:01 jnthn brrt: It's certainly not a universal good.
22:01 brrt no
22:03 jnthn cxreg: Anyway, to conclude, I see no reason we can't implement such a thing. Thanks for bringing it up; I'd been considering bitmaps for the other reason I mentioned, but this is an extra bonus.
22:03 * brrt now really afk
22:03 jnthn o/ brrt
22:03 brrt left #moarvm
22:03 jnthn cxreg: Of course, I've had a Windows upbringing and so am more used to thinking in threads that processes. ;)
22:08 cxreg jnthn++ thank
22:08 cxreg thanks! :)
22:14 timotimo aaw
22:14 timotimo no more jit progress before i go to bed? :(
22:17 jnthn timotimo: Only if you patch it ;)
22:27 * jnthn is too tired to do anything more worthwhile today
22:27 jnthn 'night
22:37 cxreg embeddable mvm is still a todo, right?
22:39 cxreg yeah, the main README says so.  ok cool
22:51 timotimo what does "embeddable" mean to you?
22:55 tadzik having a moarvm fakexecutable with the code bundled, I guess
22:56 lizmat joined #moarvm
22:58 timotimo ah, that
22:58 timotimo that's not what i think of when i hear "embeddable"
22:58 timotimo either tadzik or froggs had a prototype a month or two ago
22:58 timotimo but there was still some problem, forgot what.
22:59 cxreg i don't think that describes what I meant
22:59 cxreg I'm talking about sticking moarvm inside something ala libperl
23:00 timotimo oh
23:00 cxreg https://github.com/MoarVM/MoarVM/blob/master/src/README.md references this
23:00 timotimo yeah, that's what i think of when i hear embeddable :)
23:00 timotimo i didn't see that it was tadzik who wrote that
23:00 timotimo you have the same color in my weechat :)
23:00 cxreg heh.
23:01 timotimo well, we currently have a shared library "libmoar.so" and a very thin frontend "moar" that can "only execute moarvm files"
23:01 timotimo so it's actually already exactly what you want
23:01 timotimo and we have made sure to not have any globals, so you can have multiple MVM instances in the same process
23:04 timotimo i kinda wish for some mechanism in mvm that would allow me to supply hooks to implement loading files
23:05 timotimo that way, it'd be easy to compile MVM in a setting like NativeClient where file access needs to be "special" in some kind of way
23:05 timotimo or with emscripten
23:07 timotimo maybe a define flag that kicks out all I/O things libuv has and that moarvm uses
23:07 timotimo sockets, for example
23:08 timotimo oh, the main point was to offer a stable API for embedding
23:08 timotimo we don't have that yet, "obviously"
23:14 klaas-janstol joined #moarvm
23:16 colomon joined #moarvm
23:19 timotimo i may be reading or calculating this wrong, but during nqp's compilation, each nursery collection seems to leave the new nursery at about 3/4 filled?!
23:22 * timotimo double-checks the math
23:23 timotimo fprintf(stderr, "nursery collection left us at %d of %d\n", (char *)tc->nursery_alloc_limit - (char *)tc->nursery_alloc, (char *)tc->nursery_alloc_limit - (char *)tc->nursery_tospace);
23:23 timotimo this ought to be correct, i believe
23:23 timotimo we reset the nursery_tospace before that code
23:27 timotimo most collection in both nqp and rakudo look like this: nursery collection left us at 7604800 of 8388608
23:27 timotimo is it just that we have boatloads of long-lived objects?
23:29 timotimo that's also true for a custom script of mine
23:29 timotimo as in: an application that does stuff :P
23:44 timotimo ooooh
23:44 timotimo that's how much is *free*, not how much is *used*
23:45 timotimo well, that was a major derp
23:45 timotimo you can tell what time of day it is over here ...
23:55 colomon joined #moarvm
23:56 timotimo seems like it never leaves more than 10% alive after a minor collection, usually about 5%
23:58 timotimo .o( doesn't mean i'm not for getting rid of more extremely-short-lived containers and such )

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary