Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2013-11-30

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:25 timotimo well, i could just implement GatherIter! that's probably totally easy and simple!!!!!kkk
01:06 benabik joined #moarvm
02:24 Ben_Goldberg joined #moarvm
08:08 woolfy left #moarvm
08:18 tadzik joined #moarvm
09:01 tadzik joined #moarvm
09:19 tadzik joined #moarvm
09:40 tadzik joined #moarvm
11:37 tgt joined #moarvm
11:47 tadzik joined #moarvm
11:53 tgt joined #moarvm
12:44 jnthn timotimo: To help with GatherIter, just implement continuations...
12:46 timotimo sometimes i implement ten continuations before breakfast!
12:52 FROGGS[mobile] I implemented continuations in PHP and VBA for fun!
12:55 jnthn take_a_continuation_here()
12:56 jnthn So, no dalek reported them, but last night I pushed two commits starting to work on the serialization reposession stuff.
12:58 FROGGS[mobile] Ohh, nice
12:58 FROGGS[mobile] I have no tuits till late evening
12:59 jnthn I have some tuits this afternoon, but am kinda in recovery mode from the last weeks being busy...
13:01 jnthn So, how did repo work... :)
13:03 FROGGS[mobile] Yeah, I feel the same... >8h work plus >4h hackory plus kids is not ideal when you have a cold for weeks
13:04 FROGGS[mobile] I actually started last night glueing it to panda
13:04 FROGGS[mobile] Works nice so far
13:05 jnthn Oh, I was talking about repossession... :)
13:05 jnthn But cool that *that* repo work is going well... :)
13:05 FROGGS[mobile] Ahh, w/e :o)
14:51 FROGGS[mobile] joined #moarvm
14:52 FROGGS[mobile] joined #moarvm
15:05 grondilu joined #moarvm
15:12 jnthn Hm, a half-implemented stable reposession thing actually gets us further...
15:13 jnthn Oh...I can guess why. But we really need the other bit. :)
15:15 timotimo at least it's stable
15:16 jnthn :P
15:39 jnthn "Gaps in STable root set not allowed"
15:39 jnthn OH YES IT IS
15:42 diakopter ?
15:43 jnthn diakopter: I'm just implementing stuff
15:43 jnthn And running into my own lack of foresight. Or maybe laziness :)
15:46 jnthn Seems we reach line 2560 when loading the setting now...
16:08 jnthn Implemented slurpy/LoL-slurpy biding, but then the same thing soon runs into needed nqp::p6listitems doing. :)
16:10 FROGGS[mobile] slurpy biding?
16:10 jnthn *@foo
16:10 FROGGS[mobile] sounds like I am missing all the fun
16:32 jnthn p6listitems done
16:32 jnthn Next: p6arrfindtypes NYI
16:38 nwc10 when I run it under valgrind, I'm getting grumblings about the base64 writing routine
16:38 nwc10 simplest fix - get rid of base64? :-)
16:39 jnthn That's on my hit list at some point in the not too distant future.
16:39 jnthn We already spec'd the alternative storage mechanism in the bytecode file, after all...
16:39 FROGGS[mobile] does that play well with MVMString?
16:39 FROGGS[mobile] ahh
16:40 jnthn Well, the idea is to just shove the binary blob directly into a segment of the bytecode file.
16:42 timotimo that should improve our bytecode file sizes significantly
16:43 jnthn Aye.
16:43 jnthn And our startup time. :)
16:44 timotimo and our valgrind time
16:44 jnthn Though sorting out the bottlenecks in CORE.setting compilation is more pressing.
16:45 jnthn I need to focus on some indy stuff on Mon/Tue though.
16:45 nwc10 is killing base64 clear enough and sufficiently spec'd that someone else could do it?
16:45 jnthn Not really...
16:46 jnthn I'm not quite sure how it should look...
16:46 jnthn Like, the getting the blob between serializer and file...
16:48 timotimo stash it away until a later point, together with a stubbed forwards reference, then when emitting, spit out the data and go back in the file to fix the reference? :\
16:48 timotimo that sounds bad.
16:49 jnthn Yes, it does :P
16:49 jnthn So yeah, it's more a design task...
16:49 timotimo emit a length field and skip that many bytes when deserializing?
16:49 jnthn p6arrfindtypes done
16:49 timotimo \o/
16:49 jnthn timotimo: We probably want to just pass a compunit into the deserializer or something in that direction...
16:50 jnthn The other direction - I guess a byte array could do it...
16:50 jnthn P6opaque: no such attribute '$!storage'
16:50 jnthn Hmm. Innerestin...
17:12 jnthn d'oh...
17:12 jnthn Think I found it.
17:12 jnthn port-o
17:19 jnthn OK, up to line 3239 of the setting load now
17:19 jnthn p6setautothreader NYI
17:21 timotimo mhhh junctions
17:21 timotimo setautothreader just passes a reference to a sub to a statically known location, so that the dispatcher can find it when a dispatch fails?
17:21 timotimo you'll be able to implement that in a minute or two :)
17:21 jnthn Yeah, it won't take long
17:22 jnthn But I also should head out soon
17:22 timotimo maybe i'll do it for you :)
17:22 timotimo a few hints and i'll see what i can do
17:22 jnthn Meeting a friend for l'öl
17:22 jnthn Well, it's a binder thing
17:23 jnthn So it wants handling by turning it into a method call on the Binder in BOOTSTRAP
17:23 jnthn Which can then stash it
17:23 TimToady .oO(a 3-ring circus binder)
17:23 jnthn In a lexical.
17:24 timotimo that BOOTSTRAP is backend-agnostic?
17:24 jnthn ish
17:24 jnthn The bit in question is maor only
17:25 jnthn Writing the binder in C on Moar would be an epic disaster for my sanity... :)
17:25 jnthn Nobody should do that much CPS transforming by hand... :)
17:25 timotimo hah, ok
17:25 TimToady 'course, you wanna talk about hot spots then...
17:26 jnthn TimToady: Well, the Evil Plan is to optimize away the full-blown binder in the common case of signature binding.
17:26 TimToady sometimes sanity is overrated
17:26 jnthn TimToady: So if you end up in the full thing you're already going to be doing lots of more costly sutff anyway.
17:26 * TimToady watches out for excess foil
17:27 jnthn I'll probably actually work on that optimizing away this week, given I probably need it to for taking advantage of the indy stuff I'm plotting.
17:27 jnthn Nice that the need to do it for JVM means we get it for Moar too. :)
17:28 TimToady indystuff++
17:28 timotimo \o/
17:28 jnthn Anyway, by putting the binder into BOOTSTRAP, the symbols are all handily available and once we get the specializer stuff into Moar it should be able to do nice things on this code. It's also full of native types where possible, which is nice for JIT when we get that. :)
17:28 TimToady reminder to folks following along at home: "indy" is short for invoke dynamic
17:28 timotimo \o/
17:29 timotimo so, er, i could just implement p6setautothreader with QAST instead of C, right?
17:30 timotimo can i just find Binder with Var/WVal?
17:30 jnthn timotimo: oh, totally
17:30 timotimo it'll be a lexical, aye?
17:30 jnthn timotimo: Yes, see many other examples in src/vm/moar/Perl6/Ops.nqp
17:31 jnthn Aye, inside class Binder
17:31 timotimo perfect :)
17:32 timotimo i meant Binder, rather than the lexical for the autothreader
17:32 timotimo the latter is already a my $autothreader inside the Binder class
17:33 arnsholt What's invoke dynamic entail on Moar, OOC?
17:33 jnthn arnsholt: invoke dynamic is a JVM thing, not a Moar thing.
17:34 jnthn arnsholt: It's used to teach the JVM about other languages and how they work, so its JIT can make sensibler decisions about that to do with them.
17:34 jnthn Moar is just built with 6model stuff right in the core
17:34 timotimo do i need to update moar and nqp to get from 7c81a62..7832310?
17:34 jnthn You certainly need a new moar
17:34 timotimo OK
17:35 TimToady do any of the VMs allow storage layout at the bit level yet?
17:35 TimToady s/allow/provide/ :)
17:35 jnthn Not from Perl 6 land...I suspect the JVM must have classes in the library for bit buffers...
17:37 TimToady and I suppose we also don't have compact int arrays smaller than 8 bits yet either
17:37 timotimo i don't think we do, no
17:37 jnthn No
17:38 jnthn Well, we don't have native arrays anywhere yet really.
17:38 TimToady btw, who actually does commit bits? I have a combinations routine that runs 6 times faster
17:38 jnthn Buf and Blob are like that
17:38 jnthn Oh wow...
17:38 jnthn TimToady: github ID?
17:38 TimToady TimToady
17:38 jnthn lemme see if I can
17:38 timotimo oh that seems like a nice addition :3
17:39 TimToady well, I showed it to quietfanatic++, and he said "Why don't you make the recursive solution faster instead?"
17:39 jnthn TimToady: You should be able to push now
17:39 timotimo hah :)
17:40 timotimo well, when jnthn makes the binder "inlinable" on jvm and moar, recursive stuff should be a lot faster for everything, no?
17:40 TimToady oops, pushed to moar-support instead of nom
17:40 TimToady oh well, I guess it'll all end up the same place
17:40 timotimo eventually, sure
17:41 timotimo Internal error: invalid thread ID in GC work pass
17:41 timotimo ... huh
17:41 jnthn Yeah. Well, merge early, merge often...once you can "say 'hi'" on Moar, there'll be little reason not to merge the moar-support branch, provided it regresses no other backend.
17:41 jnthn timotimo: That error tends to mean "memory corruption".
17:41 timotimo yeah :(
17:41 timotimo so re-build without --optimize?
17:41 timotimo (trying a clean first)
17:44 diakopter .
17:44 jnthn gotta go now...bbl
17:44 jnthn &
17:45 diakopter :'(
17:52 FROGGS[mobile] joined #moarvm
17:58 cognominal joined #moarvm
18:06 timotimo this is taking sooo looong :\
18:06 timotimo 270 seconds just to parse
18:07 diakopter boo
18:07 timotimo soon i'll probably be able to implement the charrange codegen on moarvm, that'll give us ... half a second?
18:08 FROGGS[mobile] hehe
18:09 timotimo Error while compiling op p6setautothreader: Cannot invoke null object  -  oh no! :(
18:10 timotimo Var.new( :name('Binder'), :scope('lexical') ) didn't work :(
18:10 timotimo oh, wait
18:10 timotimo this is during generating the QAST
18:10 timotimo not running it
18:11 timotimo oh well. i'll work on it some more in ~30 minutes
18:30 ssutch joined #moarvm
18:34 colomon joined #moarvm
18:36 timotimo i don't understand why i'm getting "cannot invoke null object"
18:40 timotimo https://gist.github.com/timo/4232f378ec4d07f9eb8e - it's probably very obvious, but i'm not getting it
18:41 timotimo FROGGS[mobile]: an idea?
18:41 timotimo i've removed the op from the other files it existed in already
18:42 FROGGS[mobile] timotimo: getting kids to bed, no time atm
18:42 timotimo no problem
18:45 * timotimo tries adding a :want to as_mast
18:46 TimToady .oO(Two Years Before the Mast)
18:46 timotimo i don't understand that reference :(
18:47 TimToady http://en.wikipedia.org/wiki/Two_Years_Before_the_Mast
18:51 FROGGS[mobile] joined #moarvm
18:51 timotimo ah
18:54 timotimo oh, duh
18:55 timotimo of course Binder isn't available in the lexical scope
18:55 timotimo of the setting %)
18:55 FROGGS[mobile] Maybe WVal?
18:55 timotimo yup, that's what i have now
18:55 timotimo will take like 5 minutes to try, though
18:56 FROGGS[mobile] yeah :/
18:59 timotimo Segmentation fault (core dumped)
18:59 timotimo i wonder why.
18:59 timotimo but now i've changed the op's implementation anyway
18:59 timotimo another 10 minutes of compilation ... :|
18:59 FROGGS[mobile] you are using $*W.find_sym?
19:00 timotimo no
19:00 timotimo i'm getting it directly from nqp-land
19:00 timotimo which may very well be wrong
19:00 timotimo but i won't have it in the world, will i?
19:01 timotimo because $*W.find_sym is what gets used when you have a Var with :scope<lexical>
19:01 timotimo afaict
19:01 FROGGS[mobile] no, dont think you get lexicals that way
19:02 timotimo oh?
19:02 timotimo oh, there's $Binder
19:03 timotimo this time i'm relatively certain i have the right incantation
19:07 FROGGS joined #moarvm
19:10 timotimo 5 minutes to go
19:13 FROGGS timotimo: why don't you steal from p6bindsig?
19:13 timotimo that's pretty much what i have now
19:13 FROGGS k
19:16 FROGGS jnthn: here I temp_push but I never pop, where should I do that? https://github.com/MoarVM/MoarVM/commit/05eaefe8431547a1b7eabfed44b959c3051ba4d2#diff-db8462d383c5c121ed6763f510fa4804R62
19:17 FROGGS ohh, dalek is still absent :/
19:18 timotimo er, don't you perm_add there?
19:18 FROGGS ohh, I do, right
19:18 FROGGS I hope that is fine then :o)
19:18 FROGGS --vmlibs=dynext/libperl6_ops_moar.so=Rakudo_ops_init src/gen/m-BOOTSTRAP.nqp
19:18 FROGGS Internal error: invalid thread ID in GC work pass
19:18 timotimo m)  i made sub set_autothreader instead of method set_autothreader
19:19 FROGGS eww
19:19 FROGGS does it fail for you too that way?
19:19 FROGGS I guess not
19:19 timotimo yes, i tfails for me that way as well
19:19 timotimo that's why i don't have --optimize set
19:20 FROGGS ahh
19:20 FROGGS good to know
19:20 timotimo feel free to valgrind if you have a few hours to spare :P
19:20 FROGGS *g*
19:21 FROGGS I might do that later, yes
19:23 FROGGS damnit, I do that now
19:29 timotimo P6opaque: get_boxed_ref could not unbox for the given representation
19:29 timotimo at src/gen/m-CORE.setting:4499  (./CORE.setting.moarvm:frame_name_1647:11708)
19:30 FROGGS that sounds good
19:30 timotimo up from 3234 before i implemented setautothreader
19:30 FROGGS timotimo++
19:31 timotimo i didn't implement the actual autothreading fallback, though ;)
19:32 FROGGS hmmm, but an XXX there? :P
19:33 timotimo there is an XXX?
19:34 FROGGS O.o
19:34 FROGGS just put a comment there if it is too hard
19:35 FROGGS or let it die with an NYI when it hits that branh
19:35 FROGGS branch*
19:36 timotimo it already does :)
19:37 FROGGS is it pushed then? *g*
19:38 timotimo yes
19:38 FROGGS :o)
20:26 FROGGS umm, there is such an object in the worklist: {header = {owner = 27885928, flags = 0, size = 0, forwarder = 0x0, sc = 0x28000000000000}, st = 0x0}
20:27 FROGGS "sending a handle 0x1a9a958 to object 0x7ffff6ac85d8 to thread 27885928"
20:36 timotimo i don't understand ...
20:36 FROGGS this object has a threadid of 27885928
20:36 FROGGS that is why it is blowing up
20:36 FROGGS should be 0
20:37 timotimo ah
20:37 timotimo oops :)
21:08 timotimo FROGGS: are you about to fix it? should i try to valgrind as well?
21:09 FROGGS valgrind does not help
21:09 timotimo i can't spend a lot of attention on it, though
21:09 timotimo mhk
21:09 FROGGS gdb does a bit, but I am not sure what to do
21:09 timotimo :(
21:10 FROGGS I mean, it looks like we pass a pointer of the thing we actually want to add to the worklist
21:10 ggoebel110 joined #moarvm
21:10 FROGGS but I am not sure how likely that is
21:11 FROGGS the good thing is, that pointer is always the same, even when you rebuild moarvm
21:11 timotimo you can search the whole binary for that value :)
21:11 FROGGS I am bisecting now because I am out of ideas
21:11 FROGGS :o)
21:17 * jnthn back
21:17 FROGGS o/
21:18 tgt joined #moarvm
21:26 FROGGS ó.Ò
21:26 FROGGS this is to blame: https://github.com/MoarVM/MoarVM/commit/05eaefe8431547a1b7eabfed44b959c3051ba4d2
21:26 FROGGS jnthn: can you review that patch please? https://github.com/MoarVM/MoarVM/commit/05eaefe8431547a1b7eabfed44b959c3051ba4d2
21:28 FROGGS yeah, it must be that: https://github.com/MoarVM/MoarVM/commit/05eaefe8431547a1b7eabfed44b959c3051ba4d2#diff-db8462d383c5c121ed6763f510fa4804R61
21:28 FROGGS if I comment that out, the error goes away
21:45 jnthn FROGGS: wrt https://github.com/MoarVM/MoarVM/commit/05eaefe8431547a1b7eabfed44b959c3051ba4d2#diff-8cb9d62eae3a641a6647908c1c5dae81R831 how can that condition ever be false?
21:45 jnthn Given you assign them at startup?
21:49 FROGGS jnthn: it will be false for the very first time, no?
21:49 jnthn If you do
21:49 jnthn instance->stdin  = MVM_file_get_stdin(instance->main_thread);
21:50 jnthn It will never be NULL after taht?
21:50 jnthn And startup occurs before we run any ops...
21:50 jnthn oh, ahng on, I see...
21:50 jnthn But if we're only meant to call them once...
21:50 FROGGS I can do it better, yes
21:50 jnthn If's safer to complain about a second call...
21:51 jnthn Or just write a setup_std_handles function in there that is called from bootstrap that does it.
21:51 jnthn uv_run(tc->loop, UV_RUN_NOWAIT);
21:51 jnthn What's the NOWIAT actually mean here?
21:51 FROGGS it won't block if there are no pending requests
21:53 jnthn OK, seem reasonable, now I glance the docs
21:53 jnthn (I was worried it may mean "don't wait for callbacks")
22:12 * jnthn is at the get_boxed_ref also
22:21 colomon joined #moarvm
22:21 FROGGS jnthn: I pushed
22:22 FROGGS though, it does not help with my problem
22:23 jnthn Does look better, though...
22:23 jnthn What exaclty is the problem?
22:23 FROGGS ROGGS>         --vmlibs=dynext/libperl6_ops_moar.so=Rakudo_ops_init src/gen/m-BOOTSTRAP.nqp
22:23 FROGGS <FROGGS> Internal error: invalid thread ID in GC work pass
22:24 FROGGS there is an item in the worklist that has a weird owner
22:25 FROGGS if I dereference it once more, it almost looks sane (before it had size=0, afterwards it had size=168 and type=10)
22:25 jnthn Weird..
22:25 FROGGS except that its REPR name if unreadable
22:25 FROGGS yeah
22:25 jnthn Yeah, sounds corrupte
22:26 FROGGS and this seems to introduce it: https://github.com/MoarVM/MoarVM/commit/05eaefe8431547a1b7eabfed44b959c3051ba4d2#diff-db8462d383c5c121ed6763f510fa4804R61
22:26 FROGGS especially what now is in setup_std_handles
22:27 jnthn Odd. Looks like they're correctly added as perm roots...
22:28 FROGGS hmmm, when I comment out the setup_std_handles now, it complains about STDERR not beeing initialized... so something is "using" it
22:30 FROGGS ahh, that still might be the stuff in nqp
22:36 jnthn Hm, something is weird with the pow_I impl
22:40 FROGGS what exactly?
22:40 FROGGS I could imagine that Inf and -Inf are not corret
22:41 jnthn https://github.com/MoarVM/MoarVM/issues/65
22:41 jnthn That task should be accessible to somebody other than me :)
22:47 FROGGS sounds doable, yes :o)
22:50 FROGGS btw, what makes me think that my filehandles are to blame is that the corrupt item in the worklist appears after calling MVM_gc_root_add_permanents_to_worklist in collect.c line 73
22:51 jnthn Does the filehandle thing itself have a gc_mark?
22:52 FROGGS the MVMOSFilehandle? no
22:52 FROGGS err, MVMOSHandle
22:53 FROGGS so, it needs to add something to the worklist? yes?
22:53 jnthn Well, only if it refs anything...
22:54 FROGGS no, don't think so
22:57 jnthn oh wow, how on earth did we get away without having attribute auto-viv stuff this long...
22:57 FROGGS :o)
23:03 jnthn Building a hopeful fix for this...we should be past line 6000 at startup with this...
23:03 FROGGS cool!
23:08 colomon joined #moarvm
23:10 jnthn Grr.
23:10 jnthn Fixing the misisng auto-viv breaks stuff.
23:11 FROGGS :(
23:27 FROGGS jnthn: this adds the corrupt obj to the worklist: MoarVM/src/gc/collect.c:355:        MVM_gc_worklist_add(tc, worklist, &new_addr_st->WHO);
23:29 jnthn o.O
23:30 jnthn Odd
23:32 FROGGS I added that right before the mentioned line:
23:32 FROGGS if (new_addr_st->WHO && new_addr_st->WHO->header.owner > 0)
23:32 FROGGS printf("ohh noes! %d\n", new_addr_st->WHO->header.owner);
23:32 FROGGS and it comes out like:
23:32 FROGGS ohh noes! 28410216
23:32 FROGGS sending a handle 0x1b1a958 to object 0x7f2a04d085d8 to thread 28410216
23:32 FROGGS Internal error: invalid thread ID in GC work pass
23:44 FROGGS this was called by MoarVM/src/gc/collect.c:288
23:46 jnthn Thing is, it probably copied that corruption from the object's old location...
23:46 FROGGS yeah :/
23:48 jnthn Meanwhile, I might have some progress on the auto-viv container thing
23:48 jnthn But getting tired...
23:48 FROGGS me too (not the progress, the other thing)
23:49 FROGGS okay, the thing comes from the if (item_gen2) { branch
23:49 BenGoldberg joined #moarvm
23:51 jnthn grrr
23:51 FROGGS so all I know is that it already is in the worklist :/
23:51 jnthn My fix didn't help at all...
23:51 FROGGS ó.ò
23:54 diakopter jnthb: are you spawning threads?
23:55 jnthn diakopter: Me? No...though it's FROGGS looking at the thing with the owner. But no threads...
23:55 jnthn Just corruption.
23:56 jnthn Oh...I see what's up...
23:56 diakopter whew
23:56 jnthn (with the auto-viv thing)
23:57 jnthn (not FROGGS's thing)
23:57 jnthn Grr, that's gonna be some "fun"...
23:57 diakopter wut
23:57 FROGGS ":/"
23:57 jnthn I didn't get around to putting in the attribute auto-viv thing
23:57 jnthn And we're accidentally relying on it not working in one place in regex code-gen
23:58 jnthn "fun" 'cus I think it may need a trip around the bootstrap loop to fix
23:59 jnthn Anyway, too tired to do that today.

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary