Camelia, the Perl 6 bug

IRC log for #moarvm, 2013-10-03

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:24 benabik joined #moarvm
01:45 FROGGS_ joined #moarvm
04:12 diakopter o_O running  ..\moarvm nqp.moarvm t\nqp\19-file-ops.t takes 478 ms with --debug --optimize --profile
04:13 diakopter for some reason visual studio's profiler is telling me how many times it looped through the interpreter switch... o_O    11,069,894
04:14 diakopter 12 gc runs
04:14 diakopter 11% of time spent in the gc
04:15 diakopter 15% of time spent allocating, including gc runs
04:15 diakopter whoa!
04:16 diakopter almost the entire time in the gc is spent in MVM_gc_mark_collectable
04:16 diakopter that's... unexpected (until I go look in the code)
04:18 diakopter well, hm
04:23 diakopter HUNH.
04:24 diakopter of mvmarray.c, p6opaque.c and mvmstaticframe.c, each had a single call to gc_mark that took >150ms... which was >98% of the total time spent in that function in the program run
04:24 TimToady maybe they were having a concert or something
04:25 diakopter hee
04:31 diakopter .tell jnthn interestingly, after optimization, only 0.04% of time is spent in the compunit loading... o_O
04:31 yoleaux diakopter: I'll pass your message to jnthn.
04:33 diakopter 9% of time is spent in msvcrt startup o_O
04:33 diakopter loading windows dlls and such
04:35 diakopter 9% of total time spent in mvmarray.c push function!
04:35 diakopter exclusive time!
04:36 diakopter 14% of program time is spent in exclusive time in mvmarray.c
04:36 diakopter ewww
04:36 diakopter 1m calls to at_pos in there
04:36 diakopter O_O
04:37 diakopter oops, no, 16% of program time in mvmarray.c
04:38 diakopter wow.
04:39 diakopter fully 28.42% of time was spent in interp.c
04:39 diakopter not very informative, that.
04:40 diakopter 1m calls to memcmp  /o\
04:40 diakopter 400k calls to memset; 223k calls to memcpy
04:41 diakopter almost no time doing hash-related stuff.... o_O
04:41 diakopter <2% of time in string-related stuff
04:41 diakopter so... arrays are super-heavy
04:41 diakopter and the interpreter... gotta get rid of that ;)
04:43 diakopter <1% spent running NFAs
04:50 diakopter TimToady: http://i.imgur.com/RCvoa0E.png
04:56 diakopter TimToady: anything stand out to you?
04:56 diakopter JimmyZ: ?  :)
04:57 diakopter nwc10: oy
04:58 ingy o/
04:58 diakopter ingy: oy
04:58 diakopter oops sbux closing &
06:28 diakopter .
06:35 moritz ..
06:48 ssutch joined #moarvm
07:10 FROGGS joined #moarvm
07:42 foo_bar_baz joined #moarvm
08:09 ssutch joined #moarvm
08:21 Woodi joined #moarvm
09:46 cognominal joined #moarvm
09:48 diakopter jnthn: get this... if I raise the ratio to 50, no errors :/
09:48 jnthn diakopter: of course, you just never reach a gen2 collection...
09:49 diakopter t\serialization\01-basic.t ............ QAST::Block with cuid cuid_29_1380793736.057255 has not appeared
09:49 diakopter well it does say that
09:49 jnthn :S
09:49 diakopter (maybe that tests runs the gc 50 times!)
09:54 diakopter jnthn: sure enough, the gc runs 52 times :)
09:54 diakopter jnthn: I've noticed it fails usually on the 2nd run after the gen2 run
09:55 diakopter er, after the 2nd run after the gen2 run
09:55 diakopter though sometimes after the first one after the gen2 run
09:55 diakopter any ideas?
09:55 jnthn Curious...that makes it sound more invovled that simple wb issue
09:56 jnthn I wonder if something is wrong with the "I refer to the nursery" list cleanup
09:56 diakopter ah, the one part I haven't gone over with a fine-toothed combpickroll
09:57 jnthn MVM_gc_collect_cleanup_gen2roots
09:57 lizmat_ joined #moarvm
09:58 arnsholt diakopter: http://i.imgur.com/7s0ALeF.gif ? =)
09:59 diakopter arnsholt: exactly. ludicrous speed.
10:00 arnsholt As long as you don't hit plaid
10:00 * diakopter wonders if jnthn has seen that movie
10:01 jnthn no :)
10:01 jnthn Or I don't get the reference, at least... :)
10:08 arnsholt It's Spaceballs
10:08 arnsholt Very funny!
10:32 Woodi joined #moarvm
11:01 FROGGS[mobile] joined #moarvm
14:15 jnap joined #moarvm
14:41 larks joined #moarvm
14:57 FROGGS /home/froggs/dev/nqp/install/bin/nqp --target=pir --setting=NULL --stable-sc --output=NQPCOREMoar.setting.pir nqp-src/NQPCORE.setting
14:57 FROGGS Confused at line 621, near "IO Methods"
14:57 FROGGS :o(
14:57 FROGGS is this due to cursless? does somebody know?
14:58 jnthn No idea
14:58 benabik joined #moarvm
14:58 FROGGS k, will bisect it I think
14:59 FROGGS ohh, that is inside a pod block
14:59 jnthn oh...eeks
15:00 tadzik sounds like a job for a pod man
15:00 FROGGS *g*
15:04 FROGGS ... testing
15:12 FROGGS okay, 2013.09 is fine
15:48 FROGGS bisecting told me it is: https://github.com/perl6/nqp/commit/9e​e3ed7235c9f7110d119ef748cac39a62517e77
15:49 jnthn OK
15:49 jnthn That probably means NQP::Grammar is making a bad assumption somewhere...
15:49 jnthn And we got away with it before this commit
15:50 jnthn I don't think the commit itself is wrong, even if it's "to blame" for the regression
15:50 FROGGS hmmm
15:51 FROGGS same problem on nqp@jvm fwiw
15:51 FROGGS I'll dig into this
15:52 jnthn Great :)
15:58 TimToady 59.4 to 55.0 seconds, so more cores may be a factor; alternately, it throttles them harder when it's getting hot :)
15:58 jnthn Still, a welcome little improvement.
15:59 TimToady oops, ww :)
16:13 not_gerd joined #moarvm
16:13 not_gerd o/
16:13 diakopter o/
16:14 ssutch joined #moarvm
16:14 not_gerd so, I've done some thinking on how to unify my DLL and ctypes work and get it into a shape that's more amenable for eventual merge
16:15 not_gerd I probably won't have time to work on it until the weekend
16:15 not_gerd should I write something up about what I'm planning to do for $interested_parties?
16:15 jnthn not_gerd: I'd rather keep the two apart
16:16 jnthn We need extops as soon as work on the Rakudo port starts.
16:16 jnthn The ctypes stuff I'd rather not have to worry about until later.
16:16 not_gerd I'm actually planning to make ctypes my test case for VM extensions ;)
16:17 jnthn OK, but still, please make it so I can review/merge extops on its own.
16:17 not_gerd sure
16:17 jnthn thanks :)
16:18 not_gerd the thing is, a DLLSym is just another type of pointer
16:18 not_gerd if factored right, there's potential for unification
16:19 diakopter so use one from the other
16:19 diakopter you can make thd ctyoes branch vranch from extops vranch
16:20 not_gerd oO( those pesky off-by-one errors )
16:21 diakopter :D
16:21 jnthn vranch!
16:35 timotimo salad with vranch dressing?
16:45 FROGGS jnthn: it is this branch btw: https://github.com/perl6/nqp/blob/​master/src/NQP/Grammar.nqp#L90-L95
16:47 jnthn FROGGS: oh...
16:47 jnthn identifier is now declarative
16:47 jnthn .*? is also I guess :)
16:48 FROGGS and since it is declarative... ?
16:49 diakopter mmmm vranch
16:50 diakopter jnthn: did you see my various metrics from last night pasted here?
16:50 jnthn FROGGS: It will eat anything
16:50 jnthn diakopter: Glanced over them this morning...I should re-read 'em :)
16:51 diakopter mainly just see that imgur link
16:51 diakopter infer what you would
16:51 jnthn FROGGS: It'll come out longer than begin, for example
16:52 jnthn FROGGS: Because begin will match identifier, and .*? will match whatever comes after it.
16:52 jnthn FROGGS: In fact, if you do .*? then the LTMer will eat the whole document I suspect :P
16:52 jnthn FROGGS: Anyway, my guess is we never hit the 'begin' branches any more
16:53 jnthn FROGGS: Try a {} after <identifier>
16:53 * FROGGS does so
16:55 diakopter . in a token eats whitespace?
16:55 diakopter hm, I guess
16:56 jnthn Sure
16:56 jnthn . is everything :)
16:56 diakopter yeah but..
16:57 diakopter what's non-whitespace
16:57 diakopter \W ?
16:57 jnthn \S
16:57 diakopter er, duh.
16:57 jnthn ok, dinner time...
16:57 * jnthn hands diakopter another coffee :)
16:57 FROGGS dinner &
16:57 * diakopter feels the coffee fall out my toes
17:00 diakopter jnthn: mostly I realized there was almost 0 gain to be had by optimizing deserialization of strings like we talked about
17:00 diakopter .. unless someone already did it while I wasn't looking
18:19 ssutch joined #moarvm
18:31 FROGGS hmmm
18:31 diakopter ?
18:32 FROGGS ../moarvm nqp.moarvm --target=mbc --output=foo.mbc t/nqp/01-literals.t
18:32 FROGGS MVMHash representation requires MVMString keys
18:32 FROGGS it gets an MVM_REPR_ID_P6str instead
18:32 diakopter oh, it could learn to unbox I guess
18:32 FROGGS ... of an MVM_REPR_ID_MVMString
18:33 diakopter er
18:33 TimToady .oO(vranch roast coffee)
18:33 diakopter the compiler should have it unbox the keys before passing it to the hash add
18:34 FROGGS diakopter: this happenes when calling exists_key with an P6str
18:34 diakopter yeah; whatever compiler is outputting that qast needs to unbox the key
18:35 FROGGS k...
18:36 FROGGS ahh, it happens with ../moarvm nqp.moarvm --target=mbc --output=foo.mbc -e '' too
18:36 FROGGS so it is not even the test file
18:36 diakopter look for exists_key in the qast compiler
18:36 diakopter erm, I think.
18:37 diakopter jnthn should decide whether he wants the hash ops to unbox or whether the compiler should inject the unbox operation
18:43 timotimo .o( that throws a vranch in the machine )
18:51 diakopter lunchish&
18:56 FROGGS why is it called at_key_boxed when the to-be-passed key must be an MVMString?
18:57 timotimo maybe it returns to the return value?
18:58 FROGGS returns to the return?
18:58 FROGGS refers perhaps?
19:00 timotimo yes, refers to the return value's type
19:00 timotimo or kind or mode or whatevs.
19:01 FROGGS hmm, if it is so I don't see it
19:02 timotimo was just guessing
19:02 FROGGS I don't see any boxing at all tbh
19:03 FROGGS but that might be my very own problem :o)
19:07 FROGGS okay, it gets called by reprconv.c MVM_repr_exists_key
19:15 diakopter a P6Str is a box
19:15 not_gerd joined #moarvm
19:15 not_gerd FROGGS: see documentation in 6model.h
19:16 diakopter not_gerd:  :P
19:16 diakopter that's not terribly helpful :P
19:16 FROGGS not_gerd: thanks
19:17 not_gerd well, it confirms that the boxing indeed refers to the result type
19:17 diakopter .oO( is there documentation there? )
19:17 diakopter oh...
19:17 jnthn It's about the result type, yes
19:17 not_gerd it of course doesn't help with the problem of how it should handle its arguments
19:17 diakopter jnthn: should the hash ops unbox the key?
19:18 jnthn No, they should be taking a str reg, and do afaik
19:18 jnthn I think the error is from within the bowels of the MAST assembler...
19:18 diakopter hrm.
19:18 jnthn So it's something in there calling it without doing an appropriate unbox, I'll bet
19:18 diakopter actually
19:19 FROGGS I can only trace it to src/6model/repronv.c:162
19:19 FROGGS but I can't find a call to MVM_repr_exists_key with a REPR id != 0
19:19 diakopter jnthn: actually, it's a runtime error, and the keys take objects
19:20 diakopter how do you know it's exists_key
19:20 FROGGS printf
19:20 diakopter oh
19:20 diakopter jnthn: no clue why the keys take objects
19:20 jnthn Look at the stack trace
19:20 FROGGS and this error message appears only once
19:20 jnthn at nqp-src\QASTMoar.nqp:7958  (./QASTMoar.moarvm:assemble_to_file:4294967295)
19:20 jnap joined #moarvm
19:20 diakopter er, I don't remember
19:20 jnthn Then at what is there
19:20 jnthn method assemble_to_file($mast, $file) {
19:20 jnthn __MVM__masttofile($mast, self.node_hash(), $file)
19:20 jnthn }
19:21 jnthn That's the thing that calls down to the MAST -> bytecode assembler.
19:21 diakopter okay; the error message is in MVMHash.c
19:22 jnthn Sure, that's called by the assembler
19:22 jnthn existskey doesn't show up too many times in there...hmm
19:23 jnthn bah, easier to look at it under the C debubber
19:23 jnthn bugger
19:23 jnthn debugger
19:23 lizmat hehe
19:23 diakopter FROGGS: existskeyop calls exists_key
19:23 diakopter existskey op
19:25 jnthn It's in get_string_heap_index
19:25 FROGGS I added a print statement before every call to MVM_repr_exists_key, and none of them got hit
19:25 jnthn in src/mast/compiler.c
19:25 diakopter erm.
19:25 jnthn strval there is not what it should be.
19:25 jnthn I hope we're debugging the same bug here?
19:25 FROGGS ahh, there
19:25 FROGGS yes, seems like
19:26 FROGGS the line of the stacktrace you pastes is exactly like mine
19:26 FROGGS pasted*
19:26 diakopter FROGGS: lots of times it calls exists_key diretly
19:26 FROGGS diakopter: that might be the reason why my method didn't work out *g*
19:28 diakopter yeah the convenience routines (and macros) were added relatively recently, and not all calls converted for speed reasons
19:28 diakopter (paranoid superstitious speed reasons)
19:29 jnthn This is odd...
19:29 jnthn It's in a code branch that should only ever have a MAST::SVal
19:29 jnthn and
19:29 jnthn class MAST::SVal is MAST::Node {
19:29 jnthn has str $!value;
19:29 jnthn That's a native string slot
19:29 diakopter sounds like a critical bug somewhere :)
19:30 jnthn yeah
19:31 jnthn quick trip to the shop; bbiab
19:54 * diakopter is annoyed I have to nmake realclean the cross compiler when switching branches
19:54 diakopter because of some missing Makefile dependencies
19:56 japhb_ joined #moarvm
20:03 * jnthn back
20:10 diakopter At Frame 304, Instruction 181, op 'shell' has invalid number (2) of operands; needs 4.
20:10 diakopter argh. nmake realclean didn't even fix it
20:12 FROGGS shell() gets the cwd and env now too since a few weeks...
20:12 FROGGS maybe something is just outdated?
20:13 diakopter right, that's what I said
20:13 FROGGS so we agree?
20:13 FROGGS :P
20:13 diakopter no, but I have no idea what
20:13 diakopter I wish there were a git status command that ignored the .gitignore
20:14 diakopter *was
20:15 not_gerd just a guess, but what about re-running update_ops?
20:18 jnthn diakopter: git status --ignored
20:24 jnthn So, lemme look at this mbc bug a bit more...
20:29 dalek MoarVM/dll: be0fe24 | jnthn++ | src/core/interp.c:
20:29 dalek MoarVM/dll: Fix STable claiming.
20:29 dalek MoarVM/dll:
20:29 dalek MoarVM/dll: With this, t/serialization/01-basic.t now fully passes.
20:29 dalek MoarVM/dll: review: https://github.com/MoarVM/MoarVM/commit/be0fe2401e
20:29 not_gerd for got to push those yesterday
20:29 dalek joined #moarvm
21:05 jnthn diakopter: So far, discovered that somehow an SC ends up with its ->handle pointing to something not an MVMString.
21:05 diakopter oopsie
21:06 jnthn yeah
21:06 jnthn but struggling to see where
21:06 jnthn createsc ends up with the Right Thing
21:08 diakopter hrm
21:08 diakopter perhaps the quasi-weak hash
21:08 diakopter nah
21:08 jnthn it's ok in MVM_sc_create
21:09 diakopter oops  MVM_HASH_GET(tc, tc->instance->sc_weakhash, handle, scb);
21:10 diakopter oh, nm :/
21:10 jnthn yeah, added a check there and it seems fine too
21:11 diakopter well, what *is* in that slot
21:11 diakopter if not a string
21:11 diakopter junk pointer?
21:12 jnthn A P6str
21:12 diakopter must be from deserialization
21:13 diakopter well wait, what string is in the p6str
21:13 diakopter what content
21:14 donaldh joined #moarvm
21:14 diakopter does the P6str have a sc?
21:15 jnthn oh, duh...I did something silly
21:15 jnthn wait a mo
21:15 diakopter ok
21:15 jnthn yeah, false lead
21:16 jnthn We *are* somehow getting an object on a string reg
21:16 jnthn But not there
21:16 diakopter yikes :)
21:16 diakopter could theoretically put a very inefficient guard in MVM_ASSIGN_REF :)
21:17 diakopter 'course, that may never terminate...
21:24 * diakopter goes to try to repro
21:25 diakopter .. instead of just imagining...
21:25 diakopter oh yeah, I can't build the cross compiler
21:25 diakopter argh.
21:28 foo_bar_baz joined #moarvm
21:31 FROGGS diakopter: what is your nqp --version?
21:32 diakopter ARGH
21:32 diakopter rogue zombie nqp.exe
21:32 FROGGS :/
21:32 diakopter I always forget to check that.
21:33 * PerlJam wonders if "zombie nqp" is the beginning of the zombie apocalypse
21:36 not_gerd bye, #moarvm
21:36 diakopter o/
21:37 not_gerd left #moarvm
21:37 diakopter jnthn: re object in string register, I knew I put that redundant repr check in for a reason.... ;)
21:38 jnthn I'm struggling to reproduce it in a small test case
21:40 diakopter I think it'll be easier to try to trace it back by logging in giant hashes of addresses... :D :D :D
21:42 diakopter (I'm only half-kidding!)
21:46 diakopter I've wanted such a feature already
21:47 diakopter though it's replicating address sanitizer, some
22:01 jnthn this is *really* confusing... :S
22:02 diakopter well that's worrisome
22:02 diakopter any learnings?
22:03 jnthn Well, the fail starts here:
22:03 jnthn QAST::Op.new( :op('createsc'), QAST::SVal.new( :value(nqp::scgethandle($sc)) ) )
22:03 FROGGS should be :s for a natively typed str
22:03 diakopter jnthn: oh you know what
22:04 jnthn If I put something in P6opaque that makes sure that when you bindattr_s, it really is a string, then it explodes
22:04 jnthn in setting $!value inside QAST::SVal
22:04 diakopter MVM_sc_get_handle is actually wrong
22:04 jnthn oh?
22:04 diakopter yeah it needs to do the resolution check like the other sc_get things
22:05 jnthn um
22:05 diakopter er, hang on; I'll tell you what I mean
22:05 jnthn like *none* of the other things do in that file?
22:05 diakopter se
22:05 diakopter sec
22:05 diakopter :)
22:06 diakopter erm, where'd it go
22:06 * diakopter keeps looking
22:08 diakopter whaaaaaa
22:08 diakopter where'd it go :S
22:10 diakopter I swear I wrote a few lines of code to fixup the sc body of ones once they're resolved
22:10 diakopter and intended that check to go in all the accessors
22:11 diakopter oh, it's in MVM_sc_get_sc
22:12 diakopter cu->body.scs[dep] = sc;
22:12 diakopter oops, that needs a MVM_ASSIGN_REF
22:12 diakopter probably not the problem tho
22:13 diakopter also it should null cu->body.scs_to_resolve[dep]
22:13 diakopter ergh
22:14 diakopter erm
22:14 diakopter don't listen to me
22:15 diakopter *today
22:15 diakopter *except for this statement
22:17 diakopter jnthn: lemme take this on; I have a decent idea how to tackle it, at least, I think
22:18 diakopter but not at the moment
22:18 diakopter but this evening
22:19 FROGGS it is already past midnight here, so you might get the chance to tackle it
22:27 jnthn diakopter: ok
22:27 jnthn diakopter: The compiled output looks very sane, fwiw
22:28 diakopter compiled output of what?
22:28 diakopter bytecode dump of something?
22:28 jnthn yeah, --dump
22:29 diakopter of nqp.moarvm?
22:29 jnthn QASTMoar.moarvm
22:29 jnthn The thing that calls scgethandle
22:29 diakopter oh :)
22:33 diakopter bye bye dalek
22:33 dalek joined #moarvm
22:35 FROGGS well, good that dalek survived in our commit-channel :o)  (#parrot)
22:40 jnthn diakopter: Anyway, I won't hunt this any further for now.
22:40 jnthn Something's rotten, somewhere
22:44 diakopter I'm guessing it's something small
22:44 diakopter jnthn: how did you conclude an object was definitely getting into a string register
22:45 diakopter s/conclude/determine/
22:45 jnthn Let me give you my patch...
22:46 jnthn diakopter: https://gist.github.com/jnthn/6818287
22:46 diakopter :) string is not
22:47 jnthn aye
22:47 jnthn Anyway, that triggers in the place I pointed to
22:47 jnthn The reason we don't hit it without --target=mbc is 'cus we don't emit deserialization code.
22:48 jnthn If you don't put that it then we have the P6str object shoved in the str attribute, and it makes it all the way until MAST assembler, where it explodes (which is the original error we were seeing)
22:50 diakopter hrm
22:50 jnthn So anyway, that patch makes it blow up closer to the thing to blame.
22:51 jnthn I put similar in the scgethandle op, but to no aveil

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary