Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2015-09-30

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:09 vendethiel joined #moarvm
02:17 dalek MoarVM: 4df2030 | hoelzro++ | src/io/procops.c:
02:17 dalek MoarVM: Track process state when spawning children
02:17 dalek MoarVM:
02:17 dalek MoarVM: This is prepare for a fix around closing standard input and race
02:17 dalek MoarVM: conditions
02:17 dalek joined #moarvm
02:21 tokuhiro_ joined #moarvm
06:25 Ven joined #moarvm
06:55 FROGGS joined #moarvm
07:54 FROGGS joined #moarvm
07:59 brrt joined #moarvm
08:26 Ven joined #moarvm
08:39 Ven joined #moarvm
09:25 * Ven 's learning about all those weird registers things, on his way to enlightenment 'bout computers and VM...
09:25 Ven I certainly understand more thing of that big JIT file now... But damn you, gnu asm...
09:26 Ven timotimo++ for motivating me to do so :P
09:30 brrt Ven++ for trying to learn :-)
09:30 brrt can i help you?
09:30 brrt with anything?
09:30 brrt for example, with the options to convert to intel asm format? :-)
09:31 Ven I've been given this great chart: https://upload.wikimedia.org/wikipedia/commons/1/15/Table_of_x86_Registers_svg.svg which has helped me a lot already :D
09:31 brrt oh, yes, that seems useful
09:31 Ven I got my code to do what I wanted it to do (store char in the stack, increment char unless char='z', then set it to 'a'; print it using a syscall). I'm trying to rewrite it to be a function call now
09:32 brrt that will be fun
09:32 Ven seems the output (with cat -e) is "M-I^B^B", I think I'm not really there now XD
09:32 brrt :-)
09:35 nwc10 hoelzro: unfortunately 02d5241997ae7b5a2dc1ce1f1a8bd1fa3884258b creates a use-after-free bug: http://paste.scsys.co.uk/499584
09:36 nwc10 from that stack trace, I hope that the solution is simple. It looks like the use is just after the free, and so could be fixed by some re-ordering or somesuch
09:48 * Ven wonders why gcc -O2 -S sometimes generates -4(%rbp), sometimes -1(%rbp) in the program, for what seemed to be equivalent
09:48 moritz moon phase
09:50 Ven aah, I got it. I forgot to fill %eax before using %al..
09:51 * brrt wonders why Ven is using lower registers at all
09:51 Ven it's just a char, so a byte
09:51 Ven at least that's what I got..
10:10 Ven ooh, I used cmp, not cmpb. fixed now!
10:49 Peter_R joined #moarvm
10:53 zakharyas joined #moarvm
10:53 mtj_ joined #moarvm
10:58 mtj_ joined #moarvm
11:21 Ven joined #moarvm
11:22 brrt Ven: i can totally recommend intel syntax
11:22 brrt in intel syntax, it's not movb
11:22 brrt or cmpb
11:22 * masak .oO( I can call you Betty, and baby, you can call me %al )
11:22 Ven yeah, that works :)
11:22 brrt but cmp byte foo, byte bar
11:22 brrt actually, typcally
11:22 brrt cmp byte [rbp-1], al;
11:22 brrt lol masak
11:23 brrt an anyway, i find it kind of handy when things like operand size are explicit rather than encoded into the operand in a single character
11:23 Ven well, it's supposed to be an exercise for the students: if they're done with the C version, they have to rewrite it. and the school decided to tell people to use gnu asm. so, I'm writing a correction in case any of them submits a solution
11:23 Ven (they seem to prefer slacking to that. honestly, I can somewhat get it..)
11:23 brrt ....
11:23 brrt well, yeah
11:24 brrt anyway, i know this is officially a matter of taste
11:24 brrt but i find intel asm to be *much* more legible and explicit about things
11:24 brrt actually, just as explicit, but the explicit things are more visible, i guess
11:31 Ven yeah, from looking around the internet (trying to answer my questions), everybody recommended intel synta
11:32 Ven +x. but oh well. I guess it's still very valuable an experience
11:34 brrt yes, i agree
11:34 brrt and fun, if you get the hang of it
11:34 brrt well, thats my opinion at least
11:45 Ven oh, I need to take a break every few hours because my head hurts a lot, but it's amazing at the same time :-)
11:50 co-gnome joined #moarvm
12:04 co-gnome joined #moarvm
12:27 co-gnome joined #moarvm
12:31 mtj_ joined #moarvm
12:55 Ven joined #moarvm
13:15 * cognominal_ watches the register chart. Things have evolved since the 6502 and Z80...
13:15 FROGGS joined #moarvm
13:16 brrt yeah, although 'regular' code just uses xmm* and the gpr
13:43 synbot6 joined #moarvm
13:50 hoelzro nwc10++ # running address sanitizers on spec tests
13:51 hoelzro I'll revert that commit for now, and stuff it into a branch
13:52 timotimo or shall i just try to fix it?
13:52 dalek MoarVM: 9befd69 | hoelzro++ | src/ (4 files):
13:52 dalek MoarVM: Revert "Track whether or not a capture owns its callsite"
13:52 dalek MoarVM:
13:52 dalek MoarVM: This reverts commit 02d5241997ae7b5a2dc1ce1f1a8bd1fa3884258b.
13:52 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/9befd69353
13:53 tokuhiro_ joined #moarvm
13:53 timotimo hoelzro: i wonder if it would have been easier to set up the tracking of malloc info via nativecall
13:54 hoelzro timotimo: possibly, but I didn't want any possible leaks with nativecall to interfere with my measurements
13:54 hoelzro although I could've tested for that first =)
13:54 timotimo right
13:55 timotimo i'd be interested in such leaks with nativecall
13:56 dalek MoarVM/callsite-memory-leak: f7a176b | hoelzro++ | src/ (4 files):
13:56 dalek MoarVM/callsite-memory-leak: Restore "Track whether or not a capture owns its callsite"
13:56 dalek MoarVM/callsite-memory-leak:
13:56 dalek MoarVM/callsite-memory-leak: This reverts commit 9befd69353643bff3a529aa5bea8101cd574fa29.
13:56 dalek MoarVM/callsite-memory-leak: review: https://github.com/MoarVM/MoarVM/commit/f7a176b702
13:57 nwc10 hoelzro: couldn't you simply have branched that from the parent of the revert commit?
13:59 hoelzro I suppose so
13:59 hoelzro then I could've just dealt with the restoration during the merge
14:01 timotimo huh, just some simple re-ordering?
14:01 timotimo i don't actually see how it's read just after being freed …
14:01 hoelzro I think it's another situation like copy_to
14:01 hoelzro I haven't looked into this new situation, though
14:02 hoelzro when I have time, I might just add a reference count to callsites
14:02 timotimo OK :)
14:02 timotimo ... reference count :\
14:09 timotimo especially since that has some ramifications with multi-threading stuff
14:27 hoelzro eesh, good point
14:27 hoelzro well, I'll have time to think about it =)
14:27 hoelzro commute &
14:31 timotimo have a good one!
14:54 tokuhiro_ joined #moarvm
15:04 hoelzro fg
15:30 Ven joined #moarvm
15:55 Ven joined #moarvm
16:16 timotimo jnthn: i didn't actually look yet, but i'm pretty sure the code i was talking about used failure to signal the fallback "outwards"
16:17 jnthn hmm
16:17 jnthn But they should get GC'd really
16:17 timotimo where should i look to find out if backtraces are being kept around and such?
16:17 jnthn So I'm curious how that can happen
16:17 jnthn Well, see how many MVMExceptions are alive maybe
16:17 timotimo i wonder if the existence of the DESTROY method tickles a bug somewhere?
16:18 jnthn Oh, 'cus it goes on the finalization queue? Maybe but I can't see quite how offhand.
16:18 timotimo right
16:21 timotimo hmm. see how many MVMExceptions are alive ... how exactly? :P
16:21 timotimo with the heap analyzer?
16:23 jnthn Doesn't the profiler track them?
16:23 jnthn And retention/promotion rates? :)
16:23 jnthn Maybe we don't label exception creation well enough?
16:23 timotimo i built something to that effect at some point, but it remained buggy
16:24 jnthn s/label/instrument/
16:24 timotimo mhm
16:25 timotimo huh
16:25 timotimo it seems like we don't instrument those at all yet
16:25 jnthn wow :)
16:26 timotimo anything beyond newexception that needs that?
16:26 timotimo also, we don't specialize newexception into fastcreate?
16:26 timotimo ah, because it wants the stacktrace
16:27 timotimo bleh. i should have put this code into a git repository before doing so many different things with it
16:27 timotimo also, i should have it on my desktop, too %)
16:31 Ven joined #moarvm
16:38 timotimo i don't actually see a "newexception" show up in the speshlog and a perl6 -e 'my @a; for ^1000 { @a[$_] // Inf }' doesn't show any exceptions allocated
16:39 jnthn hm, curious
16:39 jnthn dinner time &
16:52 ilbot3 joined #moarvm
16:52 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
16:53 moritz joined #moarvm
16:54 psch joined #moarvm
16:55 tokuhiro_ joined #moarvm
17:01 timotimo perhaps i shouldn't just have put more and more and more 0s at the end of that number?
17:07 Util joined #moarvm
17:13 timotimo once in throw and once in THROW for each, it seems like
17:14 timotimo perhaps the nqp::isconcrete($!ex) is never 1?
18:05 timotimo @a[$_] actually just returns (Any), obviously
18:05 timotimo i would have to go for -1
18:06 timotimo the speed difference between accessing @a[-$_] and accessing @a[$_] is tremendous
18:09 timotimo at least this little "benchmark" shows the "gc times increase more and more" problem nicely
18:09 vendethiel joined #moarvm
18:10 timotimo but the gen2 roots just fluctuate rather than growing steadily
18:10 timotimo on the other hand, about 800 KB end up promoted, which i don't really understand
18:20 timotimo wait what
18:20 timotimo we're allocating 19999 instances of Perl6::Metamodel::ModuleHow ?!
18:20 jnthn o.O
18:20 jnthn wtf!
18:20 jnthn HOW is that happening?!
18:21 timotimo am i seeing this right ?!
18:21 jnthn If so, then it's an interesting find... :)
18:21 jnthn bbiab
18:21 timotimo or is it just something returning an instance of ModuleHOW that gets interpreted as allocated?
18:24 timotimo nope, that's totally the method new from the ModuleHOW that's allocating these
18:38 lizmat timotimo: is ther much difference between @a.AT-POS($_) and @a.AT-POS(-$_) ?
18:38 FROGGS[tab] joined #moarvm
18:38 FROGGS[tab] o/
18:40 timotimo lizmat: i'm just investigating in another direction right now
18:42 timotimo which is:
18:42 timotimo why do we create two new ModuleHOW whenever we fail something
18:42 timotimo oh wow
18:42 timotimo fail() is more expensive than fail("test") in that way
18:42 timotimo perhaps it's the default value for the payload argument?
18:57 tokuhiro_ joined #moarvm
18:58 cognominal joined #moarvm
19:05 dalek MoarVM: c8173b7 | TimToady++ | src/6model/reprs/NFA.c:
19:05 dalek MoarVM: log nfa offsets into string
19:05 dalek MoarVM:
19:05 dalek MoarVM: this allows us to calculate how many times a position was lexed
19:05 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c8173b7a9d
19:05 dalek MoarVM: c393c4e | TimToady++ | src/ (2 files):
19:05 dalek MoarVM: add rudimentary timing to dynvar logs
19:05 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c393c4ec34
19:05 dalek MoarVM: 3ffc453 | TimToady++ | src/ (3 files):
19:05 dalek MoarVM: do better at not counting logging time
19:05 dalek MoarVM:
19:05 dalek MoarVM: (the old code assumed that it takes no time to log dynvar lookups)
19:32 jnthn Looks like somebody's gearing up for some perf work :)
19:46 timotimo i had dreamt about building a regex performance analysis thing that'd point out how hot each of the source characters is
19:47 timotimo okay, so why would self.bless(:$exception) cause ModuleHOW.new to be invoked twice?
19:47 jnthn Why do I immediately start thinking how to construct regexes that make certain letters hot to spell naughty things out of innocent words? :)
19:47 hoelzro http://cdn.meme.am/instances2/500x/2168198.jpg
19:47 jnthn timotimo: I...no idea...what is the BUILD doing?
19:48 jnthn :P
19:50 timotimo jnthn: i'm having a hard time finding that out …
19:50 timotimo it kind of looks like a "die" is being called somewhere in there
19:50 timotimo "for good measure" ?!?
19:50 timotimo oh
19:51 timotimo Backtrace gets created via try { die() }
19:51 timotimo that at least explains why we get an X::AdHoc created for every exception we create
19:53 jnthn Every exception, or every Failure?
19:53 lizmat every Backtrace.new
19:53 timotimo what liz said
19:54 timotimo Failure's BUILD creates the new Backtrace object
19:54 lizmat I have no idea why it works that way
19:54 timotimo however
19:54 timotimo if $!exception is set, it uses its backtrace instead
19:54 lizmat that line is from 2011
19:55 lizmat comment on commit de3997a73635a627c007b: make Backtrace.new() DWIM
19:55 timotimo this still doesn't tell me how the flying fuck we end up creating new ModuleHOW instances :)
19:56 jnthn timotimo: Chase the callgraph until you find that node? :)
19:56 lizmat well, not having a die in there, would help speed up all warnings, I would think
19:57 timotimo tried to, failed
20:02 timotimo peppering the code with note("blah") at the moment
20:02 timotimo could perhaps have to do with Backtrace::Frame?
20:02 timotimo maybe we would have wanted to put that inside the Backtrace class instead?
20:04 timotimo soon it'll be food time and my search will be interrupted
20:05 jnthn timotimo: Maybe easiest is just to make ModuleHOW.new dump a backtrace every time it's called (you'll need to do it at NQP level)
20:07 timotimo i wanted to do that
20:08 timotimo i asked what the best way was, and i didn't get an answer :)
20:08 timotimo i've found the offending line
20:08 timotimo it's try { die() }
20:08 timotimo i'll have a look at how die() is implemented
20:09 jnthn Oh
20:12 jnthn try { nqp::die(''); CATCH { nqp::say(join("\n", nqp::backtracestrings($_))) } }
20:12 jnthn timotimo: ^^
20:12 timotimo thanks
20:13 timotimo that'll help a lot :)
20:18 timotimo i'm about to have noms
20:19 jnthn enjoy :)
20:19 timotimo https://gist.github.com/timo/a8ff4eb12e11f1d10db5
20:19 timotimo i think it's about the default value for die($payload = ...)
20:19 timotimo because it uses CALLER::CALLER::...
20:19 jnthn Oh...
20:20 timotimo the very same code is used in Failure and Exception a bunch of times
20:20 jnthn We should probably teach the static optimizer to simplify CALLER::CALLER::symbol
20:21 jnthn (Into the nqp ops)
20:21 timotimo something involving nqp::callerctx(nqp::callerctx(nqp::curctx))?
20:21 timotimo okay, AFK for noms now
20:22 jnthn right
20:24 vendethiel joined #moarvm
20:30 Ven joined #moarvm
20:34 colomon joined #moarvm
20:45 tokuhiro_ joined #moarvm
20:45 leont joined #moarvm
21:00 lizmat jnthn: perhaps putting in those nqp ops now would already help ?
21:00 timotimo that'd be nice, especially since this is perhaps the only place we really stumble over this
21:07 jnthn lizmat: Could do, otoh relying on an optimization in CORE.setting is a great way to make sure it keeps working :P
21:08 tadzik oh, exploitable :)
21:09 lizmat jnthn: I just wanted to speed up things now... if we can wait for the static optimizer to pick up on this, that's fine by me too  :-)
21:09 leont jnthn: what is the current state of the async io? (specially processes, but the rest too)?
21:09 leont How feasible would it be for someone who isn't particularly familiar with moar to help out on it?
21:10 jnthn lizmat: I think the static optimizer patch will be relatively easy
21:11 jnthn leont: Depends what you are familiar with, I guess. If working with the libuv API is already something comfortable, that helps.
21:11 leont I've looked over it before, it looked straightforward enough
21:12 jnthn leont: The processes bit I know a copule of people have been polishing of late, to deal with various corner caess.
21:12 timotimo lizmat: i'd personally appreciate if you put that in
21:12 jnthn leont: So far as I can tell it's in decent shape.
21:13 leont Should recompile I guess, last time I tried I hit issues
21:13 timotimo lizmat: in the mean time, i'll check @a[-$_] vs @a[$_] against @a.AT-POS(-$_) vs @a.AT-POS($_)
21:13 jnthn Also, I've been continuing to hunt down and fix pesky concurrency bugs.
21:14 jnthn (Such wonders as type finalization timing issues in concurrent GC sweeps...)
21:15 timotimo jnthn: did you see hoelzro's problem about callsites we're leaking?
21:16 jnthn timotimo: Yeah, but I simply don't have the brane for it this week with teaching, travel, etc.
21:17 timotimo totally acceptable
21:19 hoelzro timotimo: I dug in a bit earlier; I think that callsites are shared more than my patch accounts for
21:22 timotimo mhh
21:23 hoelzro my patch only deals with CallCapture objects that refer to the same callsite
21:23 hoelzro I'm pretty sure frames do a lot moar
21:24 timotimo mhh, i see
21:24 timotimo lizmat: the difference between [] and AT-POS is negligible
21:24 lizmat ok, that's good to know...
21:25 timotimo right
21:25 timotimo the difference is most probably just the crazy overhead from the whole failure thing
21:25 lizmat if there was a significant difference, we would have a problem  :0)
21:25 timotimo this is 80 seconds vs 0.8 seconds for 100_000 accesses
21:25 timotimo that's negative indices vs positive indices
21:26 lizmat and posiitive indices with [] vs positive indices with AT-POS ?
21:27 timotimo 0.84 vs 0.83
21:27 lizmat I guess you need to increase until a runtime of several seconds  :-)
21:28 timotimo sure
21:28 jnthn It'll still be similar percentage wise :P
21:28 lizmat well, it's really .74 vs .73  :-)
21:29 jnthn *nod*
21:29 jnthn Mostly these are just "wait for MoarVM's inliner to get smarter" though :)
21:30 lizmat true
21:30 timotimo 33.3s vs 29.14s for 5_000_000 accesses
21:30 timotimo these are accesses to not-yet-existing elements, fwiw
21:31 timotimo so scalars with a whence and all that
21:31 lizmat m: say .73/74; say 29.14/33.3
21:31 camelia rakudo-moar dd46c3: OUTPUT«0.009865␤0.875075␤»
21:31 jnthn Odd that the difference increaess...
21:32 timotimo could be that the gc hadn't kicked in before or something like that?
21:32 colomon joined #moarvm
21:32 jnthn Mebbe
21:35 timotimo can i simply translate CALLER::CALLER:: to nqp::callerctx(nqp::callerctx(nqp::ctx)) in CALLER::CALLER::.EXISTS_KEY(...) and ::('...')?
21:36 timotimo or do i need a ctxlexpad there?
21:36 timotimo i probably do
21:37 jnthn I'd just look for cases that end in an indexing operation
21:37 jnthn An even then maybe only constant ones
21:37 jnthn e.g. when we would never actually expose a pseudo-package
21:37 timotimo oh, i was going to do the hand-optimization first
21:38 jnthn ah :)
21:38 jnthn Then yeah, can do it that way
21:38 * jnthn should rest up for tomorrow's class
21:38 jnthn 'night
21:38 timotimo i'm apparently syntax-wronging right now
21:38 timotimo good rest, jnthn!
21:38 timotimo have fun in class :)
21:39 timotimo ah, preclimit trouble, probably
21:44 timotimo it didn't like "and", so i used && instead
21:45 timotimo time perl6 -e 'my @a; for ^100_000 { @a.AT-POS(-$_) // Inf };'
21:46 timotimo 47.61user 0.30system 0:48.03elapsed 99%CPU (0avgtext+0avgdata 851208maxresident)k
21:46 timotimo that's not quite 2x better, but it's noticable
21:46 timotimo but anyway, improving &fail is probably worthwhile
21:46 timotimo we want people to use this facility, too
21:47 timotimo in this case it's about improving die, actually
21:47 timotimo maybe we shouldn't be using a perl6-level die() to create & throw the exception we use to get the backtrace from
21:47 lizmat perhaps we need fail to be cheaper, only create the Backtrace if really necessary
21:48 timotimo whoa
21:48 lizmat whoa ?
21:48 timotimo the run time for that is 70% GC
21:48 lizmat yeah, lot's of garbage created in a fail
21:49 timotimo 9899901 Scalars getting allocated from inside BUILDALL
21:49 timotimo sorry
21:49 timotimo not scalar, BOOTCode
21:49 timotimo so closures being taken
21:50 timotimo only 299997 + 299997 Scalars from BUILDALL and another 299997 from backtrace (Exception's backtrace method; why does it allocate scalars??)
21:51 timotimo hash's AT-KEY allocates another 299997
21:51 timotimo fail also allocates scalars: 199998 of them
21:53 timotimo ah, ok, the number of gen2 roots is climbing a whole lot again there
21:53 timotimo the last ~50 gc runs (out of 719) take >100ms each
22:00 [Coke] I'm having trouble githubbing from hack.
22:01 hoelzro [Coke]: what kind of trouble?
22:01 * hoelzro just cloned rakudo on hack sans problems
22:01 [Coke] just hangs.
22:02 timotimo i wonder how that one anon block inside BUILDALL gets to deopt
22:03 hoelzro [Coke]: what operation(s) hang?
22:10 [Coke] git push; git pull --rebase
22:12 hoelzro both of them hang?
22:25 oetiker joined #moarvm
22:33 timotimo so here's a data point
22:33 timotimo using NativeCall to get at puts() is quite a bit faster than nqp::say
22:33 timotimo but NativeCall also has a whole bunch of overhead still
22:36 timotimo so we^WI really need to put in direct synchronous I/O
22:46 lizmat :-)
22:51 tokuhiro_ joined #moarvm
23:25 timotimo huh
23:25 timotimo actually
23:25 timotimo it seems like libuv also offers synchronous IO - at least on files - and we're using that
23:26 timotimo huh. but only for files, not for streams
23:27 timotimo right, our implementation of say() will epoll_wait between every instance of say
23:35 leont That makes sense
23:40 timotimo it does?
23:40 timotimo epoll_wait(5, {}, 1024, 0)              = 0
23:40 timotimo what does that even mean?
23:41 timotimo hm, well, at least it won't block or wait at all
23:47 leont It means it did a non-blocking wait for at most 1024 events
23:47 leont And it didn't return any events
23:54 timotimo ok, that's what i expected

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary