Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2014-03-07

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:00 tgt joined #moarvm
01:57 benabik joined #moarvm
02:07 ilbot3 joined #moarvm
02:07 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
02:10 woosley joined #moarvm
02:20 FROGGS__ joined #moarvm
02:21 cognominal__ joined #moarvm
02:21 hoelzro_ joined #moarvm
02:23 japhb__ joined #moarvm
02:26 timotimo joined #moarvm
02:51 woosley1 joined #moarvm
02:58 colomon MoarVM near the top of HN at the moment!
03:00 JimmyZ HN?
03:01 tgt joined #moarvm
03:06 woosley joined #moarvm
03:07 colomon Hacker News
03:08 TimToady is this good or bad?
03:08 colomon https://news.ycombinator.com/news
03:10 burganovb joined #moarvm
03:15 [Coke] looks like there's no real thread talking about it, just a linklist.
03:23 lee__ joined #moarvm
03:30 tadzik joined #moarvm
03:55 teebrz joined #moarvm
04:04 tgt joined #moarvm
05:02 woosley joined #moarvm
05:05 tgt joined #moarvm
05:48 woosley joined #moarvm
07:52 woosley1 joined #moarvm
07:52 JimmyZ joined #moarvm
07:52 _sri joined #moarvm
07:52 cognominal__ joined #moarvm
07:52 flussence joined #moarvm
07:52 lue joined #moarvm
07:52 camelia joined #moarvm
07:52 japhb joined #moarvm
07:52 benabik joined #moarvm
07:52 rurban__ joined #moarvm
07:52 sorear joined #moarvm
07:52 cxreg joined #moarvm
07:52 synopsebot joined #moarvm
07:52 chipdude joined #moarvm
07:52 tokuhirom joined #moarvm
08:14 moritz 08:41 < arnsholt_> MoarVM on the HN frontpage!  https://news.ycombinator.com/item?id=7357670
08:17 odc joined #moarvm
08:27 FROGGS joined #moarvm
08:35 tgt joined #moarvm
10:05 woosley joined #moarvm
10:06 woosley1 joined #moarvm
10:11 yawnt joined #moarvm
10:11 yawnt hi :P
10:13 FROGGS hi yawnt
10:48 avar joined #moarvm
13:11 tgt joined #moarvm
13:23 JimmyZ timotimo and anyone: https://github.com/dtrace4linux/linux :)
13:25 timotimo oh, not bad
14:40 tgt joined #moarvm
14:58 jnap joined #moarvm
15:12 FROGGS[mobile] joined #moarvm
15:33 FROGGS joined #moarvm
15:36 tgt joined #moarvm
16:13 cognominal joined #moarvm
17:27 benabik joined #moarvm
17:48 TimToady my @bar = (gather take 42) while 1;
17:48 TimToady that is a very fast leak on maorvm
17:52 flussence joined #moarvm
17:53 vendethiel joined #moarvm
18:02 isBEKaml joined #moarvm
18:03 jnthn Mebbe worth valgrinding...
18:03 isBEKaml OHHAI, I ran into this error, when I tried building MoarVM on Linux( Slackware - perl v5.10): http://sprunge.us/KGhQ
18:03 isBEKaml does moarvm require newer perl versions?
18:04 jnthn Not sure we ever laid down a requirement so far...
18:05 * TimToady has never valground, are there any experts in the house
18:05 jnthn Grrr...train is rocking a little too much on this section of hte journey for me to be able to keep looking at a screen without feeling ill :/
18:05 jnthn SJ, why you no lay track flat...
18:07 isBEKaml jnthn: I don't think it's really much to do with newer perl as much as asking for a newer perl module (File::Path, in this case).
18:09 isBEKaml it's build/probe.pm asking for unexported/absent subs in File::Path.
18:09 jnthn isBEKaml: OK. Probably whoever wrote the code had a newer version. Then, on this box I've a stock 5.16...
18:09 jnthn ...with no updated modules I know of. And it's working.
18:09 jnthn nwc10: Dunno if you know, given I think you worked on probe code?
18:10 jnthn oh dears...
18:10 jnthn src\strings\ops.c(1390) : error C2275: 'MVMCodepoint32' : illegal use of this type as an expression c:\consulting\moarvm\src\6model/reprs/MVMString.h(4) : see declaration of 'MVMCodepoint32'
18:12 dalek MoarVM: 3787171 | jonathan++ | src/strings/ops.c:
18:12 dalek MoarVM: Unbust MSVC build.
18:12 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/3787171d46
18:12 isBEKaml Ah, it's mkpath and rmtree in my version of File::Path
18:15 isBEKaml Why was the module's API changed? :/ (Modifying build/probe.pm to use mkpath and rmtree got me going, for now)
18:20 jnthn TimToady: Reproduced the leak. Maybe I'll find it.
18:22 TimToady it's a full fart leak :)
18:22 tgt joined #moarvm
18:22 isBEKaml Alright, he's experiencing that in a train :-)
18:23 TimToady I daresay the leak is going faster than his wifi...
18:28 isBEKaml https://metacpan.org/pod/File::Path#API-CHANGES -- make_path and remove_tree use mkpath and rmtree under the hood. Can we not detect versions and fallback accordingly?
18:46 lizmat joined #moarvm
18:47 jnthn Well, I found a leak, but not the leak...
18:48 TimToady cool, find a bunch moar :)
18:50 lizmat .oO( Hans Brinker to the rescue )
18:53 TimToady is possible it's a space leak instead, perhaps in the eager implied by sink, if it doesn't actively throw stuff away as it's generated
19:00 colomon joined #moarvm
19:05 nwc10 I hadn't realised how "new" make_path and remove_tree are
19:05 nwc10 fix is to rework the code to use the older functions
19:05 nwc10 I certainly don't have time today
19:05 nwc10 I won't on Sunday
19:05 nwc10 I may not tomorrow either
19:06 nwc10 it's a SMOP for anyone who knows Perl
19:06 isBEKaml nwc10: I don't :-)
19:06 nwc10 most of next week looking unlikely
19:12 isBEKaml nwc10: It's not a big deal anyway. I got rakudo-m built and ready after changing probe.pm to use mkpath and rmtree.
19:12 cognominal joined #moarvm
19:38 nwc10 yes, but I think we should patch the probe code
19:40 brrt joined #moarvm
19:45 jnthn OK, I managed to get a build with the CRT debug malloc that can do leak detection.
19:45 FROGGS coool
20:04 TimToady I'll bet my leak is "special".  :)
20:04 TimToady but I'd be glad to lose that bet...
20:08 jnthn Trouble is, we've not been too good at making sure stuff is freed at exit
20:08 jnthn So there's a ton of leakage
20:08 jnthn (Knew I'd need to take it on at some point... :))
20:11 dalek MoarVM: efa454b | jonathan++ | src/6model/reprs/MVMContinuation.c:
20:11 dalek MoarVM: Fix leak of active handlers in continuation.
20:11 dalek MoarVM:
20:11 dalek MoarVM: If a continuation is never resumed but it saved handlers, they would
20:11 dalek MoarVM: never become active again and would thus be leaked.
20:11 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/efa454bd98
20:11 dalek MoarVM: 46000f6 | jonathan++ | src/gc/gen2.c:
20:11 dalek MoarVM: Clear up gen2 memory at exit.
20:11 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/46000f6837
20:18 TimToady p5 is very picky about cleaning up when it knows it's embedded
20:18 TimToady if it's not embedded, it only cleans up enough to attempt to run all DESTROYs
20:19 jnthn Well, I'd rather cheat once we've learned to not cheat :)
20:19 TimToady since exit is a rather efficient way to clean up the rest of 'em
20:19 TimToady ayup
20:20 evdivan joined #moarvm
20:20 jnthn I'm gonna go through this list and zap the easy ones, to try and work out where the real ones are.
20:23 dalek MoarVM: 4711808 | jonathan++ | src/6model/reprs/MVMStaticFrame.c:
20:23 dalek MoarVM: Don't leak static frame instruction offsets.
20:23 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/47118085ea
20:49 dalek MoarVM: e62d7c7 | jonathan++ | src/core/ (3 files):
20:49 dalek MoarVM: Clear up frame pool at exit.
20:49 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e62d7c7a89
20:51 cognominal joined #moarvm
21:00 jnthn Somehow, hashes leak somewhat.
21:06 vendethiel left #moarvm
21:21 dalek MoarVM: b07e028 | jonathan++ | src/6model/reprs/MVMHash.h:
21:21 dalek MoarVM: Avoid leaking 72 bytes per MVMHash.
21:21 dalek MoarVM:
21:21 dalek MoarVM: In gc_free for MVMHash, due to HASH_CLEAR clearing the head pointer,
21:21 dalek MoarVM: we ended up never freeing the memory for the first node in a hash.
21:21 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b07e028d66
21:22 jnthn TimToady: So, having cleaned up this much, I can confirm that we seem to be leaking frames.
21:25 jnthn TimToady: Hm, but it doesn't seem to be enough to account for the loss
21:26 jnthn ooh, nearly home...
21:26 jnthn bbiab
22:55 * lizmat hopes jnthn got home safely
22:57 timotimo hm. i wonder if the leak reduction for mvmhash will cause better startup memory usage yet
23:01 jnthn lizmat: Yeah, just a bit wet :)
23:01 jnthn timotimo: tits
23:01 tadzik what
23:02 jnthn try it to see
23:02 jnthn :P
23:02 tadzik :D
23:02 jnthn timotimo: It will probably reduce CORE.setting comp memory
23:02 jnthn timotimo: 'cus we allocate and throw away a lot of hashes.
23:05 timotimo that's very good.
23:05 timotimo i'll try it out, actually
23:07 jnthn Yeah, I'm just doing a build here too
23:11 timotimo 89852maxresident - not really seeing a difference here
23:12 jnthn Pretty sure CORE.setting build was lower though.
23:12 timotimo oh, that
23:12 timotimo should verify.
23:12 jnthn yeah, I only eyeballed it.
23:13 timotimo i'm verifying it now
23:13 timotimo what revision should i revert moar to?
23:13 timotimo Unbust MSVC build. ← this one?
23:14 timotimo wowza. 620 megabytes of maxrss for setting compilation
23:15 timotimo absolutely speaking, that's quite a lot of memory
23:15 timotimo relatively speaking, that's probably way better than parrot
23:15 timotimo oh wow. can this be real?
23:15 timotimo 67.68user 0.28system 1:08.20elapsed 99%CPU (0avgtext+0avgdata 742696maxresident)k
23:15 timotimo 70.31user 0.22system 1:11.00elapsed 99%CPU (0avgtext+0avgdata 619296maxresident)k
23:16 * timotimo collects more datapoints
23:17 jnthn Yes, unbust is the one.
23:17 jnthn Hm, is that suggesting we went from 743 to 620 with my fixes? :)
23:18 timotimo yes
23:18 jnthn wow.
23:18 timotimo so leak
23:18 timotimo much garbage
23:18 jnthn so fix
23:19 timotimo at least the memory usage seems pretty stable in the "before" version
23:23 timotimo the numbers seem to be legit
23:23 timotimo good catch!
23:25 * jnthn does himself a leak detecting build on his main dev machine
23:26 timotimo the memory consumption numbers are quite stable
23:28 timotimo there's also leakage happening caused by string decoding and/or encoding according to valgrind
23:28 timotimo want me to run the analysis?
23:28 timotimo i wonder how long the core setting takes when running under valgrind
23:29 timotimo since we do our own allocations in our own heaps for a big chunk of all allocations, it may not be terribly slow compared to normal programs that use malloc for every little thing
23:30 jnthn timotimo: I see them here too, so no need to valgrind unless you also want to go hunting. :)
23:33 timotimo oke
23:33 timotimo í'll do it just to find out how much longer it takes
23:33 timotimo hm. usually like 20x longer, right?
23:35 jnthn no idea :)
23:36 jnthn TimToady: With the cruft cleared up, it's fairly clear we're leaking frames in the gather/take, which is why your example swallows so much memory.
23:37 TimToady yay!
23:37 jnthn mmm...rochefort 10...
23:38 [Coke] so glad I get to program in perl6 and not c when jnthn is done. :)
23:38 jnthn Only question now is why...
23:40 timotimo why rochefort?
23:40 TimToady [Coke]: well, I still note that the narcissist code equivalent to that which runs hours in Perl 6 ran in about .3 seconds in D, so we still have a ways to go...
23:43 jnthn timotimo: 'cus it tastes nice! :P
23:43 timotimo heh
23:44 timotimo hm. now i have to go. i can't suspend the laptop during the run because that'd give me wildly exaggerated measurements
23:44 timotimo ctrl-c'd it and got these numbers: ==8728==    definitely lost: 3,584,385 bytes in 98,440 blocks
23:44 timotimo ==8728==    indirectly lost: 1,226,998 bytes in 9,340 blocks
23:44 timotimo ==8728==      possibly lost: 54,773 bytes in 119 blocks
23:45 timotimo probably just means moarvm doesn't bother cleaning up when you ctrl-c it
23:45 kfarhayarok joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary