Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-02-06

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:08 pyrimidine joined #moarvm
02:11 pyrimidine joined #moarvm
02:15 KDr2 joined #moarvm
02:24 pyrimidine joined #moarvm
03:00 pyrimidine joined #moarvm
03:04 pyrimidine joined #moarvm
03:05 zakharyas joined #moarvm
03:29 agentzh joined #moarvm
04:06 pyrimidine joined #moarvm
04:59 pyrimidine joined #moarvm
05:05 pyrimidi_ joined #moarvm
06:06 pyrimidine joined #moarvm
06:33 pyrimidine joined #moarvm
07:14 KDr2 left #moarvm
08:36 pyrimidine joined #moarvm
09:55 pyrimidine joined #moarvm
10:07 jnthn moarning, #moarvm o/
10:07 yoleaux2 5 Feb 2017 20:24Z <japhb> jnthn: Did you get my response to RT #130716?  It isn't appearing for me on the RT site, but I don't know if my response got somehow routed to you and not included in the bug log, or if it just got lost entirely.
10:07 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=130716
10:07 yoleaux2 01:56Z <japhb> jnthn: Sheesh, it just took *several hours* to show up.  WTH, RT?
10:13 dogbert17_ hello jnthn
10:14 dogbert17_ I have some more comments wrt harness6 if you're interested
10:15 dogbert17_ there are at least two problems with harness6, one was discussed the other day, running out of filehandles
10:21 dogbert17_ the interesting thing with that is that this problem only occurs if TEST_JOBS=1, if set to 1+, the filehandle problem disappears
10:24 dogbert17_ on the other hand, even though TEST_JOBS=1+ avoids the filehandle problem another problem plagues that setup, i.e. memory exhaustion at least on 32 bit
10:29 dogbert17_ so running make spectest HARNESS_TYPE=6 [TEST_JOBS=1] fails due to the filehandle progrem while make spectest HARNESS_TYPE=6 [TEST_JOBS=1+] runs out of memory (2.5-3 gigs on my 32 bit machine)
10:31 jnthn Hmm, I wonder if TEST_JOBS=1 uses Proc rather than Proc::Async...
10:32 dogbert17_ for the first problem, see https://gist.github.com/dogbert17/069bf4c7614f5467850370e3a1a9293a and for the other see https://gist.github.com/dogbert17/a74e36cd2dd4843773134b6bcbc78931 which contains a tiny bit of valgrind output after having run a reduced spectest, i.e. ca 25 test files from S01 and S02
10:33 dogbert17_ for problem two the amount of 'still reachable' memory grows alarmingly
10:57 samcv hi jnthn
10:58 pyrimidine joined #moarvm
10:59 samcv .tell japhb that probably happened because you are not added to the rt list or something. send an email to That one was waiting in human moderation because you're not subscribed to the list. email perlbug-admin@perl.org and they can add you
10:59 yoleaux2 samcv: I'll pass your message to japhb.
11:00 agentzh joined #moarvm
11:00 samcv .tell japhb otherwise takes human moderation for it to go through
11:00 yoleaux2 samcv: I'll pass your message to japhb.
11:00 dogbert17_ jnthn: any ideas wrt the second gist?
11:01 jnthn dogbert17_: Well, it's the final leak source htat's the interseting one, I think
11:02 jnthn Though...does this not have --full-cleanup?
11:02 dogbert17_ nope
11:03 jnthn Hm, so hard to know if it's just poor collection timing or a real managed leak
11:03 jnthn Those buffers, I'm pretty sure, get wrapped up in a MVMArray
11:04 dogbert17_ well, when I ran a full spectest I got a MoarVM Panic, failed to allocate memory. I had run ps just before and VSZ was 3000000 and RSS 500000
11:05 dogbert17_ I can run it again with full cleanup if you want
11:05 zakharyas joined #moarvm
11:22 pyrimidine joined #moarvm
12:08 samcv hmm jnthn ok so i'm trying to implement multi-level collation modes now
12:08 jnthn Sounds fun :)
12:08 samcv nqp::unicmp_s('a', 'b', 15, 0, 0)
12:08 samcv collation_mode[0] lang_mode[48] country_mode[0]
12:08 samcv :\
12:08 samcv wat
12:09 jnthn o.O
12:09 samcv try it again and i get collation_mode[0] lang_mode[657072792] country_mode[0]
12:09 timotimo you're clearly reading uninitialized memory?
12:09 samcv they need to be MVMint64 maybe?
12:09 samcv instead of 32?
12:09 timotimo must be the right thing in both ends
12:09 jnthn Or wrong register reading?
12:10 timotimo well, the same thing
12:10 timotimo all our int registers are 64bit
12:10 samcv ok then i will change it to MVMint64
12:10 timotimo smaller registers are 64bit big but we (try to) mask out the high bits whenever they are used
12:10 jnthn unicmp_s            w(int64) r(str) r(str) r(int64) r(int64) r(int64)
12:10 samcv yeah that's what i guessed
12:11 samcv i should stay with int64 even if i don't need 64 bit?
12:11 jnthn OP(unicmp_s):
12:11 jnthn GET_REG(cur_op, 0).i64 = MVM_unicode_string_compare(tc,
12:11 jnthn GET_REG(cur_op, 2).s, GET_REG(cur_op, 4).s);
12:11 samcv i only need like 1000 max for any of them i think
12:11 timotimo yeah, that'd make it easier
12:11 timotimo hah
12:11 jnthn Notice it doesn't even read/pass the last two args
12:11 timotimo you forgot to pass all the other values
12:11 jnthn To the function
12:11 jnthn So they end up as junk
12:11 samcv ah
12:11 timotimo and C be like "didn't pass any of those arguments? doesn't matter."
12:11 samcv lol.
12:11 jnthn heh, why on earth doesn't it warn though...
12:11 samcv don't care if you passed it or not! i'm gonna make up some for you!
12:12 samcv some sane values
12:12 samcv to just toss in there
12:12 timotimo well, i wouldn't notice a warning like that, because my terminal is always spammed with that diagnostic error
12:12 jnthn Well, they're integers, so that's fine, right? :)
12:12 samcv :P
12:12 timotimo well, calling convention is "just take arguments from the registers"
12:12 jnthn :)
12:12 timotimo and registers are always "initialized" :P
12:12 jnthn Time for lunch, bbiab
12:13 timotimo so it could have been that you put an inline assembly block before the call that sets the registers manually!
12:13 samcv what do i need to change it to jnthn
12:13 timotimo in that case, it would clearly be fine and totally what you meant and want!
12:13 samcv heh timotimo
12:13 timotimo well, put GET_REG(cur_op, 6).i64, GET_REG(cur_op, 8).i64, GET_REG(cur_op, 10).i64 into the parethesis
12:13 samcv kk
12:13 jnthn Add something like: GET_REG(cur_op, 6).i64, GET_REG(cur_op, 8).i64, GET_REG(cur_op, 10).i64
12:14 jnthn To pass the 3 extra args to the function call
12:14 jnthn Really off to eat :) bbi30
12:14 timotimo at least you're already incrementing the cur_op by the right amount
12:14 timotimo even though you didn't read any of the "late" values
12:15 samcv now i get too many
12:16 samcv timotimo, https://gist.github.com/samcv/3632be9e3f90944fbb14353080740959
12:17 timotimo you only changed the function signature in the .c, not the .h?
12:17 samcv i didn't change the number in the c function for MVM_unicode_string_compare
12:17 samcv it always had that many arguments
12:17 timotimo but you turned 32 into 64
12:17 samcv i changed the type tho. will check any .h
12:17 timotimo that could get you into trouble
12:18 timotimo but since you didn't get into trouble for not passing the right amount of parameters ... ?!?
12:18 samcv oh yeah it is too little in the .h
12:18 samcv hehehe
12:18 timotimo you're supposed to get into trouble for having different stuff in the .h and in the .c
12:18 samcv there's no ints in the .h
12:18 timotimo but somehow you seem to get away with anything ever?
12:18 * samcv is in trouble
12:18 samcv basically
12:19 timotimo i suppose that's fair
12:19 timotimo though i was hoping for compile-time trouble
12:19 samcv jolly good. it compiled now
12:19 samcv perfect :)
12:23 pyrimidine joined #moarvm
12:25 samcv now to get it to work so it won't break ties if i unset quaternary
12:26 timotimo if it doesn't break ties, it just returns 0 if all else comes up as "equal"?
12:26 samcv yeah
12:27 samcv that is one of the configurable things
12:27 samcv quanternary is breaking ties by cp
12:27 samcv quanternary is not defined by unicode other than 'whatever you want to do here'
12:28 timotimo that's fair
12:28 samcv tbh idk how many cases there would be where collation would come up the same
12:28 samcv not likely i think
12:28 samcv but it is more likely if you ignore some of the collation levels probably
12:29 samcv like if you ignore diacritics (second level)
12:29 timotimo in the near future i'll never be able to trust that a sequence of strings that has been sorted and given to my program won't change when i try to sort it again inside my program
12:29 samcv you may want to unset quaternary so it won't break ties by cp
12:29 timotimo thanks unicode
12:29 samcv just don't set $*COLLATION and it should be predictable, or just use unicmp and not coll
12:29 samcv or something
12:29 timotimo no, i mean
12:29 timotimo what if i get my list of strings from a python program?
12:30 samcv oh lol
12:30 timotimo or a database of some kind
12:30 samcv python sorts by cp probably right
12:30 samcv idk. java doesn't sort by cp
12:30 timotimo no clue
12:30 samcv it sorts 'lexigraphically' which is undefined for the most part
12:30 harrow joined #moarvm
12:30 timotimo :D
12:30 timotimo well, not completely undefined, but at least underdefined
12:30 samcv i thought that was hillarious
12:30 samcv basically implying that don't trust it'll always be exactly the same
12:31 timotimo ugh
12:31 samcv though it'll always say whether they are equal or not, it may not have the same 1 or -1
12:31 samcv who knows.
12:31 timotimo so at least it'll report same for one ä and the other ä
12:31 samcv will it?
12:31 timotimo well, i certainly hope so
12:31 samcv what are these two a's
12:32 timotimo those two äs are actually the same ä
12:32 samcv what are the two ones tho
12:32 timotimo because my keyboard won't let me easily input both variants
12:32 samcv what are their names
12:32 timotimo well, a + diaresis, and &auml;
12:32 samcv or you mean one with diacritic and one just its own character?
12:32 samcv that has the :?
12:33 timotimo yeah, ä has its own character, but you can also compose it
12:33 timotimo u: ä
12:33 timotimo .u ä
12:33 yoleaux2 U+00E4 LATIN SMALL LETTER A WITH DIAERESIS [Ll] (ä)
12:33 samcv m: 'ä'.NFD.say
12:33 camelia rakudo-moar 206148: OUTPUT«NFD:0x<0061 0308>␤»
12:33 timotimo funny that the html/xml entity name is "auml", but it's "diaresis" in the unicode name
12:33 samcv m: Uni.new(0x61, 0x308).Str.ords
12:33 camelia rakudo-moar 206148: ( no output )
12:33 samcv m: Uni.new(0x61, 0x308).Str.ords.say
12:33 camelia rakudo-moar 206148: OUTPUT«(228)␤»
12:33 samcv well it recomposes
12:33 samcv at least in perl 6
12:34 samcv but if the case that it does not recompose, unicmp will show the same i believe
12:34 samcv as long as unicode collation says so
12:34 samcv which it probably does, or they are sorted very closely
12:34 timotimo in perl6 it certainly will
12:34 timotimo but what about python and others? :)
12:35 samcv heh
12:35 samcv tell them to sort better
12:35 timotimo sort out your sort?
12:35 samcv yep
12:35 samcv sort out your shit
12:35 timotimo shit your values in the right order?
12:35 pyrimidine joined #moarvm
12:36 timotimo anyway. let's spend some big part of every day high-fiving ourselves because we'll now be able to sort emoji correctly
12:41 samcv hah
12:41 samcv yes
12:41 samcv ^5
12:41 timotimo
12:42 samcv ** 5
12:42 timotimo pow($_, 5)
12:42 samcv hah
12:43 pyrimidine joined #moarvm
12:48 lucasb joined #moarvm
12:49 * jnthn back from lunch
12:49 lucasb can I hi5 too? yay
12:49 lucasb o/
12:49 samcv sweet got it working
12:49 samcv yes you can lucasb
12:49 jnthn nice :)
12:49 samcv we all have to high 5 all day long
13:06 Geth ¦ MoarVM: ab225fa94d | (Samantha McVey)++ | 3 files
13:06 Geth ¦ MoarVM: Implement configurable collation_mode for MVM_unicode_string_compare
13:06 Geth ¦ MoarVM:
13:06 Geth ¦ MoarVM: This change causes the collation_mode argument to affect how the strings are
13:06 Geth ¦ MoarVM: compared. Quaternary level will ignore differences in codepoints and only
13:06 Geth ¦ MoarVM: rely on the collation values.
13:06 Geth ¦ MoarVM:
13:06 Geth ¦ MoarVM: This also fixes a problem in interp.c where it did not pass the three integer arguments,
13:06 Geth ¦ MoarVM: and also fixes the arguments being incorrectly set to MVMint32 instead of MVMint64
13:06 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ab225fa94d
13:06 timotimo yes yes, very good
13:07 moritz we're comparing strings with Quaternions!
13:11 dogbert17_ jnthn: sry, was called away on a $meeting, will run valgrind with --full-cleanup now
13:16 jnthn dogbert17_: No hurry at all, I'm fairly tied up with other stuff today anyway
13:16 timotimo so ... you know how there's richard wagner's walkürenritt? wouldn't it be funny if someone put up a version of that slowed down to 0.05x and called it "valgrindritt" or something?
13:22 dogbert17_ lol
13:24 dogbert17_ jnthn: you'll see some (shocking ?) results shortly
13:24 lucasb timotimo: the song that plays in that movie scene in vietnam with helicopters? :)
13:24 timotimo yeah
13:24 lucasb :D
13:24 timotimo apocalypse now, isn't it?
13:25 lucasb yes, that one
13:26 timotimo BBIAB
13:36 dogbert17_ jnthn, timotimo: dramatic results can be found here: https://gist.github.com/dogbert17/a74e36cd2dd4843773134b6bcbc78931
14:37 pyrimidine joined #moarvm
14:43 pyrimidine joined #moarvm
15:01 agentzh joined #moarvm
15:26 pyrimidine joined #moarvm
15:46 lizmat joined #moarvm
16:07 pyrimidine joined #moarvm
17:09 pyrimidine joined #moarvm
17:12 dogbert17 joined #moarvm
18:00 pyrimidine joined #moarvm
18:06 pyrimidine joined #moarvm
18:21 pyrimidine joined #moarvm
18:27 pyrimidine joined #moarvm
18:53 lucasb joined #moarvm
19:03 agentzh joined #moarvm
19:07 lucasb What are the feelings of jnthn and the others about moar's documentation? any future plans?
19:07 lucasb Adding markdown files in the docs dir or creating a new doc repo would be considered useful/helpful?
19:13 TimToady it's all a trade secret!
19:14 lucasb oh :)
19:28 lucasb I was trying to learn moar by reading a little code and writing notes into markdown files.
19:28 lucasb Then I got sidetracked playing with the idea of generating a static site.
19:28 lucasb Most pages are empty and those that are not contains inacurate information from my notes: https://lucasbuchala.github.io/moredocs/
19:28 lucasb So I ask, is there any interest by the MoarVM community in adopting this documentation skeleton as a base for Moar's community doc repo?
19:29 lucasb the source of the site is a single directory with markdown files and some build scripts
20:06 pyrimidine joined #moarvm
20:28 jnthn Documentation for MoarVM is kinda a question of "who is the audience"
20:30 lucasb joined #moarvm
20:31 jnthn Tied for time, I've figured my best bet on docs so far is to try and stick comments explaining various key decisions/designs into the code. And then just spend some effort trying to make the code clear.
20:32 lucasb I see the audience as definitely not Perl 6 users; but anyone interested in Moar's internals, C source code
20:32 lucasb I noticed the very nice comments along the way, jnthn++, thanks for that
20:33 jnthn Yeah, the question is what (a) will help people in ways reading the code cannot, and (b) is likely to stand the test of time well enough to not become out-dated quickly and thus confuse people more than if it weren't there :)
20:33 jnthn I think there's plenty of such things that exist.
20:34 jnthn The MVMROOT macro and related is one example
20:34 jnthn Something on overall structure is another.
20:35 jnthn I'd probably be more inclined to link to appropriate bits of code than duplicate it in docs.
20:35 lucasb right, understood
20:36 jnthn Ah, gotta go to the store before it closes...back in 15-20 mins
20:44 lucasb The source is there for anyone interested in reading it. One could start reading it from start to end, or jump around until the person starts to retain the vocabulary of the code. Wouldn't it be nice to have a more "holding hands" experience of guiding through the code?
20:44 lucasb The intention is not to duplicate code, but just show the main pieces of it, and how they work. Sure, it can get outdated quickly, and what doesn't? It was just a try anyway :)
21:01 dogbert17 jnthn: I have some additional info wrt the problems with HARNESS_TYPE=6
21:05 dogbert17 if I run 'make spectest HARNESS_TYPE=6 TEST_JOBS=2' with an unchanged MoarVM, i.e. #define MVM_NURSERY_SIZE 4194304 it will run out of memory. Here's a 'ps' just before the crash
21:05 dogbert17 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
21:05 dogbert17 dogbert  24198 37.1 16.0 2952656 497040 pts/2  Sl+  19:50   7:32 /home/dogbert/repos/rakudo/install/bin/moar --execname=./perl6-m --libpath=/home/dogbert/repos/rakudo/install/share/nqp/lib --libpath=/home/dogbert/repos/rakudo/install/share/nqp/lib --libpath=. /home/dogbert/repos/rakudo/perl6.moarvm --nqp-lib=blib -Ilib t/harness6 --fudge --tests-from-file=t/spectest.data
21:07 dogbert17 if I change MVM_NURSERY_SIZE to, e.g., (32768 * 8) the program will NOT run out of memory. Here's a 'ps' just before it's done testing
21:07 dogbert17 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
21:07 dogbert17 dogbert  24198 37.1 16.0 2952656 497040 pts/2  Sl+  19:50   7:32 /home/dogbert/repos/rakudo/install/bin/moar --execname=./perl6-m --libpath=/home/dogbert/repos/rakudo/install/share/nqp/lib --libpath=/home/dogbert/repos/rakudo/install/share/nqp/lib --libpath=. /home/dogbert/repos/rakudo/perl6.moarvm --nqp-lib=blib -Ilib t/harness6 --fudge --tests-from-file=t/spectest.data
21:07 dogbert17 fail, pasted the first ps again it should be:
21:08 dogbert17 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
21:08 dogbert17 dogbert  31864 44.7  5.6 516208 175928 pts/2   Sl+  20:13  12:51 /home/dogbert/repos/rakudo/install/bin/moar --execname=./perl6-m --libpath=/home/dogbert/repos/rakudo/install/share/nqp/lib --libpath=/home/dogbert/repos/rakudo/install/share/nqp/lib --libpath=. /home/dogbert/repos/rakudo/perl6.moarvm --nqp-lib=blib -Ilib t/harness6 --fudge --tests-from-file=t/spectest.data
21:09 dogbert17 does the above give you any clever ideas?
21:11 pyrimidine joined #moarvm
21:16 jnthn lucasb: Yes, good point. I agree some kind of "map" would be useful. Something that explains what the different directories under src/ are about what be good and quite lasting. And some "reading tips" for each - both source code and background. For example, in src/spesh/ it's not obvious that graph.h is the most important file to start reading, and links to what a control flow graph, basic block, and SSA form are would be useful, for example.
21:17 jnthn dogbert17: Interesting. It suggests we're simply not reclaiming memory anything close to fast enough.
21:18 dogbert17 any theories as to why?
21:19 jnthn Various
21:19 dogbert17 :-)
21:20 jnthn Did you try playing with the gen2 collection thresholds here: https://github.com/MoarVM/MoarVM/blob/master/src/gc/collect.h#L6
21:20 dogbert17 no; i have only changed MVM_NURSERY_SIZE, never touched the others. Any suggestions?
21:21 dogbert17 for values I mean
21:21 jnthn Well, they control how often we do a full collect
21:21 jnthn The algorithm using them is in orchestrate.c
21:21 jnthn I think it should be relatively clear how it's doing that decision-making
21:22 jnthn But smaller means "will do gen2 sooner"
21:22 jnthn Well, full collection, which includes, gen2, really.
21:22 jnthn If that makes no difference then it suggests we might need some kind of memory pressure mechanism for the nursery
21:24 dogbert17 I'll do a quick experiment then, setting MVM_GC_GEN2_THRESHOLD_PERCENT   10
21:25 dogbert17 I'll set VM_GC_GEN2_THRESHOLD_MINIMUM  to  (10 * 1024 * 1024) as well
21:54 dogbert17 it worked with 10 instead of 20
21:54 dogbert17 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
21:54 dogbert17 dogbert  16230 39.0 11.5 1279656 359588 pts/2  Sl+  22:27   9:56 /home/dogbert/repos/rakudo/install/bin/moar ...
22:01 dogbert17 stupid question, how large can the value of 'promoted' be?
22:08 jnthn Well, it uses size_t in the calc, though it's stored as MVMuint32, so 4 gigabytes
22:08 jnthn If you mean maximum value
22:08 jnthn Or do you mean how large can it become between tests?
22:10 dogbert17 I was looking at this code
22:10 dogbert17 /* Otherwise, consider percentage of resident set size. */
22:10 dogbert17 if (uv_resident_set_memory(&rss) < 0 || rss == 0)
22:10 dogbert17 rss = 50 * 1024 * 1024;
22:10 dogbert17 percent_growth = (100 * promoted) / rss;
22:12 dogbert17 was wondering if promoted is somehow bounded in any way by some constant in the source
22:12 jnthn Oh
22:12 jnthn No, not other than the data type it's stored in
22:13 jnthn We add on to promoted every time we move an object from nursery to gen2
22:13 jnthn Both the object's body size, and then for objects that report it, any extra memory they hold indirectly that the GC won't otherwise know about
22:14 dogbert17 ok, in my runs rss is quite large so I guess it will take time before promoted is large enough to cause a full gc
22:14 dogbert17 or have I misunderstood /o\
22:16 jnthn No, you probably did
22:16 jnthn But
22:16 jnthn If there's a huge discrepancy between promoted and RSS then we might want to look into why
22:17 jnthn Uh, that ain't quite right
22:17 jnthn If there's a huge discrepancy between RSS growth and how big MoarVM thinks the heap is, then that's an issue.
22:17 jnthn As it means promoted is under-estimating.
22:18 dogbert17 how can we figure out if that is the case?
22:18 jnthn moar-ha
22:18 jnthn And --profile=heap
22:18 dogbert17 moar-ha ?
22:18 jnthn You can install the first with [zef|panda] App::MoarVM::HeapAnalyzer
22:18 jnthn moar-ha is the name of the "binary" it installs
22:19 jnthn That said, moar-ha itself slightly distorts collection rates
22:19 dogbert17 I saw a comment to that effect
22:20 dogbert17 what about these GCDEBUG_LOG statements in the code, might thay help and how do you see them?
22:20 jnthn Still, if there's a huge discrepancy between usual (non-profiled) RSS and the heap size...
22:20 jnthn Those are more usual for investigating a different class of bugs mostly
22:20 jnthn *more useful
22:21 jnthn For example if there are problems with inter-thread coordination of GC runs
22:21 dogbert17 aha
22:21 jnthn I've not needed them for quite some months, thankfully
22:21 jnthn (Because any problem I'm hunting with those turned on will typically be horrible.)
22:21 dogbert17 with some luck most of the horrible problems are gone now
22:22 dogbert17 I had better read up on the heap analyzer, didn't you blog about that?
22:26 jnthn Yes, the README in its github repo has a link to the blog post
22:27 dogbert17 will look into to it
22:28 jnthn Cool
22:28 * dogbert17 might have gotten in over his head ...
22:28 jnthn One other thing to check; if you lower the threshold to 10%, what is the CORE.setting build time difference?
22:28 jnthn (I tuned the numbers using that)
22:29 dogbert17 I didn't check that
22:32 dogbert17 did you see the --full-cleanup gist I posted earlier?
22:32 dogbert17 i.e. https://gist.github.com/dogbert17/a74e36cd2dd4843773134b6bcbc78931
22:35 jnthn Yes, it confused me some though
22:35 jnthn As it left behind things I'd not expect --full-cleanup to...
22:35 jnthn Like nurseries
22:36 dogbert17 is it because valgrind died, it says 'killed' on the last line
22:36 jnthn Oh, yes, it will be
22:37 jnthn m: say 60_489_728 / 923
22:37 camelia rakudo-moar 4efcc2: OUTPUT«65536␤»
22:37 jnthn That's a magic number :)
22:37 jnthn I thought that I'd already fixed procops.c to correctly track sizes though
22:38 dogbert17 maybe something erroneous has been mistakenly reintroduced
22:49 jnthn Mebbe, though it was a pretty recent patch...
22:49 jnthn Sleep time for me; 'night
22:52 timotimo nite jnthn
22:53 timotimo it would have been nice to know about the "killed" before :)
22:54 dogbert17 yup, my bad :(
22:56 dogbert17 timotimo, did you read about the latest findings above?
22:56 timotimo i read a little bit
22:57 timotimo we're promoting things that we don't calculate the correct size of, and therefor don't do a full gc?
22:58 dogbert17 that could be the case, running make spectest HARNESS_TYPE=6 TEST_JOBS=2 eats all memory and crashes, making the nursery smaller removes the problem
22:58 pyrimidine joined #moarvm
22:58 timotimo interesting indeed
22:59 timotimo were you able to run it under the HA?
22:59 timotimo er, i mean, with the heap profiler output turned on
22:59 dogbert17 haven't tried it yet although I'm slighly sceptical, trying it on a oneliner is quite slow ...
23:00 timotimo it depends verily on how much gcing it does
23:00 timotimo each time it gcs, it records a full snapshot of the object graph (but leaves out anything but inter-object connections)
23:01 dogbert17 so should I make the nursery slightly smaller so that it barely avoids crashing or should I set it really small?
23:02 timotimo for the purpose of getting a moar-ha thing?
23:02 dogbert17 and to see if there's something fishy with the heap/rss/promoted
23:03 timotimo if you set it really small, it'll likely promote more stuff into gen2
23:04 dogbert17 yes, perhaps that is not what we want, with a normal nursery that doesn't seem ta happen too often since it runs out of memory
23:05 timotimo oh, i think if it crashes it won't write out the heap dump, i'm not sure about that
23:05 timotimo and it'll take a bunch of memory at the end to write out the heap snapshots
23:05 timotimo but also: it won't cross-reference system-reported RSS with the individual snapshots
23:06 timotimo maybe it'd be a better idea to just go through all the reprs and look at their managed_size functions?
23:06 dogbert17 ah, ofc, perhaps I should change spectest.data soo that it doesn't contain so many files, don't think we need to run through all of them
23:06 timotimo mhm
23:07 dogbert17 maybe it's enough to go through, let's say, 30 files in order to get good data
23:09 dogbert17 I'll read up on the heap analyzer and try it tomorrow, bedtime for me
23:09 dogbert17 night
23:11 timotimo nite
23:27 zostay_ joined #moarvm
23:30 ilmari_ joined #moarvm
23:30 konobi_ joined #moarvm
23:40 timotimo joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary