Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-10-11

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:48 ilbot3 joined #moarvm
01:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
05:11 geekosaur joined #moarvm
05:35 domidumont joined #moarvm
05:39 domidumont joined #moarvm
06:02 domidumont joined #moarvm
08:34 harrow` joined #moarvm
08:36 psch_ joined #moarvm
08:40 domidumont Weird, arm64 tests pass on a qemu chroot, but hangs on real arm64 HW
08:41 domidumont The hangs involve precompilation phase. It hangs only when the precompiled files are missing. See http://paste.debian.net/864333/
08:41 domidumont When the test hang, the CPU is idle
08:41 domidumont HTH
08:42 brrt good * #moarvm
08:43 brrt that definitely sounds like a deadlock too me
08:43 brrt domidumont: ^
08:46 domidumont brrt: I've seen similar symptoms on qemu/arm64 chroot before this fix https://github.com/MoarVM/MoarVM/commit/17f33bf03586941a70a25b0273501fa347920a64
08:47 brrt I'm puzzled as to why that should make a difference
08:49 domidumont it may be a mix of different issues. We also had test failure on armhf. As far as I know, only arm64 lead to tests hang
08:49 dalek joined #moarvm
08:49 flaviusb joined #moarvm
08:49 synopsebot6 joined #moarvm
08:49 JimmyZ joined #moarvm
08:49 BinGOs joined #moarvm
08:55 domidumont Hmm, test don't hang with MVM_SPESH_DISABLE=1
08:56 domidumont anything else I should try ?
10:36 FROGGS joined #moarvm
11:12 cygx joined #moarvm
11:12 cygx o/
11:12 cygx another fun Unicode issue: https://github.com/MoarVM/MoarVM/issues/427
11:12 jnthn Yup, they went and added prepends...
11:13 jnthn Only a problem since Unicode 9
11:13 cygx and forgot to give a sensible update for the definition of base char
11:13 jnthn Not sure on ord yet
11:14 jnthn Question is if we want ord($c) and ords($c)[0] to be different
11:14 jnthn But as for deriving character properties, it's fairly clear we don't want to base *those* on the prepend
11:15 jnthn Same for the tc/lc/uc/fc
11:17 cygx keeping .ord as an alias for .ords[0] sounds ok to me (assuming a replacement like .base gets added)
11:17 jnthn .base is taken :)
11:17 jnthn I could probably go with changing .ord also
11:17 cygx .base-char, .unibase, .first-non-prepend-char ;)
11:18 jnthn I don't think we promised .ord and .ords[0] equivalence anywhere
11:20 jnthn https://docs.perl6.org/routine/ord#(Str)_routine_ord says "base char" :)
11:20 cygx then, all good ;)
11:21 jnthn aye :)
11:21 jnthn Talking of good things...lunch :) &
11:31 stmuk_ joined #moarvm
12:49 zakharyas joined #moarvm
13:00 cygx jnthn: in my long tradition of throwing out half-baked ideas, here's another one on how to deal with synthetics representing denormal and compatibility input: https://github.com/cygx/p6-newio/blob/master/synth32.h
13:12 ilmari joined #moarvm
13:54 timotimo i'm watching Nicholas Ormrod's talk on "std::string at facebook" and they (in their fbstring) and clang (in their std::string) both have a representation of strings where short strings can live directly in the space that used to be for pointer, size, and capacity
13:54 timotimo we don't actually have a capacity value, so we'd have to make do with only 2/3rds as much space for "small strings"
13:54 timotimo though perhaps the member that'd usually encode what kind of storage we're using can become a part of this mess
13:54 timotimo oh, and of course their characters are probably 8bit
13:55 timotimo ok, i think this approach is not right for us.
13:57 timotimo they replaced std::string with fbstring and measured a 1% performance win across all of www.facebook.com
14:03 cygx well, we do special case ASCII strings, and for those, inlining the data into the MVMStringBody might be a win
14:03 cygx depending on architecture and whether we want to cache the hash code or not, there's place for 10+ chars
14:04 timotimo we *do* special case them ... but we don't have any code that deals with ascii strings, if i'm not mistaken
14:04 timotimo like, i don't think a single 8-bit-per-char string is alive in any moar instance out there
14:05 timotimo especially since we still have to turn every string into a flat MVMGrapheme32 string in order to calculate hashes
14:05 cygx we do get a fwe special cases in strings/ops.c
14:05 cygx *few
14:05 timotimo OK
14:05 timotimo ideally we'd be using the grapheme iterator api for everything
14:08 cygx or go the opposite direction: always use 32-bit codepoints, but flag the range (ascii-only, latin-1, 16-bit single-codepoint graphemes, 32-bit single codepoint graphemes)
14:08 [Coke] https://github.com/perl6/doc/pull/964/files#diff-0 - that's not a twigil.
14:08 [Coke] ... wrong window
14:09 cygx we could also put those into a flexile array mamber to reduce the number of allocations
14:09 cygx *member
14:09 timotimo i'm not sure what you mean by that
14:10 timotimo you mean "this string never contains anything outside of 0..127" for example?
14:10 cygx yes
14:10 cygx could be used for various optimizations
14:11 timotimo it'll allow us to fast-fail for same-length strings where one contains silly characters and the other doesn't and we want to eq them
14:11 cygx or encoding ASCII-only to UTF-8
14:11 timotimo mhm
14:13 cygx a flexible array member might not be a win if we then have to move the string contents around during GC
14:13 timotimo i don't know what "a flexible array member" is?
14:15 cygx struct { size_t size; uint32_t codes[]; }
14:15 cygx pre C99, you used zero-length array for that
14:16 timotimo oh
14:16 timotimo we can't have that kind of thing yet
14:16 timotimo our GC doesn't understand that
14:17 cygx the GC does not support variable-sized objects?
14:17 cygx where does the size live - the STable?
14:17 timotimo let me stop speaking about this and have a look before i embarass myself :)
14:18 timotimo the size actually lives at the end of the header of the object
14:18 jnthn The object's size can't change during its lifetime, but the size is stored in the header. But it's not intended you'll allocate large arrays directly into the nursery.
14:18 jnthn (The size is actually stored in 16 bits)
14:19 timotimo the only examples i know where we can have different sizes for object building is p6opaque and cstruct
14:19 timotimo in p6opaque, we allocate st->size bytes for the object
14:19 cygx I see
14:20 timotimo the thing about putting a string's body (or the same for arrays) inside the object itself is that it's only worth it if it's sufficiently small
14:20 timotimo on the other hand, that technique is usually combined with the fact that the objects are often allocated on the stack
14:21 cygx what could be tried is this: always have 32-bit units in regular strings but mark the range and embed short latin-1 strings directly
14:21 timotimo so you don't have to go from stack to heap at all, whereas we currently always allocate on the heap
14:21 cygx you could reserve a couple of bytes for that purpose at the end of the structure
14:21 arnsholt Putting the characters in the body of the string would save you a level of indirection though
14:24 cygx you would end up with regular 32-bit strings of RANGE_FULL, RANGE_ASCII, RANGE_LATIN1, RANGLE_BASIC16, RANGE_BASIC32, strands and embedded short latin-1 strings instead of the current variants GRAPHEME_32, GRAPHEME_ASCII, GRAPHEME_8, STRAND
14:24 cygx might be a loss if there are a lot of long-ish
14:24 cygx ... GRAPHEME_ASCII, GRAPHEME_8 strings
14:24 timotimo i had a bit of code lying around for caching strings that are a single grapheme
14:24 timotimo it didn't seem to make a noticable difference when i was combing through a large string
14:24 jnthn The point of the ASCII/_8 variants is largely for people doing processing of really long but ASCII-based strings
14:25 jnthn e.g. bio-inf folks
14:26 timotimo maybe in the benchmark i was using all those strings were actually strands-based rather than actually containing the storage. maybe that's why the cache didn't get hit?
14:27 cygx is the grammar engine able to/supposed to be able to deal with such byte strings?
14:27 jnthn Well, the grammar engine just uses string ops
14:27 jnthn So "provided the ops support it"
14:27 timotimo the regex engine will always run "flattenropes" first, which currently forces a string into the grapheme32 storage format
14:27 jnthn We'll need to finish decouploing hash from flattening strings though
14:27 timotimo yes, very.
14:27 jnthn Once *that* is done, then a flat 8-bit string is fine for the hash
14:28 jnthn uh
14:28 jnthn for the regex engine
14:28 jnthn And it'll only be strandy things that need the flattening
14:31 cygx if byte-sized strings are supposed to become first class, they should stick around
14:33 cygx MVMStrings.h also speculates about byte strings with synthetics - that would require the introduction of string-specific synthetic table (or rather tables for translation to the global one), correct?
14:33 jnthn No
14:34 jnthn More than in reality there'll probably not be that many synthetics
14:34 jnthn But perhaps it's better to just always go 32-bit once there are
14:34 jnthn Since if you do have synthetics, you'll probably have other non-8-bit things too
14:34 jnthn I suspect GRAPHEME_8 might want to go away
14:35 jnthn There was some thinking that you could do ASCII + use the negatives to compactly represent other codepoints
14:35 timotimo yeah, that'd totally be possible
14:35 jnthn So, for example, you can store something alphabetic like Russian in an 8-bit string
14:35 timotimo there'd have to be measurements
14:36 jnthn Which would be cool but...that's an optimization for some years into the future :)
14:39 ilmari joined #moarvm
15:04 timotimo jnthn: could it be the cross thread write log has some trouble staying threadsafe?
15:05 timotimo https://gist.github.com/timo/a586bda6065285dc2ff8725370b66d8c
15:06 FROGGS joined #moarvm
15:07 timotimo i'm not sure about the lifecycle of this stuff
15:07 FROGGS o/
15:08 timotimo why was spesh_graph_destroy freeing something that bytecode_finish_frame was allocating?
15:09 jnthn That's odd indeed
15:09 timotimo oh
15:09 timotimo here's a check that's probably bitrotted
15:09 jnthn I thought initial instrumentations took place under lock
15:09 timotimo "frees handlers array if different from the static frame"
15:10 timotimo if (!g->cand && g->handlers != g->sf->body.handlers)
15:10 timotimo probably wants to learn about not-yet-fully-decoded static frames?
15:11 timotimo or perhaps the first place we ask for the handlers from has to cause the sf to be fully decoded
15:24 dalek MoarVM: c1ada4b | timotimo++ | src/instrument/crossthreadwrite.c:
15:24 dalek MoarVM: include debug_name in crossthreadwritelog
15:24 dalek MoarVM:
15:24 dalek MoarVM: shows what type of object has been acted upon
15:24 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c1ada4bbac
15:25 timotimo i'd also like to give a bit more info on the operation performed. like what attribute name was accessed for example
16:25 FROGGS @all: is here somebody who has an arm system where rakudo compiles?
16:26 FROGGS (and I'm not talking about a qemu system)
16:27 FROGGS (nor a chroot)
16:27 FROGGS (which would be qemu)
17:31 domidumont joined #moarvm
17:45 domidumont FROGGS: arm64 hang-up are getting weirder: https://github.com/MoarVM/MoarVM/issues/428
17:48 FROGGS domidumont: can you post the output of lsof of the time it hangs?
18:08 FROGGS domidumont: another option would be to run the hanging command in gdb
18:09 FROGGS so it would be: time gdb --args /usr/bin/moar --execname=./perl6-m --libpath=/usr/share/nqp/lib --libpath=/usr/lib/perl6 --libpath=. perl6.moarvm t/04-nativecall/02-simple-args.t
18:09 FROGGS and when you ctrl+c, you should be in gdb IIRC and would be able to generate a backtrace
18:37 domidumont FROGGS: I've added the output of lsof in gh ticket
18:38 domidumont FROGGS: the test does not hand when run under gdb (timing issue ?)
18:39 domidumont s/hand/hang/
18:47 FROGGS domidumont: I have no clue why it hangs :o(
18:48 domidumont can't I active a trace, logs, to have a better idea what going on when it hangs ?
18:53 FROGGS timotimo: do we have something suitable?
18:54 domidumont it blocks on epoll_pwait(5,
18:54 domidumont I can provide the whole strace output
18:54 FROGGS that'd be awesome
18:56 domidumont github won't like it :o) (~ 1800 lines)
18:57 domidumont FROGGS: done
19:00 domidumont FROGGS: I can send you a file by mail if it's more convenient for you
19:00 FROGGS no, got it in my editor
19:00 timotimo if you have a tc
19:00 timotimo you can "call MVM_dump_backtrace(tc)"
19:00 timotimo in all of your threads
19:00 FROGGS well, but it hangs
19:13 FROGGS domidumont: does it help prepending "no precompilation;" to t/04-nativecall/CompileTestLib.pm ?
19:14 FROGGS also, could you output "tree t/04-nativecall/.precomp/" after a hang?
19:14 FROGGS (I' need to by an arm64 machine)
19:14 domidumont let me try (I'd guess yes, since the test does not hang once the precompilation has been done)
19:14 FROGGS I'd*
19:15 nwc10 FROGGS: http://geizhals.de/raspberry-pi-3-modell-b-a1400349.html -- Raspberry Pi 3 Modell B ab € 33,53
19:16 nwc10 although to be fair, I think that only FreeBSD currently tries to run as 64 bit, and I don't know how stable it is
19:16 FROGGS nwc10: and 1GB ram is enough?
19:16 nwc10 Might need to swap more for 64 BIT
19:17 nwc10 I think I had some swapping buidling on a Pi 2 (which is ARMv7, so only ever 32 bit) and 1G
19:17 nwc10 I had an external drive for swap, to avoid wearing out the SD card
19:17 nwc10 anyway, the disclaimer on this is "€ 33,53" but might be more trouble than your time is worth to get it working
19:18 FROGGS I once tried to build rakudo on an old raspberry 1 b+ with 512mb ram, and it just froze when compiling core setting
19:18 FROGGS domidumont: what type of machine do you have?
19:19 nwc10 took 24 hours on a 256MB early Pi 1 B, but with an external HD
19:19 nwc10 that was over a year ago
19:22 FROGGS domidumont: what we also could do is to run the test when the env var RAKUDO_MODULE_DEBUG is set to 1
19:25 domidumont same result with "noprecompilation": it does hand (but not always)
19:27 domidumont I'm using this machine : https://db.debian.org/machines.cgi?host=asachi
19:29 domidumont I've posted .precomp content on github
19:31 domidumont is that normal that .precomp has file even if "no precompilcation" is specified ?
19:32 FROGGS no, should be empty I think
19:34 FROGGS I end up with two files less when I prepend "no precompilation;" to said file
19:34 domidumont getting a hang with RAKUDO_MODULE_DEBUG is harder ... but I got one
19:35 FROGGS awesome
19:35 FROGGS would should compare a success and a fail
19:35 domidumont do you need the full trace ?
19:35 FROGGS yes
19:44 domidumont sent on github.
19:44 domidumont And that's it for today. It's getting late ...
19:45 domidumont Let me know if you need more, I'll resume tomorrow
20:08 timotimo FROGGS: when something hangs, just ctrl-c it a few times and see where it travels
20:08 FROGGS well, it is not hanging on my box, sadly
20:09 timotimo damn
20:11 FROGGS and there seems to be no output after hitting ctrl+c
20:14 timotimo in gdb, of course?
20:14 timotimo ctrl-c should drop you to the prompt where you can go through the threads and print their backtraces
20:18 FROGGS it does not hang under gdb

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary