Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-10-08

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:48 ilbot3 joined #moarvm
01:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
07:02 FROGGS joined #moarvm
07:37 domidumont joined #moarvm
07:42 domidumont joined #moarvm
08:33 nebuchadnezzar MoarVM segfault on x32 (debian amd64 with kernel option “syscall.x32=y”) https://lists.alioth.debian.org/pipermail/pkg-rakudo-devel/2016-October/000983.html
08:33 nebuchadnezzar Using libffi or dyncall by the way
08:39 nine nebuchadnezzar: is there any way you could give be a backtrace of that segfault?
09:03 FROGGS o/
09:08 camelia joined #moarvm
09:10 domidumont joined #moarvm
09:16 nebuchadnezzar nine: sure, I'm openning a github issue to track it
09:18 nebuchadnezzar nine: here https://github.com/MoarVM/MoarVM/issues/423
09:21 FROGGS nebuchadnezzar: would be nice to recompile moar with --debug=3
09:21 nebuchadnezzar FROGGS: ok
09:21 FROGGS this way the backtrace should show more information
09:26 nebuchadnezzar FROGGS: do you think this should be integrated in the packaging? It would produce more useful -dbgsym package?
09:27 FROGGS well, this slows things down
09:27 FROGGS so it might make sense to enable this in a debug build, but not in the normal package
09:29 nebuchadnezzar ok
09:30 nebuchadnezzar I have to go for now, will do that in the begining of afternoon
09:30 nebuchadnezzar thanks for the advices
09:30 FROGGS nebuchadnezzar++
09:44 domidumont joined #moarvm
09:57 ggoebel joined #moarvm
10:07 TimToady joined #moarvm
10:26 TimToady joined #moarvm
10:57 domidumont nine: Hello, here are the rakudo/moar test issues I've found on armhf qemu chroot: https://github.com/MoarVM/MoarVM/issues/424
11:06 FROGGS hmmm, interesting
11:10 jnthn Left a note but I suspect https://github.com/MoarVM/MoarVM/issues/423 is a configure issue (should not build with JIT on x32)
11:10 jnthn So I suspect it explodes upon executing the first JITted function
11:11 jnthn (Due to ABI mismatch)
11:37 nwc10 [A
11:38 timotimo yes
11:49 nine Failed test 'sizeof(MyStruct)' is quite simple I dare say
11:50 nine The 8s in CUnion's compute_allocation_strategy are quite suspicious :)
11:50 FROGGS hmmm
11:51 FROGGS nine: the eights are about bytes vs bits
12:04 nine Then it's this: /* The structure itself will be the multiple of its biggest element in size. So we keep track of that biggest element. */
12:04 FROGGS now that's just a comment :P
12:04 nine Obviously not true if the size is 28
12:05 nine and the struct contains e.g. a num64
12:05 FROGGS yeah, we potentially have to guess the rules again
12:05 FROGGS no, wait... are we guessing 28 or 32?
12:06 FROGGS if we are guessing 28, then it might be easy to spot
12:06 FROGGS I'm building debian-arm64 right now btw
12:10 nine Oh wait indeed, we're guessing 28!
12:10 nine expected 32 and got 28 for is nativesizeof(MyStruct), SizeofMyStruct()
12:15 FROGGS # uname -a
12:15 FROGGS Linux froggs-Latitude-E6530 4.2.0-42-generic #49-Ubuntu SMP Tue Jun 28 21:26:26 UTC 2016 aarch64 GNU/Linux
12:15 FROGGS \o/
12:18 nine FROGGS: I think the comment is still wrong, or rather that the comment is right and the code does not do what the comment says. Because we only track the alignment rules of the members, not their size.
12:18 nine And if a num64 has to be aligned at 4 byte boundaries, that's what we use to multiply.
12:19 FROGGS ohh, yeah
12:19 nine OTOH we only seem to use multiple_of for rounding up.
12:20 FROGGS if (bits / 8 > total_size)
12:20 FROGGS total_size = bits / 8;
12:20 FROGGS but bits is about its size, no?
12:20 FROGGS wait
12:21 FROGGS we are not talking about CUnion, right?
12:21 FROGGS -.-
12:21 FROGGS lemme try that:
12:21 FROGGS if (bits / 8 > multiple_of)
12:21 FROGGS multiple_of = bits / 8;
12:22 FROGGS (inc CStruct.c)
12:23 nine That would bring it back in alignment (pun intended :) with the comment
12:23 FROGGS hehe
12:28 FROGGS bbi30
12:49 FROGGS building moar on arm64 now...
12:50 nebuchadnezzar jnthn: thanks for the advice, you were right
12:51 FROGGS nebuchadnezzar: now the question is: how we detect that to disable the jit when running Configure.pl
12:51 nebuchadnezzar the JIT support is detected by Configure.pl or should we do something in the packaging?
12:51 FROGGS we should fix Configure.pl
12:51 nebuchadnezzar ok
12:55 nebuchadnezzar FROGGS: maybe detecting the size of pointer (4 bytes) and the availablity of 64 bits features (https://en.wikipedia.org/wiki/X32_ABI)
12:55 FROGGS yeah, I was thinking the same
12:59 nebuchadnezzar ok, for now I disable JIT explicitely in the packaging
12:59 nebuchadnezzar I found https://paste.debian.net/858210/ from https://sites.google.com/site/x32abi/, I don't know if it help
13:00 cygx joined #moarvm
13:00 cygx o/
13:00 FROGGS nine: my changes pass on amd64
13:01 FROGGS nine: now trying x86 and then arm64
13:01 * cygx needs some help with 6model guts
13:02 cygx background: I want to be able to call nqp::rebless on an MVMOSHandle
13:02 nebuchadnezzar FROGGS: I reopen #423 to track the Configur.pl
13:02 cygx that works, but now it fails when trying to serialize the type object for a class with that repr
13:03 cygx cf https://gist.github.com/cygx/9c9fa30c43967512b8c228f3bcc31bb1
13:03 cygx jnthn: ^
13:03 FROGGS nebuchadnezzar++
13:05 nebuchadnezzar FROGGS: on this packagers have the easy part, let's try to build and complain ;-)
13:06 FROGGS hehe
13:06 nebuchadnezzar Should we track x32 tests failures (https://github.com/MoarVM/MoarVM/issues/423#issuecomment-252422990) in #424 like with amhf or in a dedicated ticket? Some failures may be related.
13:07 nebuchadnezzar I don't see what's the reponsibility of MoarVM, NQP or rakudo
13:08 FROGGS nine: it seems to be slightly more complicated than my proposed patch: https://github.com/MoarVM/MoarVM/commit/f2ae11b5
13:08 FROGGS nebuchadnezzar: a dedicated ticket might be better
13:12 nine FROGGS: maybe you need both?
13:12 nine max(bits/8, align)
13:16 FROGGS no
13:16 FROGGS if a struct is in an attr slot we cant to that
13:18 nebuchadnezzar Does building with --debug=3 and strip debugging symbol (they are in a dedicated package) produce a slower moar binary?
13:26 nebuchadnezzar FROGGS: héhé https://wiki.debian.org/X32Port#Detecting_X32
14:19 jnthn cygx: Why do you need rebless on that?
14:19 jnthn cygx: I'm really, really not keen on supporting it more widely than on P6opaque
14:19 cygx jnthn: not need, but want :p
14:19 cygx cf https://github.com/cygx/p6-newio/blob/master/NewIO.pm
14:20 jnthn Yes, but you also wanted to do runtime mixins to IO handles a while back, which is just no.
14:22 jnthn Long story short, any time you rebless, you deopt the whole callstack, except to actually do it right we should probably really deopt the callstacks of all running threads in the future unless we can prove we don't need to
14:23 jnthn I see a nice absence of those in https://github.com/cygx/p6-newio/blob/master/NewIO.pm :)
14:23 jnthn Oh, except the one rebless :P
14:23 jnthn Is there really no better design, though?
14:24 cygx a wrapper object
14:24 jnthn That'd be preferable
14:24 cygx I can do that
14:25 jnthn fwiw though, your patch to implement rebless on MVMOSHnadle does look correct.
14:26 cygx it's just a bit sad that there's all this 6model goodness, but in the end, it still comes down to wrapper objects
14:26 cygx if I wanted that, I could do Java instead ;)
14:26 cygx but of course the requirements for CORE stuff are a bit different...
14:27 jnthn At the end of the day, a lot of optimization - even of dynamic code - relies on it settling into relatively static behavior
14:27 jnthn I guess there may be clever things we can do with tracking "this type usually gets mixed in to" and other such things
14:28 jnthn But yes, for now it's nice if CORE stuff is written in terms of stuff we can optimize
14:28 jnthn Rather than causing opts to be backed out
14:29 jnthn About uniread and so forth, I'd been wondering a bit about passing norm => ... during open
14:30 jnthn And then things like .get and .lines might return Str or Uni
14:31 jnthn Which feels like it'd make refactoring a tad easier if you start out with Str and then realize you need more control
14:31 jnthn The default for norm being Str, but other options being Uni, NFC, NFD, etc.
14:32 cygx my reasoning was that in principle, if you take care of properly maintaining your buffers (eg the .restore-bytes method on Decoder), there's nothing stopping us from intermixing binary, grapheme-based and codepoint based IO
14:32 cygx so the 'low-leve' methods always return a consisten type and can be used at any time
14:32 jnthn You are I assume re-using the encoder design I already posted? :)
14:32 cygx high-level things liky lines/sput/slurp are multis that accept :uni or :bin and dispatch accordingly
14:32 jnthn And partly implemented?
14:33 cygx nothing implemented so far, just playing around with the API
14:33 jnthn https://gist.github.com/jnthn/4b2ce730fe7b505d9104d157291de572
14:33 jnthn OK
14:34 cygx I'm starting at the top to see what the bottom needs to look like
14:35 cygx eg I'd like for the bytes corresponding to codes/chars held back in the normalization buffer to stick around so we can losslessly call a byte-based function
14:37 jnthn urgh
14:37 jnthn That'll need quite some tracking
14:38 cygx I already did something like this at the vm level (cf https://github.com/MoarVM/MoarVM/pull/319 ), but I needed to unnecessarily fseek() when the buffers could just be restored
14:38 cygx wasn't there this thing about torturing the implementor ;)
14:38 jnthn Yes, though I think this falls under "hard things possible" :P
14:39 jnthn Do you actually have some concrete use cases?
14:39 jnthn Torturing the implementor for something nobody's ever going to need is a little pointless.
14:40 cygx it's switching encodings on-the-fly on steroids
14:43 cygx most real-world use cases I can think of can be handles by just doing binary IO and manual encoding
14:44 cygx but from the 'artistic' perspective, I still like the idea that I do not need to care if I'm dealing with a binary, text or codepoint-based file
14:46 jnthn The real world use case that comes most immediately to mind is HTTP, but in that case you'll most likely be dealing with the data "reactively"
14:46 jnthn (e.g. it'll be arriving over the socket in chunks)
14:57 jnthn grmbl, headache...afk for a bit
14:58 domidumont joined #moarvm
15:02 cygx nasty things, headaches
15:02 cygx in my youth, I used to get hit with them fairly reguarly, so my sympathies
15:36 FROGGS nine: I guess we need more like: min(max(align, bits/8), sizeof(void *))
15:41 nine That's 4 bytes on x32
15:42 nine Which I think is exactly what we currently get :)
16:06 FROGGS are you sure? bits/8 should be 8 for our use case here
16:07 FROGGS so the result will be...
16:07 FROGGS damn
16:07 FROGGS maybe we just do min(max(align, bits/8), 8) ?
16:07 FROGGS I guess that might be sane
16:19 FROGGS this passes on amd64 and x86 so far: https://gist.github.com/FROGGS/9421a660bd7ce4473ad9faae34e76440
16:19 FROGGS arm64 is still building rakudo
16:23 FROGGS Stage parse      : 666.390
16:23 FROGGS heh
16:24 timotimo %)
16:24 timotimo seems like a devilishly slow device
16:27 arnsholt nine, FROGGS: The size of a struct can be pretty much whatever you damn well want it to be
16:28 arnsholt For example struct { int32; char}  will be 9 bytes
16:28 arnsholt The *alignment* requirement is of course the alignment of its member with the larges alignof, though
16:29 arnsholt OTOH struct { char; int32 } will be 8 bytes, due to the alignment requirements of the int32
16:29 synopsebot6 joined #moarvm
16:30 cygx arnsholt: struct size is always a multiple of the alignment
16:30 geekosaur not entirely true, actually. struct {double, int32, double} is usually 20 not 24
16:30 cygx otherwise, you'd have to introduce padding between array elements
16:31 arnsholt geekosaur: How does that happen? Isn't double 8-byte aligned?
16:31 geekosaur ...but alignment depends on the ABI, and there are indeed ABIs where the alignment is 8 instead of 4
16:32 geekosaur it is, and a special alignment is used just for that while other things are 4-byte aligned on the ABIs that work that way
16:34 arnsholt cygx: Point
16:35 FROGGS okay, then my patch is wrong
16:38 geekosaur arnsholt, you might want to look at "man offsetof" which shows what I was referring to. x86_64 may differ; their example is i386
16:40 geekosaur actually my prior example was wrong because of the second double, yeh.
16:40 geekosaur should have included a second int as well so the second one would show the align being 4 for those
16:44 dalek MoarVM/overflow_exception_mvmarray: cc71759 | timotimo++ | src/6model/reprs/MVMArray.c:
16:44 dalek MoarVM/overflow_exception_mvmarray: clip size at biggest possible, or throw exception for growing
16:44 dalek MoarVM/overflow_exception_mvmarray:
16:44 dalek MoarVM/overflow_exception_mvmarray: when a ridiculously large size is requested, first give the exact
16:44 dalek MoarVM/overflow_exception_mvmarray: maximum malloc and realloc will let us specify, and if we're asked
16:44 dalek MoarVM/overflow_exception_mvmarray: to grow past that, throw an exception.
16:44 dalek MoarVM/overflow_exception_mvmarray:
16:44 dalek MoarVM/overflow_exception_mvmarray: Have fun waiting for the zeroing of 2**62 or 2**63 slots ...
16:44 dalek MoarVM/overflow_exception_mvmarray: review: https://github.com/MoarVM/MoarVM/commit/cc7175966a
16:46 geekosaur it also at least used to be possible to define an ABI where doubles were 4-byte aligned, but that came at a performance penalty (extra clocks to access unaligned values) and iirc Intel removed at least some unaligned access ability in later cpus although other unaligned accesses can still be enabled in the cpu flags/control word
16:47 * geekosaur still remembering horrid SCO 3.2 kernel ABI that didn't match the default userspace ABI re alignment
16:48 dalek MoarVM/overflow_exception_mvmarray: 0573c98 | timotimo++ | src/6model/reprs/MVMArray.c:
16:48 dalek MoarVM/overflow_exception_mvmarray: use SIZE_MAX, which actually exists. also include limits.h
16:48 dalek MoarVM/overflow_exception_mvmarray: review: https://github.com/MoarVM/MoarVM/commit/0573c98169
16:49 timotimo ah, damn.
16:49 timotimo that'll apparently silently give you a smaller array when you ask for a specific size and not asplode at all
16:50 timotimo BBL
16:50 geekosaur welcome to the hell that is C >.>
16:51 timotimo well, no, that's not C's fault
16:51 timotimo that's how our stuff works
16:51 timotimo maybe i do throw the exception but it gets caught :P
16:51 timotimo who knows
16:51 timotimo anyway, BBL for now
16:51 timotimo this went into a branch for a good reason
17:56 FROGGS arnsholt: struct { int32; char} would be 12 bytes, no?
17:57 japhb FROGGS: How do you get 12 bytes?  There's only 5 bytes worth of data there.
17:57 FROGGS hmmm, right
17:57 FROGGS should be eight bytes also
17:57 FROGGS no...
17:58 FROGGS well yes
18:02 * FROGGS reads http://www.catb.org/esr/structure-packing/
18:02 cygx struct { int32; char[3]; int16; } would be 12 bytes, with one padding byte after the char[3] and two after the int16
18:11 geekosaur on x86 that is 8 bytes
18:11 geekosaur sorrt, I mean the int32/char
18:12 geekosaur cygx is correct about theirs, again on x86 (with its standard ABI, not weirdshit like the old SCO kernel ABI)
18:14 geekosaur and this is why perl 5's stuff (and haskell's, and pretty much every other FFI out there) goes through C and asks the C compiler to report exact offsets for struct members etc.
18:16 geekosaur it depends on the processor family, it depends on the ABI, there is no single hard and fast rule that is guaranteed to work everywhere. even in the modern world where you can mostly pretend only x86_64 and arm exist, there are ABI differences between POSIX systems and Windows, and every ARM generation has slight ABI differences
18:16 geekosaur (and that doesn't help those of us who support big companies that still run SPARC and have no intention of getting rid of it)
18:17 geekosaur (or likewise IBM POWER/PowerPC)
18:21 cygx https://gist.github.com/cygx/fec5e5aa03df84483e81fb5c25a74131
18:21 cygx ^ compute size/alignment if you know size/alignment of the members
18:22 cygx (hopefully ;))
18:28 dalek joined #moarvm
18:41 FROGGS hmmm
18:41 FROGGS cygx: I can try your formula
18:56 FROGGS cygx: seems to work on amd64, x86 and arm64 (all linux) so far...
18:57 FROGGS will check x86 and amd64 on windows now... (gcc and msvc)
18:59 FROGGS though, I got another issue:
18:59 FROGGS # Failed test 'returning char works'
18:59 FROGGS # at t/04-nativecall/03-simple-returns.t line 19
18:59 FROGGS # expected: '-103'
18:59 FROGGS #      got: '153'
18:59 FROGGS ... on arm64
18:59 FROGGS is this the same that's failing on osx?
18:59 nwc10 'char' is unsigned on arm (presumably arm64 too)
19:06 cygx yup - 'char', 'signed char' and 'unsigned char' are 3 different types
19:06 cygx two of them have the same representation, but which will depend on the ABI
20:09 FROGGS think I found the (un)signed char issue
20:10 cygx_ joined #moarvm
20:23 dalek MoarVM: 9723396 | FROGGS++ | src/core/nativecall_dyncall.c:
20:23 dalek MoarVM: make sure we mean "signed char" when we say "char"
20:23 dalek MoarVM:
20:23 dalek MoarVM: This fixes an issue on arm64 and probably on arm and osx also.
20:23 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/9723396e06
20:59 nebuchadnezzar In MoarVM build log I see several “warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]”, could it cause some issue on arch like x32?
21:06 dalek MoarVM: b2ed8e4 | FROGGS++ | src/6model/reprs/C (2 files):
21:06 dalek MoarVM: fix calculating structure sizes for arm64 and others
21:06 dalek MoarVM:
21:06 dalek MoarVM: cygx++ for the algorithm. Tested on:
21:06 dalek MoarVM:   amd64, x86, arm64 (linux, gcc)
21:06 dalek MoarVM:   amd64 (windows 7 x64, msvc + gcc)
21:06 dalek MoarVM:   x86 (Windows XP, msvc)
21:06 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b2ed8e4ed1
21:06 FROGGS nebuchadnezzar: yes
21:07 FROGGS nebuchadnezzar: err, cause? I guess not
21:10 Ven_ joined #moarvm
21:10 nebuchadnezzar FROGGS: I'll testing your last commit on x32, is there anything I can look for without builing everything to see if --no-jit is automatically added?
21:12 nebuchadnezzar ho, it looks like this has nothing to do you the jit stuff, just some failing tests ;-)
21:12 FROGGS right
21:13 FROGGS the jit-detection is untouched
21:13 FROGGS though, arm64 is fine now on my box
21:13 nebuchadnezzar thanks
21:13 FROGGS pleasure :o)
21:13 jnthn FROGGS++
21:14 nebuchadnezzar you fixed #424, right?
21:15 FROGGS I think so
21:15 FROGGS though, gimme a se to verify
21:15 FROGGS sec*
21:16 nebuchadnezzar no problem, it's time to sleep for me ;-)
21:17 FROGGS well then, off to bed :o)
21:17 FROGGS gnight nebuchadnezzar
21:40 lizmat FROGGS: fwiw, looks like compiling MoarVM is a lot quieter now on OS X
21:41 FROGGS really? hmmm, not sure it is due to my patches...
21:41 lizmat Files=45, Tests=609,  8 wallclock secs ( 0.26 usr  0.11 sys + 32.71 cusr  3.68 csys = 36.76 CPU)   # make test
21:41 FROGGS but it is good anyway :o)
21:41 timotimo https://github.com/libtom/libtommath/pull/56  -  something about mp_rand
21:42 lizmat FROGGS: ah, it was already quieter before
21:42 lizmat timotimo: yup, looks like the problem to me
21:44 timotimo not entirely sure what exact defines we have set in our libtommath configuration when we build moar
21:45 * lizmat neither, but it looks like skids was already on it
21:47 lizmat Files=1146, Tests=53273, 250 wallclock secs (14.49 usr  4.64 sys + 1504.16 cusr 135.98 csys = 1659.27 CPU)  # spectest
21:47 timotimo i just posted to the mailing list about this
21:47 lizmat on a non-cold machine, tomorrow will have better numbers
21:50 timotimo oh!
21:50 timotimo it was skids who opened that pull request!
21:50 timotimo now i notice what you meant when you said skids is on it
21:55 * geekosaur recalls perl<6 once upon a time had special tests of rand() because some systems lied about the range and others were just much smaller than expected, etc.
21:57 timotimo ah
21:58 geekosaur and I see that pull mentions some linux only using half the range (no negative numbers)
21:58 geekosaur siiigh.
21:59 dalek MoarVM: c8d9ea6 | FROGGS++ | Configure.pl:
21:59 dalek MoarVM: disable JIT on x32, resolves #423
21:59 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c8d9ea6e60
22:04 FROGGS joined #moarvm
22:40 dalek MoarVM: 082c989 | FROGGS++ | Configure.pl:
22:40 dalek MoarVM: fix pointer size unit in configure message
22:40 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/082c989416
22:42 FROGGS building a debian-x32 now while going to bed...
22:42 FROGGS gnight
22:45 timotimo good night!

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary