Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2015-06-24

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
06:20 FROGGS joined #moarvm
06:49 Ven joined #moarvm
07:29 nebuchadnezzar FROGGS: new package uploaded this morning, waiting for build logs ;-)
07:31 nebuchadnezzar FROGGS: If I understand correctly, you add libffi support in place of dynasm which should support all architecture but hurd?
07:32 FROGGS nebuchadnezzar: \o/
07:32 FROGGS nebuchadnezzar: exactly :o)
07:32 FROGGS I'm about to push it right now
07:36 TEttinger woo!
07:41 JimmyZ not dynasm ,it dyncall
07:41 dalek MoarVM/libffi: cfdd245 | FROGGS++ | / (12 files):
07:41 dalek MoarVM/libffi: support libffi besides dyncall (switchable via Configure.pl --has-libffi)
07:41 dalek MoarVM/libffi:
07:41 dalek MoarVM/libffi: Libffi is a very portable backend for foreign function calls. So this will
07:41 dalek MoarVM/libffi: unblock MoarVM from reaching platforms like mips, s390x, sparc, and
07:41 dalek MoarVM/libffi: potentially others.
07:41 dalek MoarVM/libffi: review: https://github.com/MoarVM/MoarVM/commit/cfdd245591
07:42 FROGGS nebuchadnezzar: I need to test this branch on windows, and I hope somebody can test it on osx...
07:43 FROGGS and I need to setup mips/mipsel/s390x/sparc vms, to tweak our build system
07:54 zakharyas joined #moarvm
08:15 nebuchadnezzar FROGGS: I may have some access to porter machines with my guest Debian account
08:16 FROGGS I guess there will be some build/setup.mp tweaking needed to make it work on these platforms
08:16 FROGGS build/setup.pm*
08:17 nwc10 I take it that if one wants to use libffi, then one needs libffi installed already?
08:18 nebuchadnezzar FROGGS: so kfreebsd-any is OK :-D
08:20 FROGGS \o/
08:21 FROGGS nwc10: and you need to pass --has-libffi as of yet... but I'll make it the default for platforms that dyncall cannot reach atm
08:26 colomon joined #moarvm
09:16 vendethiel joined #moarvm
09:48 nwc10 oh gnoes. jnthn has failed to bootstrap, because he no longer has a coffee maker.
09:51 FROGGS /o\
10:17 Ven joined #moarvm
10:41 FROGGS joined #moarvm
11:30 brrt joined #moarvm
11:31 brrt \o
11:45 FROGGS hi brrt
11:46 JimmyZ o/
11:46 brrt hi FROGGS, JimmyZ
11:58 FROGGS grrr, I cannot run mips or sparc because qemu either hangs at some point or segfaults
12:24 jnthn The coffee situation was solved by CEO collecting me from home and bringing me to the office, where there is coffee.
12:25 jnthn (Yes, some details omitted; it's funnier without them. :P)
12:28 nwc10 :-)
12:28 nwc10 do you have a backup plan for tomorrow morning?
12:28 nwc10 Or are you just going to stay up all night?
12:29 jnthn Um, no plan for tomorrow
12:29 jnthn But I have to finish some whisky tonight to prevent it going to waste :)
13:19 * brrt is still a beer man
13:21 brrt jnthn: i'm pondering how i can prevent an explosion of IR representations
13:21 brrt the way i'm seeing it now
13:22 brrt i need to build a machine-level IR (think: load, store, arithmetic, conversion, branching, condition flags, etc)
13:22 brrt then i need tiles working at that level to match the IR tree to a specific piece of machine code
13:23 brrt these should then fit into the existing jit node structure
13:23 brrt with a single 'expression' node being a collection of tile nodes?
13:23 brrt hmmm
13:24 brrt that gives me a lot of entities, few of which are compatible with the old JIT
13:24 brrt if any
13:24 nwc10 brrt: so this is not your thing? http://act.yapc.eu/ye2015/wiki?node=SherryTasting
13:25 brrt it'd be the first time i tasted sherry :-)
13:25 brrt sherry is made from grapes?
13:25 brrt i thought it was made from.... cherries
13:26 tadzik sherries :P
13:27 brrt tell me it did not make sense :-o
13:29 jnthn Sherry always felt to me like a drink that wanted to be port... :P
13:30 brrt i suppose port is also wine then?
13:30 jnthn Fortified wine
13:30 jnthn I quite like it
13:30 JimmyZ_ joined #moarvm
13:35 jnthn brrt: Maybe look elsewhere for inspiration? Like, what nodes does tcc have could be worth checking
13:35 jnthn brrt: Generally though, the QAST experience is that you often don't need so many nodes.
13:43 brrt tcc is tiny, right
13:43 brrt luajit has only a single IR, famously, but it's also quite low-level
13:44 brrt hmmm
13:46 brrt ok, let's start with the simplest question
13:47 brrt what would happen if we directly use the spesh graph as IR
13:48 brrt if i did that, then i could potentially map values to registers, do some simple register allocation
13:48 brrt what i could not do so simply is stuff like CSE
13:49 brrt assuming in this case that on a low level there are more commonalities for CSE to eliminate than on the moar-bytecode level
13:49 jnthn I think the spesh graph is too high level, though..
13:49 brrt yes, that is my suspicion too, but my point is figuring out why
13:49 brrt :-)
13:50 jnthn Well, I'd take a look at the sort of things we spit out in the JIT today that are "chunks"
13:50 jnthn And also things we know we'd like to do
13:51 brrt in general, we do stuff like: load, calculate, store
13:51 jnthn Especially how we JIT things like the fast-path memory access things that attribute reads are lowered into
13:51 jnthn And how we could JIT array accesses
13:51 brrt hmmm
13:51 brrt ok
13:52 brrt well, the way i see it in literature, is that the operation b = a[i] is first lowered into
13:53 brrt store(b, load(addr(a + i * sizeof(*a))))
13:53 brrt which is then compiled into (if possible):
13:54 brrt mov reg(b), [reg(a)+reg(i)*scale-of(a)+offset-of(a)]
13:54 brrt scale-a and offset-of are constants in this case :-)
14:00 jnthn Something like that lowered level sounds like what you'd want as the "JIT graph" or so...
14:04 brrt yes
14:05 brrt the main reason, i think, to have the tile structures too, is that a tile is something that makes a single 'atomic' chunk
14:06 brrt ultimately, it's a buffering issue
14:06 brrt there are multiple ways to tile a graph, and you want to select the best one (fsvo best)
14:07 brrt you do that by reducing the graph from lower to higher level nodes
14:07 brrt given that you can 'always' take really low-level tiles, you want to 'override' them with higher-level nodes as is possible
14:07 brrt e.g.
14:08 brrt i could implement b = a[i] as: mul i, scale-of(a); add a, i; mov b, [a];
14:08 brrt but the one-opcode version is significantly more efficient
14:10 brrt the tree reduction algorithm should then replace the lower-level operations with the single higher-level instruction
14:10 brrt that is only possible if we haven't irreversibly committed the lower-level instructions to the stream just yet
14:11 brrt given as we use dynasm (rather than manipulating the instruction stream itself), we need a stand-in object
14:11 brrt does that make sense?
14:11 brrt if we *could* buffer-and-select the instruction stream, we wouldn't need to represent the tiles seperately
14:12 brrt another thing i kind-of want to do is register allocation
14:12 brrt but we don't need tiles for that, we can just assign the registers in the IR for that
14:21 brrt anyway, i'll try to sketch something out :-) then we can evaluate that
14:21 * brrt afk
14:21 brrt left #moarvm
14:21 timotimo looking forward to that :)
14:48 zakharyas1 joined #moarvm
15:15 TEttinger joined #moarvm
15:18 nwc10 FROGGS: when I   origin/libffi
15:19 nwc10 FROGGS: when I try building origin/libffi on both x86_64 Linux and (old) OS X, it's failing
15:19 nwc10 src/core/nativecall_dyncall.c:464:47: error: ‘MVM_NATIVECALL_TYPE_CHAR’ undeclared (first use in this function)
15:19 FROGGS ups
15:20 FROGGS yes, I need to fix that
15:20 nwc10 :-)
15:20 nwc10 how come it worked for you?
15:20 FROGGS thanks for reporting :o)
15:20 FROGGS nwc10: I forgot to test the case where one does not pass --has-libffi :o)
15:21 nwc10 aha!
15:22 nwc10 now, the sort of sideways question from this is, will we able to configure Travis to build both with libffi and with dyncall, and check that both still work
15:22 nwc10 or at least, fail identically
15:22 FROGGS hmmm
15:23 FROGGS libffi should be installable via apt on the travis boxes...
15:29 nwc10 #include <ffi.h>
15:29 nwc10 on this OS X, it's in /usr/include/ffi
15:31 nwc10 oh, and it's too old to have _ffi_closure_alloc and _ffi_prep_closure_loc, it seems
15:37 FROGGS user@debian-mips:~$ git clone https://github.com/MoarVM/MoarVM.git
15:37 FROGGS Cloning into 'MoarVM'...
15:37 FROGGS nwc10: okay, so we keep using dyncall there... I'm pretty fine with that
15:38 FROGGS I dont want to replace dyncall on platforms where it works
15:41 japhb FROGGS: Is there a reason to keep both rather than converting to one that works everywhere?  Or is the platform support of dyncall not a subset of libffi?  (Or is there some other reason we want to keep dyncall around?)
15:42 FROGGS japhb: they overlap
15:42 FROGGS but libffi does not work on Windows IIRC
15:42 FROGGS japhb: and it is always good to have more than one choice
15:42 FROGGS like with backends
15:43 dalek MoarVM/libffi: 30b3bbc | FROGGS++ | src/core/nativecall_dyncall.c:
15:43 dalek MoarVM/libffi: unbreak dyncall build, nwc10++
15:43 dalek MoarVM/libffi: review: https://github.com/MoarVM/MoarVM/commit/30b3bbc6d6
15:43 JimmyZ_ more than a way to do it
15:43 FROGGS aye
15:50 japhb Oh, fair enough.  Just wanted to know if it was a "keeping options open" or "libffi is only a fallback if you can't use dyncall, because dyncall has more features" or "neither one has a complete set of platform support" or what have you.  Sounds like #1 and #3 right now.  :-)
15:51 FROGGS correct :o)
15:53 JimmyZ_ what ffi java use on windows
15:53 FROGGS I dont know
15:55 TEttinger JimmyZ_: I'm pretty sure OpenJDK on windows needs a microsoft toolchain, they may use non-portable APIs on windows
15:55 TEttinger I know it needs a weird version of DirectX for Swing UI
15:56 FROGGS ohh, libffi states it works for  Windows/Cygwin and Windows/MingW
15:57 JimmyZ_ no MSVC..
16:00 FROGGS JimmyZ: though, there is a msvcc.sh in the tarball that cares about MSVC written by Timothy Wall <twalljava@dev.java.net>
16:00 TEttinger ironic that it's a .sh
16:00 FROGGS his names also show up in the jna or jni docs
16:00 FROGGS TEttinger: probably used in a cygwin shell that does some magic
16:01 FROGGS ahh, it translates gcc flags to msvc flags
16:01 JimmyZ_ andthen supports msvc?????
16:02 FROGGS somehow...
16:03 FROGGS It's also possible to build libffi on Windows platforms with
16:03 FROGGS Microsoft's Visual C++ compiler.  In this case, use the msvcc.sh
16:03 FROGGS wrapper script during configuration like so:
16:03 FROGGS path/to/configure CC=path/to/msvcc.sh CXX=path/to/msvcc.sh LD=link CPP="cl -nologo -EP"
16:10 nwc10 FROGGS: yes, works on my machine, and "my" machine
16:12 FROGGS :D
16:24 dalek MoarVM: d8c65d1 | FROGGS++ | src/6model/reprs/CPointer.c:
16:24 dalek MoarVM: silence cast warning on 32 bit systems
16:24 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/d8c65d1f09
16:25 FROGGS nebuchadnezzar: the last commit solves the first warning shown here: https://qa.debian.org/bls/packages/m/moarvm.html
16:26 FROGGS nebuchadnezzar: but it is not clear to me what the second means
16:26 FROGGS nebuchadnezzar: what LDFLAGS need to be respected?
16:32 nebuchadnezzar FROGGS: I think it's about the hardening flag, and it happens only on one architecture
16:32 nebuchadnezzar I'll see with other perl6 maintainers
16:33 nebuchadnezzar FROGGS: for now I was looking at the reproductibility guide, it looks like there is something that change each time the package is build
16:45 nebuchadnezzar FROGGS: for the hardening flag I'll run blhc directly on the log to have more informations
16:49 nebuchadnezzar https://paste.debian.net/260299/
16:51 nebuchadnezzar looks like building minilua.c is missing some LDFLAGS
16:53 nebuchadnezzar So, the question is why minilua.c is build on kfreebsd-amd64 and not on the other architectures?
17:37 jnthn Because our JIT only works on 64-bit, and we only need that for the JIT
17:38 jnthn I'd expect it to be built on 64-bit with Linux kernel... Also it's only a build dependency; we don't ship the minilua
17:51 FROGGS nebuchadnezzar: one thing that changes every time is src/gen/config.c
17:51 FROGGS it contains an unordered hash
17:51 colomon joined #moarvm
17:51 FROGGS and we record the build time I think
17:54 FROGGS hmmm, there is no build date time, so that's good
17:55 FROGGS I wonder if there is something else, besides the config.c
18:01 dalek MoarVM: 69050de | FROGGS++ | Configure.pl:
18:01 dalek MoarVM: sort config.c keys to have predictable output
18:01 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/69050de4e7
18:01 FROGGS nebuchadnezzar: ^^
18:02 FROGGS if I would now get the same sha1 for a release tarball, we should be fine, right?
18:06 nebuchadnezzar FROGGS: I'll try to build a package two times to see if it works
18:07 nebuchadnezzar jnthn: ok, so as the amd64 is already uploaded by dod we do not have the build log, I'll check one locally
18:07 FROGGS hmmm, it still changes for me
18:15 FROGGS nebuchadnezzar: I think I'll know what is going on in a minute
18:16 FROGGS files that change: Makefile, src/gen/config.c, libmoar.so
18:22 FROGGS I think I got it
18:28 japhb FROGGS++  nebuchadnezzar++  # Thanks for the excellent packaging work, guys!
18:29 japhb Debian++  # High standards that make packaged products *better*
18:30 dalek MoarVM: 49f0c1c | FROGGS++ | Configure.pl:
18:30 dalek MoarVM: sort thirdparty libs/objects etc
18:30 dalek MoarVM:
18:30 dalek MoarVM: This is the second patch aiming a reproducable build.
18:30 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/49f0c1c884
18:30 FROGGS nebuchadnezzar: I think I'm done
18:32 japhb The goal on this one is to build twice and get a bit-exact identical tarball?
18:33 japhb Is there a plan for attacking the same problem on NQP and Rakudo?  (Or do we no longer use timestamps for block IDs and such?)
18:33 nwc10 japhb: technically I think that a tarball can't be bit-exact, because it countains file timestamps. But I think that it is byte-for-byte on the file names and contents
18:34 FROGGS japhb: yes, that's the goal, and no, we can't easily achieve that for nqp/rakudo yet
18:35 japhb nwc10: Hmmm.  In theory we could work around that, but understood
18:35 japhb FROGGS: gotcha
19:17 vendethiel- joined #moarvm
19:37 colomon joined #moarvm
19:40 zakharyas joined #moarvm
20:02 FROGGS user@debian-mips:~/MoarVM$ LD_LIBRARY_PATH=. ./moar --version
20:02 FROGGS This is MoarVM version 2015.06-26-g30b3bbc
20:02 FROGGS user@debian-mips:~/MoarVM$ uname -a
20:02 FROGGS Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips GNU/Linux
20:03 FROGGS with --has-libffi moarvm will just build on mips without any further tweak
20:18 nebuchadnezzar FROGGS: thanks
20:32 jnthn FROGGS++
20:33 jnthn The compiler toolchain uses timestamps in various places.
20:34 colomon joined #moarvm
20:39 jnthn I can think of two of hand, of which one can possibly trivially go away
21:13 TEttinger FROGGS++, nice work fixing support for a super-obscure arch!
21:16 FROGGS_ joined #moarvm
21:36 colomon joined #moarvm
22:20 nebuchadnezzar TEttinger: sure FROGGS++, he does a lot to help the Debian packaging
22:57 japhb nebuchadnezzar: s/the Debian packaging// and it's still true.  :-)
23:27 FROGGS joined #moarvm
23:30 vendethiel joined #moarvm
23:52 LLamaRider joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary