Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2014-07-23

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:03 TimToady o/
00:46 ventica joined #moarvm
01:14 FROGGS__ joined #moarvm
02:57 ventica1 joined #moarvm
05:27 lizmat joined #moarvm
06:15 sergot hi \o
06:37 woolfy joined #moarvm
07:03 FROGGS[mobile] joined #moarvm
07:05 FROGGS[mobile] jnthn++
07:08 ventica1 Hi all... I'm interested in contributing to MoarVM... what are "biggest needs" (links/pointers to reading are welcome)? I'm most interested in working on implementing unimplemented or only-plumbed-in features. My background is computer hardware (CPU microcoding and CPU core validation)... I'm very comfortable with parallelism, threading, events, etc.
07:10 ventica1 computer hardware engineering*
07:12 FROGGS[mobile] joined #moarvm
07:14 zakharyas joined #moarvm
07:16 FROGGS__ hi ventica1
07:17 FROGGS__ ventica1: here is a roadmap with bigger things to implement: http://moarvm.org/roadmap.html
07:18 FROGGS__ ventica1: but there are many many little things that Perl 6 (rakudo) needs, which want to be supported by MoarVM directly
07:18 FROGGS__ ventica1: here you can see missing features of Perl 6: http://perl6.org/compilers/features
07:18 FROGGS__ err, rakudo
07:21 hatseflats I'm looking for some info as well, I was introduced to MoarVM by a friend who claimed that the meta object model is crafted from 3 'super'baseclasses, but I can't seem to find proof of this in the code, am I blind?
07:21 hatseflats He mentioned Cool, Mu, and another class, but I can't seem to find references to this setup
07:22 ventica1 FROGGS: Thx, I'll diff those two lists... :)
07:23 TimToady Cool, Mu, and (probably Any) are not metaclasses, just base classes; metaclasses would be more like ClassHOW, RoleHOW, or fundamentally, KnowHOW
07:24 TimToady m: say 42.^mro
07:24 camelia rakudo-moar 6504db: OUTPUT«(Int) (Cool) (Any) (Mu)␤»
07:24 TimToady those are the parents of an Int
07:25 TimToady m: say 42.HOW
07:25 camelia rakudo-moar 6504db: OUTPUT«Perl6::Metamodel::ClassHOW.new()␤»
07:25 TimToady that's a metaclass object
07:25 hatseflats right, mixed terminology is one of the reasons I'd like to read up on the subject(s), is there any spec available?
07:26 TimToady the P6 specs talk primarily about base classes, and tend to hide everything behind the "HOW" wall.  I'm sure there's some 6model docs lying about somewhere though that describe the HOW stuff
07:27 ventica1 concurrency looks interesting... and looks like Moar still needs a bit of work on that?
07:27 TimToady sure, it's still a work in progress, and has bugs, and doubtless misfeatures in the specs even
07:27 hatseflats 'kay, I'll try digging a bit further then :)
07:28 TimToady have the appropriate amount of fun!  :)
07:28 FROGGS__ here are docs about 6model implemented in nqp/rakudo: https://github.com/perl6/nqp/tree/master/docs/6model
07:28 ventica1 o/
07:29 FROGGS__ ventica1: it has concurrency support, which is "almost" stable
07:29 hatseflats Juerd must've mentioned the two in passing without mentioning what 6model really was a part of, thanks FROGGS__
07:31 ventica1 Nice. So, in your estimation, what's the biggest "hole"? I know that's a subjective question but I'm out of the loop and just trying to get a feel of where my efforts could be best placed
07:31 TimToady we're very careful to keep our class hierarchy independent of our metamodel, and both of those independent of the representational polymorphism model
07:32 ventica1 FROGGS: I see "asynchronous file I/O" listed as due 8/2014 in the first list you sent me
07:32 FROGGS__ ventica1: I think feature wise shaped arrays/hashes might be very worth it, but multi threaded hypers would be awesome too
07:32 TimToady was just gonna mention them
07:33 TimToady the hypers
07:34 TimToady probably needs the compact shaped array support before we can feed the hypers to your GPU though...
07:34 hatseflats I don't really understand what a representational polymorphism model means, what makes it representational?
07:34 FROGGS__ hypers are an awesome primitive to do something in parallel without getting insane starting and joining threads
07:35 TimToady not even the class knows how its attributes are laid out in memory, basically; could be stored in a hash, or in a C struct, or whatever
07:35 FROGGS__ hatseflats: I think what is meant here is that you have representations, that define how an object is layed out in memory
07:35 hatseflats ah, in that sense, I see
07:36 TimToady basically, instead of doing like most dynamic languages and force conversion of foreign objects into the representation model of the language
07:36 FROGGS__ like an Array in Perl 6 can be of repr CArray[uint8], which means that its elements are really just bytes in memory in a row
07:36 TimToady we abstract that away, so a P6 object could, in fact, simply be a mapping of a C+ object, say
07:36 TimToady C++
07:36 TimToady or Java, or whatever
07:36 FROGGS__ so an $carray_of_uint8[3] would read the fourth byte of that memory area
07:37 FROGGS__ for the normal Array type you would have a list of pointers to Perl 6 objects that again known how they are layed out in memory
07:37 TimToady basically, the repr writes your accessors for you so you don't have to care
07:38 FROGGS__ yes
07:38 FROGGS__ and it works very well :o)
07:38 FROGGS__ for example, you can cast a C structure to an Perl 6 object just using the object's class definition
07:41 hatseflats that sounds really clever
07:41 FROGGS__ here is the positional access of an CArray implemented btw: https://github.com/MoarVM/MoarVM/blob/master/src/6model/reprs/CArray.c#L218
07:41 ventica1 FROGGS: Cool... I'll go read up on those. Any hints/links/docs?
07:41 brrt joined #moarvm
07:42 hatseflats clever in the good sense that is, not 'clever code' is stupid
07:42 FROGGS__ S03:4173
07:42 synopsebot Link: http://perlcabal.org/syn/S03.html#line_4173
07:42 * TimToady has wanted representational polymorphism ever since he first hacked expat into P5 to support XML.
07:42 FROGGS__ ventica1: see that link
07:43 klaas-janstol joined #moarvm
07:43 ventica1 thx
07:43 brrt \o
07:43 FROGGS__ but in short: @a »+« @b will add @a[0]+@b[0] and all other elements in parallel (in any order), and return a list with the original order
07:44 brrt btw, welcome hatseflats, ventica1 :-)
07:44 FROGGS__ * where in parallel depends on the machine, if you cannot do MT, you will only have on thread of course
07:44 * brrt truly wonders if that will ever give an advantage aside from specialised (GPU) solutions
07:44 FROGGS__ but if we can make use of CUDA on some platforms, that would be awesome of course
07:45 xiaomiao s/cuda/opencl/ ;)
07:45 ventica1 I vaguely remeber watching a talk on YT TimToady gave on the hypers
07:45 ventica1 remember*
07:45 hatseflats brrt: thanks, unlike many other channels, I feel welcome as well :)
07:45 FROGGS__ brrt: I had a prototype of »+« on parrot, and it was like 3.5 times faster
07:46 brrt truly? i've implemented the same idea in go once, and it was 2x slower :-)
07:46 brrt that may have been go's lousy locking, though, i don't know about that
07:46 ventica1 brrt: o/
07:46 brrt also, i
07:46 FROGGS__ brrt: but that involved an array of native ints, so I could just add them without many dispatch
07:47 brrt i understand, but i'd think locking overhead would be too large
07:47 brrt of course, that may still have had more to do with my specific solution
07:50 brrt (all this not to say that it isn't very cool, which it is)
07:52 TimToady well, of course it's gonna depend on the granularity of your memory model, but for compact storage I could see just dividing the work into lock-free regions that are far apart enough not to need interlocks
07:52 TimToady maybe with a second pass to fill in the inbetweens that were necessary to cushion the first pass
07:53 brrt tfair enough, but that looks like more work than it seems at first sight :-)
07:54 FROGGS__ brrt: here was my attempt: https://gist.github.com/FROGGS/6127100
07:54 TimToady Our aim is to torment the implementors on behalf of the users.  :_
07:54 TimToady :)
07:54 FROGGS__ I am sure there was a parrot or nqp issue, but I can't find it
07:55 TimToady parallelizing lists is a different matter, of course; you need some kind of divide-and-conquer representation
07:56 * brrt mumbles about being happy not to implement a JIT for parrot
07:56 TimToady so Lispy cons lists are really bad that way
07:56 brrt that depends on if you can update a head pointer atomically and cheaply
07:56 brrt no, that's not right
07:56 TimToady that's why P6 doesn't specify linked lists for its basic list type
07:57 brrt scrap that
07:57 ventica1 so, does Moar have a vector representation for lists, internally? Or a flex representation between vector and LL?
07:57 brrt ok, i'm going to figure out why windows build are broken
07:57 ventica1 or just LL only?
07:57 brrt well... good question
07:57 brrt let's see
07:58 brrt it's an array
07:58 FROGGS__ ahh, also here https://github.com/rakudo/rakudo/commit/71f8958169debcaef615e341a926274594748af8
07:59 TimToady currently basically a vector of generators, since they're lazy by default, but reified storage is array, yes
07:59 FROGGS__ though, there was a nice discussion that is probably lost :(
07:59 brrt ventica1: https://github.com/MoarVM/MoarVM/blob/master/src/6model/reprs/MVMArray.h
08:00 TimToady we need to refactor our demand model to better communicate to a list what its contextual policy is, lazy, eager, hyper, race
08:00 ventica1 so, then we can assert that a list needs to be reified into an array before getting divided up for use by a hyper operator? (forgive me for n00b questions....)
08:00 brrt fun stuff FROGGS__
08:00 TimToady that's what hyper should be able to do, but we don't have it yet
08:01 TimToady pmichaud++ was gonna do that refactor, but real life intervened
08:01 ventica1 brrt: Thx - n00b q... diff between int and num?
08:01 brrt i can actually imagine a list with a different type of repr associated, but i wouldn't know how that should be done
08:01 brrt num is floating point, int is integer :-)
08:01 ventica1 ah, yes, thx
08:02 TimToady and the lowercase ones are unboxed natives, while the uppercase ones are boxed
08:02 FROGGS__ boxed and not machine size limited
08:02 TimToady well, except the denominator of Rat
08:03 ventica1 REPR is representation?
08:04 TimToady which is Int / uint64 basically
08:04 TimToady yes
08:04 ventica1 thx
08:07 ventica1 TimToady: OT question... how do u feel about Perl adoption/retention in the broadest possible sense? I know that's a Big Hairy Question but just curious about ur feelings on it. Just reading the comp news, one can get the feeling that Py/Rb/flavor-of-the-month are the future and Perl is falling into disuse. I know that's not true but just curious of your 1/2-sentence feelings on it.
08:07 ventica1 1-or-2 sentence*
08:09 TimToady We've said all along that it will take as long as it takes, and we're not gonna rush it.  Perl 6 will certainly succeed on its merits over the long term, I believe.
08:10 ventica1 cool... I agree with that assessment
08:10 TimToady Over the short term, people will say the darnest things. :)
08:11 ventica1 lol - and Perl 5?
08:11 jnthn morning, #moarvm
08:11 ventica1 o/
08:11 TimToady Perl 5 has already succeeded on its merits, and continues to acquire more of 'em.  :)
08:11 ventica1 :)
08:12 TimToady it will never, however, be able to mutate into Perl 6 without a massive number of (presumably unacceptable) deprecation cycles
08:13 brrt ventica1: i should add that this is mostly a 'what are people talking about' kind of trend than 'what is actually used' one
08:13 ventica1 we'll just write a pure-Perl 6 version of Perl 5 and deprecate the old code base... haha jk
08:14 brrt i.e. there are still very effective companies working with perl5
08:14 ventica1 brrt: indeed... but sometimes it's nice to hear from the ppl involved in order to cut thru all the noise
08:14 TimToady well, that's what v5 is, which you should ask FROGGS__++ about
08:14 ventica1 not always easy to know what to believe
08:14 ventica1 aha
08:14 ventica1 so there's a use v5;?
08:14 TimToady what to believe is what you yourself are willing to accomplish; as they say, the best way to predict the future is to invent it :)
08:15 brrt well, you can believe that whatsapp, which used (among other things) perl internally, was acquired for... a number that was so high that i can't recall, but postfixed by a dollar sign
08:15 ventica1 haha yeah that buyout was insane
08:15 brrt and yes, FROGSS__++ is working on that indeed
08:15 brrt but they were using perl
08:15 brrt (i recently had the same discussion of my former manager)
08:15 brrt with my former manager
08:16 ventica1 my experience is that corp. mgmt mostly doesn't comprehend the Perl concept at all
08:16 brrt duckduckgo, the darling of paranoid hackers everywhere, is using perl
08:16 ventica1 It's "just a script"
08:16 TimToady m: use v5; $_ = "a b c"; /(A B C)/i; print $1
08:16 camelia rakudo-moar 6504db: OUTPUT«===SORRY!===␤Could not find Perl5 in any of: /home/p6eval/rakudo-inst-2/languages/perl6/lib, /home/p6eval/rakudo-inst-2/languages/perl6␤»
08:16 TimToady aww
08:16 ventica1 I cud never beat it into their heads that, yes, it's "REAL CODE" (R)
08:17 * brrt is rebooting
08:17 ventica1 Or, as Robert Anton Wilson dubbed them, the Mgt
08:18 ventica1 not to insult people of smaller stature
08:18 TimToady well, this mgt should probably rest his eyeballs, now that the smart people are coming back online :)
08:19 TimToady zzz &
08:22 jnthn 'night, TimToady
08:23 ventica1 ok another n00b q... i'm seeing function def'ns in  MVMArray.c but no corresponding declarations in MVMArray.h... am I blind or should I be looking somewhere else?
08:24 jnthn ventica1: Near the bottom of that file, you'll see the functions all get shoved into a table.
08:25 ventica1 oh my, I have never seen anything like that before... amazing
08:25 FROGGS__ ventica1: yes, there is a 'use v5' but I am rewriting it right now, so that it is installable using panda, our module installer
08:25 jnthn When interpreting, representation stuff is late-bound.
08:25 jnthn Well, when naively interpreting :)
08:25 ventica1 so... if I'm an array and I want to say "How am I represented right now?" what call do I make to answer that question?
08:26 jnthn At a Perl 6 level, .REPR. Though note Array there is actually a rather more complex object that has a reified part and an unevaluated part, to support laziness.
08:27 ventica1 FROGGS: Nice... I think that will be key to wider Perl 6 adoption when it gets to that point
08:27 jnthn A Blob is lower level, however.
08:27 jnthn m: say "abc".encode('utf-8').REPR
08:27 camelia rakudo-moar 6504db: OUTPUT«VMArray␤»
08:29 ventica1 jnthn: ok, so, I think I kind of understand what u mean about the reified thing... but this was raised in the context of me trying to understand hypers (since this is an area needing development work), and my understanding is that we would basically need to convert the list into a reified form for use by the hyper op
08:29 jnthn On areas to contribute, async file IO may be an easy-ish on-ramp, since it's mostly adding code that looks a good bit like stuff that already exists. In fact, part of it may be factoring things out.
08:29 ventica1 So, just trying to wrap my head around that... seems like the first step in the flow would be "How am I represented right now? Am I reified or not? If not, get myself into reified representation in preparation for passing to hyper op"
08:30 ventica1 jnthn: OK, that sounds like a good match to me, too... I'm fairly comfortable with asynch stuff
08:33 jnthn On reification, though, it's already triggered by the implementation of hyper ops, which before they do any work check the number of elements on each side, which forces the reification.
08:33 brrt joined #moarvm
08:34 ventica1 ah, that makes sense, link reification to a size-check
08:34 jnthn We implement a lot of Perl 6 in Perl 6, and try to only create abstractions in the VM where they are needed or where they offer significant performance benefits.
08:34 ventica1 so, is asycn i/o in /src/core ?
08:35 jnthn I think so far as hypers go, for non-native data types, we can just use existing threadpool infrastructure to do the work division and handle it in Perl 6 code.
08:35 jnthn For native types, however, it's probably worth pushing it down to the VM in so far as we can take advantage of SIMD style things.
08:35 ventica1 So, rakudo must then have some kind of Perl 6 core running under the hood?
08:35 jnthn Yes and no.
08:36 jnthn The built-ins are just written in Perl 6 itself, with an initial bit of wiring together.
08:36 jnthn The compiler is written in NQP, which is a small, more optimization, dialect of Perl 6.
08:36 brrt i wonder if the JIT can be a part of the repr's one day
08:36 jnthn uh, more easily optimization
08:36 jnthn brrt: Well, REPRs already can do spesh things
08:37 jnthn brrt: But it'd occurred to me that the JIT could de-virtualize a load of REPR calls.
08:37 jnthn Not an immediate priority, but worth it at some point.
08:38 brrt interesting
08:38 brrt visual inspection, btw, doesn't show anything obviously wrong with the win64 code, so i suspect i'll have to arrange some kind of VM
08:38 jnthn ventica1: Anyways, async sockets are implemented so far. There's some code in src/core/IO/Socket/Async.pm that is the Perl 6 level API. It uses various nqp::op things which are an abstraction layer over the VM.
08:38 brrt or reinstall the hard disk containing windows
08:39 jnthn ventica1: The actual I/O itself is handled by Moar. src/io/asyncsocket.c contains the heart of it.
08:40 jnthn It's using libuv, and the VM runs an event loop thread which it uses to service async I/O requests.
08:41 ventica1 who enforces ordering? libuv or Moar?
08:42 jnthn Can you be more precise about ordering?
08:42 ventica1 i.e. two reads to same I/O or two write to same I/O
08:42 jnthn Ah
08:43 ventica1 or i guess OS maybe
08:43 jnthn There's two things at play there. The first is that IO handle operations are protected by locks. That handles two threads trying to start an operation on the same I/O handle at the same time.
08:44 jnthn The second part is that for async things we have a single event loop thread; since it's a single thread, it's only ever processing one notification at a time.
08:45 jnthn Typically it does the immediate things it needs to to get the stuff from libuv, and drops a notification into a concurrent queue, where one of the Perl 6 thread pool threads will pick it up and get to work.
08:45 jnthn (And if there's threads with nothing to do, they'll be waiting on a condvar, and one will be signalled)
08:46 ventica1 wow... so the request can potentially move up a level (to Perl 6) and back down to the VM (once the thread is assigned)?
08:47 jnthn I'm talking about callbacks or completion notifications there.
08:47 ventica1 ok
08:47 jnthn So there's no movement back up to the VM
08:47 jnthn Unless the code wants it
08:48 jnthn We send a "message" in the queue containing the Perl 6 callback to run along with the data.
08:48 jnthn So the thread that ends up with the work has all it needs.
08:50 * ventica1 reading up on libuv
09:05 brrt anybody has experience with running MSVC on wine?
09:05 brrt well, that doesn't work, so i don't think anybody has
09:09 FROGGS__ no, I dont think that is a good idea :o)
09:10 ventica1 sounds painful in the extreme
09:12 FROGGS__ install virtualbox + windows 7 + msvc express -> done (sort of)
09:12 FROGGS__ you need the windows sdk's of course
09:12 FROGGS__ and activeperl
09:13 brrt uhuh... ok, will do that
09:14 brrt but that means i'll have to get a windows 7 image fist
09:14 brrt first
09:15 jnthn brrt: I'll have a bit more time to look into the reason it explodes on Windows this evening, if you'd rather work on other bits...
09:15 brrt hmm... other bits are easier
09:15 brrt :-)
09:16 brrt if you would, i'd be happy
09:16 jnthn I wanted to look into it last night, but fixing the OSR lexicals occasional-bug took a while.
09:16 brrt it might be something really silly (i hope it is)
09:16 zakharyas joined #moarvm
09:16 brrt i have enough on my mind wrt to extops
09:16 jnthn extops and OSR are the two things that really block moar-jit helping the benchmarks.
09:17 brrt ok, well, i'll need to know a bit more about OSR before i know what to do with that
09:18 jnthn Sure, just ask when you're ready.
09:19 jnthn I think an initial step may be marking out which extops are JIT-friendly.
09:20 brrt non-invokish ops are most friendly
09:20 brrt jumpish ops are least friendly
09:20 brrt also, throwning stuff, we should handle that some day, to
09:20 brrt o
09:21 ventica1 is there a test that I can run to exercise the async socket code?
09:22 jnthn t/spec/S32-io/IO-Socket-Async.t
09:26 jnthn (If you didn't "make spectest" in Rakudo yet, it may not have been fetched)
09:27 ventica1 yeah, i haven't built anything at all yet, still dizzy and its bedtime
09:29 jnthn Ah :)
09:29 jnthn Well, rest well, and hope all of this doesn't seem like a bad dream in the morning. :)
09:32 ventica1 so, i guess what I'm looking for is some kind of enumeration of the kinds of things that go out through asyncsock i/o
09:32 ventica1 in or out
09:33 ventica1 is there a libuv func or funcs that all asyncsoc i/o reqs go thru?
09:33 ventica1 or maybe that's the wrong q
09:35 ventica1 anyway, afk for zzz... will attempt a build + test run in the next couple days
09:36 jnthn Well, an async socket write is an easy one to consider. An async write op is done on the handle; this in turn places the work into the async tasks queue. The thread running the libuv event loop (eventloop.c) receives this, issues the request with libuv. Later, libuv comes back with success or error. We package up a message and put it into a queue supplied with the initial request, where a Perl 6 worker thread handles it.
09:36 jnthn OK, sleep well :)
09:37 brrt ok, seems doable
09:37 brrt (the OSR stuff)
09:38 brrt basically, in MVM_spesh_osr_finalize, care has already been taken of inlining
09:40 brrt a spesh cand already exists from which i can hang the jitcode
09:40 brrt what i need to do is map the osr finalization 'deopt index' to a label in the JIT
09:40 brrt and update the bytecode to the magic bytecode (that invokes the JIT)
09:40 brrt and... i think i'll be golden, then
09:41 brrt basically the same game as in invoking stuff in the first place :-)
09:41 jnthn I think you'll already have the jitcode even
09:41 jnthn 'cus specialize triggers producing that iirc
09:42 jnthn Note that the OSR 'deopt index' in question is actually marked out specially
09:42 jnthn MVM_SPESH_ANN_DEOPT_OSR
09:43 brrt oh, you're quite right
10:04 brrt osr doesn't kick in early, does it
10:10 jnthn "kick in early"?
10:11 brrt it takes 99 iterations before OSR starts logging :-)
10:11 jnthn Right
10:12 jnthn Oh, and I guess the number somewhere is 100, and we < it, or something :)
10:13 brrt i guess so :-)
10:13 brrt what are characterististics of OSR deopt points?
10:16 jnthn Well, for one they are never used as part of deopt :)
10:17 jnthn They use the same "remember the offset" logic, though.
10:17 brrt ok
10:17 jnthn So they have an entry in the familiar deopts table
10:17 jnthn It's just that we do the lookup "backwards" to the usual use of the table.
10:17 brrt so basically, any instruction may be a osr deopt point?
10:18 brrt i see
10:18 jnthn osrpoint instructions exist in the unoptimized code.
10:18 jnthn These are turned in the logging phase into sp_osrfinalize or so
10:18 jnthn And then totally removed (and thus the annotation moved) in the final phase.
10:19 brrt what about deopt inline? are they ever used as 'real' deopt points
10:19 brrt ... oh
10:19 brrt that's an annoyance
10:20 jnthn What is?
10:20 jnthn The annotation is still there on the target place to jump into the bytecode
10:21 brrt well, if they're removed, then how will i know (at jit level) where i should push the label
10:21 jnthn It's just that it's on the instruction in the optimized code to go to
10:21 brrt ok, so the annotation is moved?
10:21 brrt oooooh
10:21 jnthn Right
10:21 brrt no problem for me then
10:21 jnthn "and thus the annotation moved" :)
10:21 brrt i see
10:21 jnthn But yeah, deleting instructions shuffles annotations.
10:21 jnthn MVM_spesh_manipulate_delete_ins does it
10:21 * brrt should look for his glasses embarassed
10:23 brrt ok, so ehm...
10:24 brrt the annotation is moved to the next instruction, so the label should be inserted before the instruction
10:24 brrt i can do tht
10:24 jnthn *nod*
10:33 brrt hmmwait
10:40 brrt lunch&
11:35 brrt joined #moarvm
11:49 brrt can there be multiple osr finalization points?
11:51 jnthn Potentially; nested loops could cause that.
11:52 carlin joined #moarvm
11:53 jnthn Yeah, seems log.c just translates them all
12:02 brrt hmm ok
12:02 brrt then my current approach is simplistic
12:03 brrt no matter
12:03 brrt that can be fixed
12:09 dalek MoarVM/moar-jit: 2f37da4 | (Bart Wiegmans)++ | src/ (10 files):
12:09 dalek MoarVM/moar-jit: Initial support for OSR
12:09 dalek MoarVM/moar-jit:
12:09 dalek MoarVM/moar-jit: This supports OSR for those frames that only have
12:09 dalek MoarVM/moar-jit: a single OSR deopt point. That isn't true of all frames,
12:09 dalek MoarVM/moar-jit: so it needs some fixing to support multiple OS depot points.
12:09 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/2f37da4157
12:11 carlin joined #moarvm
12:11 * jnthn wonders if that helps with a simple NQP benchmark like while_empty
12:15 brrt probably, yes
12:15 brrt it will probably also break on some benchmarks
12:15 brrt errands&
12:21 jnap joined #moarvm
12:42 brrt joined #moarvm
13:09 brrt joined #moarvm
13:19 brrt errands made me lose my concentration
13:19 jnthn aww
13:20 brrt allow me to rubbeduck :-)
13:21 brrt if i need to support multiple osr points, i need to find the right label to jump into
13:21 brrt i find the right label on basis of the current position of the interpreter
13:22 jnthn Could do it with a jump table mechanism, and pass in an index?
13:22 brrt well, yes, that's pretty much what i'm doing, except that i pass the pointer to the jit code segment directly so as to avoid annoying lookups and being bound to the table
13:23 jnthn ah, OK
13:23 jnthn That works too
13:25 brrt e.g. invokish ops aren't normally associated with a label, but i can use a 'local label' and load this directly into the jit reentry address
13:26 brrt and for the one-osr-point csae, i used a global label, which is stored in the global_labels array that dasm use and i had to setup
13:27 brrt but for multiple osr points, i need to use dynamic labels, which are allocated in a different array
13:27 brrt so what i think i'd need to do is associate the current offset with the entry label
13:28 brrt and make sure i output the correct /dynamic/ label at the correct osr point
13:28 brrt which means that this association should already exist at emitting-time
13:30 FROGGS[mobile] joined #moarvm
13:30 jnthn I guess however you do it, you'll need to have some analysis on this ahead of emitting...
13:33 brrt indeed
13:59 Ven joined #moarvm
14:06 brrt is there a way to know ahead of time how many osr points there are?
14:07 jnthn Conut them?
14:07 brrt ugh... yes i guess so
14:07 jnthn But aside from that, not really...
14:07 brrt ok, that's just how it will be then
14:07 jnthn Are you not already making a pass through the graph for some other things, anyway? Maybe not... :)
14:08 brrt no, i'm not doing that now
14:08 brrt graph creation is just a single pass
14:08 brrt linear too
14:08 brrt (yes, very very very boring :-) but i kind-of like it that way
14:10 jnthn I like simple things :)
14:11 brrt it's enough for now
14:12 brrt it may not be enough in the future, but who can tell?
14:27 btyler joined #moarvm
14:41 brrt oh, i know a line of c that is just evil
14:43 * brrt hopes you can find it in my next commit
14:43 jnthn Did I write it? :)
14:43 jnthn hah
14:43 jnthn :P
14:48 jnap joined #moarvm
14:49 brrt no, i've just written it
14:54 brrt and yes, you should mock it
14:56 * jnthn wonders if he'll also find a bug in it :P
14:56 * timotimo is annoyed the page-down button isn't bringing him to the spot where moar-jit works for everything
14:57 jnthn timotimo: I think it'll need using the other buttons quite a bit... :P
14:57 timotimo brrt: you probably already answered this before, but: is eliminating loads and stores between all instructions still doable for this summer? if not, does it have to wait a year for the next GSoC? :)
14:57 brrt ehm.... well.... the ambitious version of that, will have to wait a bit i'm afraid
14:58 brrt not-so-ambitious versions could be done, but there's still plenty to do before i get to that
14:58 timotimo OK
14:58 brrt it is my ambition to patch dynasm still this summer
14:58 brrt but moar has priority
14:59 timotimo well, it'll already be a big help if we can turn a simple "store This Here, load That Here" into a "use This instead of That in the following instruction"
14:59 brrt hmm
15:00 brrt you can (on the moarvm level) work on eliminating sets, and that would certainly help
15:00 brrt basically, it depends on how complex you want to get it
15:00 timotimo i've tried that once before and it blew up in an "interesting" fashion
15:00 jnthn None of it is interesting until the JIT is handling typical NQP and Perl 6 code :)
15:00 brrt such true
15:00 timotimo er ... of course
15:00 jnthn timotimo: heh, I have a patch that tries to kill sets too, and it also exploded things :S
15:01 brrt i think i handle quite a bit of typical nqp already, but that may be my arrogance
15:01 brrt well, there's a bug for sure :-)
15:02 brrt ah, that's my second most evil line that blows up
15:06 brrt hmm.. i see
15:07 timotimo well, code that only does things like access object properties, deciding things with boolean logic, printing to stdout ... those certainly are jittable right now :)
15:10 timotimo (not actually serious; i didn't look at bail reports in a long while)
15:18 brrt hey hey :-)
15:18 brrt we can invoke, we can handle invokish operators, loop, and now it seems we can OSR
15:20 * brrt realises there is still a lot to do indeed :-(
15:21 TimToady there's been a lot to do for 14 years now, but we still keep doing it... :)
15:21 brrt tests passed, time to commit
15:21 timotimo i don't know what it is, but there seems to be something that keeps us going
15:21 timotimo oh yay, the hunt for the evil line of c is on!
15:22 * TimToady wonders if the second-most-evil line is still evil...
15:26 dalek MoarVM/moar-jit: 9822cde | (Bart Wiegmans)++ | / (9 files):
15:26 dalek MoarVM/moar-jit: Support multiple OSR entry points
15:26 dalek MoarVM/moar-jit:
15:26 dalek MoarVM/moar-jit: Use the current bytecode offset to find the correct label
15:26 dalek MoarVM/moar-jit: to jump to, which are associated at compile time by use
15:26 dalek MoarVM/moar-jit: of the spesh annotation
15:26 dalek MoarVM/moar-jit: review: https://github.com/MoarVM/MoarVM/commit/9822cde480
15:26 brrt second most evil line is no longer so evil
15:27 timotimo brrt: is the osr_offsets sorted? you should use a binary search there!!!k
15:28 timotimo the evil line seems to have a comment pointing out what it is
15:28 jnthn It's unlikely to have more than 2 or 3 entries :P
15:28 brrt osr_offsets ehm.. i don't know that, i process the spesh graph in linear time
15:29 brrt linear order
15:29 brrt ugh
15:29 woolfy left #moarvm
15:29 timotimo well, time to process the spesh graph in logarithmic order insteak
15:29 timotimo instead!*
15:29 timotimo i applaud your use of *= in the last few characters of a line
15:32 jnthn I'm guessing something gets in the way of not putting the code in a test-osr sub?
15:32 jnthn (e.g. the mainline contains an op we don't JIT yet?)
15:35 brrt the OSR needs a caller
15:35 brrt i'm not sure if the mainline has that?
15:35 brrt (so that's why i didn't put it in mainline, btw)
15:36 brrt in other words, we seem to have OSR now \o/
15:36 * brrt dinner &
15:36 brrt left #moarvm
15:37 jnthn Yeah, it does have a caller
15:37 jnthn Otherwise OSR'd not help the benchmarks, and it does... :)
16:08 lizmat joined #moarvm
17:10 zakharyas joined #moarvm
18:26 brrt joined #moarvm
18:30 Ven joined #moarvm
18:31 * Ven recompiles to get masak's AST.Str
18:39 Ven Missing or wrong version of dependency 'gen/moar/stage2/QRegex.nqp'
18:39 Ven I did git pull && perl Configure.pl --gen-nqp --gen-moar --backends=moar && make
18:40 brrt hmmm
18:40 brrt i suppose it still has the old ones installed?
18:41 * brrt has no idea about how --gen-nqp and --gen-moar are implemented
18:41 * Ven makes clean
18:42 jnthn Was gonna say, was make install done also...
18:42 vendethiel oh yeah, forgot to put it in the list. it was
18:42 carlin joined #moarvm
18:43 Ven works, nice :-)
18:52 * brrt is getting a bit more excited about perl6 every day
18:55 * Ven is getting a bit more excited about the JIT every day :-)
19:02 ventica_desktop joined #moarvm
19:03 ventica_desktop should I clone MoarVM repo or rakudo?
19:04 jnthn If you clone Rakudo one you can --gen-moar to Configure.pl and it'll go grab the dependent repos
19:05 ventica_desktop ok
19:21 jnthn brrt: Did you get chance to look into how we bail on the mainline of an NQP program, btw? :)
19:21 brrt i'm not sure that we do
19:21 brrt but i'll see about it
19:22 brrt frankly, it never seems to reach the JIT graph creation at all
19:23 jnthn Hmm
19:23 jnthn tbh I didn't even check it for NQP, just for Rakudo
19:23 jnthn But they both compile to nqp::while
19:24 brrt nqp::while?
19:25 jnthn QAST::Op.new( :op('while'), ... )
19:25 lizmat joined #moarvm
19:25 jnthn Is how you'd normally see it
19:25 jnthn But
19:25 jnthn m: nqp::while(1, say 'omg')
19:25 camelia rakudo-moar b20535: OUTPUT«(timeout)omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤omg␤o…»
19:25 jnthn Actually works
19:25 brrt awesome
19:26 brrt but there's no op called while
19:26 jnthn Right, this is nqp:: op level
19:27 jnthn What I was getting at (badly :)) is that it's the code-gen from nqp::while that inserts osrpoint
19:27 jnthn So it should be working just the same between NQP and Rakudo.
19:27 brrt i see
19:28 brrt but ehm, in my osr.nqp, MVM_spesh_osr_finalize is never called
19:29 jnthn Oh?
19:29 brrt that is, without the containing osr-test sub
19:29 brrt it is called when i have put it in there
19:29 brrt funny, because MVM_spesh_osr is called, though
19:30 brrt (fprintf debugging ftw)
19:30 jnthn Weird
19:30 brrt yes
19:34 brrt wait
19:36 brrt has caller, does not have interned callsite
19:36 jnthn Ah
19:36 brrt might we be able to change that to has_caller /or/ has interned callsite
19:52 Ven joined #moarvm
20:41 jnthn I'll have a little Perl 6 / MoarVM time in a little bit; will look at the bug queue unless somebody has something more pressing for me :)
20:43 lizmat joined #moarvm
20:43 lizmat_ joined #moarvm
20:45 Ven mmh, we can't very well give a compile-time error for a missing mandatory named, can we ?
20:46 jnthn On a sub perhaps yes
20:46 jnthn On a method, no
20:47 Ven and what about too many positionals, maybe ?
20:47 Ven (yes, all on a sub)
20:48 jnthn m: sub foo($a) { }; foo(1, 2)
20:48 camelia rakudo-moar b20535: OUTPUT«[31m===[0mSORRY![31m===[0m Error while compiling /tmp/oikepEaKON␤Calling 'foo' will never work with argument types (Int, Int)␤    Expected: :(Any $a)␤at /tmp/oikepEaKON:1␤------> [32msub foo($a) { }; [33m⏏[31mfoo(1, 2)[0m␤»
20:48 jnthn Already do that :)
20:48 Ven m: sub c(:$c = 5) { $c }; sub { c(3) } # that I mean
20:48 camelia rakudo-moar b20535: ( no output )
20:49 jnthn The analyzer bails at present if it sees there's named params
20:49 Ven oh, alright =)
20:49 jnthn It's actually trying to prove properties that will let it compile-time resolve multi dispatches, or do inlinings.
20:50 jnthn And the errors are driven when it proves the call can never work.
20:50 jnthn *given
21:03 brrt left #moarvm
21:25 jnthn I'm suspecting that https://github.com/MoarVM/MoarVM/issues/114 also relates to the #116 fix I did yesterday
21:27 * jnthn leaves a note to say so
21:28 jnthn On #111 too
21:29 jnthn Less sure about that one, but it seems very viable
21:41 lizmat jnthn: if you needed to describe spesh in 1-2 lines, what would it be?
21:42 FROGGS__ jnthn: I'll retest the tickets I've created tomorrow
21:46 timotimo lizmat: since most code that is flexible/polymorphic at the time of writing ends up being static/monomorphic at run-time, it's helpful to optimistically re-write your bytecode to assume staticness and bail out if it turns out not to be the case. That's what spesh does. ← my attempt
21:47 lizmat timotimo++  :-)
21:47 jnthn That's not bad :)
21:47 timotimo i've learnt from the best :3
21:49 jnthn Spesh takes code that is highly dynamic, with a lot of late binding and polymorphism, and - based on the actual types that show up at runtime - generates specialized versions of the code that do away with the costly late-binding.
21:49 lizmat jnthn++
21:49 jnthn (Which works well because, as timotimo++ said, most potentially polymorphic code is monomorphic)
21:50 lizmat I will use that
21:50 jnthn The "bail out" bit is kinda worth mentioning too perhaps.
21:50 jnthn The fact that Moar can deoptimize is as important as its ability to optimize, in many senses. :)
21:52 timotimo we've seen a few times how not being able to properly deoptimize can affect the behavior of a program
21:52 jnthn brrt: So far, the Windows JIT failures are gone with JIT disabled, so it's certainly a JIT-related issue. OSR and inline being disabled don't affect things.
21:52 jnthn brrt: So seems quite confined to the JIT.
21:53 jnthn It *looks* like we end up storing bogus stuff in an attribute slot.
21:53 jnthn The segv happens if we GC mark it
21:53 jnthn But other times we get this:
21:53 jnthn P6opaque: invalid native binding to object attribute
22:51 jnthn Well, too tired to nail this tonight...sleep &
22:52 FROGGS__ gnight
23:12 lizmat joined #moarvm
23:21 lizmat fwiw, I just found out that you cannot install MoarVM on a Mac (and possibly other machines)
23:21 lizmat when you have a space in the names of any of the directories in which you're compiling
23:22 lizmat Piers traced this back to a problem in the makefile of dyncall
23:22 lizmat where there is apparently a call to "dirname" without quoting the parameter
23:22 lizmat what would be the best way to fix this?
23:22 lizmat afk&
23:22 TimToady destroy all macs?
23:23 lizmat could  be that it is the same on other OSes
23:24 TimToady destroy all mac users?
23:25 TimToady oh wait, windows users also use spacs...
23:25 lizmat spacey creatures  :-)
23:28 TimToady space, the final frontier...
23:33 lizmat :-)
23:33 lizmat afk&

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary