Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2016-01-12

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:12 timotimo i think so
01:36 colomon joined #moarvm
02:35 colomon joined #moarvm
02:49 ilbot3 joined #moarvm
02:49 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:34 hoelzro I feel like I'm at the end of this bug, but I can't quite get it
03:34 hoelzro I don't think I really understand the relationship between bytecode loading and deserialization
06:53 domidumont joined #moarvm
06:58 domidumont joined #moarvm
07:24 FROGGS joined #moarvm
08:19 zakharyas joined #moarvm
08:36 brrt joined #moarvm
08:42 brrt JimmyZ: why do you say that (MoarVM shall have tests), I don't know of that?
08:47 timotimo hello brrt :)
08:47 brrt hi timotimo
08:48 brrt scroll shadows in high contrast mode *facepalm*
08:48 brrt not well thought out this is
08:48 brrt timotimo: will you be at FOSDEM perchance?
08:48 timotimo nope :(
08:48 timotimo "scroll shadows"?
08:51 brrt yes, in applications, if you reach the end
08:51 brrt of scrolling
08:51 brrt too bad
08:56 timotimo oh
08:57 timotimo so with high contrast, they aren't actually changed at all so they just lower the overall contrast and legibility of things, eh?
08:57 brrt the shadows don't change, no
08:57 brrt the rest of the stuff does
08:57 brrt which is handy sometimes :-)
08:58 brrt how are things? still plans?
08:58 timotimo hm? what kind of plans?
09:01 brrt to hack :-)
09:03 timotimo ah, right
09:04 timotimo well, i'm behind one day on the weekly, but i want to contribute a little patch to krita (an open-source drawing/painting program) just so i don't have my head in perl6 all day every day
09:06 brrt :-)
09:06 brrt that's well enough
09:06 * brrt is in the finishing stages of his thesis now :-)
09:06 brrt i found a limiting case of python
09:07 brrt a loop of 50 lines long.. i can't tell which block belongs where
09:07 timotimo to be fair, braces don't help much there, do they?
09:07 brrt so i'm resorting to counting indentation, this sucks
09:07 timotimo i have a vim plugin that colors areas by indentation depth; it may not be high-contrast enough for you, though
09:08 brrt braces would help; they always have a corresponding other brace
09:08 brrt so i can have emacs tell me where the closing brace responds to
09:08 brrt corresponsds
09:09 brrt anyway
09:09 brrt i haven't really hacked on moarvm htese last few weeks
09:13 timotimo there ought to be something in emacs that'll give you that same feature with python
09:14 brrt hmmm
09:14 brrt probably, yes :-)
09:14 timotimo i have no clue if there is. but it ought to be made if there isn't :)
09:19 kjs_ joined #moarvm
09:20 kjs_ joined #moarvm
09:35 timotimo humpf. my simple five-line patch idea just turned into what amounts to a swim through potentially a big amount of code >_>
09:35 timotimo i'm not thrilled
09:39 JimmyZ brrt: not really me, jnthn++ said it 2 years ago, IIRC.
09:40 brrt well, it hasn't become a priority yet
09:40 brrt i think testing unit-ish things like string ops, that may be plausible
09:40 JimmyZ yes, so, it will
09:40 timotimo yeah
09:40 brrt why are you so sure? the hardest part are the most difficult to test, it seems
09:41 brrt stuff like gc, concurrency etc
09:41 brrt thats already tested by nqp
09:41 FROGGS I think that that running nqp's testsuite is good enough
09:41 brrt although i'd agree with the sentiment that we would like a bit more
09:41 FROGGS you dont even need to build it, you could even check out stage0 and run the tests
09:41 brrt but thats still for me personally to figure out, how to test the jit better
09:47 JimmyZ like, test some code whether it is jitted well enough after we changed the spesh/jit code
09:47 JimmyZ or s/it/it still/
09:48 brrt that is a good idea; however i think that can live in the nqp testsuite
09:49 brrt moar doesn't really have a separate personality next to nqp
09:50 timotimo agreed
09:51 timotimo i've sat next to a discussion about giving moarvm a little bootstrapped moar+nqp of its own for build system purposes before
09:52 brrt hmmpf
09:52 timotimo with that nqp copy, a freshly-built moar could run its own test suite which would then be written in nqp
09:52 brrt turtles will have to stop somewhere
09:52 JimmyZ we did have  some test in moarvm and only for moarvm around 2013
09:53 timotimo :)
09:53 JimmyZ it uses nqp to generate some moarvm only code.
09:53 JimmyZ like fib
09:53 JimmyZ something like use QAST
10:10 leont joined #moarvm
10:24 virtualsue joined #moarvm
10:24 zakharyas joined #moarvm
10:37 kjs_ joined #moarvm
11:11 kjs_ joined #moarvm
11:20 TimToady joined #moarvm
12:19 brrt joined #moarvm
12:42 virtualsue joined #moarvm
13:03 brrt joined #moarvm
13:15 jnthn hoelzro: Bytecode loading triggers a "we're being loaded" frame to be interpreted, which somewhere calls a deserialize op. It's actually up to that code when exactly deserialize takes place. So the two things are rather decoupled.
13:16 jnthn I don't think that we'll have a Moar test suite that covers the same set of things the NQP test suite does, since the NQP test suite does an increasingly good job of covering all the ops.
13:16 timotimo especially thanks to pmurias' work
13:16 jnthn Yes, pmurias++ for that.
13:17 jnthn We may have some tests for MoarVM that stress GC, or maybe optimization-related tests. I imagine we'll write them in NQP; since they're stress rather than sanity tests, it doesn't matter hugely if we can't run them until we build an NQP. :)
13:53 hoelzro jnthn: so a .moarvm file contains bytecode, which is arranged in the structures described in the documentation.  One of the frames is the deserializer, which calls deserialize with a string in that bytecode file's heap; is that correct?
13:53 jnthn Used to be
13:54 jnthn It actually passes the op a null string
13:54 jnthn And that means "go find the serialization blob from a special segment in the bytecode file"
13:54 jnthn Which avoids having to base-64 it to shove it in the strong heap
13:54 hoelzro ooooh, I was wondering why there was a null_s in the moar dump
13:55 hoelzro jnthn: which segment is that?
13:57 hoelzro ah, I see it
13:57 hoelzro the SC data, it seems
13:57 jnthn oh, actually I lied
13:57 timotimo i had this random idea: can we annotate the gc_mark functions of our reprs with something that'll put all of them together in a special section in the final binary?
13:58 jnthn /* Special frames. */
13:58 jnthn MVMStaticFrame  *main_frame;
13:58 jnthn MVMStaticFrame  *load_frame;
13:58 jnthn MVMStaticFrame  *deserialize_frame;
13:58 jnthn deserialize_frame is run unconditional of the "load mode"
13:58 timotimo perhaps that can improve strain on the code cache during GC mark runs?
13:58 jnthn load is run if it's loaded in any way other than as the mainline
13:58 jnthn main is run if it's loaded as the program entry point
13:59 jnthn timotimo: Not run into such an annotation
14:00 jnthn timotimo: Also not sure there's enough gc_mark routines to have huge impact
14:00 timotimo mhm
14:00 jnthn (The reasoning is sound, just not sure if it'll amount to measurable.)
14:00 timotimo and the gc always has to go through the REPR pointer anyway
14:00 timotimo probably not
14:00 hoelzro jnthn: sorry for all the questions, but could you clarify the difference between an MVMFrame that's loaded as a result of deserialize_frames in the bytecode loading, vs an MVMFrame loaded as a result of deserialize_contexts in SC loading?
14:00 * timotimo is just spewing random ideas at this point :)
14:00 timotimo things that come to me during showers, or while falling asleep
14:00 hoelzro I'm at my wits' end with this bug, and it all comes down to that
14:01 jnthn hoelzro: Confusing terminology is confusing :)
14:01 jnthn /* The various static frames in the compilation unit, along with a
14:01 jnthn * code object for each one. */
14:01 jnthn MVMStaticFrame **frames;
14:01 jnthn And
14:01 jnthn static MVMStaticFrame ** deserialize_frames(MVMThreadContext *tc, MVMCompUnit *cu, ReaderState *rs) {
14:01 jnthn Those are actually both about *static* frames
14:02 jnthn deserialize_contexts is about actual MVMFrames
14:02 jnthn MVMStaticFrame contains all the static things about a call frame (mapping of lexical names to indexes, the bytecode, etc.)
14:02 jnthn MVMFrame is an actual invocation record
14:03 jnthn An MVMFrame points to an MVMStaticFrame
14:03 jnthn Does that help clear things up a little?
14:03 hoelzro ah ha
14:03 hoelzro yes, it does
14:03 hoelzro they both seem to carry lexical information, though
14:03 jnthn Maybe the language used in the code was clearing up
14:04 jnthn An MVMFrame carries the actual set of values lexicals are bound to
14:04 jnthn It doesn't know anything about names, though. It's just a bunch of indexed slots.
14:04 jnthn MVMStaticFrame carries the metadata on what types of values are in the slots and the name that goes with each slot.
14:04 hoelzro what's confusing to me is that deserialize_context itself creates a MVMStaticFrame/MVMFrame pair
14:05 hoelzro so I'm not sure where the MVMFrames from bytecode decoding come in
14:06 jnthn MVMFrames aren't produced by bytecode decoding...
14:06 jnthn (There's no MVMFrame mentioned at all in bytecode.c)
14:07 hoelzro hmm...let me see if I can find the code that made me think that
14:07 hoelzro it may make more sense after reading your explanation
14:07 jnthn Sure. Feel free to point me at any other bits of code you want me to explain further also.
14:08 hoelzro I suppose it was MVM_bytecode_finish_frame
14:08 hoelzro but that has to do with static frames
14:08 jnthn Yes
14:08 jnthn That's just laziness
14:08 hoelzro got it
14:08 jnthn (As in, putting off work until we have to do it)
14:08 hoelzro well, let me describe my issue, and maybe you can point me in the right direction
14:08 jnthn Well, and the name is typing laziness too :P
14:08 hoelzro heh =)
14:09 hoelzro basically, if I create an anonymous sub at BEGIN time in a module, that sub cannot make calls to subs in CORE at runtime
14:09 jnthn We probably shouldn't pun on two so easily confused things, except we often get away wiht it...
14:09 hoelzro it can make method calls
14:09 hoelzro or qualified sub calls
14:09 jnthn Ah...I fixed something like that before for constants...
14:10 jnthn git log --author=jnthn --grep=constant
14:10 hoelzro in my example, the frame->outer chain is three frames longs; if I remove the BEGIN, it's like six
14:10 hoelzro moar --dump says that the outers chain is longer than I see in practice
14:10 jnthn hmm, that don't find it
14:11 hoelzro here's the actual RT: https://rt.perl.org/Ticket/Display.html?id=127089
14:12 jnthn Ah, I was misremembering
14:12 jnthn 76204fa58a7
14:12 jnthn Is the Rakudo commit I last touched in this area
14:13 jnthn I think it's the same kind of issue.
14:13 hoelzro ah ha
14:13 jnthn Maybe we shouldn't be teaching BEGIN/role bodies specially
14:13 jnthn Maybe we should be doing this on *any* code we trigger at compile time.
14:13 hoelzro that makes sense to me
14:13 jnthn s/teaching/treating/
14:14 jnthn I'm not sure off hand how hard it will be, other than "it'll be hard because this stuff is always a pain"
14:14 hoelzro =/
14:14 hoelzro well, at least I should be able to fix my issue for now
14:14 hoelzro if not, I know where to find you =)
14:14 hoelzro thanks a ton, jnthn
14:15 timotimo thanks jon-a-ton
14:15 hoelzro haha
14:15 jnthn :P
14:17 * jnthn wonders if there isn't a more elegant solution to this kind of problem...
14:17 jnthn So far I've only come up with solutions I hate less than what came before them. :)
14:17 hoelzro as long as hate is a strictly non-increasing function over time, it's ok =P
14:22 virtualsue joined #moarvm
14:24 timotimo don't want to get stuck in a spiral of hatred
14:27 jnthn .oO( How does a hate spiral compare with a love triangle? )
14:27 moritz less edges, more hateful
14:27 timotimo probably also has spiders in it
14:27 nwc10 more dimensions
14:28 nwc10 a triangle is at best a plane
15:12 FROGGS joined #moarvm
15:23 domidumont joined #moarvm
16:53 ilmari https://gist.github.com/ilmari/2a23f96373163a1fa45b
16:53 ilmari Draft MoarVM magic(5) file ^^
16:54 hoelzro ilmari++
16:54 ilmari dunno how relevant all the sizes/counts are
16:54 ilmari maybe just display the bytecode and serialization versions?
16:55 hoelzro I would probably just do that
16:56 jnthn They're nice to have in a sense.
16:57 jnthn But maybe I think that 'cus Moar doesn't provide a good way to get at them already :)
17:02 ilmari I notice the bytecode itself doesn't have a version. does moar check for unknown ops at load time?
17:04 jnthn It's part of validation
17:04 jnthn We should at some point establish version policy around adding new ops.
17:05 jnthn Probably some point soonish given we're in a post-6.c world and we'd like to be able to have people easily upgrade their MoarVM without upgrading the things running on it.
17:05 ilmari ah, in get_info()
17:06 ilmari as long as we add new ops at the end and don't remove existing ones, that would Just Work™
17:07 jnthn Oh, that direction works already, yes
17:07 jnthn It's more that it'd be better to up-front detect accidents like somehow running a newer bytecode file on a Moar that can't handle all the ops in it.
17:08 ilmari afaict validation.c does that
17:08 jnthn Yes, with the caveat that we don't validate bytecode for a frame until we actually execute that frame.
17:08 jnthn So the error can come a good way down the line.
17:08 ilmari ah
17:08 ilmari that's LTA
17:09 ilmari also, the error message is a bit LTA too: "Bytecode validation error at offset %u, instruction %u:\ninvalid opcode %u"
17:09 jnthn Right. It's not that we don't catch it, but a better versioning policy would let us give a better experience.
17:09 ilmari being a) up-front and b) explicit about "your moar is too old" would be MA
17:09 jnthn Agree
17:10 jnthn New ops are pretty rare already, so we could think about making it policy already.
17:10 ilmari well, I added two only the other day
17:10 jnthn OK, they're more rare than they maybe were before :)
17:11 jnthn I think mostly I mean they're uncommon enough that I don't mind the extra developer time overhead of having to bump a number.
17:11 kjs_ joined #moarvm
17:16 ilmari yeah, but there currently doesn't exist such a number (unless you want to bump the top-level number each time)
17:24 jnthn I'm not sure there's a downside to bumping the top level number for that
17:25 ilmari but then, why the separate serialization version?
17:27 jnthn Because the serializer can be used independent of bytecode files (in fact, we used to store the serialized blob on the string heap)
18:20 leont joined #moarvm
18:27 vendethiel joined #moarvm
18:29 domidumont joined #moarvm
18:58 kjs_ joined #moarvm
19:00 FROGGS joined #moarvm
19:23 leont joined #moarvm
19:28 virtualsue joined #moarvm
19:36 jnthn Oh noes...I just tried to build Rakudo on my desktop after being away for a while and... msinttypes/inttypes.h is missing
19:45 zakharyas joined #moarvm
19:46 virtualsue joined #moarvm
19:50 FROGGS :S
19:52 dalek MoarVM: 940850e | jnthn++ | Configure.pl:
19:52 dalek MoarVM: Correct installation of msinttypes header test.
19:52 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/940850e669
19:53 FROGGS jnthn++ # I merged the PR that broke it
19:53 FROGGS (sorry)
19:53 jnthn No worries, wasn't a hard fix
19:53 jnthn Somewhat curious why things seem slower on this box than before...
19:54 FROGGS do you clean your install/?
19:54 jnthn (make test in NQP is up a bit, build times for Rakudo also)
19:54 jnthn Yeah
19:54 FROGGS I mean, because I did not notice the breakage
19:54 FROGGS k
19:54 jnthn Ah
19:54 jnthn You'd get away with it then
19:54 FROGGS dunno about the slowdown though
19:54 jnthn That's why I didn't notice it on my laptop
19:55 * ilmari is inordinately amused by the filename moar.magic
19:55 jnthn :D
19:55 FROGGS .magic?
19:55 FROGGS where is that from?
19:55 * geekosaur flips the switch...
19:56 geekosaur FROGGS, libmagic which is the file type detection stuff used by file(1)
19:56 geekosaur it's driven by config files describing how to identify a given file type
19:57 FROGGS I know mime magic... though I did not know about .magic files
19:58 geekosaur (the original file command had a file of "magic numbers" from back when most data files started with a binary value indicating what kind of file it was)
20:04 ilmari FROGGS: https://gist.github.com/ilmari/2a23f96373163a1fa45b
20:04 FROGGS ilmari: ahh, now I understand
20:04 ilmari save that, do file -m moar.magic somefile.moarvm
20:04 ilmari you'll get nice details
20:04 FROGGS awesome
20:04 FROGGS will try
20:05 ilmari and geekosaur spotted the reference
20:05 moritz is anybody trying to get moar.magic upstream into file(1)?
20:06 moritz or how are such things handled?
20:06 ilmari moritz: I'm planning to submit it to the mailing list once we agree it's in suitable shape
20:06 ilmari FROGGS: http://catb.org/jargon/html/magic-story.html
20:06 moritz ilmari: great, ilmari++
20:09 ilmari hm, wonder if the magic language is flexible enough to extract the HLL name
20:09 ilmari but not now, hometime now
20:10 ilmari should have left half an hour ago, got distracted :-/
20:12 ilmari yes, it can do multiple levels of indirection
20:13 ilmari \o/
20:26 FROGGS CORE.setting.moarvm: MoarVM bytecode (version 5), 7 serialization contexts, 16 extension ops, 10733 frames, 352 callsites, 19921 strings, 5307976 bytes serialized data, 3783540 bytes bytecode, 505272 bytes annotations
20:26 FROGGS I love it :o)
20:26 FROGGS ilmari++
20:40 zakharyas joined #moarvm
20:51 FROGGS joined #moarvm
20:53 kjs_ joined #moarvm
21:02 [Coke] nice!
21:31 virtualsue joined #moarvm
21:51 ilmari FROGGS: oh, that version doesn't have the serialized data version, the one I have at $ork adds that
21:51 FROGGS now my sword glows blueish
22:01 ilmari I was trying to extract the hll name, but that would require looping over the string tables, which I'm not sure it can do
22:02 ilmari since the strings are stored as a sequence of <length>,<data> pairs without an index
22:02 FROGGS hmmmm
22:03 FROGGS the string heap, yeah
22:06 ilmari the hll name is just stored as "the Nth string"
22:19 ilmari oh, and the length of each string is stored shifted left one, with the bottom bit indicating whether it's latin1 or utf8
22:42 kjs_ joined #moarvm
22:53 timotimo i'm back
22:54 virtualsue joined #moarvm
23:05 kjs_ joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary