Camelia, the Perl 6 bug

IRC log for #moarvm, 2013-08-04

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:40 lizmat joined #moarvm
02:07 colomon Fixed ABC module.  Turns out I long ago used "our method" to declare a few of the methods, and early last week some Rakudo change broke that.
02:08 timotimo what was that supposed to do?
02:11 colomon heck if I remember
02:11 colomon probably just to export the method out of the module
02:12 colomon might even have been an attempt to work around a bug that got left in the code by accident.
02:12 colomon at any rate, it worked like just plain "method" would have a month ago in Rakudo, and still does in Niecza.
02:13 timotimo :)
02:41 colomon bet the flower module failure is the same thing, I see it's using "our submethod" in a couple of places.
02:44 colomon and perl6-web-template is failing because flower is.
02:45 benabik .oO( Wonders if colomon knows this is #moarvm and not #perl6 )
02:45 colomon benabik: doh!
02:45 benabik Not that there isn't a notable amount of overlap...
02:47 colomon benabik++
02:47 * colomon slinks back to #perl6....
04:33 lizmat joined #moarvm
05:49 FROGGS joined #moarvm
06:00 FROGGS joined #moarvm
06:31 crab2313 joined #moarvm
07:08 FROGGS joined #moarvm
07:22 JimmyZ Good afternoon
07:23 sorear o/
07:25 JimmyZ \o
09:26 FROGGS[mobile] joined #moarvm
09:28 FROGGS[mobile] o/
09:29 FROGGS[mobile] sorear: long time no see
09:32 jnthn morning, #moarvm
09:34 JimmyZ morning, jnthn
09:36 JimmyZ jnthn: how about merge readlineintfh2 branch? it works very well on windows, with ctrl+key(ctrl + r, etc)
09:37 jnthn JimmyZ: I...didn't even see what branches have happened while I was gone yet:)
09:37 JimmyZ hehe
09:59 JimmyZ jnthn: how about remove 'type_object' args in MVM_decode_C_buffer_to_string and other decode functions?
10:08 FROGGS[mobile] JimmyZ: you propose to change the nqp ops too?
10:11 jnthn JimmyZ: In favor of what?
10:12 jnthn Building latest MoarVM fails for me :(
10:12 FROGGS[mobile] oh dear
10:13 FROGGS[mobile] how?
10:13 jnthn Loads of linker errors related to APR
10:14 FROGGS[mobile] I hab to remove apr's build dir at some point
10:14 jnthn Like
10:14 jnthn apr-1.lib(filedup.obj) : error LNK2019: unresolved external symbol __imp_setvbuf referenced in function apr_file_dup2
10:14 * jnthn tries that
10:15 FROGGS[mobile] I believe there is a Release dir in apr/, remove it and build afresh
10:16 FROGGS[mobile] dunno why though, I'm building rarely on windows
10:18 jnthn Yeah, git clean -fdx 3rdparty helped
10:36 JimmyZ jnthn: just looks like needless
10:36 jnthn JimmyZ: The param is being used, no?
10:37 JimmyZ jnthn: just always tc->...VMstring
10:39 JimmyZ re: apr build failed, for implementing getenvhash, I changed to link libcmt, instead of msvcrt
10:41 JimmyZ jnthn: the param looks like tc->...oshandle before in fileops.c
11:29 JimmyZ so for type_object param, Will there be any value instead of 'tc->instance->VMString' ?
11:30 jnthn JimmyZ: Probably not for the time being.
11:31 jnthn Potentially some day it'd come out of HLL config but we've no need/use for that right now.
11:31 jnthn Str in Perl 6 will point to a VMString.
11:32 jnthn So we could probably go same way as we did with fileops.c
11:33 JimmyZ oh yeah, that's want I want to know also, jnthn++
11:33 JimmyZ s/want I/what I/
12:18 JimmyZ jnthn: I was trying compiling QASTOperationsMAST.nqp, but got https://gist.github.com/zhuomingliang/6102349, the cause is the 'nqp::splice' line
12:19 jnthn JimmyZ: k, will look, thanks
12:20 JimmyZ jnthn: thanks :P
12:20 jnthn Just working on porting the compile time bind analysis stuff on JVM at the moment.
12:22 JimmyZ good
12:46 timotimo i'm heating up my benchmarking machine already, jnthn :)
14:19 birdwindupbird joined #moarvm
14:39 FROGGS joined #moarvm
15:03 crab2313 joined #moarvm
15:24 benabik joined #moarvm
15:25 diakopter sorear: hi! :)
15:32 diakopter jnthn: ping
15:33 diakopter jnthn: at oscon I met Damian Conway
15:33 diakopter we talked about MoarVM and things
15:33 jnthn diakopter: o/
15:33 jnthn diakopter: nice
15:33 diakopter he said that after a couple weeks from now, he (suddenly) has the rest of the year open, so he's looking for things to tackle
15:34 jnthn wow :)
15:34 diakopter I wasn't able to get him super-excited about anything in particular, but I'm guessing you or pm could
15:35 diakopter he was certainly interested
15:36 diakopter we couldn't think of an ultra-compelling demo use case for the p5interop - something that showcased the stackless cross-vm callbacks and/or .. hm, now I don't remember the other thing
15:37 diakopter jnthn: another thing from oscon:
15:37 diakopter discussed including the sregex library in MoarVM with a couple folks
15:37 diakopter (agentzh's regex JIT)
15:38 diakopter I think it would be VERY worth it to include that in MoarVM before any other JIT
15:38 diakopter since regexes are generally abominably slow compared to p5 currently
15:38 lizmat why would you want to include another regex engine when rakudo has one?
15:38 diakopter since it'd be thousands of times faster
15:39 lizmat aren't there severe compatibility concerns ?
15:39 lizmat yes, faster but wrong ?
15:39 diakopter surely you're kidding
15:39 lizmat let me rephrase that: why does rakudo have its own regex engine, when it could be done with sregex
15:40 diakopter "but won't you incorporate it badly?"
15:40 diakopter "it" is defined poorly there
15:40 diakopter "it could be done"
15:40 diakopter I wasn't saying "ALL regexes" could be done
15:40 lizmat maybe I dpn't understand what sregex is then
15:40 benabik diakopter: "but doesn't P6 have a different definition of regexen than other things"
15:40 diakopter just the ones that sregex can handle
15:40 diakopter okay.
15:41 timotimo is there a way to weave slow parts into sregex to make it cover all of p6 rules?
15:41 diakopter no; it'd be an optimization, just like the NFA one
15:42 diakopter we don't say "but doesn't P6 have a different definition of regexen than what NFA can handle?"
15:42 diakopter we simply don't use the NFA optimization when it can't be used
15:42 diakopter we don't say "why does rakudo have its own regex engine, when it could be done with a native NFA engine?"
15:42 timotimo ah. but a complex rule could be sped up even if it has sregex-incompatible parts by using sregex on the whole rest? does that work?
15:42 diakopter right, just like the nfa optimization
15:43 timotimo that clears up everything for me
15:43 timotimo sounds very nice to have, indeed.
15:43 diakopter maybe.
15:43 diakopter but it's at least worth experimenting with
15:45 diakopter lizmat: you don't see value in making 80% of regexes 900000% faster?
15:45 lizmat I do, I do
15:45 diakopter because it's unfair to the other 20%?
15:46 lizmat no no no
15:46 timotimo stop arguing, you've convinced us ;)
15:46 diakopter I'm just speculating possible objections
15:46 diakopter because it makes the other 20% seem cripple and not useful
15:46 diakopter crippled*
15:46 diakopter punitive
15:47 lizmat I sorta assumed "including" meant replacing, like you can sorta replace the regex engine in perl5
15:47 diakopter oh
15:48 diakopter it would need extended to integrate with my libicu reimplementation
15:48 diakopter (which could ruin its entire speed benefit)
15:49 diakopter (if it uses jump/lookup tables of ascii/ansi size all over the place)
15:49 lizmat well, let me put it this way: I'd rather see effort put into making the regex engine as a total faster
15:49 lizmat than going for a quick performance benefit that may be lost in the long run when doing e.g. NFC
15:49 diakopter I'm not very confident that's possible
15:50 diakopter NFC?
15:50 diakopter it could be integrated with an NFG implementation without detriment
15:50 lizmat NFG is what I meant
15:51 diakopter well it would add another translation layer of index, but that's ok
15:51 lizmat when I hear you talking about ascii based jump tables, it makes me shudder in that respect
15:52 diakopter I was definitely admitting that making it very unicode aware might kill any benefit...
15:52 timotimo let's invent an encoding that puts all of unicode into just 8 bits
15:52 diakopter timotimo: actually, the storage optimization TimToady came up with for NFG will do that for NEARLY ALL strings
15:53 diakopter .. if not every string ever seen in actual non-pathological input
15:53 timotimo that sounds great
15:53 arnsholt diakopter: Using high-bit characters as index carriers, or something along those lines?
15:53 diakopter but it does require another translation layer
15:53 arnsholt (the storage optimisation, that is)
15:53 JimmyZ sregex? looks good
15:54 diakopter arnsholt: yes, to segment the point space into two areas, the lower area that map directly, and the upper area that map through another lookup table
15:55 arnsholt Yeah, that makes sense
15:56 diakopter so the only thing needing to be stored is that upper space lookup table (4,8,16,32,64,128 codepoint/nfg mappings) and the number of bits used for the high area
15:56 jnthn JIT compiling the NFAs used for LTM may help some, but that's not the dominant cost today
15:56 diakopter I wasn't talking about just for LTM
15:57 JimmyZ agentzh said sregex is based on http://swtch.com/~rsc/regexp/regexp2.html
15:57 diakopter jnthn: it can handle captures
15:58 * JimmyZ saw Thompson and pike Implementation in that link
15:58 diakopter jnthn: what's the dominant cost today?
16:00 jnthn diakopter: Everything else after we decided where we're going, I think :)
16:00 jnthn diakopter: Building match objects is one notable cost, iirc
16:02 timotimo jnthn: could it be tweaked like the cursor object has been?
16:02 diakopter decided where we're going?
16:03 jnthn timotimo: Perhaps some more, I think CAPHASH is partly where the cost is. I already did the LHF there :)
16:03 jnthn diakopter: The NFAs only decide which alternation or protoregex to go down.
16:03 timotimo until then, maybe i can speed up rakudo parsing by throwing dots into more subrules that are not used in the actions?
16:04 diakopter jnthn: right.. what happens after that
16:04 jnthn diakopter: We run the code that the regex compiler produces. :)
16:05 diakopter okay, that's hilarious, but why.. what do we need from it? just capture offsets?
16:05 diakopter (if we had all the capture offsets, could you generate the match offsets)
16:05 diakopter er, generate the matches?
16:06 diakopter (make all the matches nested captures)
16:07 FROGGS joined #moarvm
16:07 jnthn Probably, I'm just saying I'm not sure that (a) finding the offsets is the slow bit today, and (b) a normal JIT would not make a decent job of some of the code we generate anyway, given various bits of it are just nudging positions forward and calling into ops to do comparisons, char class lookups, etc.
16:08 jnthn If there's anything that sregex may do a LOT better than we do today, it's almost certainly handling the backtracking cases.
16:08 diakopter I think you're right about (b)
16:08 diakopter but I think that's so far away, and this could be a lot sooner
16:08 jnthn But grammars don't backtrack, so while it'd be a boon for regexes, it'd not for grammars, or at least that's my expectation.
16:08 diakopter and wouldn't take a huge amount of work
16:09 diakopter why don't grammars backtrack?
16:09 jnthn 'cus keeping state to do that would be costly/really slow.
16:09 jnthn token and rule mean "don't backtrack"
16:09 jnthn Or rather
16:09 jnthn Don't store information needed to backtrack
16:10 diakopter I know why token doesn't need to backtrack, but why doesn't rule need to backtrack?
16:10 jnthn rule = token + sigspace.
16:10 jnthn (the thing that turns whitespace into a <.ws> call)
16:10 diakopter hm, ok
16:11 jnthn It's only regex that has backtracking on by default.
16:11 jnthn And that shows up only a few times in the Perl 6 grammar.
16:11 diakopter I guess I don't get why rakudo regexes are so slow then, if it's not regex processing itself
16:11 diakopter *regex processing logic
16:11 diakopter sorry, rakudo grammars
16:12 * jnthn suspects method call overhead and match/cursor construction overhead.
16:12 diakopter why wouldn't a regex JIT (that converted entire grammars to regexes) eliminate all of those?
16:13 diakopter er, entire rules in grammars
16:13 jnthn Perl 6 grammars are, by design, a mix of declarative (probably easy to regex JIT) and imperative (harder as we can get recursion)
16:14 jnthn Those that are declarative (that we build the NFA for today) probably can be handed off to a regex JIT.
16:14 jnthn One thing Pm and I discussed is whether in certain cases the NFA thing we do today can just say "oh, don't bother doing the work again once you get into this rule, just go right to this char"
16:15 jnthn (The answer is it almost certainly can, but finding a neat factoring is harder.)
16:16 diakopter I was aware of all of that; I was asking what about the imperative portions makes them impossible to JIT to one of these engines
16:17 diakopter (just recursion? b/c there are ways to eliminate that)
16:17 jnthn Partly recursion, partly the need to evaluate code blocks
16:17 diakopter well, in the Perl6 Grammar, are there really inline code blocks?
16:17 diakopter (if so, *why*)
16:19 diakopter I'd think the special things like bracket matching are just a special case of backreferences
16:19 jnthn Yes, including in <.ws>
16:19 jnthn Which means you can hit 'em pretty quick :)
16:19 diakopter what in the world is in .ws
16:20 jnthn https://github.com/perl6/st​d/blob/master/STD.pm6#L607 is the STD one
16:21 diakopter so, what if we put fastfail death traps in there.. such as when it hit a point where it would normally do something imperative.. just fall out of the fast thing and back to the slow thing
16:21 diakopter the only imperative things I see there are optimization attempts
16:22 diakopter and error handling
16:22 diakopter and the deadly inline comments
16:23 diakopter which need vast simplification
16:24 jnthn "deadly inline comments"?
16:24 diakopter "deadly" with a wink for how they cause grammar slangs
16:25 diakopter pod should be moved up to the top level of .ws
16:26 diakopter so the rest can be one big declarative thing
16:27 diakopter oh boy, there's lots of work of automatic grammar refactoring to be done
16:27 diakopter commute &
17:03 diakopter well hi
18:27 japhb o/
18:32 diakopter japhb: hi
18:32 japhb Hey there!  How goes it?
18:36 diakopter good; how are you?
18:37 japhb Not too bad.
18:37 japhb Nice to be coming back to the community, let me tell you ...
18:37 diakopter heh.
18:39 diakopter anyone: my nqp-cc Configure.pl is failing with git fetch failure
18:39 diakopter Permission denied (publickey).
18:39 diakopter fatal: Could not read from remote repository.
18:40 diakopter why would publickey fail to perl6/nqp
18:44 diakopter whatever.... rebuilding nqp. :(
18:46 benabik diakopter: Sounds like an issue between you and github.  Or perhaps file permissions in your ~/.ssh dir or something.
18:46 diakopter no
18:46 diakopter it didn't like that my nqp dir was empty, but its .git was there
18:46 diakopter silly corner case.
18:47 diakopter no clue what deleted nqp/
19:00 diakopter .. and of course parrot configure failed too. D: :( :( not my day I guess
19:00 diakopter only error from parrot configure:
19:00 diakopter During configuration the following steps failed: 01:  init::manifest
19:03 diakopter init::manifest -      Check MANIFEST...No such file: examples/compilers/Makefile
19:03 diakopter No such file: examples/embed/Makefile
19:03 diakopter No such file: examples/tools/Makefile
19:03 diakopter argh.....
19:04 diakopter manually restored those files from git...
19:05 benabik Sounds like you have a file deleting gremlin.
19:34 diakopter and now.. none of nqptest succeeds
19:34 diakopter jeez.
19:36 japhb diakopter, I'd just move that entire moarvm tree away and clone and build a completely new one.
19:37 japhb Too much strangeness going on, with too many mysterious "fixes"
19:37 diakopter turns out there was a rogue nqp/moarvm still running
19:37 diakopter hidden
19:37 japhb hidden?
19:37 diakopter not visible from ther terminal
19:37 japhb ?!?
19:37 diakopter happens all teh time on windows
19:38 japhb Oh.  Yeah.  WinFail, I forgot you used that.
19:38 diakopter bah, no iculib.
19:38 diakopter that's a fail.
19:39 japhb (TBC, I wasn't calling Windows WinFail, I was referring to the thing that Windows *does* on a regular basis)
19:39 * japhb separates himself about a micron from fanboydom
19:39 diakopter yes, I use those. ;)
19:56 Alpha64 joined #moarvm
19:57 diakopter FROGGS: ping
20:04 FROGGS diakopter: pong
20:05 diakopter FROGGS: how do I reproduce your splice problem
20:08 diakopter nm got it
20:08 FROGGS k
20:09 FROGGS $ nqp nqp-moar-cc.nqp -e 'my @a; nqp::splice(@a, @a, 0, 0) if 1;'
20:09 FROGGS cannot box a void register
20:10 diakopter whoa; nice golf
20:10 FROGGS had to compile first to make sure to dont paste crap
20:22 diakopter actually, there's an easy fix here.
20:23 FROGGS diakopter: is it the right one?
20:24 FROGGS diakopter: see https://github.com/MoarVM/MoarVM/commit/​9a2b79c6df0065584fc305e14f278207ddbfc27f
20:27 diakopter no
20:36 diakopter nqptesting
20:45 diakopter FROGGS: pushed
20:45 dalek MoarVM: 9b197c2 | diakopter++ | nqp-cc/src/QASTOperationsMAST.nqp:
20:45 dalek MoarVM: this actually works just fine.  in void context the code to generate the result is thrown away anyway....
20:45 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/9b197c22d9
20:45 colomon joined #moarvm
20:46 diakopter FROGGS: what was that blocking?
20:49 FROGGS diakopter: this: http://irclog.perlgeek.de/m​oarvm/2013-07-29#i_7383597
20:49 diakopter right, but..
20:49 Alpha64_ joined #moarvm
20:51 diakopter FROGGS: try with that patch
20:51 diakopter git pull
20:52 FROGGS currently doing
20:53 diakopter oh wait, that's not quite right
20:54 diakopter meh, it's close enough :)
20:54 FROGGS :o)
20:59 diakopter well, that made QASTOperationsMAST.nqp fully compile
20:59 FROGGS cool
20:59 dalek MoarVM: 18ce71d | diakopter++ | nqp-cc/tools/build/Makefile.in:
20:59 dalek MoarVM: cross-compile QASTOperationsMAST.nqp
20:59 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/18ce71d5fb
21:00 diakopter beat ya
21:02 diakopter what next
21:03 FROGGS make it selfhost nqp
21:03 FROGGS ;o)
21:03 diakopter yeah but what's next to cross-ompile
21:03 FROGGS no idea
21:04 FROGGS I dunno at all what is needed in order to do that
21:04 diakopter I guess MASTTesting and MASTCompiler
21:05 crab2313 joined #moarvm
21:06 diakopter I like how the *.moarvm are 1/3 the size of the corresponding *.pbc
21:06 FROGGS yeah
21:07 diakopter so.. how do I use this preprocessor to fudge things for moarvm
21:07 diakopter does #?if moarvm   work?
21:10 diakopter erm, how is this cross compiler working if it was able to cross-compile QASTCompilerMAST.nqp before it could do QASTOperationsMAST.nqp
21:26 arnsholt Magic!
21:26 diakopter arnsholt: I guess it just ignores missing symbols?
21:28 arnsholt Or that. I'm gonna go with magic, since it's past my bedtime =)
21:38 diakopter anyone know if jnthn wants moarvm integrated as a backend in the nqp source tree?
21:38 diakopter jnthn: do you know?
22:00 diakopter nm, I see how to do it for now..
22:01 jnthn diakopter: When I did it for JVM, I had a small script that mangled the QASTCompilerMAST equivalent a bit.
22:02 jnthn diakopter: You compile them a second time, is the trick.
22:02 jnthn diakopter: The nqp-jvm-prep repo probably has leftovers from how I did it there.
22:10 diakopter well I certainly won't be able to tackle the bootstrapping
22:10 diakopter but I think I can make the stage1
22:10 diakopter or is it stage0
22:11 diakopter heh, I wrote 0x1G
23:10 diakopter jnthn: how do I generate the initial NQP.pm?
23:11 diakopter gen_cat?
23:24 diakopter nm;
23:31 benabik joined #moarvm
23:33 jnthn yes, that
23:49 diakopter jnthn: seems it relies on nqpp6qregex
23:53 jnthn diakopter: correct, that needs cross-compiling, then nqp
23:53 jnthn Though it was a rather soft target in JVM port, since it doesn't demand all that much that won't already be in place.
23:57 benabik joined #moarvm
23:58 ggoebel joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary