Perl 6 - the future is here, just unevenly distributed

IRC log for #phasers, 2011-10-05

| Channels | #phasers index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:25 colomon joined #phasers
00:28 colomon_ joined #phasers
02:56 colomon joined #phasers
03:29 colomon joined #phasers
07:09 colomon joined #phasers
09:08 [Coke] joined #phasers
10:45 colomon joined #phasers
15:32 benabik joined #phasers
16:00 colomon joined #phasers
16:33 mberends joined #phasers
16:51 moritz what I did this week
16:51 moritz * lots of ticket triaging, closing, testing, fixing: down to < 750 open bugs, also thanks to [Coke]++
16:52 moritz * a bit of exception hacking: Exception now has a .backtrace emthod
16:52 moritz * applied MAIN/USAGE patch by japhb++
16:52 moritz * fixed tests that assume some dispatches will fail at run time, even though jnthn++'s smart optimizer can catch them at compile time
16:53 moritz * created a version of the mandelbrot-color benchmark that uses native ints to increment some indexes; noticably faster
16:53 moritz * implemented native num ops
16:53 moritz What I'm doing right now:
16:54 moritz * trying to clean up the mess I made by the test changes mentioned above
16:54 moritz What I plan to do:
16:54 moritz * more testing, more exception hacking, more assisting with the optimizer
16:54 moritz * test my tests on niecza
16:54 moritz EOR
17:01 [Coke] did: dug through RT queue, trying to triage the 1 year old+ tickets. down to <750 tickets, as moritz said, also < 700 if you discount testsneeded tests. fudging nom, niecza as time permits, keeping moritz honest.
17:01 [Coke] EOR
17:04 jnthn DONE
17:04 jnthn Rakudo:
17:04 jnthn * Added 'is required' trait for parameters
17:04 jnthn * Better private method error reporting
17:04 jnthn * Implement .+"$foo" style dispatches
17:04 jnthn * Fixed lcm/gcd infinite recursion
17:04 jnthn * Added error for declaring placeholders in the mainline
17:04 jnthn * Implemented $$foo, @$foo and %$foo
17:04 jnthn * Improved performance of string ranges a bit
17:04 jnthn * Improved performance of list assignment a bit
17:04 jnthn * Tracked down and fixed various issues in list handling, especially MapIter, that caused performance issues. for 1..1000000 { $i++ } now runs ~40 times faster.
17:04 jnthn * Various other small fixes
17:04 jnthn Optimizer:
17:04 jnthn * Added line numbers to CHECK failures, and grouped instances of the same error
17:04 jnthn * Implemented basic inlining of multis and onlies in some simple cases; enough for many core ops, though not restricted to them
17:04 jnthn * Initial native operators; moritz++ added some more
17:04 jnthn * CHECK-time detection of various calls to only subs that could never possibly work
17:04 jnthn * Extended type analysis to catch more "could never work" situations at CHECK time
17:04 jnthn TUIT JAR
17:05 jnthn * My $dayjob was relatively quiet in September, but this and the next couple of weeks will be decidedly busier. After that, it'll quieten down again. I'll have tuits in the next couple of weeks, but less of them than I have in recent weeks.
17:05 jnthn WORRIES
17:05 jnthn nom currently has significant missing chunks in regex support, and some nasty list bugs. These are areas that pmichaud++ has tended to take care of, and I've not been so involved with; the division of focus has tended to work well. Unfortunately, Pm's tuits seem in short supply, I've little idea when that will change, and bugging Pm for patches is the last thing we should be doing.
17:05 jnthn While I can find and often fix a wide range of issues in Rakudo in O(minutes), for me to sort out list bugs is more on the scale of O(hours). Me being able to do significant hacking on the regex engine is more like O(days).
17:05 jnthn Mostly, your options if you want to see progress on list/regex issues are...
17:05 jnthn * Be patient and wait until Pm has tuits again
17:05 jnthn * Be patient while I try to piece together an understanding of these less familiar bits of Rakudo, so I can make things better. Yes, I understand lazy lists OK and I went to my finite automata class like a good computer science student, but these things are still a bit tricky to get in to. :-)
17:05 jnthn * Dive in and try to fix them ;-)
17:05 jnthn I'm somewhat "feeling the pressure" on Rakudo nom at the moment; things really aren't where I hoped they'd be at this point. I don't feel there's an immediate risk of that translating to burnout, though I may use my couple of few-day teaching trips in the next couple of weeks as short escapes from Perl 6 work also.
17:05 jnthn So, don't worry if I disappear for a couple of days, and thanks for understanding.
17:05 jnthn EOR
17:06 jnthn moritz++ [Coke]++ # amazing ticket work!
17:06 moritz jnthn++
17:07 jnthn Also moritz++ for company in optimizer land :)
17:10 [Coke] jnthn: you're doing the hard work, I feel like I'm just mucking out the stable. ;)
17:11 [Coke] but you're welcome.
17:13 colomon Mucking out the stable, doesn't that make you Hercules?  ;)
17:13 * PerlJam reads jnthn's text above as "niecza ... now's your chance!"   ;-)
17:13 moritz PerlJam: niecza suffers from the same underlying problem: too low bus number in many important areas
17:14 PerlJam yeah
17:15 PerlJam As as no one goes the way of pugs ...
17:15 PerlJam s/As/As long/
17:16 mberends The answer to low bus numbers is within each of us. When choosing how to use our precious tuits, let bus-number-lowering be a high priority.
17:16 * moritz tries to keep the bus number on the test suite > 0.9
17:17 PerlJam mberends: part of the problem is that there is insufficient knowledge about the critical matters spread across our developers (IMHO anyway)
17:17 pmichaud my tuit supply is still... uncertain.
17:18 pmichaud I may have some time on Friday; weekend looks unlikely; early next week is possible.
17:18 masak joined #phasers
17:18 pmichaud is there a list of missing regex features?
17:18 jnthn pmichaud: The one that scares me most is protoregexes.
17:18 pmichaud okay, other than that one?
17:19 masak pmichaud: I could play around a bit during the evening and see what I come up with.
17:19 jnthn pmichaud: backtracking, before and <&foo>
17:19 pmichaud I thought <&foo> was done.
17:19 masak pmichaud: there was a worrying backtracking bug the other day.
17:19 masak pmichaud: basically, quantifiers don't backtrack in captures.
17:20 PerlJam does the spectests no longer hang on infinite now?
17:20 jnthn pmichaud: oh, so it is.
17:20 pmichaud backtracking into subrules/subcaptures might not be implemented yet, no.
17:20 pmichaud not hard for me to do, though.
17:20 jnthn pmichaud: oh...I got mixed up...I think it was actually passing args to regexes.
17:20 jnthn <foo(1,2,3)>
17:20 masak sounds familiar.
17:21 pmichaud did ng support defining regexes w/args?
17:21 pmichaud or are we talking about passing args to methods and invoking them from regexes?
17:21 jnthn pmichaud: We depend on it in Perl6::Grammar
17:22 jnthn pmichaud: I mean, it supported it to some degree, as we used it Perl6::Grammar.
17:22 jnthn Also there's a spectest I'm pretty sure ng ran.
17:22 pmichaud oh, that's still using the old engine, though, yes?
17:22 jnthn Yes.
17:22 pmichaud okay
17:22 jnthn Also, there's some list issues.
17:23 pmichaud anyway, args to regexes/methods isn't a long coding effort (for me)
17:23 jnthn Yes, the "(for me)" comes up quite a bit in these things ;)
17:23 pmichaud assign me tickets and I'll hack away at 'em  :)
17:23 masak pre-report: finally pushed my work so far on macros D1 to a branch in the rakudo repo. one known bug and a few possible improvements. will blog, but no idea when there might be time for that. no word yet on the grant application. EOR
17:23 jnthn pmichaud: for 1..$n { } was accidentally quadratic.
17:24 pmichaud "was"?
17:24 masak /o\
17:24 jnthn pmichaud: Yeah. When my C profiler told me that for 1..1000000 { } was spending 95% of it's time in memmove, I went hunting. :)
17:24 pmichaud seems odd that it'd be quadratic, after I worked on making sure that   for 1..20000 { }   was reasonably fast.
17:25 moritz pmichaud: I just assigned you two tickets (one lists, one regexes)
17:25 pmichaud moritz++
17:25 jnthn pmichaud: In eager mode, we reify 100000 or so items at a time
17:26 pmichaud yeah, because that's faster than reifying 1 item at a time 100000 times :)
17:26 jnthn pmichaud: MapIter was doing .munch one of those at a time. Each time, that does a shift on an RPA
17:26 jnthn A shift on an RPA means memmoving everything in the RPA one element to the left.
17:27 jnthn A memmove of 99999 items takes a while. The 99998 takes quite a while too. ;-)
17:27 pmichaud so, what was the fix?
17:27 pmichaud or is it not fixed yet?
17:27 jnthn pmichaud: Probably the wrong one :(
17:27 jnthn pmichaud: munch all we're going to work with in one go
17:27 jnthn pmichaud: And then just keep a position.
17:27 pmichaud hmmmm
17:28 TimToady .oO(shifts are very cheap in Perl 5 by keeping an extra pointer...)
17:28 Util Report: No Rakudo work this week; none expected next week due to $WORK.
17:28 pmichaud I'll have to look at that.  the problem I see there is that means that MapIter might not be as lazy as it needs to be
17:28 Util Q: Date of next nom-based Rakudo Star?
17:28 Util EOR
17:28 pmichaud i.e., it'll consume too much
17:28 jnthn pmichaud: If that's very wrong, feel free to undo it.
17:28 jnthn pmichaud: Yeah, I worried a bit too
17:28 jnthn pmichaud: tbh, the main time was spent on understanding the list implementation more and diagnosing what was wrong.
17:29 jnthn Both of which are beneficial even if my patch doesn't hold up.
17:30 moritz so, it's #phasers time
17:30 pmichaud I'll have to think of a way to speed up MapIter there.
17:31 moritz any reports from people who haven't reported yet?
17:31 colomon sure, I've got one queued up here and ready to go
17:31 pmichaud report:  tuit-starved here.  Some tuits possibly on Friday.  Weekend unlikely.  Next week probable.
17:31 moritz colomon: fire then
17:31 pmichaud EOR
17:31 colomon Pre-report: (no longer pre- because I didn't want to interrupt the conversation)
17:31 colomon * Mucked about with mberends++ GTK stuff in Niecza
17:31 colomon * Started implementing my "tune reminder" project
17:31 colomon * (Goal is to help remind me to practice the tunes I know)
17:31 colomon * Currently it probably works with Nom, but I've been developing under Niecza...
17:31 colomon * ...because I plan to use GTK for a user interface for it
17:31 colomon * In the process, discovered open :w was not implemented and got a crude version of it working
17:31 colomon Hope to:
17:31 colomon * Blog on end result of trig work
17:31 colomon * Blog on tune reminder
17:31 colomon * Get tune reminder prototype up and reminding
17:31 colomon EOR
17:32 PerlJam I have nothing to report other than I'm doing the compiler release this month. EOR
17:32 pmichaud PerlJam++
17:32 moritz colomon++
17:32 moritz ++PerlJam
17:32 pmichaud also tadzik++ and others++ for the release this past weekend
17:32 moritz so, questions
17:32 moritz Util queued one, I also have one
17:32 mberends q1q
17:33 Util ETA for nom-based Rakudo Star?
17:33 moritz "I hope soon"
17:33 moritz but currently many modules don't work on nom yet
17:33 pmichaud when there's a consensus that nom is ready for it
17:33 pmichaud (which I don't think we have yet)
17:33 Util Fair enough.
17:33 PerlJam pmichaud: and then back to timed releases?
17:33 pmichaud PerlJam: yes.
17:34 PerlJam bueno
17:34 Util EOQ
17:34 moritz I hope we can make the compiler releases from now on timed and on schedule again
17:34 pmichaud probably monthly to begin with again, as we make rapid updates to nom and modules; then back down to 3-month again
17:34 moritz +1
17:34 pmichaud I think compiler releases can return to normal schedule now
17:34 moritz ok, my question...
17:35 moritz how do we deal with exercising compile-time failures in roast?
17:35 moritz problem is that eval_dies_ok evals in a different lexical scope
17:35 moritz so symbols from the caller are not visible to it
17:35 moritz so I went with   nok eval('...'), 'description';
17:36 moritz which breaks niecza, which correctly does not catch exceptions in eval()
17:36 moritz so I tried   nok try { eval '...' }, 'description';
17:36 moritz and that breaks rakudo, because the return value from a failed eval() isn't a proper Perl 6 object
17:36 moritz I get segfaults and stuff
17:36 pmichaud fix rakudo's eval()
17:36 moritz pmichaud: I did, in a branch
17:37 moritz pmichaud: but it needs also try { } fixed
17:37 moritz and i don't know how to do that
17:37 moritz (branch eval-throws fwiw)
17:38 moritz is a fix for try { } returning the exception (or a Failure wrapping the exception) possible in the nearish future?
17:38 moritz (in case of an exception thrown, that is)
17:38 pmichaud that seems like it wouldn't be too difficult (more)
17:38 pmichaud try should already be wrapping any exception it receives.
17:38 moritz the thing in $! is properly wrapped
17:38 pmichaud I suspect the problem is try's return value.
17:38 moritz correct
17:39 pmichaud that ought to be a simple-ish fix
17:39 * tadzik 's here
17:39 moritz $ ./perl6 -e 'say try { die "foo" }'
17:39 moritz Null PMC access in find_method('gist')
17:39 moritz such a fix would be greatly appreciated
17:39 pmichaud right... I bet try isn't returning anything atm.
17:39 moritz EOQ
17:39 pmichaud thus the NPA
17:39 ashleydev joined #phasers
17:39 moritz mberends: you too had a question?
17:40 mberends my question: DateTime.now hangs on 32-bit nom, because of some data type problem.  We have to comment out 5 test files in spectest.data as a result.  This itch wants me to scratch it, but I'm worried that the problem may be pretty deep.
17:40 mberends what to do?
17:40 moritz the problem *is* pretty deep
17:40 pmichaud I think out-of-range ints should autopromote to num
17:40 moritz it's that we don't have enough prevision in a Rat to do the Num -> Rat conversion on 32 bit, because our ints aren't big enough
17:40 pmichaud oh, but our Rats are using 'int', yes?
17:41 moritz don't think so
17:41 pmichaud oh
17:41 moritz nope, untyped objects
17:41 pmichaud but the constructors are typed.
17:41 pmichaud hmm
17:41 moritz we *could* store Num's in there
17:41 pmichaud anyway, autopromote int->num is my suggestion.  there's even an opcode to simplify it now.
17:41 moritz but we'd need to provide alternative, unsafe constructors
17:43 jnthn sorry, had phone call
17:43 pmichaud another completely bizarre possibility is to have our Int types box num natives
17:43 jnthn Well, real solution is bigints :)
17:43 pmichaud I don't see "real solution" in O(weeks) though.
17:43 pmichaud if our Int types boxed num natives, we'd also have the ability to store Inf and NaN in Int
17:43 jnthn Not unless it becomes a priority for somebody, no.
17:44 jnthn pmichaud: Yeah but...they'd not really be Ints then ;) We could in theory try it but I dunno what the fallout would be :)
17:44 pmichaud I can't imagine that switching from int to num internally would pose that big a speed differential
17:44 colomon issue is precision
17:44 pmichaud why wouldn't they be "Int"s?   Int is something of an opaque type anyway
17:45 jnthn I'm not so much worried about speed as I am about the int/Int relationship.
17:45 moritz well, we'd have to switch all those _i opcodes to _n
17:45 jnthn And it's precision as colomon mentions.
17:45 colomon a 64-bit int has more precision than a num can hold
17:45 pmichaud a 64-bit integer has more precision than an int can hold, on a 32-bit system :-P
17:46 jnthn One thought on this.
17:46 jnthn We're going to need a set of ops for handling bigints anyway.
17:46 jnthn I could potentially implement those just with 64-bit ints (even on 32-bit platforms) for now.
17:46 cotto q1q
17:46 jnthn And fill it in with a real bigint implementation later.
17:47 pmichaud jnthn: pondering
17:47 tadzik q2q
17:47 pmichaud what ops would we need?  and do we still want to deal with 64-bit overflow somehow?
17:47 jnthn It's a kinda stepping stone to real bigint support. Which will have to come sooner or later.
17:47 tadzik (may overlap with cotto++'s)
17:48 cotto likely
17:48 jnthn pmichaud: The "usual set" :)
17:48 * jnthn guesses tadzik and cotto have comments relevant to bigint :)
17:48 jnthn If so, go ahead.
17:48 cotto nope
17:48 jnthn oh :)
17:48 tadzik (:
17:48 pmichaud jnthn: ooc, how would you change the Int representation?
17:48 tadzik . o O ( q3q )
17:49 jnthn pmichaud: Int is a P6opaque. It's the representatioin it "inlines" that would need to change.
17:49 jnthn *representation
17:49 pmichaud jnthn: thus I'm asking, how would that changes?
17:49 pmichaud currently we have
17:49 pmichaud Int.HOW.add_attribute(Int, BOOTSTRAPATTR.new(:name<$!value>, :type(int), :box_target(1)));
17:50 jnthn That :type would almost certainly change.
17:50 pmichaud would it become :type(bigint) ?
17:51 pmichaud and then we have some way of mapping "bigint" to (for now) 64-bit integer in C ?
17:51 sorear did: mostly hacked on /serialize.  extreme tuit shortage now, I'll backlog the meeting later
17:51 jnthn pmichaud: Something like that. Though the name "bigint" would really just be a vessel for an appropriate representation.
17:51 PerlJam for jnthn's proposal, seems like it would be :type(slightly-bigger-int)
17:51 pmichaud we could always do   int64
17:51 jnthn ooh :)
17:51 jnthn Yes, we could :)
17:52 mberends there is some almost ready bigint code in zavolaj/examples/biggishint.[ch]
17:52 jnthn mberends: ooh
17:52 jnthn pmichaud: Anyway, I'm thinking this really needs the same stuff I need to handle compact structs.
17:53 pmichaud jnthn: any idea of ETA?
17:53 jnthn pmichaud: Which needs me to work out the details of how that's going to look.
17:53 PerlJam jnthn: is that the same stuff needed for NCI (or FFI)?
17:53 pmichaud sounds like "not quick"
17:53 jnthn pmichaud: Depends what "quick" means :)
17:53 jnthn pmichaud: I suspect once I get a chance to think it through I'll work out how it should look quite quickly.
17:54 pmichaud jnthn: well, I agree there :)
17:54 jnthn pmichaud: It's an ideal task for a long train journey, and I've got four of those in the next 2 weeks. :)
17:54 pmichaud okay, so O(2 weeks) at least :)
17:54 jnthn So provided I have a functional brane on one of them... :)
17:54 jnthn Yeah. I'm not likely to get to much Perl 6 stuff in the next week or so :(
17:54 pmichaud okay -- if there's a ticket for the datetime 32-bit issue, assign it to me.
17:55 * masak doesn't dare point out to the uni prof that that's not how you use O()
17:55 pmichaud if there's not a ticket, create one and assign it to me
17:55 * masak submits rakudobug
17:55 pmichaud switching int to autopromote to num is fairly straightforward
17:55 moritz there is a ticket already
17:55 masak oh, ok.
17:55 masak standing down, then.
17:56 jnthn oh, q1q...the queue is getting quite long now :)
17:56 pmichaud I'll patch up Rat to accept integral Num values
17:56 pmichaud with a big XXX beside it so we know we need to come back to it
17:56 jnthn pmichaud: Ah, just patching up Rat worries me a lot less than patching up Int.
17:57 pmichaud anyway, it'll get us around this issue (and others) on 32-bit systems for a short while yet
17:57 pmichaud I'm done with this topic.
17:57 jnthn *nod*
17:57 moritz pmichaud: #100136 assigned to you
17:57 moritz ok, cotto had a question
17:58 cotto tadzik mentioned that Rakudo was having some issues because an installed parrot couldn't be relocated.  Can someone elaborate on that?
17:58 tadzik that's the reason Star cannot be packaged easily
17:58 masak seems to be mostly an issue for package managers.
17:58 tadzik it's also a reason why Rakudo on windows always has to be in C:\Rakudo
17:58 cotto that makes sense now
17:59 [Coke] tadzik: I own a ticket about that. I disbelieve.
17:59 [Coke] (has to be C:\ on windows)
17:59 [Coke] I agree you probably cannot build once, move anywhere, though.
17:59 jnthn A Windows user
17:59 jnthn He wants his Rakudo here
17:59 jnthn But Parrot says no
17:59 jnthn :)
17:59 masak jnthn++
18:00 pmichaud that's a long-standing Parrot issue -- parrot installs aren't relocatable
18:00 cotto It's worth looking at how to fix in Parrot.
18:00 jnthn .oO( if in doubt, explain with a haiku... )
18:00 [Coke] A terse bug report
18:00 tadzik Haikus are easy. But sometimes they don't make sense. Refrigerator
18:00 [Coke] with very little detail
18:00 moritz jnthn: it misses the reference to a time of your to be a proper Haiku :-)
18:00 [Coke] unlikely get fixed
18:01 TimToady Refrigerator obviously refers to the time of midnight snacks
18:01 masak tadzik: it's the only autopun haiku I know.
18:01 jnthn I dunno about other platforms, but Windows users tend to expect they can download a binary installer, and tell it where to install ot.
18:01 jnthn *to
18:02 moritz well, linux users like that too :-)
18:02 jnthn OK :)
18:02 masak Mac users don't care.
18:02 jnthn masak: A general statement? ;-)
18:02 masak or rather, they like everything in /Applications/
18:02 Util If Parrot fixes its relocation problems, will that completely resolve Rakudo's reloc issue?
18:03 tadzik I suppose so
18:03 moritz Util: it might not immediately, but it's a required step
18:03 jnthn Util: We install stuff in the "standard" Parrot places afaik.
18:03 jnthn So there's a decent chance it'll get us much of the way there.
18:03 TimToady absolute paths can sneak in at any level
18:03 tadzik the problem with Star is "we can't build modules before we install Rakudo"
18:04 moritz cotto: question answered?
18:04 cotto moritz: yup
18:04 cotto thanks
18:04 moritz then on to tadzik's questions
18:04 tadzik first one is already answered
18:04 tadzik I'll now stick in my short report:
18:05 TimToady wow, that's, like, 0 lines :)
18:05 cotto actually, would an environment variable be enough to make windows users happy, a la PERL5LIB
18:05 tadzik done: .mobi version of Using Perl 6 book, required a few fixes in the result of 'make html', not sure if it's our fault, or PseudoPod's. Got some feedback, will probably give it more love
18:06 tadzik also, some minor patches and some ticket mangling, nothing worth mentioning I suppose
18:07 tadzik now, to the question: release went flawlessly, but I received some negative feedback from people packaging Rakudo for Linux distros. The problem is that Rakudo 'Riga' NQP_REVISION was newer than NQP 2011.09 release
18:07 tadzik we give much care about releasing Rakudo, but I suppose we have to give at least some of that love to NQP releases as well
18:07 TimToady cotto: env vars can present security issues
18:07 moritz tadzik: agreed
18:07 tadzik I can find the relevant irclog
18:08 pmichaud should've had a 2011.09.1 release, I suspect.
18:08 cotto I'll see what we can come up with in Parrot and check back when we have something for Rakudo to play with.
18:08 tadzik afaik, but not sure, what stopped the maintainer from just using --gen-all-the-things was the relocatibility issues
18:08 moritz ... or the nqp tarball I uploaded to perl6/nqp/downloads
18:08 moritz ... after the issue
18:09 PerlJam tadzik: are linux dists packaging nqp separately?
18:09 tadzik PerlJam: it appears so, yes
18:09 pmichaud rakudo (nom) releases should always target a nom release
18:10 pmichaud if that means we have to issue a separate nom release, then do that :-)
18:10 tadzik not sure what was the blocker for using just --gen-nqp with previously installed Parrot
18:10 moritz tadzik: distro policy
18:10 tadzik pmichaud: s/nom/nqp/?
18:10 tadzik oh, right
18:10 tadzik makes sense
18:10 pmichaud s:2nd/nom/nqp
18:10 pmichaud I'll rewrite.
18:11 pmichaud nom releases should always target a nqp release.  If that means we have to issue a separate nqp release, then do that.
18:11 tadzik makes sense
18:11 pmichaud I suspect the release guide needs to be updated with that detail.
18:11 PerlJam Um ... when do NQP releases happen?
18:11 moritz pmichaud: aye
18:12 pmichaud PerlJam: between the parrot and rakudo release :)
18:12 PerlJam (oddly, I'd never considered "nqp releases" until this conversation)
18:12 pmichaud we've mused about nqp releases before.  we've also thought about extending the rakudo release to be "up to 4 days after parrot release" to give time for nqp stuff to happen.
18:14 PerlJam how about just make the rakudo release the following week?
18:14 pmichaud anyway, I'd say that nqp releases happen just prior to or in conjunction with rakudo releases
18:14 pmichaud we can separate the timing if we decide we need to do that
18:14 pmichaud (separate the timing later)
18:15 pmichaud I'm fine with saying "rakudo release following week", too.  given that parrot has talked about making changes in their release/deprecation cycles, it makes sense for us to rethink the rakudo/star/nom cycles as well
18:16 pmichaud s/nom/nqp/  # grrrr
18:16 pmichaud tadzik: I know it's hindsight, but would a separate 2011.09.1 nqp release have helped the situation?
18:16 tadzik pmichaud: yep
18:17 pmichaud then that should be our policy going forward, I think.
18:17 tadzik pmichaud: moritz++ uploaded another tarball and it fixed the issues
18:17 tadzik that should be the Justin Case policy for sure
18:17 tadzik but I'd rather have no problems than having to fix problems
18:17 pmichaud well, I think we have to recognize that from time to time we'll want to make .1 and .2 releases
18:17 pmichaud things happen.
18:18 tadzik sure
18:19 sorear moritz: we changed the spec there, eval is no longer supposed to catch exceptions
18:19 pmichaud okay, I'll make sure to check that the release guide gets updated.  (If someone wants to draft updates for me to check, that'd be awesome)
18:19 moritz sorear: I know
18:19 pmichaud (thus I said  "fix eval()"  :-)
18:19 moritz sorear: I'm sorry for having busted so many spectests for niecza
18:20 moritz sorear: I'll fix it, but it might need some time
18:20 pmichaud forgiveness > permission, I think  :)
18:20 sorear and I'm now backlogged on #phasers.
18:20 sorear if anyone has a question for me to answer in the next 5 minutes...
18:21 pmichaud same here -- I will have to depart shortly
18:21 moritz I think jnthn++ had one more question
18:21 jnthn Yes
18:21 jnthn Optimizer branch. I want to merge it at some point BUT would prefer to avoid chaos. :)
18:22 jnthn I'm pondering something like this.
18:22 jnthn There's a --optimize option. It defaults to "1". We start of classifying any non-very-comfy optimization as "2". We build the setting with --optimize=2 because we have good test coverage of the setting.
18:23 jnthn That way we immediately get the CHECK time analysis.
18:23 masak sorear: how far along are you with the /serialization branch?
18:23 pmichaud jnthn: I have no objection to that.
18:23 masak sorear: and what will it mean, exactly, when it lands?
18:23 moritz jnthn: +1
18:23 jnthn Over time, optimizations get migrated into "normally on" as we're comfortable with them. --optimize=2 always gives you the "most benefits, most risk"
18:24 jnthn And --optimize=0 totally disables it, so we can know if the optimizer is guilty of causing issues :)
18:24 pmichaud I might suggest  --optimize=1|2|3
18:24 pmichaud 3 is the "most benefits, most risk"
18:24 pmichaud 2 is the default, the ones we're comfortable with
18:25 pmichaud 1 is "very stable optimizations" -- i.e., those that are known to be good based on experience
18:25 jnthn pmichaud: That also works for me.
18:25 pmichaud i.e., in case we make a mistake moving one from 3 to 2, the end-user still has an option for --optimize=1
18:25 jnthn *nod*
18:25 benabik And -Ofun gives which optimizations?  ;-)
18:25 pmichaud (without losing all optimizations)
18:25 jnthn pmichaud: Good point. Thanks, will do that.
18:26 jnthn I need to clear up a few things, but it's potentially landable within a week.
18:26 pmichaud +1
18:27 jnthn It does optimize the setting in various places.
18:27 jnthn Note that we have an incentive to (carefully) add a bit more type info in the setting after this lands.
18:27 jnthn Just need to be careful not to over-constrain things.
18:28 pmichaud or to supply suitable multi candidates :)
18:28 moritz now that we have ops for native ints in the setting, it might be a big win to change some of the loop variables to int
18:28 jnthn If there's $a + $b and it knows both are Int, for example, then it will inline the addition, and not have to make a dispatch at all.
18:28 jnthn moritz: Yes, it is
18:28 jnthn tadzik++ posted some numbers on this not so long back.
18:28 jnthn It's an order of magnitude faster.
18:29 moritz it shove several seconds off of the mandelbrot benchmark
18:29 jnthn Yeah, the MapIter change also did...wonder what happens if we combine that :)
18:29 jnthn *those
18:29 moritz and since ops of natives are often inlined, we can make the Complex stuff less ugly
18:29 moritz jnthn: that loop didn't use MapIter at all
18:30 jnthn moritz: mandelbrot uses MapIter a lot though.
18:30 jnthn moritz: I meant, that change plus the optimizer together might bring us down under 40s or something :)
18:33 moritz I think we have reached EOP (where P = Phasers)
18:33 tadzik may be
18:33 jnthn pmichaud: oh, just found these: https://gist.github.com/1265262
18:33 jnthn pmichaud: Numbers before/after my MapIter change.
18:34 pmichaud jnthn: okay, I'll definitely think about how to fix up MapIter then
18:34 pmichaud we definitely need it to be able to preserve laziness, though.
18:35 jnthn pmichaud: My reasoning was "if we're already eager..." - but I can see how that may not be enough.
18:35 pmichaud ...how do we know that we're eager, though?
18:35 pmichaud and how do we know that the body doesn't abort in the middle?
18:35 pmichaud (thus losing the munched-but-not-processed items from the list)
18:36 jnthn pmichaud: I took care to make sure we don't lose them in the patch.
18:36 jnthn pmichaud: But we do certainly over-munch in that case.
18:36 pmichaud the point being that they're no longer on the list, I think.
18:36 pmichaud (source list)
18:37 pmichaud that might actually be okay; I have to think a bit about the .map setup.
18:37 jnthn pmichaud: I stick 'em back into the source list.
18:37 pmichaud even with an exception?
18:37 pmichaud hmmmm
18:37 jnthn Look for "uneaten_saved" in the patch :)
18:37 jnthn :)
18:38 * masak .oO( zombies )
18:38 jnthn pmichaud: I'm not attached to the patch. I am kinda attached to the speed improvement though ;)
18:38 pmichaud I agree we need the speed improvement.
18:38 pmichaud but correct is still as important as speed :)
18:38 jnthn I agree we need the right semantics. :)
18:39 jnthn I mostly hoped I could get away with the things my patch did, and the spectest didn't tell me I couldn't. :)
18:39 pmichaud well, it's one of those things I suspect is underspecced
18:39 pmichaud it may very well be that MapIter ends up needing a very close relationship with List
18:39 jnthn If it's allowed to go peeking, it may have an easier time being efficient, yes.
18:40 pmichaud for the version I put together, I tried to keep a little distance in case we're MapIter'ing something that isn't a List.
18:40 jnthn *nod*
18:40 jnthn Oh, I also noticed that we seem to end up wrapping things in two levels of List/Parcel when doing a for loop
18:40 pmichaud often that's true, because we need to .flat
18:41 pmichaud so one of the List's is the part that does flattening
18:41 jnthn OK, I was probably missing something; it looked like one level too many.
18:41 jnthn But I'm still figgering it all out :)
18:41 pmichaud It may be one level too many, yes.
18:41 pmichaud but we have to ensure that things are flattened properly (more)
18:42 jnthn I quite like it as I'm working through it; it's just a little mind-bending at times ;)
18:42 pmichaud also, believe it or not, MapIter comes from before when .map was specced to always .flat
18:42 jnthn oh :)
18:42 pmichaud now that .map always flattens, we may be able to optimize a bit further than I was dealing with at first.
18:42 pmichaud (instead of wrapping in a flattening list)
18:43 pmichaud it's very possible that .map simply becomes a method on List that is really smart about iterating.
18:43 pmichaud (and that MapIter may go away entirely)
18:43 masak \o/
18:44 pmichaud i.e., perhaps ListIter can be given the mapping smarts.
18:44 jnthn Well, if MapIter were going to have such a cosy relationship with List anyway, that could make a lot of sense.
18:44 pmichaud right
18:44 pmichaud otoh, MapIter tends to want to consume its list, while ListIter doesn't
18:44 jnthn yeah, it munches.
18:45 pmichaud I'll think about it a bit more
18:45 jnthn OK :)
18:45 pmichaud I agree the O(n^2) memmoves is a problem that we have to avoid.
18:45 pmichaud (did I get that O(...) correct this time, masak++?  ;-)
18:45 jnthn Yes. The C profile was...incredible.
18:46 pmichaud we really do want to be shifting things somewhere
18:46 pmichaud otherwise we end up with huge map memory footprints
18:47 pmichaud i.e., for   1..1000000 { }     we'd really prefer not to keep the 1-million element list around for the whole loop, or to construct it all in the first place
18:47 jnthn Yes, but we could throw build and throw it away in 10000 element increments.
18:47 pmichaud right
18:47 jnthn s:1st/throw//
18:47 pmichaud that's what the old implementation did -- it tended to limit itself to smaller increments
18:49 jnthn I guess "how big" is a matter of tuning.
18:49 jnthn Or we find a way to shift cheaply :)
18:49 pmichaud or munch cheaply
18:49 jnthn Which may mean moving away from RPAs.
18:49 pmichaud we can keep RPAs if munch becomes cheap.
18:49 jnthn Or keeping an RPA and an index.
18:50 pmichaud I think making munch cheap is the bigger win.
18:50 pmichaud (which likely involves an RPA and an index, yes.)
18:50 jnthn Yeah, munch is where we were being killed.
18:51 jnthn well, our performance was :)
18:51 pmichaud then we can leave MapIter alone-ish.
18:51 jnthn That said, https://github.com/rakudo/rakudo/commit/3bc839c00ee03491a7bf8c74923198779a8cc2bb also helped a lot :)
18:51 pmichaud heh
18:52 jnthn Actually I still need to go back and fix that..all the pushing leads to a load of time in realloc.
18:52 pmichaud I think shiftpush was written before I "fixed" RPA's splice to not be quadratic
18:52 jnthn RPA's splice used to be...omg!
18:52 pmichaud RPA's splice was quadratic when I started working on lists...
18:52 jnthn o.O
18:52 pmichaud just a sec, I'll find the patch :)
18:52 jnthn Please say splice wasn't implemented in terms of shift and push... :)
18:53 pmichaud https://github.com/parrot/parrot/commit/f12d1201585132dd8eb36907c0fc786f31683aa9
18:53 pmichaud it was implemented in terms of keyed_int
18:54 jnthn Still, youch!
18:54 pmichaud yeah
18:54 pmichaud so if you were inserting at the beginning of a list, you ended up doing O(n) vtable calls via keyed_int.
18:54 pmichaud instead of a memmove
18:54 jnthn ugh
18:55 pmichaud so shiftpush was written because splice was not yet fixed in whatever version of Parrot I was working with :)
18:55 pmichaud (also because it's quite convenient)
18:55 jnthn It is also quite convenient :)
18:55 jnthn :P
18:56 pmichaud okay, I'll work on fixing munch.
18:56 jnthn OK, great. :)
18:56 pmichaud I suspect that if we simply keep a pointer, and the splice whenever the pointer reaches a certain point or we're doing more reification, that'd be enough.
18:56 pmichaud s/the/then/
18:57 jnthn Sounds reasonable.
18:57 jnthn Oh, one of the list bugs seems to happen when you end up iterating a list that's currently being reified.
18:57 jnthn $!rest then ends up as Mu rather than an RPA
18:57 pmichaud yeah, there's an incorrect initialization or reset there somewhere
18:58 pmichaud assign me a ticket :)
18:58 jnthn Yeah, I looked a little yesterday, but to no avail
18:58 jnthn OK
18:58 pmichaud okay, gotta run here
18:58 pmichaud be back... sometime
18:59 jnthn ok, take care o/
18:59 pmichaud (probably fri morning)
18:59 moritz \o
19:18 masak left #phasers
21:19 benabik left #phasers
22:29 [Coke] joined #phasers

| Channels | #phasers index | Today | | Search | Google Search | Plain-Text | summary