Perl 6 - the future is here, just unevenly distributed

IRC log for #parrotsketch, 2009-11-24

| Channels | #parrotsketch index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
04:52 davidfetter joined #parrotsketch
04:54 rin1024 joined #parrotsketch
05:28 davidfetter left #parrotsketch
12:27 bluescreen joined #parrotsketch
12:56 davidfetter joined #parrotsketch
16:48 japhb DONE:
16:48 japhb * Keep up with pmichaud's NQP-rx improvements (and push for more :-)
16:48 japhb * Keep up (mostly) with fperrad's bug reports and metadata updates
16:48 japhb * Various Glue.pir improvements
16:48 japhb * A few more tests
16:48 japhb WIP:
16:48 japhb * Big refactoring of main "brains" to OO classes
16:48 japhb * Document refactored code
16:49 japhb * About 50% done with planned refactors before next feature push
16:49 japhb MAD PROPZ:
16:49 japhb * fperrad++   # Copious metadata updates, suggestions, and bug reports
16:49 japhb * pmichaud++  # NQP-rx feature-o-rama
16:49 japhb * dukeleto++  # Initial pass at qx() tests
16:49 japhb NEXT UP:
16:49 japhb * More of the big refactoring.  I'm currently thinking the depsolver next.
16:49 japhb BLOCKERS:
16:49 japhb * I think I've got enough unblocked tasks to use all my tuits this week.
16:49 japhb * By next week IWBNI NQP-rx had full hash constructor syntax; there are several subprojects that are blocked on this feature.
16:49 japhb EOR
16:55 Tene I seem to be recovering enough from being sick to do some work again.  Poked a little bit at rakudo-ng.  KTHXBAI
16:56 Util joined #parrotsketch
16:56 * Util has nothing to report, will miss the meeting today, and has negative tuits until Monday.
17:26 NotFound joined #parrotsketch
17:51 pmichaud joined #parrotsketch
18:04 darbelo joined #parrotsketch
18:09 cotto_work # Done:
18:09 cotto_work * Continued converting pprof2cg to nqp.
18:09 cotto_work * npq syntax is nicer than perl syntax.  For string processing and efficiency, not so much.
18:09 cotto_work * It's still incomplete but it's way slower than the perl version (9:00 vs 1:30 for same input)
18:09 cotto_work * Note to self: there may be some sort of profiling tool out there that can be used to narrow the gap.
18:09 cotto_work # Will do:
18:09 cotto_work * Eat turkey.
18:09 cotto_work * Finish pprog2cg port and be even more disappointed about how slow it is.
18:09 cotto_work * Start profiling that sucker.
18:10 cotto_work # Blockers:
18:10 cotto_work * Turkey.
18:10 cotto_work eor
18:12 whiteknight joined #parrotsketch
18:13 NotFound What I did:
18:13 NotFound - parot
18:13 NotFound * A few fixes and test coverage improvements.
18:13 NotFound * Improve X11 library search in Xlib example module.
18:13 NotFound - Winxed
18:13 NotFound * Big refactor: the compiler written in C++ is now a 'stage 0' compiler,
18:13 NotFound and the compler driver is now a winxed program, able to be compiled
18:13 NotFound to .pbc and pbc_to_exe'd. This clears the path towards a self-hosting
18:13 NotFound compiler.
18:13 NotFound * Backward incompatible syntax change: now the new operators uses
18:13 NotFound parentheses for his arguments, and can't be omitted for winxed classes.
18:13 NotFound * Lots of improvements in parser example, the future 'stage 1' compiler.
18:13 NotFound Now is able to generate code for some simple programs, and allow
18:13 NotFound float (N registers) variables, which the stage 0 doesn't.
18:13 NotFound What I will do:
18:13 NotFound * No fixed plan.
18:13 NotFound EOR
18:15 whiteknight WHAT I DID:
18:15 whiteknight (pla) some miscellaneous tweaks and improvements to the test suite
18:15 whiteknight (matrixy) pass more tests, fixes from the messy merge.
18:15 whiteknight (matrixy) Improved and reorganized test suite
18:15 whiteknight (parrot) started work on a new "mystery project". Details to come eventually.
18:15 whiteknight WHAT I WILL DO:
18:15 whiteknight * More cleanup work on Matrixy and Parrot-Linear-Algebra
18:16 whiteknight * I have some ideas for improvements to nqpTAP that I want to try
18:16 whiteknight * Work on the :call_sig stuff more
18:16 whiteknight * Work on my mystery project
18:16 whiteknight WHAT I AM BLOCKING ON:
18:16 whiteknight * Ongoing computer problems
18:16 whiteknight EOR
18:21 mikehh joined #parrotsketch
18:23 kj joined #parrotsketch
18:26 mikehh What I did in the last week:
18:26 mikehh * building and testing parrot on amd64/i386, with gcc/g++, with and without --optimize
18:26 mikehh * fixing tests
18:26 mikehh * checking skipped test for passes or able to TODO
18:26 mikehh What I intend to do in the next week:
18:26 mikehh * testing and fixing
18:26 mikehh * continue working on checking skipped tests
18:26 mikehh .eor
18:32 Coke joined #parrotsketch
18:34 Tene MEETING TIME
18:34 cotto_work HELLO
18:34 NotFound hola
18:34 mikehh hi there
18:34 Tene It's many-people-are-absent day today, so... anyone have questions or reports to give?
18:35 Tene Turkey to share?
18:35 Coke Belated report: pmichaud started on partcl-nqp; hacking on that.
18:36 whiteknight hello
18:36 cotto_work This could be the shortest #ps ever.
18:37 Tene I seem to be alive again, and I've got a lot of spare time during the long weekend, so if anyone has any tasks they'd like me to work on, please harass me.
18:38 whiteknight Who's driving this train?
18:38 Tene Me!
18:38 whiteknight go for it
18:39 Tene Well, I did.  Everyone has reported, nobody has questions...
18:39 Tene Am I missing anything?
18:39 Coke priorities for the next week? I vote "turkey"
18:39 darbelo Roadmap review?
18:40 whiteknight more optimizing
18:40 Tene Oh, right, there actual plans.  Man, I'm an incompetent leader today.
18:40 * Tene reads topic link
18:40 cotto_work Could you re-set that?  It's blank for me.
18:40 Topic for #parrotsketchis now Vision for 2.0: Production Users | Priority for 1.8: Testing Sprint | Priority for this week: close RT tickets | https://trac.parrot.org/parrot/w​iki/ProposedParrotsketchProtocol | Note: This channel is only for our Tuesday status meetings; you probably want #parrot instead.
18:41 cotto_work Thanks
18:41 Tene okay, there's a bunch of HLL items on the roadmap... there's been no progress on those recently.
18:41 NotFound Looks like "close RT tikcets" is no longer a priority ;)
18:41 Coke (thanks to everyone who worked on that!)
18:41 Tene I didn't see japhb around... anyone else want to talk about seed libraries?
18:42 Tene parrot_debugger?
18:43 Tene Priorities?
18:43 Tene We've got one vote for turkey, and one vote for optimization.
18:44 NotFound I had plans to take some looks on parrot_debugger after pcc refactor, but Winxed has distracted me.
18:44 mikehh kid51++ removed a load of RT references in parrot
18:45 cotto_work I like optimization.
18:45 whiteknight optimization++
18:45 cotto_work -Oturkey
18:45 Tene Oh, right, 1.8 was released, so we're looking at 1.9 milestones too.
18:45 Tene Who's working on bytecode tools?
18:45 mikehh but don't forget the big caveat
18:46 darbelo A too optimized turkey can outrun you?
18:46 Tene How did the testing sprint mentioned in the topic go?  Do we have better tests?
18:47 mikehh Util I think
18:48 Tene There's a documentation sprint on the roadmap... is there a list of underdocumented places?
18:48 mikehh a couple but there did not seem to be that much enthuasiam
18:48 whiteknight a list of documents needing improvement would be good
18:48 allison the tasklist is coming together on https://trac.parrot.org/parrot/wiki/DocsTasklist
18:48 whiteknight (don't have one to my knowledge)
18:48 mikehh darbelo: how about get it workin' first
18:49 * Tene turns the meeting over to allison.
18:49 NotFound A list of documents that doesn't need improvement will be a lot shorter.
18:50 allison specific tasks are easier to pick off than "improve X file"
18:50 allison that is, pretty much any documentation file could be improved in some way
18:50 mikehh a prioritizeed list perhaps
18:50 whiteknight test all example cod
18:50 whiteknight code
18:50 Tene Coke said that the AST documentation was sorely lacking.
18:51 Tene I'm not quite sure what's missing there.
18:51 Coke (all example code) it's already mostly tested.
18:51 allison start with a goal, for 2.0, "improve user documentation"
18:51 mikehh some examples don't work - need fixin'
18:51 allison we're getting a lot of reports lately of trouble with the "getting started" tutorials
18:51 Coke Tene: something for people who are starting from scratch.
18:51 Coke allison: exactly. =-)
18:51 allison right, examples that don't work
18:52 whiteknight allison: yes, and the make_language_shell.pl utility
18:52 NotFound Someone is working on, or using, SDL?
18:52 allison whiteknight: yup
18:52 whiteknight Notfound: SDL?
18:53 NotFound whiteknight: runtime/parrot/library/SDL
18:53 Tene Is there an index of the various "getting started" tutorials scattered around the internet?
18:53 allison Tene: no, and that would be a good place to start
18:54 allison Tene: pulling them into one place inside the Parrot repository
18:54 allison Tene: (at least where we can)
18:54 Tene That's always been one of my worries about docs out on wikis, etc.
18:54 NotFound We can list only the ones we know someone is trying to keep up to date.
18:54 Tene mk_language_shell is rather out of date, I believe, and really needs to be fixed.
18:55 allison Tene: supposedly it can be replaced by fperrad's tool, but for backward compatibility we still need mk_language_shell to work for now
18:55 Coke and once it's fixed, it probably needs to be updatd to nqp-rx.
18:55 allison do we have a ticket for that?
18:56 pmichaud (back)
18:57 allison okay, so need a ticket for mk_language_shell
18:57 Tene I'd love it if we had a tool that generates configs that support 'make install' and includes plumage metadata.
18:58 NotFound Also a tool that checks plumage metadata will be helpful.
18:58 allison Tene: that'd be a good extension to the language shell generators
18:58 pmichaud my intent is to create a new language shell generator in nqp.  Not sure when it will happen, but should be before the 1.9 release.
18:59 pmichaud (since the procedure for creating a language in nqp-rx is somewhat different from the previous language shell generators)
19:00 allison we may be looking at a replacement then.
19:00 whiteknight is that covered under the deprecation policy?
19:00 allison depends on if you consider utility scripts part of the public interface
19:01 pmichaud the nqp-rx tool will likely live (initially) in ext/nqp-rx
19:01 pmichaud it can of course be moved somewhere else :-)
19:01 allison for 2.0 it's fine if mk_language_shell only works on the old NQP, but it should work
19:01 pmichaud q1q
19:02 NotFound I think that programs intended only for interactive usage must not be under deprecation policy.
19:03 allison I'd agree they aren't part of the official deprecation policy
19:03 allison but, they also shouldn't lie around broken
19:03 allison so, either work or be removed
19:03 NotFound Provided that no one will make things like adding an option called --help that does an rm -rf, of course ;)
19:03 allison :)
19:04 allison on roadmap review: what are the chances of calling the hll-interop done by 2.0?
19:04 pmichaud allison: good
19:04 pmichaud I've made a lot of progress on it this week
19:04 allison excellent
19:05 allison prune c data structures we've made good progress, and I'd be willing to drop it off "roadmap" and down into "ongoing cleanup"
19:05 Tene I plan to do a lot of work on hll interop as well.
19:05 pmichaud also, hll-interop documentation is a deliverable for my hague grant :)
19:05 allison documentation sprint has been going, I've seen good progress on it
19:05 allison pmichaud: also good
19:06 allison the debugger documentation looks like a good addition to the docs tasklist
19:07 allison ENOdukeleto for an update on his task
19:07 allison work on that task
19:07 allison okay, questions
19:07 allison pmichaud?
19:09 pmichaud not a question so much as a pointer to a very interesting item
19:09 pmichaud last week chromatic did some analysis of compilation using pct
19:09 pmichaud the thread is at
19:09 pmichaud http://irclog.perlgeek.de/p​arrot/2009-11-20#i_1751189  # for those who wish to go back and read
19:09 pmichaud but the line I wish to point out
19:09 pmichaud chromatic 88.81% of the execution time is *marking* Capture.
19:10 pmichaud the takeaway from the discussion was that GC is killing us
19:10 pmichaud also, it's not the creation of short-lived objects that is the time eater
19:10 pmichaud it's the long-lived ones that hurt
19:10 pmichaud in particular, this run resulted in 3.2 million PMCs being created due to subroutine/method calls
19:11 allison we do have an intensely primitive GC at the moment
19:11 japhb bak ... I see some plumage highlights in the backlog, let me know if there are any outstanding questions
19:11 pmichaud but those 3.2 million PMCs only accounted for about 30K marks
19:11 allison sounds like the GC refactor just got bumped up in priority
19:11 pmichaud the 24K Context PMCs (the point of the compilation) resulted in 500K marks
19:11 dukeleto joined #parrotsketch
19:11 * dukeleto is late but here. for a bit.
19:12 pmichaud in short, we at one time were concerned that switching contexts to PMCs would hurt GC pressure (more)
19:12 NotFound BTW I think that currently we have a problem instantiating PMC.
19:12 pmichaud that appears to not be the case -- they're short-lived, and the GC pressure only comes when we do a gc run, and then what matters is the number of live contexts, not the number of contexts that had been created
19:12 pmichaud but gc runs on long-lived data structures are really problematic
19:12 allison dukeleto: just wanted a quick update on parrot_debugger documentation, what's done and still needs to be done. You can add it to https://trac.parrot.org/parrot/wiki/DocsTasklist if that's better
19:12 NotFound The PMC are not instnatiated directly, a pmcproxy is created in the cases I checked.
19:13 pmichaud (and 24K PMCs is not really a lot of PMCs in the overall scheme of things.)
19:13 allison the pmcproxies do need to go
19:13 NotFound That may mean millions of uneeded proxies for simple tasks.
19:13 allison NotFound: actually, only one proxy is created for each PMC type
19:14 allison NotFound: so, no more than 100 total
19:14 pmichaud (at one time there was a bug that caused multiple proxies to be created for various PMC types... I cleared those up late spring 2009)
19:14 allison but, they're only actually used during subclassing operations, so, could be left off the rest of the time
19:14 NotFound allison: one proxy for type, one proxy instance for object.
19:15 pmichaud NotFound++ is correct that every instance of a class also gets a proxy instance
19:15 pmichaud (iiuc)
19:15 NotFound No, I talk about plain and simple parrot PMC
19:15 allison should be one proxy for Class and one for Object
19:15 pmichaud oh, that would be wrong then.
19:15 NotFound Very wrong.
19:16 NotFound I looked at http://tapir2.ro.vutbr.cz/cover/cover-resu​lts/42796/c_cover/src-pmc-boolean-pmc.html
19:17 pmichaud anyway, back to my "question" (which got hijacked) -- I just wanted to point out that chromatic's analysis indicates that 4/5ths of compiler execution is spent in GC.
19:17 NotFound The init_pmc has no coverage on value null, but the test must exercise it.
19:17 NotFound It doesn't, because is instantiated via pmcproxy
19:17 pmichaud I'm hoping we can dedicate some effort to improve that.
19:18 pmichaud (or at least verify that the 88% figure is erroneous somehow)
19:18 pmichaud eoq
19:18 NotFound pmichaud: well, if we are creating a lot of unneeded pmc, stop creating them will mean a lot less pressure on the gc.
19:18 pmichaud NotFound: you missed my point.
19:19 allison NotFound: the GC is slow, there's no question about that
19:19 pmichaud the point is not how many pmcs are created
19:19 allison and, yes this is an important priority
19:19 pmichaud the point is that 24K pmcs were sufficient to completely bog down processing, and 24K is not a large number.
19:20 allison it's helpful to have more details on where the problem in PCT is
19:20 pmichaud it's not PCT
19:20 pmichaud I guarantee that PCT isn't the issue.
19:21 pmichaud the pmcs in question are just the parse, PAST, and POST nodes
19:21 allison well, that's what I mean, "what is it that causes the compiler to be slow?" is a question we've been asking for a while now
19:21 pmichaud ...unless you're suggesting we shouldn't be using trees :)
19:21 whiteknight it's not just GC being slow. Specifically it's GC mark
19:22 allison if it's GC, at least we have something concrete to fix
19:22 whiteknight that's the part that's hard to avoid
19:22 pmichaud if chromatic++'s analysis is correct, what is causing the compiler to be slow is the fact that a long-lived data structure (the parse and ast trees) causes gc to be slow
19:22 whiteknight we can be judicious with how much we sweep, but we need to acurately mark things
19:22 allison this argues for generational GC
19:22 whiteknight my conclusion exactly
19:22 allison that is, we shouldn't be sweeping those long-lived PMCs so often
19:23 allison certainly not on every mark run
19:23 pmichaud and although we can perhaps do things to improve the data structure a bit, the point is that trees with a total of 24K nodes isn't all that big for any sort of program
19:23 whiteknight It's not sweeping, it's marking that's the problem
19:23 whiteknight according to chromatic (and I need to verify) ~80% of execution time was spent in Capture.mark
19:23 pmichaud yes, verification of the result would also be helpful.
19:24 whiteknight if we marked the Capture PMCs less, we would also have to mark fewer of their children. A lot of stable things could be grandfathered in
19:24 pmichaud I have trouble believing it, myself, but I trust chromatic more than my own intuition on this point :)
19:24 whiteknight increasing pool size yet again might reduce the number of mark+sweep phases we need to trigger for basic operation
19:25 pmichaud fwiw, this particular profile showed 92 GC runs
19:25 pmichaud anyway, I'll leave it at that and hope it prompts some conversation/design work :)
19:25 allison sounds like something we can get excited about
19:26 pmichaud speeding up compilation by a factor of 4 would get me excited, yes :)
19:26 pmichaud heck, even a factor of 2 would be nice :)
19:26 allison indeed :)
19:26 allison any other questions?
19:27 allison alright, for those in turkey territory, enjoy the festivities, and the rest of us, have a good week!
19:27 japhb oh, for anyone still wondering about the plumage + mk_language_shell stuff,
19:28 japhb fperrad has been doing a lot of work on autogenerating plumage infor and setting up project shells.
19:28 Tene japhb: link about that?
19:28 japhb Also, I have it as a task item for Plumage (during all this refactoring) to write a stronger verifier for plumage metadata.  One that can eventually be considered canonical.
19:29 japhb Tene, sorry, I only know about it because he's been talking a lot about it in his numerous personal emails this last week.
19:29 Tene Ah. :)
19:29 Tene I look forward to seeing it.
19:29 japhb (He's pounding on Plumage pretty hard, finding lots of issues.)
19:29 cotto_work I'm verifying that mark number.  It should be ready soon.
19:29 japhb I think a lot of it is checked in to the parrot repo, just not "advertised" well.
19:30 japhb Anyone else have any plumage-related questions for me?  I've got to run again pretty soon.
19:30 japhb Also, Allison, we need to pull in the discussion from last week about API docs into DocsTasklist.  If you don't get to it by this afternoon, let me know, and I'll try to throw some tuits at it.
19:31 japhb (summarizing that discussion to the wiki page, I mean)
19:32 allison japhb: ok, thanks, yes, that would be helpful
19:32 japhb allison: will try
19:33 japhb OK, I'll assume no more plumage questions, anyone ask me on #parrot if you think of something, I'll backlog there.
19:33 japhb afk
19:44 PacoLinux left #parrotsketch
19:49 cotto_work The meeting seems to have fizzled out.
19:50 Tene 12:27 < allison> alright, for those in turkey territory, enjoy the festivities,  and the rest of us, have a good week!
19:50 cotto_work oh
19:50 cotto_work gg then
19:51 NotFound left #parrotsketch
19:52 pmichaud left #parrotsketch
20:43 bluescreen joined #parrotsketch
20:52 Coke left #parrotsketch
21:41 simcop2387 joined #parrotsketch
21:42 darbelo left #parrotsketch
23:24 davidfetter joined #parrotsketch
23:44 mariano joined #parrotsketch

| Channels | #parrotsketch index | Today | | Search | Google Search | Plain-Text | summary