Camelia, the Perl 6 bug

IRC log for #parrot, 2009-06-08

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 skids Yes, my one useful contribution so far was fixing the one-off error that was breaking INITIAL_BUCKETS=4 :-)
00:00 bacek :)
00:02 Infinoid Whiteknight: Are you running it on linux/x86 (32-bit)?
00:03 Whiteknight nope, x86-64
00:03 Infinoid Yeah, that doesn't work.
00:03 * Infinoid checks out trunk on x86-32
00:06 Whiteknight yeah, i figured that would be the problem
00:08 nopaste "skids" at 71.192.212.78 pasted "for hash profiling statistics" (103 lines) at http://nopaste.snit.ch/16826
00:08 skids hand edited diff of an out of date file on an out of date file.  Maybe it'll even apply :-)
00:09 skids Not sure I kept the STRING stuff.
00:14 skids My personal take on the path out of the madness: implement a sparse array, tie orderedhash to that to get it out of hash's hair, then reimplement hash.
00:16 skids Also, since all of hash, array, and str need efficient small-size implementations, adding infrastructure to aid with stuff that stores in the PMC and then later adds a data alloc when it grows would be wise.
00:18 skids (and what I pasted yesterday, http://nopaste.snit.ch/16809)
00:24 Infinoid skids: Oh, did I break something when I set INITIAL_BUCKETS=4?  Or was your fix already in at that point?
00:24 bacek We don't standardize on
00:24 bacek Unicode internally, converting all strings to Unicode strings, because for the
00:24 bacek majority of use cases it's still far more efficient to deal with whatever
00:24 bacek input data the user sends us.
00:24 skids Already in.
00:24 bacek *sigh*
00:25 Infinoid skids++
00:26 patspam joined #parrot
00:26 skids bacek: also since the majority of internally used strings are ascii.
00:27 Infinoid Whiteknight: Well, I re-ran mk_native_pbc on x86-32, but it doesn't appear to have changed any files.  And t/pmc/packfile*.t are all passing here on a fresh trunk checkout
00:27 bacek and majority of external string are unicode.
00:27 * Infinoid flips back to x86-64 to test there
00:28 skids There could be a case made that only ascii strings should qualify for using a short in-PMC mini-string (thus no need to store encoding in the mini-string version)
00:29 Infinoid skids: Is there anything we can do to make "Key" less insane, at the same time? :)
00:30 * skids still does not fully understand "Key".
00:30 * Infinoid does not think it's possible to fully understand "Key".  That's part of the problem.
00:30 bacek no one understand "Key"
00:30 bacek Can we just kill it?
00:30 skids Not if you want hash iterators.
00:31 Infinoid Killing it would probably break the HLLs.
00:31 skids (unless you replace them)
00:31 Infinoid But if you're reimplementing hashes, reimplementing keys on top of that would make sense
00:32 skids s/make sense/be required/
00:33 skids I seem to recall it relies heavily on the type of hash used.  e.g. it must be a chained bucket store.
00:33 skids And if I got the gist, the special key stuff is to keep track of where you are in the store.
00:34 skids If hash is to be reimplemented, though, a sane behavior of hash iterators when the hash is modified during the iteration should be defined.
00:34 Infinoid yeah
00:34 skids And it probably should be what languages need, if any language specifies that (probably the iterator gives you what was in the hash when it was instantiated)
00:35 bacek Infinoid: my fault. I forgot to update mk_native_pbc to recreate t/native_pbc/annotations.pbc :/
00:35 bacek (Because I don't understand shell to good extent)
00:36 Infinoid I can fix it if you like, I just haven't gotten to it yet
00:37 dalek parrot: r39445 | bacek++ | branches/io_rewiring/t/native_pbc/annotations.pbc:
00:37 dalek parrot: Recreate annotations.pbc
00:37 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39445/
00:37 bacek I just commit new annotations.pbc to io_rewiring branch.
00:37 bacek If you can fix mk_native_pbc it would be helpful
00:39 Infinoid You just ran "./parrot -o t/native_pbc/annotations.pbc t/native_pbc/testdata/annotations.pir", right?
00:40 bacek yes
00:40 Infinoid Then the upcoming nopaste will probably work
00:40 nopaste "infinoid" at 24.182.55.77 pasted "Simple patch to rebuild annotations.pbc." (13 lines) at http://nopaste.snit.ch/16827
00:40 bacek Looks good for me
00:41 Infinoid It'd probably be nicer if the pir sources were folded into the script with a heredoc, like the previous stuff... but I'm not entirely sure parrot would know to interpret it as pir or pasm
00:41 Infinoid I'll try it and see what breaks
00:42 snarkyboojum joined #parrot
00:50 dalek parrot: r39446 | bacek++ | trunk/docs/pdds/pdd28_strings.pod:
00:50 dalek parrot: [docs] Fix typo - UTF-16 isn't fixed-width encoding. Use UCS-2 instead.
00:50 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39446/
01:12 Whiteknight Infinoid: I didn't merge into trunk, I merged trunk into the branch
01:13 Whiteknight the branch is where the packfile tests are failing
01:13 Whiteknight but, I'm heading to bed now. I'll talk to you later
01:13 dalek parrot: r39447 | Infinoid++ | trunk (2 files):
01:13 dalek parrot: [tools] Regenerate annotations.pbc as part of the mk_native_pbc run.
01:13 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39447/
01:13 Infinoid Oh, ok, I can refix the branch
01:13 Infinoid bacek: I guess we can get rid of annotations.pir now
01:16 bacek Infinoid++
01:17 bacek Heh. We can reimplement mk_native_pbc in pir :)
01:17 * bacek hides
01:17 Infinoid Since it only works once parrot is already built, that actually makes sense
01:17 bacek It was joke actually. Chicken and eggs dilemma
01:17 Infinoid (though it does rebuild parrot a couple of times, heh)
01:18 bacek we can limit it to generate only "current parrot native" version
01:19 Infinoid I think that's what the --noconf switch is for
01:19 bacek yes.
01:19 Infinoid But its detection is broken; it generates *something* when you run it on x86-64 but the output doesn't seem to be useful
01:19 Whiteknight Infinoid++
01:19 Whiteknight and goodnight!
01:20 bacek What is current policy for adding new core pmcs?
01:20 bacek And DynPMCs?
01:20 purl DynPMCs are treated the same way as pmcs, as far as dumps go
01:21 Infinoid policy in what sense?  Deciding which to add?
01:21 bacek deciding
01:21 bacek I really want FakeSub
01:21 Infinoid I don't know what the official policy is, but I think dynpmcs are often used for linking with third party libraries that we don't want to link against unless the hll/pir requests it
01:22 Infinoid e.g. pcre, gdbm, decnum
01:22 nopaste "bacek" at 114.73.13.169 pasted "FakeSub to add" (106 lines) at http://nopaste.snit.ch/16828
01:22 Infinoid I don't know what decides whether something can be in core or not.  Probably how often we plan to use it
01:23 bacek This FakeSub used to implement PIR compiling in PIR :)
01:24 Infinoid That sounds useful.  But to be honest, I think a feature like that is more likely to be accepted into core if it doesn't have a name containing "Fake" :)
01:26 bacek But it's fake! :)
01:27 Infinoid But it isn't immediately clear *why* you'd want a fake sub.  Maybe PirSub?  or SubCompiler?
01:28 Infinoid Anyway, it was just a suggestion, not important.
01:30 bacek SerializeableSub is more close to semantic. "Fake" is implementation details :)
01:32 Infinoid Is it something we can just extend Sub to do?
01:32 Infinoid serialization sounds like something that could be generally useful
01:33 bacek Infinoid: no. Otherwise we can easily broke real executable Sub.
01:33 bacek e.g. "set_start_offset" for real sub is very dangerous.
01:34 Infinoid Ah.  So it's only for PIR subs?
01:34 Infinoid (Do NCI functions use the Sub interface too?)
01:36 bacek http://irclog.perlgeek.de/p​arrot/2009-06-06#i_1215578
01:36 dalek parrot: r39448 | Infinoid++ | branches/io_rewiring/t/native_pbc (5 files):
01:36 dalek parrot: [t] Regenerate native_pbc files.
01:36 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39448/
01:36 Infinoid Thanks
01:36 bacek Yes, this is for PIR only. Just for implement self-hosted PIR compiler in PCT
01:39 Infinoid Sounds delightfully hackish
01:39 bacek heh :)
01:39 Infinoid My first reaction is, of course, "fix IMCC".  But you know how far that argument usually gets you
01:40 bacek Dark and evil land of some macroassembly language where many heroes were died
01:40 Infinoid I think when we finally get rid of IMCC, quite a lot of code hacking around it can simply vanish
01:41 bacek I checked pirc code. It's very nice, structured, documented, etc.
01:41 bacek One problem - it doesn't work :(
01:42 Infinoid Yeah, but that's just an implementation detail
01:42 * bacek rotfl
02:31 Theory joined #parrot
02:35 janus joined #parrot
02:37 Andy joined #parrot
03:12 bacek joined #parrot
03:30 Andy joined #parrot
03:43 pmichaud message bacek  objection to pmc_i_ops branch merge noted in parrot-dev mailing list
03:43 purl Message for bacek stored.
03:47 Infinoid pmichaud: Paralleling Perl 5's I/O speed would be nice.  But the branch didn't really change I/O speed at all; it just removed a bunch of PCCINVOKE nonsense on the way to calling the I/O functions
03:47 Infinoid Seems like speeding up I/O is all about keeping the rest of parrot out of the way
03:51 pmichaud Infinoid: Sure.  The main reason I cite Perl 5 is because it seems like the P5 vm and Parrot vm have about the same amount of work to do in each case
03:51 pmichaud so the Parrot VM really ought to be able to be on the same order as P5
03:51 Infinoid that seems reasonable (though I know nothing about perl5's internals)
03:52 pmichaud I know very little about them.
03:52 Infinoid Then again, we don't match p5's speed in other areas either
03:52 Infinoid (yet)
03:52 pmichaud actually, at one time we did.
03:53 Infinoid I'll bet we were less Correct back then when we were Fast...
03:53 pmichaud well, we are talking about very simple operations here.
03:54 pmichaud for this particular benchmark.
03:54 pmichaud I'd be very curious to know where Parrot is spending all of its time for this benchmark
03:54 * Infinoid loves profiling
03:54 Infinoid I'll see if I can find out
03:55 pmichaud I mean, it's not as if there are a lot of explicit PCC calls here
03:55 pmichaud the benchmark just uses the built-in opcodes, which ought to be fairly fast
03:56 Infinoid They weren't fast at all.  Even the basic "read" and "write" were methods called through PCCINVOKE
03:56 pmichaud right, but is that still true in the branch?
03:56 Infinoid Now we call through VTABLE_* if we recognize the base_type, and fall back on PCCINVOKE
03:57 Infinoid That's what the branch sped up
03:57 pmichaud right, but it still doesn't explain the 2x difference with p5
03:57 pmichaud I mean, there really shouldn't be any GC pressure, afaict
03:58 Infinoid I think comparing p5's runloop to parrot's is sort of apples vs. oranges
03:58 pmichaud that's possible.
03:58 Infinoid But we shall see.  I'll post profiling results when I have some
03:58 pmichaud and I'm not running an optimized parrot.
03:58 pmichaud I could try that.
04:03 Infinoid I've started being more careful about that, recently.  Seems building with --optimize turns on some important gcc warnings, too
04:09 Infinoid Plenty of overhead, running this in callgrind is gonna take a while.
04:11 Infinoid I suppose that implemented in C, this could be optimized down to about 10-15 assembly instructions.  I doubt Parrot's ever going to do it that well
04:12 payload left #parrot
04:17 payload joined #parrot
04:27 Zak joined #parrot
04:37 pmichaud well, I'm not sorried about matching C's performance.  :-)
04:38 pmichaud running with --optimize on my system gets 10.816s
04:39 Infinoid I'm still waiting for callgrind :)  Looks like it'll take around half an hour in total
04:40 Infinoid You were saying something about inplace math ops using clone.  I suppose that would cause a lot more GC pressure for this benchmark
04:41 Infinoid Therefore it should be pretty easy to see the effect
04:41 pmichaud no, because this benchmark (the i/o one) isn't using any PMC ops
04:41 Infinoid It's decrementing the integer
04:42 pmichaud sure, but that's not a PMC
04:42 Infinoid oh.
04:44 Infinoid if I have time tomorrow morning, I'll convert to an Int PMC and try to compare trunk vs branch performance
04:44 pmichaud does the io_rewiring branch also include the pmc_i_ops work?!?
04:45 Infinoid No.  I meant comparing trunk to pmc_i_ops
04:45 pmichaud oh, that.
04:45 pmichaud It may not make a significant performance difference for just that case.
04:45 pmichaud I'm more concerned about the case where someone has subclassed Integer and changed the meaning of 'clone'
04:45 pmichaud (or Float, or Complex, or ...)
04:45 Infinoid well, 2.5 million extra created and destroyed PMCs should count for something
04:45 pmichaud except that it won't be "extra"
04:46 pmichaud in both the branch and trunk, 2.5 million new PMCs get created
04:46 pmichaud the main question is whether that happens using pmc_new or VTABLE_clone
04:48 Infinoid The difference is in the amount of additional init overhead from subclasses, right?
04:48 Infinoid Or is it a behavioral change?
04:48 Infinoid valgrind --tool=callgrind --dump-instr=yes --trace-jump=yes ../test/parrot   2135.25s user 13.67s system 90% cpu 39:21.63 total
04:48 pmichaud it's a behavioral change.
04:49 pmichaud that was supposed to be the main point of my message -- did I not make that clear enough?
04:49 * pmichaud re-reads.
04:50 pmichaud maybe I should've said "substantial" instead of "real" in my last sentence.  :-|
04:51 Infinoid No, I'm just sleep-deprived and unable to read emails, apparently
04:52 Infinoid https://quack.glines.org/upload/i​nfinoid/kcachegrind-Parrot-IO.png
04:52 Infinoid No really big single hog, but we are spending most of our time in things called by Parrot_io_putps
04:53 pmichaud Parrot_print_p_i seems biggish
04:55 Infinoid It's spending all its time in the callees
04:56 Infinoid around 100 million samples within the function itself, compared to 17 billion within Parrot_sprintf_c and 26 billion within Parrot_io_putps
04:57 Infinoid Overall, we appear to be spending about 70% in IO and string-processing functions, and 25% in PMC creation and destruction
04:57 Infinoid But there's probably some overlap there
04:57 cottoo joined #parrot
04:57 pmichaud I'm a little surprised there's so much pmc creation/destruction, then
04:57 pmichaud that must be from the PCC_INVOKE?
04:58 pmichaud but what is there to be PCC_INVOKED?
04:59 pmichaud am I reading this correctly that there are 30 million calls to pmc_new?
05:00 Infinoid yes
05:00 pmichaud W... T.. F...  ?!?!?!?!
05:00 Infinoid the biggest thing called by PCCINVOKE (which isn't part of PCC itself) looks like FixedIntegerArray.set_integer_native(), strangely
05:01 cottoo I think that's used in the calling conventions.
05:01 pmichaud cottoo: sure, but I'm not making use of any calling conventions in this test.
05:02 pmichaud obviously *something* is, but my test program itself isn't.
05:02 nopaste "pmichaud" at 72.181.176.220 pasted "benchmark program (for cotto)" (20 lines) at http://nopaste.snit.ch/16829
05:02 Infinoid ok.  Parrot_PCCINVOKE was called 5 million times by Parrot_io_putps, so apparently we can still optimize some more I/O
05:03 pmichaud I think I need a callgrind / kcachegrind tutorial.  :-|
05:03 pmichaud so I can do some of this.
05:03 Infinoid oh, hang on.  I think I ran it on trunk, not io_rewiring
05:04 Infinoid ok.  First, get valgrind
05:04 Infinoid Then, set up your shell with an alias: cgp='time valgrind --tool=callgrind --dump-instr=yes --trace-jump=yes ./parrot'
05:04 Infinoid Then, invoke it like: cgp x.pir
05:04 Infinoid It will generate a dumpfile with the pid in the filename, like: callgrind.out.10329
05:04 Infinoid Then run: kcachegrind callgrind.out.10329
05:05 Infinoid That's it.
05:05 pmichaud Infinoid++  # thanks, just what I needed
05:05 pmichaud running now, on branch
05:06 pmichaud needs about 35 minutes, yes?
05:06 Infinoid For trunk, yes.  Branch should be more like 10 minutes
05:06 chromatic joined #parrot
05:06 pmichaud (because it's 4 times faster... got it :-)
05:06 chromatic function kcg {
05:07 chromatic kcachegrind "callgrind.out.$1" 2> /dev/null &
05:07 chromatic }
05:07 chromatic That way you can write `kcg 10329`
05:07 Infinoid You can monitor the progress with "wc -l test-pir.txt".  The finished file has around 78 thousand lines for this benchmark
05:08 Infinoid chromatic: Thanks for introducing me to this tool.  I'm making stupid mistakes right now, but I love the amount of data it gives me
05:09 chromatic You're welcome.
05:09 chromatic I need to write a little tutorial; still plan to do so.
05:09 pmichaud other way to monitor progress is filesize -- final filesize is 9003865 bytes :-)
05:09 chromatic Maybe I can convince someone to take notes on what I say at YAPC if someone wants to listen to me walk through an optimization.
05:09 pmichaud I'll take notes.
05:09 pmichaud I might even bring a camera.  :-)
05:09 pmichaud (video)
05:10 chromatic It should be obvious why I think callgrind-style output is useful for Parrot's PIR profiler.
05:10 Zak joined #parrot
05:10 Infinoid I'm bummed.  I'm moving to the east coast next week, and I still don't think I'm gonna be able to make YAPC
05:11 pmichaud 50% done
05:12 Infinoid Ok.  In branch, Parrot_PCCINVOKE got called a total of 8 times. :)
05:13 pmichaud that's much more like it.
05:13 pmichaud how about pmc_new ?
05:14 Infinoid 1219 times
05:14 chromatic That's more like it.
05:14 pmichaud agreed.
05:15 chromatic Any time we can remove PCCINVOKE from a hot path, we win.
05:17 Infinoid We're spending all our time under Parrot_print_p_i, processing printf format strings and creating STRINGs.  A dedicated itoa() type of function would speed up this particular benchmark, but I don't know what it would do for parrot as a whole.
05:18 chromatic How many STRINGs do we create?
05:18 Infinoid 7.5 million
05:19 chromatic From Parrot_str_append or Parrot_str_concat?
05:19 Infinoid from Parrot_sprintf_c, actually
05:19 chromatic Neither append nor concat?  That surprises me, but that's fine.
05:20 Infinoid Not directly.  5 million calls to string_make from Parrot_sprintf_format, 2.5 million from Parrot_vsprintf_c
05:20 Infinoid I'll walk up the stack a bit to see if append or concat is involved
05:20 chromatic Look in the right pane at the call graph.
05:20 chromatic It should be obvious.
05:21 Infinoid Yeah, neither of those are in the All Callers pane.
05:21 chromatic Fair enough.
05:22 Infinoid I almost want to use a static STRING in Parrot_print_p_i with a buffer that we know is big enough, and just make it call sprintf() directly
05:22 chromatic I mention them because they have some weird co-recursion between them that I've never quite been able to unravel.  It seems like fixing that would speed up parts of PGE.
05:22 Infinoid But that's just hacking around the fact that our I/O interface can't handle cstrings
05:22 chromatic Can you create a string with the right buffer size at the start?
05:23 chromatic It doesn't have to be static (and look out concurrency), but it could avoid reallocating and resizing.
05:23 pmichaud might also have to worry about COW stuff
05:23 Infinoid How would that skip reallocation?  We allocate a new STRING every time Parrot_print_p_i is called, it's the first thing that op does
05:24 Infinoid s/reallocation/allocation/
05:24 chromatic This all almost makes me want to use ropes instead of strings instead.
05:24 chromatic I might be misunderstanding what you're suggesting.
05:24 pmichaud Infinoid: what would it take to get the I/O interface to be able to handle cstrings, ooc?
05:24 pmichaud or is that explicitly what was taken out in the I/O refactor?  ;-)
05:25 pmichaud (the earlier I/O refactor)
05:26 pmichaud somehow the idea that int/num have to go through STRING in order to be output to a file handle is.... weird.
05:26 Infinoid It does make per-filehandle encoding coercion easier, I guess, if we're doing that yet
05:26 pmichaud I agree they have to be stringified.  I'm just not sure they have to be made into STRINGs.
05:26 pmichaud otoh, I'm not sure it's that big a win.
05:26 pmichaud It might be over-optimizing to this specific case.
05:27 pmichaud in Rakudo's case, we'd rarely be outputting integer registers -- we'd almost always be doing PMCs
05:28 Infinoid chromatic: What I was suggesting was a hack.  (A static STRING structure that just contains a buffer, and we just overwrite the buffer once per invocation, rather than allocating a new STRING.  And break reentrancy entirely in the bargain.)
05:28 Infinoid Although... is it possible to allocate STRINGs directly on the stack?
05:28 pmichaud right, but that hack might not work if others expect to use the STRING structure across multiple calls
05:29 Infinoid This single-use pattern would have a lot less GC churn if we could do that
05:29 chromatic Stack allocation should be fine, except that marking stack-allocated STRINGs will be problematic.
05:30 Infinoid True.  We'd need a way to detect that.  And raise a big huge flag if something in the heap holds a reference to something on the stack
05:30 pmichaud what I would almost prefer would be a way for PMCs to directly write themselves to a file buffer
05:30 chromatic That does seem saner.
05:30 Infinoid pmichaud: What would that look like?
05:30 pmichaud Infinoid: well, think about it in C (more)
05:31 pmichaud if I do    printf("%d", 123);     -- it doesn't normally write the integer to a buffer and then copy the buff
05:31 pmichaud it actually does the equivalent of several putc calls to write each digit directly to the filebuffer
05:32 pmichaud (at least, that's how it once did it)
05:32 Infinoid right, it just uses putc for the output-a-character callback function
05:32 pmichaud right
05:32 pmichaud but that's a lot different from doing "convert-to-string-and-then-output-string"
05:32 pmichaud especially in our case, wehre "convert-to-string" is in fact a bit expensive
05:33 pmichaud so, what would be nicer is something (probably a vtable function) that allows a PMC to write itself to a filehandle
05:33 Infinoid So in this case, print would be a method on the integer?
05:34 pmichaud then  print ofh, $P0     is really a vtable function on $P0, not on ofh
05:34 Infinoid cool.  Then that vtable can call a vtable "putc" type of function on ofh
05:36 pmichaud and yes, the default behavior can still be  "stringify the PMC and send it to the filehandle", but for specific and common cases like Integer and Float it can be optimized a lot better.
05:36 flh joined #parrot
05:37 Infinoid I like it.  Though I think it'll get bloated when someone requests hex and octal and binary versions of print
05:37 pmichaud I don't know that we'd end up with those.  Or, more to the point, those would go through sprintf (as they do now)
05:37 pmichaud by "sprintf" I mean "sprintf opcode"
05:39 pmichaud I just know that "output integer to a file" is something that can happen a lot.
05:39 pmichaud and "output float to a file" also happens a lot
05:39 pmichaud "output string to a file" happens more often, but that doesn't (shouldn't) require extra calls to string_make
05:40 pmichaud hmmm, that's an interesting comparison -- just a sec
05:41 pmichaud okay, I'm shocked.
05:42 nopaste "pmichaud" at 72.181.176.220 pasted "benchmark, without outputting any integers" (29 lines) at http://nopaste.snit.ch/16830
05:42 pmichaud looks like we can't put too muchof the blame on the integer->string comparison.
05:43 chromatic Which opcode does print ofh, '000' become?  print_p_sc?
05:44 pmichaud yes.
05:44 pmichaud I'm running callgrind on that one now
05:45 Infinoid That part of the profile shouldn't have changed much
05:45 Infinoid it'll just output a few more bytes per write
05:45 pmichaud well, it did run significantly faster in callgrind
05:46 Infinoid That tweak seems to have sped things up about 8x
05:46 pmichaud why didn't it speed it up on my system, though?
05:46 Infinoid What, you mean the "real" time?
05:46 pmichaud yes
05:47 Infinoid I dunno.  Is it syncing to disk after every write?
05:47 pmichaud hard to say
05:47 Infinoid It is!
05:47 pmichaud I didn't request it to do so, no.
05:47 Infinoid write(3, "000,000,000,000,000,000,000,000,"..., 8192) = 8192
05:47 Infinoid fsync(3)                                = 0
05:51 pmichaud well, apparently the sync is responsible for the bulk of the real-time cost (in the branch)
05:51 Infinoid Yes.  I'm trying to track down what's calling it
05:51 pmichaud trunk's cost is dominated by the PCCINVOKE
05:52 iblechbot joined #parrot
05:52 chromatic I thought we had buffered output.
05:52 chromatic We've certainly had bugs in buffering output.
05:53 Infinoid We do
05:53 Infinoid It buffers it, and then flushes it explicitly
05:53 nopaste "pmichaud" at 72.181.176.220 pasted "same non-integer output benchmark for p5" (22 lines) at http://nopaste.snit.ch/16831
05:53 Infinoid I'm not so sure Parrot_io_write_buffer should *ever* be calling Parrot_io_flush, but it does
05:53 pmichaud note that if the sync problem is resolved, then I suspect the parrot version will be several times faster than p5
05:55 pmichaud (for the case of not converting integers to strings)
05:55 chromatic There's a FIXME in Parrot_io_write_buffer; that may be a sign.
05:59 pmichaud fwiw, all of this isn't developed as an arbitrary benchmark (more)
06:00 pmichaud I wrote this benchmark because it's actually very similar to what pbc_to_exe is doing, which is what caused me to notice things being slow in the first place
06:00 Infinoid Parrot_io_flush_filehandle calls both Parrot_io_flush_buffer (to do the write) and PIO_FLUSH (to call fsync).  That doesn't seem easily separable
06:00 pmichaud (i.e., it writes 2.5 million comma-separated integers to a file)
06:01 Infinoid But after commenting out PIO_FLUSH, it runs the benchmark fast
06:01 Infinoid infinoid@chirp io_rewiring % time ./parrot x.pir
06:01 Infinoid ./parrot x.pir  0.38s user 0.06s system 61% cpu 0.721 total
06:01 pmichaud yeah
06:01 pmichaud that's more like it
06:03 Infinoid Thing is, I thought calling fsync() is the only thing calling Parrot_io_flush() did
06:03 Infinoid So that's a little confusing.
06:04 pmichaud well, through Parrot my system seems to get a maximum I/O output of about 1MB/sec.  Not very good.
06:05 Infinoid Commenting out src/io/filehandle.c:742 should improve that.  But this I/O API is seriously misleading
06:06 pmichaud well, I'll leave it to you to figure out how to fix things from here.  :-)
06:06 Infinoid Time for bed, goodnight :)
06:06 pmichaud same here :)
06:07 pmichaud thanks for the quick callgrind/kcachegrind tutorial -- I can make use of those.
06:24 uniejo joined #parrot
06:33 Zak joined #parrot
06:37 flh joined #parrot
06:37 Zak joined #parrot
06:45 bacek joined #parrot
06:54 viklund_ joined #parrot
07:21 snarkyboojum joined #parrot
07:21 chromatic Commenting out PIO_FLUSH there *doubles* the speed of make coretest.
07:21 chromatic That's wallclock speed, but still.
07:28 Andy joined #parrot
07:38 clunker3 joined #parrot
07:39 * Tene tries again to debug GC segfaults, fails badly.
07:40 Tene I don't feel right posting this blog entry about inter-language programs on Parrot when some of the examples require running Parrot with GC disabled, though.
07:40 Tene :(
07:43 chromatic Agreed.
07:44 Tene Adding an ASSERT to that macro should only fail when the bad value is read from the context, not when it's written, afaict.
07:44 Tene So that didn't seem like it would help very much.
07:45 Tene I was trying to convince gdb to watch the relevant memory location for me, but I failed at gdb.
07:45 chromatic It helps narrow down where it gets written.
07:45 Tene Will try again sometime tomorrow.
07:50 muixirt joined #parrot
08:31 ttbot joined #parrot
08:31 ttbot bacek: Parrot trunk/ r39321 MSWin32-x86-multi-thread make error http://tt.ro.vutbr.cz/file/cmdout/24948.txt
08:31 ttbot bacek: Parrot trunk/ r39321 i386-linux-thread-multi make error http://tt.ro.vutbr.cz/file/cmdout/25045.txt
08:31 ttbot bacek: Parrot trunk/ r39321 i386-freebsd-64int make error http://tt.ro.vutbr.cz/file/cmdout/25126.txt
08:34 mj41 Hi. I just started ttbot (TapTinder bot) to report build failures - http://tt.ro.vutbr.cz/buil​dstatus/pr-Parrot/rp-trunk .
08:38 cotto mj41++
08:42 chromatic Huh.  There were more optimizations in CodeString.
08:42 chromatic Looks like a 3.5% performance improvement in parsing long files.
08:53 muixirt chromatic, you might want to revisit http://use.perl.org/~chromatic/journal/35333
08:54 muixirt and report the progress since then
08:56 mikehh_ joined #parrot
08:56 chromatic I'm not sure I want to, at least until after a couple of branches land in trunk.
08:57 * muixirt smiles innocently, holds back the diabolic grin ;-)
09:00 payload joined #parrot
09:01 chromatic Hm, 2.61% improvement, but I'll take that.
09:01 masak joined #parrot
09:04 bacek OH HAI
09:04 * bacek spent few hours on Keys, Iterators and Hashes...
09:05 bacek I have one question - who designed it? So I can visit him with baseball bat in hands...
09:07 chromatic I'm not sure anyone designed it.
09:08 bacek oh shi...
09:10 dalek parrot: r39449 | chromatic++ | trunk/src/pmc/codestring.pmc:
09:10 dalek parrot: [PMC] Avoided unnecessary STRING copying in emit() and lineno() methods by
09:10 dalek parrot: using String PMC attributes directly -- as CodeString extends String, this is
09:10 dalek parrot: safe.
09:10 dalek parrot: Avoided mostly unnecessary calls to string_ord() to coalesce \r\n sequence into a single logical newline.
09:10 dalek parrot: This combination speeds up Rakudo's actions.pm processing by 2.61%.
09:10 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39449/
09:10 bacek Can we put in DEPRECATED.pod something like "Whole Iterators, Containers and Keys will be refactored after 1.4"?
09:12 chromatic I *think* we have to deprecate them by 1.4 to fix them after 2.0.
09:12 chromatic Or fix them *in* 2.0.  Coke knows much better.
09:12 bacek ouch... Fix after 2.0 mean about a year to deployment...
09:13 chromatic Could be worse.  Could be Perl 5.
09:14 bacek no way...
09:14 purl WAY!
09:14 chromatic Make up a plan for what to do, and we'll see what we can do.
09:15 bacek ok. I'll try to make some .plan without using my favourite sentences about technical design.
09:16 chromatic "You are all idiots.  I've known monkeys who can write code better than you.  You are the universe's version of technical debt.  I've sneezed better programs than you could ever write."
09:17 bacek Something like this :)
09:17 chromatic "There is a picture of your program next to the word CRACK in the dictionary."
09:17 gaz joined #parrot
09:17 bacek Small mistake. It's after "CRAP" :)
09:18 chromatic "If dmr read your program, he would invent time travel to prevent himself from inventing C so that you could never perpetuate such horror upon the world."
09:19 bacek dmr?
09:19 purl well, dmr is Dennis Ritchie, author of Unix and C; he is our Grey Eminence, and King. or at http://cm.bell-labs.com/cm/cs/who/dmr/
09:19 chromatic "The only thing worse than your code is the compiler which allowed it."
09:21 bacek wow, we did you get it? I need this source of wisdom for my day-to-day technical discussions!
09:21 bacek s/we/where/
09:22 donaldh joined #parrot
09:23 chromatic I made them all up.  My other job is professional writer....
09:23 * bacek requesting the book "Famous quotes by chromatic"
09:24 bacek I actually quite enjoyed reading MPB and other your posts.
09:25 chromatic Thanks.  I try to throw in some over the top rhetoric so people know I'm not red-faced and ranting.
09:28 bacek btw, I work with some guy who have almost same sarcastic manners in speech (code comments, bug reports, etc). Many times I suspect that you and him are same persons :)
09:29 bacek (And he is one of greatest developers I've meat)
09:29 chromatic Don't tell, but there's a franchise.
09:31 bacek :)
09:35 bacek What is semantic behind "provides foo"? How I can check it and use it?
09:36 bacek And how it differ from "does foo"?
09:39 chromatic I don't recall that it does, but it's too late for me to remember clearly anyway.
09:41 bacek found it.
09:42 bacek VTABLE_does check provides_str which is generated from "provides foo"
09:46 mikehh joined #parrot
09:55 snarkyboojum joined #parrot
10:26 payload joined #parrot
11:13 payload joined #parrot
11:13 clinton joined #parrot
11:21 donaldh joined #parrot
11:43 clunker3 joined #parrot
11:54 skids joined #parrot
12:25 Andy joined #parrot
12:40 payload joined #parrot
12:45 whoppix joined #parrot
12:48 barney joined #parrot
12:58 gryphon joined #parrot
13:02 UltraDM joined #parrot
13:18 Whiteknight joined #parrot
13:21 Infinoid Whiteknight: You'll love this next commit.
13:21 dalek TT #746 created by coke++: pdd19 fails t/codingstd/pdd_format.t
13:22 Whiteknight really?
13:22 * szbalint readies the gift cookies
13:23 dalek parrot: r39450 | Infinoid++ | branches/io_rewiring/src/io/buffer.c:
13:23 dalek parrot: We really don't need to call fsync() after every write().
13:23 dalek parrot: Several functions in src/io/buffer.c would call Parrot_io_flush() to
13:23 dalek parrot: write buffered but unwritten data.  The problem is, that would also
13:23 dalek parrot: call PIO_FLUSH(), forcing the OS to push the data all the way to disk.
13:23 dalek parrot: While OS-level flushes are useful, we don't want to do them all the
13:23 dalek parrot: time, as doing so severely hurts performance.  Ensuring disk consistency
13:23 dalek parrot: wasn't part of the contract of these functions anyway, they were just
13:23 dalek parrot: doing it to make the buffer pointers and lengths consistent so they could
13:23 dalek parrot: do their jobs.  If we call Parrot_io_flush_buffer() directly instead of
13:23 dalek parrot: Parrot_io_flush(), we write out the buffered data, but don't call the OS
13:23 dalek parrot: flush, so life is good.
13:23 dalek parrot: Removing the OS flush sped up "coretest" by a factor of 2, for chromatic.
13:23 dalek parrot: Apparently my Thinkpad's disk is slower than his, so the difference is
13:23 dalek parrot: even more striking for me:
13:23 dalek parrot: before: make coretest  84.97s user 61.91s system 32% cpu 7:35.52 total
13:23 dalek parrot: after : make coretest  75.22s user 43.93s system 67% cpu 2:57.62 total
13:23 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39450/
13:24 Whiteknight Damnit Jim, I'm a coder not a speed-reader
13:25 Infinoid Sorry.  I had written it up as a ticket, before realizing how simple the solution was and deciding to just commit it directly.  But apparently I still felt the need to rant a little
13:25 Whiteknight so that's a factor-of-2 speedup on top of the 4x speedup that some benchmarks are showing?
13:26 Infinoid That's a wallclock time speedup, on top of your cpu speedup
13:26 szbalint Infinoid++ # you deserve more karma for this :)
13:27 Whiteknight So basically it took less then three minutes, as opposed to over seven and ahlf?
13:27 Infinoid yes
13:27 whoppix joined #parrot
13:27 Whiteknight Infinoid++ # Holy shit
13:27 Infinoid It eliminates a lot of dead time waiting for the OS to catch up the data we've written
13:28 Whiteknight wait, what command did you run to get those results?
13:28 Infinoid I had originally just commented out the call to PIO_FLUSH.  This commit does it in a safer way
13:28 Whiteknight what was your benchmark?
13:28 Infinoid "time make coretest"
13:29 Whiteknight So you cut coretest time to 38% of what it used to be?
13:29 ruoso joined #parrot
13:29 Infinoid So it seems
13:29 Infinoid I'm going to redo pmichaud's benchmark to get the numbers, but there was an insane speedup there too
13:30 Infinoid Something on the order of 9 seconds -> 0.7 seconds
13:30 Infinoid (but I haven't done it with this safer version of the patch yet)
13:30 szbalint fsync is expensive, no wonder
13:30 Infinoid yeah, and completely unnecessary
13:30 Infinoid It's nice to have, but not by default.
13:32 szbalint fsync only makes sense after accumulating a largish chunk of data anyways
13:32 Whiteknight so what's the total speedup of the branch now versus trunk?
13:32 Infinoid fsync only makes sense for databases and mail servers ensuring atomicity
13:33 szbalint yeah
13:33 Infinoid here's stats for pmichaud's benchmark:
13:33 Infinoid trunk : ./parrot x.pir  12.37s user 1.67s system 56% cpu 24.747 total
13:33 Infinoid branch: ./parrot x.pir  0.42s user 0.06s system 64% cpu 0.741 total
13:34 * Infinoid comes up with some numbers for coretest (this time ensuring his laptop won't change cpu speeds)
13:36 Infinoid Uck, with all the disk I/O I can't fairly do both checkouts in parallel
13:36 Infinoid Have you ever noticed how often "make test" sits there at almost 0% CPU?  This is why it did that.  (I've been wondering about that for a while)
13:38 Whiteknight irclogs
13:38 Whiteknight irclogs?
13:38 purl i guess irclogs is http://irclog.perlgeek.de/parrot/today or see also: infrared clogs
13:38 Infinoid pmichaud++ for making me look closer at this
13:44 Infinoid trunk : make coretest  63.07s user 36.61s system 26% cpu 6:15.27 total
13:45 Infinoid branch: make coretest  61.79s user 31.34s system 63% cpu 2:26.49 total
13:45 Infinoid (the timings I put in that commit message were a little high, because my laptop was doing cpufreq nonsense)
13:52 estrabd joined #parrot
13:54 Whiteknight amost unbelievable
13:55 Infinoid I didn't see any difference from rakudo test (but I guess that's compilation-heavy, not io-heavy)
13:56 Whiteknight list?
13:56 purl rumour has it list is http://groups.google.com/group/parrot-dev or take that, moose-heads
13:56 pmichaud good morning, #parrot
14:01 dalek partcl: r440 | coke++ | trunk/t/cmd_inline.t:
14:01 dalek partcl: Hopefully someone can suggest a way to rewrite this PASM invocation
14:01 dalek partcl: to avoid using 'end'.
14:01 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=440
14:03 uniejo joined #parrot
14:03 Infinoid morning pm
14:04 Infinoid I checked in a version of that fsync cleanup which I think is safe, but if you have a moment, more eyeballs on it can't hurt
14:08 Whiteknight purl Infinoid?
14:08 purl Infinoid is Mark Glines <mailto:mark@glines.org> or likes shiny things
14:08 Whiteknight purl Infinoid is also the master of the universe
14:08 purl okay, Whiteknight.
14:08 Whiteknight Infinoid++
14:09 Infinoid hah, thanks
14:10 Infinoid I tried to hock it once, but they said they won't accept hot merchandise.
14:10 Andy joined #parrot
14:17 Whiteknight so much work to do still on the IO system!
14:18 PacoLinux joined #parrot
14:19 Infinoid I wish I had time to actually do some in-depth tweaking... for instance, I'm still not sure src/io/filehandle.c should exist at all (I'd like to push that code into filehandle.pmc)
14:19 Infinoid Unfortunately, this week will be even crazier for me than the last one
14:21 Infinoid Anyway, again, when you do the branch merge, please just remove the Pipe and PipeHandle PMCs, I haven't had the chance to test them at all, so I assume they don't work
14:25 Whiteknight Okay, I can pull them out. I'm hoping to merge this branch tonight
14:25 Andy mmmm, branch merging
14:25 Whiteknight we can start another branch to work on improving sockets and pipes
14:25 Whiteknight and filehandle stuff definitely needs some lovin
14:26 Infinoid Awesome, thanks.
14:26 Whiteknight So long as I can get started on AIO in July, I don't care what else we have to do before that
14:26 * Infinoid has no idea how these fixups will affect AIO
14:28 Whiteknight I think it's all going to have a very positive effect
14:29 Whiteknight I'm generally envisioning a system, at least two start, that has a completely separate backend for the asynchronous stuff
14:29 Whiteknight we can unify things and pretty them up over time
14:31 dalek rakudo: 5c065e0 | pmichaud++ | docs/spectest-progress.csv:
14:32 dalek rakudo: spectest-progress.csv update: 399 files, 11428 passing, 0 failing
14:32 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/5​c065e0db6e5d6718c93d1696e458e621b6c1820
14:34 particle joined #parrot
14:36 dalek parrot: r39451 | barney++ | trunk/t/library/pcre.t:
14:36 dalek parrot: [t] a saner name for a variable
14:36 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39451/
14:45 particle1 joined #parrot
14:48 dalek TT #747 created by barney++: Constants in library.h are not available as PASM constants
14:50 payload joined #parrot
14:50 dalek partcl: r441 | coke++ | trunk/ (9 files):
14:50 dalek partcl: - make tools/spectcl a generated file to track parrot location
14:50 dalek partcl: - move all .in files to config/misc.
14:50 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=441
14:52 dalek parrot: r39452 | barney++ | trunk/t/library/pcre.t:
14:52 dalek parrot: Add trac ticket number to a TODO comment
14:52 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39452/
15:05 dalek partcl: r442 | coke++ | trunk/t/cmd_expr.t:
15:05 dalek partcl: Add a todo test that causes some failures in the expr spec tests.
15:05 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=442
15:18 Theory joined #parrot
15:21 donaldh joined #parrot
15:34 dalek partcl: r443 | coke++ | trunk/t/cmd_expr.t:
15:34 dalek partcl: Make it clear that this isn't just a round edge case, but a floor.
15:34 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=443
15:35 HG` joined #parrot
16:01 Su-Shee joined #parrot
16:01 Su-Shee hi.
16:26 Infinoid Hi, Su-Shee
16:29 viklund_ joined #parrot
16:29 payload joined #parrot
16:29 Psyche^ joined #parrot
16:35 flh joined #parrot
16:37 barney joined #parrot
16:39 particle1 what made you decide on the newark job?
16:39 particle ...stupid irc client...
16:47 dalek tracwiki: v16 | cotto++ | ParrotQuotes
16:47 dalek tracwiki: bacek++ and chromatic++ commiserate over code quality
16:47 dalek tracwiki: https://trac.parrot.org/parrot/wiki/Pa​rrotQuotes?version=16&amp;action=diff
16:56 chromatic joined #parrot
16:56 dalek parrot: r39453 | barney++ | trunk/t/pmc/array.t:
16:56 dalek parrot: TT #671  Rewrite of t/pmc/array.t to PIR
16:56 dalek parrot: Also reenable the commented out TODO tests.
16:56 dalek parrot: Curtesy of bobw.
16:56 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39453/
16:58 antiphase joined #parrot
17:02 dalek TT #671 closed by barney++: [PATCH] Rewrite of t/pmc/array.t to PIR
17:09 antiphase Is there much PIR anywhere that I can look at for examples? I'm not getting very far with the official docs
17:10 Tene antiphase: t/ has a lot of small examples
17:10 Tene in the parrot tree
17:10 antiphase Ah, thanks
17:11 cotto antiphase, you can also pass --target-pir to most Parrot-based compilers to see what PIR they generate.
17:11 cotto although that code isn't designed to be human-readable
17:12 antiphase I'm more trying to get into writing it by hand
17:14 NotFound antiphase: there are examples in the directory examples/ (suprprise!)
17:14 * cotto was just going to mention that
17:15 * antiphase swears at his terminal colour scheme for having barely visible directory entries
17:16 NotFound unalias ls
17:17 payload joined #parrot
17:30 darbelo joined #parrot
17:43 dalek rakudo: dbebac0 | pmichaud++ | src/parser/grammar.pg:
17:43 dalek rakudo: Update "module Foo;" to allow statements before it.
17:43 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/d​bebac006933cb30b4d3bc9908182641a86fb644
18:02 AndyA joined #parrot
18:16 nnunley joined #parrot
18:32 clinton joined #parrot
18:41 bacek joined #parrot
18:44 sekimura joined #parrot
18:44 clinton left #parrot
19:04 Tene I prefer alias ls='ls -F'
19:06 bacek joined #parrot
19:20 donaldh joined #parrot
19:32 dalek partcl: r444 | coke++ | wiki/SpecTest (2 files):
19:32 dalek partcl: Update 'make spectest' information
19:32 dalek partcl: - collapse separate status files back into one.
19:32 dalek partcl: - update to reflect current status on feather.
19:32 dalek partcl: - mark tests we should skip with @SKIP
19:32 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=444
19:35 mj41 /msg purl karma mj41
19:36 dalek partcl: r445 | coke++ | trunk/tools (2 files):
19:36 dalek partcl: Don't maintain the skip list in two locations.
19:36 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=445
19:52 Whiteknight joined #parrot
19:52 Whiteknight Infinoid: ping
19:58 Infinoid ohai
19:58 Whiteknight I'm not online for long here, but have you heard any other complaints about the io_rewiring branch?
19:58 Whiteknight because I'm going to merge it tonight unless any issues have popped up
19:59 Infinoid No, sounds like partcl and rakudo work, haven't heard anything else about it at all
19:59 Whiteknight okay, hearing nothing is good enough for me
19:59 Whiteknight I want it in so we can hash out any issues tomorrow at #ps
20:00 Infinoid And impress everyone with our performance, hopefully :)
20:00 Whiteknight hopefully
20:00 Whiteknight The 1.3 release should be quite an impressive one
20:01 Whiteknight at least in terms of performance
20:01 dalek partcl: r446 | coke++ | wiki/ParrotIssues.wiki:
20:01 dalek partcl: add another memory related TT.
20:01 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=446
20:02 Su-Shee left #parrot
20:14 Whiteknight okay, talk to you later
20:38 particle1 joined #parrot
21:03 Whiteknight joined #parrot
21:06 Whiteknight Infinoid: ping again
21:06 purl I can't find again in the DNS.
21:13 Whiteknight wow, coretest IS significantly more speedy on my system too
21:13 chromatic Try it with parallel testing.  It's even better.
21:13 Whiteknight how do you do that?
21:14 Whiteknight make -j2 coretest I assume?
21:15 chromatic mj coretest TEST_JOBS=5
21:15 chromatic The important part is TEST_JOBS=n
21:16 chromatic Set that environment variable to tell TAP::Harness now many parallel tests to run.
21:16 chromatic -j2 only affects how many parallel jobs make will run.
21:16 Whiteknight I don't have mj on my system. How to get it?
21:16 chromatic alias mj='make -j9'
21:16 Whiteknight ah
21:17 Whiteknight holy crap this is amazing
21:17 Whiteknight chromatic++
21:17 Whiteknight you may have just optimized me
21:18 chromatic I can run coretest in 45 seconds with a few of the PIO optimizations.  That's a huge improvement.
21:19 Whiteknight how long was it taking you before?
21:19 cotto oh wow
21:19 Whiteknight I think I just ran in in 64 seconds, it used to take me about 10 minutes
21:19 dalek rakudo: acd4cfb | jnthn++ | src/ (2 files):
21:19 dalek rakudo: Fixes and improvements to .^parents, to get it passing all of the current tests in S12-introspection/parents.t. Should now be much more stable than before.
21:19 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/a​cd4cfb44ab76235538ba8792726d91950189c9e
21:19 dalek rakudo: a18b1d3 | jnthn++ | t/spectest.data:
21:19 dalek rakudo: Add S12-introspection/parents.t to spectest.data.
21:19 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/a​18b1d34872c1dd69c9156775b533b496c755939
21:19 Whiteknight maybe closer to 7 minutes, but same ballpark
21:21 chromatic I could run it somewhere between 90 - 120 seconds before.
21:22 dalek parrot: r39454 | whiteknight++ | branches/io_rewiring (60 files):
21:22 dalek parrot: [io_rewiring] Merge from trunk r39444:39453
21:22 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39454/
21:22 Whiteknight We're squeezing that IO system pretty tightly, I don't think there are too many more huge gains to be had from it
21:23 Whiteknight a few small points here or there, but I think we got the bulk of it
21:23 chromatic I'll try to fix a couple of bugs in the calling conventions rewiring branch soon; I have some ideas.
21:24 Whiteknight Last time I talked to allison she said she had a lot of uncommitted changes in that branch, so I haven't been touching it
21:24 Whiteknight but I do hope that one lands soonish
21:24 chromatic She committed those changes but asked that no one commit to it unless they can demonstrate that the patch solves real problems.
21:24 cotto I think she committed them with the caveat they some might need to be rolled back
21:24 Infinoid There's a "FIXME: This is badly optimized, will fixup later" comment in Parrot_io_write_buffer(), I think we could see some reasonable gains from working on that
21:25 chromatic Yeah, that looked like it flushes too aggressively.
21:25 Whiteknight Infinoid: Yeah, there are some places we can squeeze some more performance, but nothing that's going to be "holy shit" faster
21:25 Infinoid chromatic: Did you see my commit from this morning?
21:25 Infinoid chromatic: https://trac.parrot.org/parrot/changeset/39450/
21:25 japhb joined #parrot
21:26 Infinoid heh, yeah.  That fsync() stuff was pretty awful.
21:27 Infinoid The more junk we can get out of the way, the faster IO will happen, that's the name of the game right now
21:27 Whiteknight true, I plan to start another branch tomorrow to work on more of that
21:29 pmichaud fwiw, the faster I/O isn't likely to affect rakudo execution speed much.  It will help with building speed, however.
21:29 pmichaud and I'll want to re-benchmark pbc_to_exe
21:29 Whiteknight pmichaud: any speedups are good speedups
21:29 Whiteknight and then we move to the next subsystem and optimize like maniacs
21:31 Infinoid Whiteknight: oh, uh, pong
21:32 Whiteknight Infinoid: can you delete the Pipe and PipeHandle PMCs from the branch?
21:33 Whiteknight If I do it I'm going to have to rebump PBC_COMPAT and make the packfiles again, and borkage will happen
21:33 Infinoid oki
21:33 Whiteknight thanks (sorry to keep bothering you today!)
21:37 Infinoid Ok.  We still want the PBC_COMPAT bump because of Handle, but I think I can just leave that as-is
21:38 Infinoid (bumping it twice within the same dev branch doesn't seem necessary to me)
21:38 Whiteknight okay, that's fine
21:38 * Whiteknight disappears
21:42 ruoso joined #parrot
21:48 dalek rakudo: 063f3d5 | jnthn++ | src/parser/ (2 files):
21:48 dalek rakudo: Returns traits should parse a fulltypename, not a typename. Fixes a bug spotted by pyrimidine++.
21:48 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/0​63f3d54d284122cd12c7497d57f0c8d7295b61d
21:49 pmichaud also, I should note that I didn't actually test rakudo on the io_rewiring branch -- I just tried out my benchmark.
21:50 jonathan pmichaud: It'll be a win for Rakudo programs doing I/O, which is a nice thing.
21:51 pmichaud jonathan: I'm not sure it'll be a big win (more)
21:51 pmichaud iiuc, the speedup is when doing I/O for native types
21:51 pmichaud when not doing native types, we still end up making PCCINVOKE calls, which dominate the overall cost
21:51 jonathan I thought the biggest win was removing so many calls to fsync?
21:52 pmichaud I can give an idea... just a sec
21:53 Limbic_Region joined #parrot
21:54 pmichaud okay, it does seem to give a speedup even for PMCs
21:54 pmichaud checking more.
21:54 pmichaud a nice speedup, even
21:55 particle1 fsync alone should result in <50% previous timings
21:55 particle1 er, ~50%
21:55 pmichaud not if the previous timings were due to the cost of converting integers to strings, or making method calls on the FileHandle object
21:55 particle left #parrot
21:56 pmichaud Let's put it this way.
21:56 bacek joined #parrot
21:56 pmichaud In trunk, my benchmark of writing 2.5 million integers takes 50+ sec
21:56 particle1 yes, for i/o related activities, of course
21:56 pmichaud before making the fsync fix, the branch had that down to 11 seconds
21:57 pmichaud so that tells me that 39 seconds of time was spent in conversion, not fsyncs (since the number of fsyncs should remain the same in both cases)
21:57 eternaleye joined #parrot
21:57 pmichaud fixing the fsync issue brings it down to about 4 seconds
21:58 chromatic How about when running with -G on the branch now?
21:58 pmichaud well, the benchmark doesn't make pmcs, so I suspect -G doesn't make a difference.
21:59 pmichaud running some tests now.
22:00 particle1 it does if it calls functions
22:00 nopaste "pmichaud" at 72.181.176.220 pasted "Pm's 2.5 million integer benchmark" (20 lines) at http://nopaste.snit.ch/16833
22:00 pmichaud it doesn't call functions.
22:00 pmichaud in trunk, that benchmark requires 48.5 seconds (wall)
22:01 pmichaud in io_rewiring, it requires 8.5 seconds (wall)
22:01 pmichaud but most of that speed improvement is due to avoiding the internal PCCINVOKEs
22:02 pmichaud surprisingly!
22:02 pmichaud converting the benchmark to output a PMC Integer instead of a register integer causes it to run in 3.5 seconds
22:02 pmichaud I have no clue why.
22:02 particle faster with pmc's.
22:02 particle interesting.
22:03 jonathan How is it with a PMC in trunk?
22:03 jonathan Before the io_rewiring?
22:03 nopaste "pmichaud" at 72.181.176.220 pasted "running with PMC instead of integer register" (30 lines) at http://nopaste.snit.ch/16834
22:03 chromatic That sounds like a job for callgrind.
22:04 pmichaud That tells me that   Parrot_io_print_p_i is very suboptimal somehow.
22:04 pmichaud if an Integer PMC can be stringified faster than an int register.... that's.... weird.
22:04 pmichaud trying in trunk
22:04 chromatic STRING * const s = Parrot_sprintf_c(interp, INTVAL_FMT, $2);
22:04 chromatic Parrot_io_putps(interp, $1, s);
22:05 chromatic Oh look, varargs.  Hooray.
22:05 pmichaud aha!
22:05 pmichaud intger.pmc uses
22:05 pmichaud return Parrot_str_from_int(INTERP, SELF.get_integer());
22:05 pmichaud obviously Parrot_str_from_int is faster.
22:06 chromatic Lots.
22:06 jonathan Any reason not to use that in the op?
22:06 pmichaud in trunk:  37.2 seconds using the PMC   (was 48.5 seconds using the int register)
22:06 pmichaud I'll try converting the op and see what we get
22:06 chromatic I'm testing it too.
22:06 Infinoid pmichaud: It's still taking 8 seconds in branch for you?  fsync knocked it down to 0.7 here
22:07 Infinoid (yes, the majority of the improvement was still PCCINVOKE)
22:07 pmichaud Infinoid: was that the "print an integer" test or the "print a '000,' string"  one?
22:07 pmichaud when printing only strings I got down to 0.7, yes.
22:07 pmichaud but printing integers is still slow (always was)
22:08 Infinoid Actually, I didn't try integers today
22:08 pmichaud right, I know that printing strings is fast.  No conversion needed.
22:08 Infinoid ok, I remember now, converting integers required allocating lots of STRINGs
22:08 chromatic So does printing PMCs.
22:10 chromatic All tests pass with that op change.
22:10 pmichaud changing the op on my system brings the time down to 3.3 seconds (was 8.5)
22:11 pmichaud (I only made the conversion in the one op -- looks like much of io.ops needs updating)
22:11 cognominal left #parrot
22:11 pmichaud notably, making those changes means that Parrot's IO is faster than p5's
22:11 pmichaud (for this benchmark)
22:12 chromatic Are you using an optimized Parrot?
22:12 particle whee!
22:12 pmichaud no.
22:12 chromatic Assume it's some 15% faster then.
22:12 pmichaud the equivalent p5 on my system requires 4.6 seconds
22:13 chromatic If your Perl 5 is a threaded build, Parrot's still faster than unthreaded Perl 5 (if you build Parrot with optimizations).
22:14 pmichaud anyway, I think we probably want to do a search for INTVAL_FMT and see if there are many places where we can change it to use Parrot_str_from_int  instead
22:14 chromatic Let's merge in the branch first and then gild it!
22:14 chromatic Not geld, mind you.  We're ungelding it.
22:14 pmichaud looks like not too many instances -- just the few in io.ops
22:15 pmichaud two, really.
22:17 pmichaud so, back to the bigger picture -- the biggest slowdown in trunk remains the use of PCCINVOKE for I/O, which the branch eliminates
22:18 pmichaud fixing fsync gets us a bit faster than that, but the fsync improvement isn't as big as the PCCINVOKE one
22:21 particle that makes sense, as there are generally more function calls than writes
22:21 bacek joined #parrot
22:22 bacek good morning #parrot
22:23 Theory joined #parrot
22:27 eternaleye joined #parrot
22:27 kid51 joined #parrot
22:28 GeJ Good morning everyone
22:31 cognominal joined #parrot
22:32 rg joined #parrot
22:33 sekimura_ joined #parrot
22:33 * particle updates from 36154 to head
22:38 dalek parrot: r39455 | Infinoid++ | branches/io_rewiring (2 files):
22:38 kid51 seen allison?
22:38 purl allison was last seen on #parrot 5 days, 21 hours, 1 minutes and 21 seconds ago, saying: and, yes, base_type comparisons will work fine for this (and be very fast)  [Jun  3 01:33:45 2009]
22:39 dalek parrot: Add a couple of files to branch that are in trunk (and MANIFEST) but apparently weren't svn added.
22:39 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39455/
22:39 dalek parrot: r39456 | Infinoid++ | branches/io_rewiring/src/pmc (2 files):
22:39 dalek parrot: Remove Pipe and PipeHandle PMCs, they aren't ready for prime time yet.
22:39 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39456/
22:39 dalek parrot: r39457 | Infinoid++ | branches/io_rewiring/t/native_pbc (5 files):
22:39 dalek parrot: Regenerate native_pbc files.
22:39 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39457/
22:39 dalek parrot: r39458 | Infinoid++ | branches/io_rewiring/PBC_COMPAT:
22:39 dalek parrot: PBC_COMPAT version 4.8 just gets Handle, not Pipe or PipeHandle
22:39 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39458/
22:39 Infinoid Whiteknight: It's all yours.
22:55 Austin_Hastings left #parrot
22:58 darbelo cotto: ping
23:03 cotto darbelo, pong
23:07 tetragon joined #parrot
23:09 darbelo I was trying to gauge the DecNum completion by comparing it with the other numeric PMCs and I think implementing visit, freeze and thaw are our only unimplemented VTABLEs.
23:09 cotto That
23:09 cotto That's quite possible.
23:10 darbelo So I figured I might as well implement them. And then I realize I don't really know what the visit VTABLE is supposed to be doing.
23:11 cotto it's a little confusing.  I remember having some trouble with it too.
23:11 cotto lemme dig up my notes
23:12 darbelo So far, the src/dynpmcs dir wasn't very helpful. Only one pmc there implements it and it's only call to SUPER().
23:13 Coke joined #parrot
23:13 darbelo Looking at src/pmc now.
23:13 Coke joined #parrot
23:14 Coke hio
23:15 chromatic Did you try Partcl with the fsync optimization on the IO branch?
23:16 Infinoid By the way, are there any HLLs out there other than Rakudo which can build from a separate (non-installed) Parrot build directory?
23:16 Infinoid I've tried partcl and lua, and they both require an installed Parrot.  (Which I don't want to do 5 times a day whenever I want to test something.)
23:17 chromatic Pheme can, but it doesn't do much.
23:17 chromatic Pynie probably can.
23:19 Tene Infinoid: steme can
23:19 Tene cardinal
23:19 purl hmmm... cardinal is http://mail.freesoftware.fsf​.org/pipermail/cardinal-dev/ or the Ruby-on-Parrot project. or http://xrl.us/uyz3
23:19 Infinoid Cool, thanks.  That'll help widen my test pool
23:19 Tene Infinoid: I just install to ~/parrot
23:19 Tene fwiw
23:19 cotto darbelo, VTABLE_visit is called before both freeze and thaw and should do one of two things, depending on visit_info->what:
23:20 nopaste "kid51" at 68.237.12.237 pasted "io_rewiring branch: failures in 2 test files" (31 lines) at http://nopaste.snit.ch/16835
23:20 donaldh joined #parrot
23:21 Coke Infinoid: installing parrot is not that much of a lift.
23:21 chromatic Pipe is gone, so the test should disappear too.
23:21 Infinoid Coke: It is when I want to compare trunk vs branch, side by side
23:21 Infinoid chromatic: Just committed that.
23:21 kid51 Nothing at all in this file in io_rewiring branch:  t/op/arithmetics_pmc.t
23:21 dalek parrot: r39459 | Infinoid++ | branches/io_rewiring/t/pmc/pipe.t:
23:21 dalek parrot: Pipe is gone, remove pipe.t.
23:21 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39459/
23:22 cotto actually, you don't need to worry about visit for a non-aggregate PMC.
23:23 Coke chromatic: I just did a plain run of the spec suite to get a baseline.
23:23 Coke chromatic: it's been 4.5 months since I did that; will run against the io branch shortly.
23:23 kid51 What about arithmetics_pmc.t ?
23:23 Infinoid kid51: That and an NQP test were improperly merged in from trunk, I'll fix that
23:23 kid51 Infinoid:  Thanks.  I wanted to time that branch's make coretest but couldn't due to failures.
23:24 darbelo cotto: "don't need to worry" == "Leave unimplmented" ?
23:24 cotto yup
23:24 darbelo One down, two to go. ;)
23:24 cotto Your code should be similar to the freeze/thaw code in src/pmc/integer.pmc
23:24 dalek parrot: r39460 | Infinoid++ | branches/io_rewiring (2 files):
23:24 dalek parrot: Copy in 29-self.t and arithmetics_pmc.t from trunk; the files I added in r39455 were apparently empty.
23:24 Infinoid kid51: If you can update to r39460, it will hopefully only update tests and not require a clean/realclean to re-test
23:24 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39460/
23:25 cotto To freeze, just push stuff onto visit_info->image_io.
23:26 Infinoid kid51: I haven't compared without --optimize, but the difference should be striking either way
23:26 darbelo What sould I do with the DecNumContext pointer? Freezing that seems like asking for a segfault to me.
23:26 cotto Hmm.
23:27 chromatic What's it store?
23:27 darbelo A pointer to another PMC.
23:27 cotto Storing it is pretty simple (just a bunch of ints iirc), but it's a singleton.
23:31 jonathan I think freezing pointers to other PMCs is what visit is about.
23:31 jonathan But last time I played with a visit routine I just made lots of segfaults... :-/
23:35 * kid51 doesn't --optimize
23:35 cotto That means that thawing a DecNumber PMC will either result in context inconsistency or the side-effect of changing the current context.
23:37 davidfetter bon appetit, kid51_at_dinner
23:37 cotto I guess the context and number PMCs will need to be frozen/thawed separately and that the docs will need to warn about posible inconsistency if the programmer isn't careful.
23:41 particle feh.  on windows, nmake smoke ; nmake smoke rebuilds parrot
23:41 darbelo cotto: I could change from storing a PMC pointer in an ATTR to always pmc_new()ing the context.
23:41 * particle bikes 11 miles &
23:41 patspam joined #parrot
23:42 cotto darbelo, I'd look into the efficiency of that.  It might make more sense to use pmc_new to populate the ATTR when thawing the context.
23:43 darbelo Efficiency is why I put the ATTR there :)
23:44 darbelo pmc_new on thaw sounds good, but I'm not sure how thaw()ing a singleton works.
23:44 darbelo I someone creates a context and then thaws a frozen one, does it get overwritten?
23:45 cotto That's tricky.  The answer is either "yes" or "no, and there may be a context mismatch".
23:47 jonathan What sort of things does the context hold?
23:47 jonathan And what would the upshot of a mismatch be?
23:50 darbelo A pointer to a structure. Thing is the semantics of the PMC rely on the singleton-ness of the context.
23:50 cotto http://speleotrove.com/decimal/dncont.html - number of digits, exponent min/max, error info, rounding mode
23:50 cotto I think the best option may be to implement thawfinish as a place to do a sanity check on the newly-thawed PMC.
23:51 cotto (thawfinish is called after thaw to do any cleanup work)
23:51 cotto actually, you could do that during thaw too
23:51 cotto s/too/instead/
23:52 pmichaud jonathan: do you expect to be around tomorrow?
23:52 darbelo I guess I could just pmc_new() a location for us and overwrite the structure, but I'm not sure a context can survive a gc run without a DecNum to mark it as live.
23:53 cotto Without looking at the code, I'd assume that singletons wouldn't get GC'd until the interp exits.
23:53 * cotto goes to check
23:57 cotto Whiteknight, ping
23:58 jonathan pmichaud: Yes
23:59 jonathan pmichaud: Got Slovak class early afternoon, may have to run an errand to the bank in the morning.
23:59 pmichaud I'm planning to work on rakudo building from installed parrot tonight -- I'll probably need some testing on windows platforms
23:59 jonathan pmichaud: But other than those, should be.
23:59 jonathan OK
23:59 jonathan Do you plan to keep it possible to build Rakudo against a Parrot build tree too, or installable only?
23:59 pmichaud I hope to be able to use both

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced