Camelia, the Perl 6 bug

IRC log for #parrot, 2009-05-30

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:04 kesselhaus joined #parrot
00:05 kid51 joined #parrot
00:16 dalek decnum-dynpmcs: r71 | darbelo++ | trunk/src/pmc/decnum.pmc:
00:16 dalek decnum-dynpmcs: Add pow, pow_int and pow_float VTABLEs.
00:16 dalek decnum-dynpmcs: review: http://code.google.com/p/decn​um-dynpmcs/source/detail?r=71
00:21 bacek purl: stupid girl
00:21 purl bacek: sorry...
00:21 bacek purl forget libraries
00:21 purl bacek: I forgot libraries
00:25 Whiteknight watching chromatic++ patching leaks is better then watching any TV
00:25 dalek TT #720 closed by jkeenan++: [PATCH] Typos in pdd19_pir.pod
00:26 Whiteknight well, maybe not as good as watching Lost, but he's only one man
00:27 bacek I hope he will not lost
00:29 dalek parrot: r39244 | jkeenan++ | trunk/docs/pdds/pdd19_pir.pod:
00:29 dalek parrot: Correct POD errors, including those described by Klaus in
00:29 dalek parrot: https://trac.parrot.org/parrot/ticket/720.
00:29 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39244/
00:33 Whiteknight darbelo is kicking ass
00:33 Whiteknight darbelo++
00:34 darbelo Heh. Actually it was apretty slow day coding-wise.
00:36 darbelo Most VTABLE have adirect mapping to a decNumber function. They're pretty easy to do.
00:36 particle2 joined #parrot
00:46 whoppix joined #parrot
00:49 contingencyplan joined #parrot
00:49 dalek parrot: r39245 | jkeenan++ | trunk/src/oo.c:
00:49 dalek parrot: Make file to c_parens coding standard.
00:49 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39245/
00:50 kid51 Does anyone know how to fix failures in t/codingstd/c_arg_assert.t?
00:50 kid51 We've got 9 failures in this file:  include/parrot/runcore_api.h
00:50 bacek kid51: make headerizer?
00:51 kid51 msg chromatic include/parrot/runcore_api.h has errors in t/codingstd/c_arg_assert.t
00:51 purl Message for chromatic stored.
00:51 eternaleye joined #parrot
00:56 dalek parrot: r39246 | jkeenan++ | trunk/lib/Parrot/Manifest.pm:
00:56 dalek parrot: Make file pass perlcritic by adding coda.
00:56 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39246/
01:09 whoppix joined #parrot
01:09 dalek parrot: r39247 | chromatic++ | trunk/src/interp/inter_create.c:
01:09 dalek parrot: [src] Cleaned up the dynops destruction invocation code so that prototypes
01:09 dalek parrot: match.
01:09 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39247/
01:09 dalek parrot: r39248 | chromatic++ | trunk/src/runcore/main.c:
01:09 dalek parrot: [src] Annotated runcore functions to pass c_arg_assert.t coding standards test.
01:09 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39248/
01:10 wayland76 joined #parrot
01:11 amoc joined #parrot
01:15 bacek here will be dragons.
01:16 dalek parrot: r39249 | bacek++ | trunk/src/pmc/scalar.pmc:
01:16 dalek parrot: [pmc] Scalar.bitwise_xor isn't MULTI.
01:16 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39249/
01:16 dalek parrot: r39250 | bacek++ | trunk/src/pmc/bignum.pmc:
01:16 dalek parrot: [pmc] Cleanup BigNum's float handling a bit.
01:16 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39250/
01:16 dalek parrot: r39251 | bacek++ | trunk/t/pmc/multidispatch.t:
01:16 dalek parrot: [t] Mark t/pmc/multidispatch tests with TODO TT#452 for future review and remove.
01:16 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39251/
01:16 dalek parrot: r39252 | bacek++ | trunk/src/pmc/bigint.pmc:
01:16 dalek parrot: [pmc][cage] Fix BigInt pmc_reuse calls. We can't reuse Null.
01:16 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39252/
01:20 dalek parrot: r39253 | bacek++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
01:20 dalek parrot: Initial version of switch dispatch for MULTI
01:20 dalek parrot: For rationale of this commit see TT#452. It should give massive speed boost
01:20 dalek parrot: for basic (arithmetic) functions which uses only core PMCs. E.g.
01:20 dalek parrot: examples/benchmark/primes2.pir for 5000 elements executed in ~10 seconds
01:20 dalek parrot: instead of ~150 seconds.
01:20 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39253/
01:20 dalek parrot: r39254 | bacek++ | trunk/t/dynpmc/foo.t:
01:20 dalek parrot: [t] Mark dynpmc failing test.
01:20 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39254/
01:20 bacek msg chromatic can you review r39253 please?
01:20 purl Message for chromatic stored.
01:27 bacek afk # will be back in few hours
01:35 uniejo joined #parrot
01:40 dalek parrot: r39255 | whiteknight++ | branches/io_rewiring/src (3 files):
01:40 dalek parrot: [io_rewiring] convert the Parrot_io_reads function
01:40 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39255/
01:42 * afk_Coke does a partcl build to see if anyone's been causin' trouble.
01:44 dalek parrot: r39256 | bacek++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
01:44 dalek parrot: [pmc2c] Fallback to MMD for non-core PMCs.
01:44 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39256/
01:46 bacek joined #parrot
01:46 Coke bacek: I think you broke partcl.
01:47 bacek Coke: It works. After r39257.
01:47 bacek make test still running.
01:48 Coke I just updated to r...4
01:48 Coke reupdating...
01:48 bacek r..7 was a fix for dynpmcs.
01:48 dalek parrot: r39257 | bacek++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
01:48 dalek parrot: [pmc2c] Fix handling of dynpmc in switch-based VTABLE. Fix pcc signatures for fallback.
01:48 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39257/
01:48 dalek parrot: r39258 | bacek++ | trunk/t/dynpmc/foo.t:
01:48 dalek parrot: [t] Remove passing todo in dynpmc/foo.
01:48 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39258/
01:48 bacek here. see! :)
01:49 bacek now definitely have to go.
01:49 bacek Feel free to comment invocation of gen_switch_vtable in PMCEmitter.pm if it still broken.
01:51 Coke hurm. again, feels faster.
01:51 Coke need to stat timing more things. =-)
02:00 Coke I am assuming I'm not freeing some references to some PMCs. is there a util to show me where all the references are?
02:02 dukeleto joined #parrot
02:04 Coke hurm. I'm not running out of memory quite so soon, it would seem.
02:15 dalek partcl: r395 | coke++ | trunk/config/makefiles/root.in:
02:15 dalek partcl: add a shortcut for checkout out the entire tcl repo from CVS.
02:15 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=395
02:16 Coke I read my commit messages and sends and am horrified by my grammar.
02:24 * kid51 quits to do Safari upgrade
02:39 dalek partcl: r396 | coke++ | trunk:
02:39 dalek partcl: ignore generated dir
02:39 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=396
02:39 dalek partcl: r397 | coke++ | trunk:
02:39 dalek partcl: ignore correct generated dir
02:39 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=397
02:39 dalek partcl: r398 | coke++ | trunk:
02:39 dalek partcl: ignore generated file
02:40 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=398
02:40 dalek partcl: r399 | coke++ | trunk/README:
02:40 dalek partcl: hey, we can build now.
02:40 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=399
02:41 janus joined #parrot
02:43 whoppix joined #parrot
02:53 mikehh_ joined #parrot
02:53 tetragon_ joined #parrot
03:01 pmichaud fwiw, Rakudo seems to have gotten faster since r39239
03:01 pmichaud (which itself was a significant speed improvement over r39238)
03:04 pmichaud as of r39239, rakudo "make spectest" was 28m45.   Now it's 26m36
03:05 dalek rakudo: 7d75524 | pmichaud++ | build/PARROT_REVISION:
03:05 dalek rakudo: Bump PARROT_REVISION to get some more speed improvements.
03:05 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/7​d75524aebd74fbd6dc512475dc0ea67b20eee96
03:08 ZeroForce joined #parrot
03:29 ZeroForce left #parrot
03:29 ZeroForce joined #parrot
03:31 ZeroForce joined #parrot
03:34 wayland76 joined #parrot
03:57 Theory joined #parrot
04:02 dalek rakudo: 764684b | pmichaud++ | src/builtins/op.pir:
04:02 dalek rakudo: Fix cross-meta for user-defined infix ops.
04:02 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/7​64684b042379b0c0da26e456141ee07e3a3fbf4
04:29 darbelo joined #parrot
04:52 darbelo joined #parrot
04:57 tetragon joined #parrot
05:01 snarkyboojum joined #parrot
05:34 wayland76 joined #parrot
06:11 Theory joined #parrot
06:39 cognominal joined #parrot
06:47 wayland76 joined #parrot
06:58 muixirt joined #parrot
07:19 iblechbot joined #parrot
08:45 david joined #parrot
09:13 snarkyboojum joined #parrot
09:16 snarkyboojum hi all, I've downloaded and built parrot '1.2.0-devel built for nojit'  on OS X and am trying to 'make' rakudo, but it's dying in the first step 'make: *** [perl6_s1.pbc] Segmentation fault' - how do I begin to debug?
09:16 snarkyboojum any ideas?
09:21 ZuLuuuuuu joined #parrot
09:29 pjcj joined #parrot
09:29 cotto Parrot doesn't come with Rakudo.  Are you using --gen-parrot on Rakudo Configure.pl or are you working with an installed Parrot?
09:30 cotto also, where did you get Parrot from?  Rakudo doesn't currently work against an installed Parrot unless the original source is there too.
09:33 cotto Your best bet it to down a Rakudo release (or get it from git) and run perl Configure.pl --gen-parrot.  That'll download and build a known-good version of Parrot for you.
09:34 snarkyboojum yep - using --gen-parrot on Rakudo Configure.pl
09:35 snarkyboojum grabbed the latest tar.gz from http://github.com/rakudo/rakudo/downloads
09:36 snarkyboojum so parrot builds ok, i.e. running perl Configure.pl --gen-parrot, but running make to build Rakudo fails with above message
09:37 cotto ok.  You don't appear to be doing anything unusual.
09:37 snarkyboojum e.g. running ~/rakudo-2009-05/parrot/parrot -V gives me 'This is Parrot version 1.2.0-devel built for nojit. ...' etc
09:38 cotto Is there any way there's something left over from a previous build?
09:38 snarkyboojum I have a previous parrot installed from source I think
09:38 snarkyboojum -- /usr/local/bin/parrot
09:38 cotto That shouldn't be messing anything up.
09:39 snarkyboojum which is a 'This is parrot version 0.5.0-devel (r23586)'
09:39 cotto *shouldn't*
09:39 snarkyboojum yeah..
09:39 cotto I'd ditch that anyway.  That's ancient.
09:39 snarkyboojum yeah :)
09:41 cotto I saw a similar error on my machine (jaunty x86) when I was building Rakudo with an installed Parrot but had modified the source without installing the changes.
09:42 snarkyboojum googled it and found some references to that error on Linux boxes, but didn't get anywhere with a solution
09:42 snarkyboojum was wondering how I'd start debugging such a problem
09:43 snarkyboojum it's definitely 'parrot  -o perl6_s1.pbc perl6.pir' which is failing
09:43 snarkyboojum but no core dumps.. etc
09:43 cotto First, I'd get rid of that dusty old Parrot, run make realclean and rebuild Parrot and Rakudo.  If the problem persisted, I'd use gdb.
09:43 snarkyboojum ok
09:44 snarkyboojum not even sure what to hook gdb into
09:44 cotto Have you used gdb before and do you have it installed?
09:44 snarkyboojum used it to debug C programs
09:44 snarkyboojum it's installed on my OS X machine yeah
09:45 cotto If it explodes again, feel free to nopaste a backtrace.
09:45 snarkyboojum cheers, will check it out a bit later
09:45 snarkyboojum thanks
09:46 cotto ok.  I'll probably be alseep by then, but it's not hard to find someone helpful here.
09:50 Austin_Hastings joined #parrot
09:54 wayland76 joined #parrot
10:08 dalek parrot: r39259 | fperrad++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
10:08 dalek parrot: [codingstd] remove trailing space
10:08 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39259/
10:17 snarkyboojum hi cotto - you still around?
10:18 cotto yup
10:18 snarkyboojum I'm attempted to use gdb to get a backtrace
10:18 snarkyboojum haven't cleaned the old parrot yet
10:20 snarkyboojum I'm basically done a 'gdb parrot' and then done 'run -o perl6_s1.pbc perl6.pir'
10:20 snarkyboojum not sure if that's the correct way to do things :D
10:20 cotto that's what I'd do
10:20 snarkyboojum ok - and I get this error
10:20 snarkyboojum Program received signal EXC_BAD_ACCESS, Could not access memory.
10:20 snarkyboojum Reason: KERN_INVALID_ADDRESS at address: 0x72704620
10:20 snarkyboojum 0x72704620 in ?? ()
10:21 snarkyboojum oops - apologies for multiline spam
10:21 snarkyboojum can I post a backtrace here?
10:21 cotto we prefer nopaste
10:21 cotto nopaste?
10:21 clunker3 http://pasta.test-smoke.org/ or http://paste.husk.org/ or http://nopaste.snit.ch:8001/ or http://rafb.net/paste or http://poundperl.pastebin.com/ or http://paste.scsys.co.uk/
10:21 purl i heard nopaste was at http://nopaste.snit.ch/ (ask TonyC for new channels) or http://poundperl.pastebin.com/ or http://paste.scsys.co.uk/ or App::Nopaste or tools/dev/nopaste.pl or at http://www.extpaste.com/ or http://paste.scsys.co.uk (for #catalyst, #dbix-class, #moose  and others) or http://gist.github.com/
10:21 snarkyboojum ok
10:22 snarkyboojum eek - which one is appropriate for here?
10:22 Austin_Hastings Whichever.
10:22 purl MAKE A DECISION
10:23 cotto any works.  You can also use tools/dev/nopaste.pl, which is nice if you've got your output in a file
10:23 Austin_Hastings You paste to the site, it gives you a temporary url, you paste the temp url here.
10:23 snarkyboojum http://poundperl.pastebin.com/d3c30f64e
10:23 snarkyboojum will that work?
10:23 Austin_Hastings I see a stack tract
10:23 Austin_Hastings * trace
10:23 snarkyboojum coolio
10:23 snarkyboojum there you have it
10:24 Austin_Hastings Looks like it goes off the rails in mmd.
10:24 Austin_Hastings Were you just running standard code, or have you changed anything?
10:24 snarkyboojum multidispatch.c 711 :) nfi
10:24 snarkyboojum standard yeah
10:25 snarkyboojum essentially d/l http://cloud.github.com/downloads/r​akudo/rakudo/rakudo-2009-05.tar.gz and followed the README on OS X
10:26 Austin_Hastings Okay. I don't know what the protocol is for this.
10:26 cotto I don't immediately know what the problem is.  I'd recommend trying to catch someone more familiar with OSX.
10:27 cotto Coke might be able to help.
10:28 snarkyboojum okydoke
10:28 snarkyboojum any bug tracking type place I can put a dump?
10:30 Austin_Hastings trac.parrot
10:33 snarkyboojum ok - thanks again
10:34 dalek parrot: r39260 | fperrad++ | trunk/lib/Parrot/Pmc2c/PMCEmitter.pm:
10:34 dalek parrot: [codingstd] fix perlcritic
10:34 dalek parrot: String delimiter used with "split"
10:34 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39260/
10:50 mikehh joined #parrot
11:03 Infinoid good morning
11:03 purl Here I am, brain the size of a planet, and all they say is 'Good Morning'
11:12 david That address looks suspect: 0x72704620. Each byte is ascii ('rpF ' in big endian, ' Fpr' in little endian). Not sure that means anything to anyone...
11:20 Infinoid msg whiteknight Hmm.  src/io/io_private.h does have some flags for file/pipe/socket; but of course the interface to access those (Parrot_io_set_flags/Parrot_io_get_flags) is specific to FileHandles right now
11:20 purl Message for whiteknight stored.
11:24 Infinoid msg whiteknight Oh, there's a console bit too.  That's interesting to me, there's a whole bunch of tty access crap (ioctls and functions and constants and such) that we can foist off into a subclass and keep the core clean
11:24 purl Message for whiteknight stored.
11:26 iblechbot joined #parrot
11:28 skids joined #parrot
12:47 bacek joined #parrot
12:47 Whiteknight joined #parrot
12:48 bacek joined #parrot
12:50 bacek hi there
12:50 purl niihau, bacek.
12:53 nopaste "Infinoid" at 24.182.55.77 pasted "packfile test failures in io_rewiring branch: apparently the PBC thaw code doesn't like a "type" value of 0" (35 lines) at http://nopaste.snit.ch/16727
12:53 Infinoid (whatever that means.)
12:54 jonathan privet, bacek # can't be bothered to go find rusky keyboard ;-)
12:54 jonathan Infinoid: Half the time, those disappear after a make realclean.
12:54 bacek jonathan: добрый вечер :)
12:55 bacek No complains about my "switch-based" VTABLE for MULTI?
12:55 jonathan вечер? Tam mozet byt. ;-)
12:55 Infinoid jonathan: Interesting.  Any idea what the mechanism behind it is?
12:55 Infinoid (they don't disappear here, some branch change caused it)
12:56 jonathan Infinoid: Normally it's PBC compability breaks that cause it.
12:56 jonathan Infinoid: It's possible to cause it by screwing up a freeze/thaw routine though.
12:56 jonathan Or visit.
12:57 Infinoid well, we've been screwing with various I/O-related PMCs
12:57 Infinoid and the lower level functions in src/io/
12:57 Infinoid There are also some tweaks to not use PCCINVOKE as often
12:58 * Infinoid tries dcommit on a branch for the first time
12:58 jonathan Infinoid: Which test fails?
12:59 Infinoid t/pmc/packfile*
12:59 jonathan It's possible that the I/O code is broken enough to have read the Wrong Thing but I suspect you'd have seen much more serious issues by now.
12:59 jonathan Ah.
12:59 Infinoid the "_pbc()" helper function fails to thaw a PBC file into memory for testing, so all the tests relying on that fail
13:00 jonathan Is that PBC file something you've generated?
13:00 jonathan Or a pre-existing?
13:01 Infinoid it's pre-existing.  t/native_pbc/number_1.pbc
13:01 Infinoid Maybe we just need to regenerate that
13:01 jonathan Oh yes
13:01 jonathan You've been re-organizing PMCs?
13:01 jonathan Removing and adding some?
13:01 jonathan In that case you will have broken PBC compatibility, should add an entry to PBC_COMPAT and should regenerate those.
13:02 Infinoid yeah, mostly adding
13:02 Infinoid ok, thanks
13:02 jonathan This will bump up the bytecode version.
13:02 jonathan Way still want a make realclean after that (as will everyone post-branch-merge probably)
13:03 Infinoid realclean is a good policy anyway
13:03 jonathan aye :-)
13:03 dalek parrot: r39261 | Infinoid++ | branches/io_rewiring (6 files):
13:04 dalek parrot: [io] Create an abstract close_piohandle() function in the main I/O API, so the various PMCs' destroy() vtables can call that directly.
13:04 dalek parrot: (This needs an implementation on win32; I've only handled unix so far.)
13:04 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39261/
13:04 dalek parrot: r39262 | Infinoid++ | branches/io_rewiring (4 files):
13:04 dalek parrot: [io] Make PIO_INVALID_HANDLE available to more than just the FileHandle PMC.
13:04 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39262/
13:04 dalek parrot: r39263 | Infinoid++ | branches/io_rewiring (3 files):
13:04 dalek parrot: [io] Add a pipe creation function.  (This needs implementation on win32; I've only handled unix so far.)
13:04 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39263/
13:04 dalek parrot: r39264 | Infinoid++ | branches/io_rewiring/src/pmc (3 files):
13:04 dalek parrot: [io] Minor cleanups:
13:04 dalek parrot: * The Handle PMCs do not appear to need_ext.
13:04 dalek parrot: * The Socket POD had some cutpaste errors from FileHandle.
13:04 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39264/
13:04 dalek parrot: r39265 | Infinoid++ | branches/io_rewiring/src/pmc/pipehandle.pmc:
13:04 dalek parrot: [io] Create a PipeHandle PMC representing a pipe/fifo endpoint.
13:04 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39265/
13:04 dalek parrot: r39266 | Infinoid++ | branches/io_rewiring/src/pmc/pipe.pmc:
13:04 dalek parrot: [io] Add the Pipe PMC.
13:04 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39266/
13:04 dalek parrot: r39267 | Infinoid++ | branches/io_rewiring (2 files):
13:04 dalek parrot: [io] Implement Parrot_io_close_piohandle() and PIO_PIPE() on win32, too.
13:04 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39267/
13:05 Whiteknight thanks jonathan++
13:05 jonathan :-)
13:05 * jonathan goes back to his pile of Slovak emails
13:05 Whiteknight I sort of figured the answer was something like that, but I haven't bumped PBC_COMPAT yet
13:06 Whiteknight Infinoid++ !!
13:08 Whiteknight This IO work is going to be a big improvemet, I think
13:09 * Infinoid should probably write some pipe tests
13:10 Whiteknight Once we get pipes working, we could probably rewrite parts of the test harness to use them, to avoid relying on Perl5
13:10 Whiteknight Parrot calling Parrot, what a thing!!
13:11 Infinoid strange concept, indeed
13:11 * bacek passing pipe with good tobacco to Infinoid. Test it, it's really good :)
13:11 bacek Whiteknight: check pmc_pct branch. Parrot building Parrot :)
13:14 Whiteknight bacek++ quite impressive
13:15 Whiteknight I'm about to get on a plane here, so I can't check too mch
13:15 Whiteknight and I want to save some battery for the flight so I can parrothack midflight
13:15 Infinoid have a safe trip
13:15 Whiteknight bacek: How close is the pmc_pct branch to being merged to trunk?
13:15 Whiteknight thanks!
13:15 Whiteknight I'm actually going to disappear now. I'll talk to you guys later
13:16 bacek Whiteknight: it can be merged right now.
13:16 bacek It's just not very useful ATM.
13:16 Whiteknight okay, thanks
13:29 kid51 joined #parrot
13:37 bacek Infinoid: around?
13:39 dalek parrot: r39268 | bacek++ | trunk/src/pmc/integer.pmc:
13:39 dalek parrot: [pmc] Reuse "dest" pmc in Integer arithmetics when possible.
13:39 dalek parrot: This reducing GC pressure for basic Integer operations. primes2.pir now executed
13:39 dalek parrot: 30% faster for calculating 5000 primes.
13:39 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39268/
13:42 burmas joined #parrot
13:44 jsut joined #parrot
13:45 Infinoid bacek: yeah, what's up?
13:45 burmas left #parrot
13:46 bacek Infinoid: I'm going to make "reuse_or_new" from r39268 as new public functions. Any objections>
13:46 bacek ?
13:48 Infinoid I think the semantics are a little weird... in one case (reuse) other references to the same PMC will also be affected, in the other case (new) they won't be
13:50 bacek Not quite true. It's useful for 3-args ops in PMCs.
13:50 bacek For example currently most PMCs will fail on "add", when will try to add to "ro" pmc.
13:51 bacek It's just helper function to check corner cases
13:51 Infinoid true.  (then again, "ro" support seems to be really sketchy in parrot; I honestly don't even know how to create one of those.)
13:51 Infinoid actually I like what you've done in that r39268 patch, will reduce GC churn from creating new objects
13:52 bacek t/pmc/ro.t
13:52 purl t/pmc/ro.t is failing for me. It was passing for me last night
13:52 bacek purl: forget t/pmc/ro.t
13:52 purl bacek: I forgot t/pmc/ro.t
13:53 bacek Infinoid: I actually want it "public" because I want to use it in all PMCs.
13:54 Infinoid Making the function global probably wouldn't hurt.  It also wouldn't hurt to ask the list how useful it is
13:54 bacek without copy-pasting every time...
13:54 Infinoid It will need some nice POD of course, so the callers can understand the side effects :)
13:54 bacek It is useful :)
13:55 bacek But I'll fail in "nice POD". English in forth language, remember? :)
13:55 Infinoid maybe you'd be more comfortable trying Rude English or Very Very Rude English :)
13:57 bacek "We are f**cing going to kill dest. Try to preserve some guts"
13:57 jonathan bacek++
13:58 jonathan (That's only Rude English though. ;-))
13:58 bacek jonathan: there is always way to improve something. Even my Rude English :)
13:59 * jonathan remembers the time a Ukrainian guy tried to teach him that "жопа" means "heart"
14:00 * bacek ROTFL
14:01 bacek Parrot_pmc_try_reuse or Parrot_pmc_reuse_or_new?
14:02 bacek any better names?
14:02 * jonathan prefers the first.
14:03 Andy joined #parrot
14:04 bacek deal
14:05 Infinoid Does rakudo use parrot threads for much, yet?
14:05 bacek Infinoid: not yet.
14:06 jonathan Infinoid: No, because last time I tried I just got segfaults and oddness.
14:06 Infinoid Ah.  Thanks, I was wondering whether these segfaults and oddness I'm seeing were due to my having used it wrong
14:07 jonathan Infinoid: I owe allison an attempted implementation of async for Rakudo, so she can see the issues we're running in to. Just didn't get to it yet.
14:08 Infinoid I'm writing a test for pipes, but I get an assertion failure when creating a thread.  It can't find Test;Builder in some class hash
14:08 pjcj joined #parrot
14:13 Infinoid seems if I only clone the code and not all the other clone bits (see runtime/parrot/include/cloneflags.pasm) it gets farther, but that's a yak I don't need to shave right now
14:13 barney joined #parrot
14:18 dalek parrot: r39269 | bacek++ | trunk (3 files):
14:18 dalek parrot: [pmc][core] Add function Parrot_pmc_try_reuse.
14:18 dalek parrot: It is reneralised version of C<reuse_or_new> from Intrger PMC. Will be
14:18 dalek parrot: used in other PMCs to reduce GC pressure.
14:18 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39269/
14:22 dalek parrot: r39270 | bacek++ | trunk (2 files):
14:22 dalek parrot: [core] dest can be null in Parrot_pmc_try_reuse.
14:22 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39270/
14:29 bacek oh shi...
14:30 bacek Do we have ticket about non-fully VTABLE copy to derived class?
14:31 autark joined #parrot
14:48 dalek parrot: r39271 | bacek++ | trunk/src/pmc.c:
14:48 dalek parrot: [core] Preserve semantic of non-core PMCs in Parrot_pmc_try_reuse.
14:48 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39271/
14:48 bacek ETOOMANYTYPOS...
14:50 pmichaud I'm getting a huge number of failures in trunk.
14:51 nopaste "pmichaud" at 72.181.176.220 pasted "test failures in trunk" (19 lines) at http://nopaste.snit.ch/16728
14:52 bacek pmichaud: r39270?
14:52 pmichaud yes.
14:52 pmichaud trying 39271 now.
14:52 bacek it should fix it :/
14:53 bacek (at least it passed on my box...)
15:00 whoppix joined #parrot
15:01 pmichaud yes, 39271 is much better, thanks.
15:03 bacek sorry for this...
15:05 bacek anyway, rakudo's make spectest should be slightly faster at 39271
15:07 * Coke needs to unleash bacek on partcl. =)
15:09 bacek Coke: one problem - I don't know tcl at all...
15:41 dalek parrot: r39272 | pmichaud++ | trunk/compilers/pct/src/PCT/HLLCompiler.pir:
15:41 dalek parrot: [pct]:  Update transcode option to HLLCompiler
15:41 dalek parrot: * Allow multiple transcoding attempts
15:41 dalek parrot: * Allow changing encoding as well as charset
15:41 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39272/
15:54 dalek parrot: r39273 | bacek++ | trunk/src/pmc/scalar.pmc:
15:54 dalek parrot: [pmc] Reuse C<dest> when possible in Scalar.
15:54 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39273/
15:54 dalek parrot: r39274 | bacek++ | trunk/src/pmc/bigint.pmc:
15:54 dalek parrot: [pmc] Reuse dest if possible in BigInt ops.
15:54 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39274/
15:54 dalek parrot: r39275 | bacek++ | trunk/config/gen/makefiles/root.in:
15:54 dalek parrot: Run nqp_test after testing Parrot's core itself.
15:54 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39275/
15:55 pmichaud bacek: does the nqp tests run only if all of the core tests pass?
15:55 bacek pmichaud: yes
15:55 pmichaud excellent  bacek++
15:56 bacek I prefer to move all compiler's test (except imcc) behind core tests. But not today.
15:58 dalek parrot: r39276 | NotFound++ | trunk/examples/nci/xlibtest.pir:
15:58 dalek parrot: [examples] add save as json feature to xlibtest.pir
15:58 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39276/
15:59 bacek pmichaud: my latest commit shaved another 1.5-2 seconds from rakudo's "make test". I'm too tired to run spectest to check impact... But it should be slightly faster.
16:00 pmichaud bacek: excellent, that's a good improvement.  I'm working on a fix that already shaves 2 minutes off of 'make spectest'
16:00 pmichaud (actually shaves 2 minutes off of just one test file :-)
16:03 bacek pmichaud: technically, this 2 minutes came from "revamped tt452_reduce_multi" branch.
16:03 pmichaud bacek: no, I'm sure it didn't.
16:03 pmichaud because I'm comparing times between my local changes using the same parrot in each instance.
16:04 pmichaud i.e., I'm not comparing an older version of parrot with a newer one.
16:05 bacek pmichaud: it is :) I just start implementing "switch-base VTABLE for MULTI" in Pmc2c (instead of doing it manually as in branch)
16:05 pmichaud bacek: you're not reading what I'm writing.  But no matter.  You're wrong.
16:05 pmichaud :-)
16:06 bacek But I'm fast! :)
16:06 pmichaud bacek:  I start with a parrot.  I time the test.  I make a change *in Rakudo*.  The test runs 2 minutes faster.
16:06 pmichaud That tells me that the 2 minutes improvement didn't come from Parrot.
16:08 bacek hang on. Can you check rakudo before r39253 and at r39257?
16:09 pmichaud not easily.
16:09 pmichaud I'll be more specific.
16:09 pmichaud I run the test with parrot 39271.
16:09 pmichaud I time the test (2m28)
16:09 pmichaud I make a change to Rakudo
16:09 pmichaud I time the test again (30 sec)
16:10 pmichaud both tests are using parrot 39271
16:10 bacek Ah, Ok. It's different :)
16:10 pmichaud ergo, the 2 minute improvement is not due to the pmc2c changes
16:10 pmichaud I'm not saying the pmc2c changes didn't produce an improvement, I'm simply saying that's not what I was measuring.
16:11 pmichaud or that pmc2c changes aren't responsible for the 2m improvement that I'm seing.
16:11 bacek I'm talking about http://irclog.perlgeek.de/p​arrot/2009-05-30#i_1192684 (pmichaud
16:11 bacek fwiw, Rakudo seems to have gotten faster since r39239)
16:11 bacek :)
16:11 pmichaud sure, it did get faster.
16:11 pmichaud And yes, it got about 2m faster.
16:12 pmichaud but that was a different 2 minutes.
16:12 pmichaud And that was 2 minutes for the entire suite, not for just one test file.
16:12 dalek TT #723 created by ingmar++: [PATCH] Parrot doesn't find ICU 4.2
16:13 bacek yes, this is performance boost that I've got from tt452 branch with manual converting MULTI to VTABLE-switch.
16:13 pmichaud which I find somewaht amusing, since allison did a lot of work last year trying to get the vtable-switches to be multis.
16:14 bacek So I just assumed that it was same things (just because they are same)
16:14 pmichaud (and yes, things slowed down significantly when that was done.)
16:15 Random joined #parrot
16:15 pmichaud still, I'm very happy for the speed improvements/recovery.
16:16 bacek "At your service" :)
16:16 Random ah, just came in the right second... I wanted to ask about the speed of loops in rakudo/parrot
16:16 Random following up on a post un use.perl.org I tested a simple while loop just incrementing an interger
16:17 Random in perl5 thats instant
16:17 Random in perl6 it's a whole-lot-of-memory and time
16:17 Random the code is as simple as: while ( $i < 1000000 ) { $i++; }
16:17 bacek Random: try very latest rakudo on top of very latest perl. It should be fast now
16:18 pmichaud "faster", maybe, but not "fast"
16:18 Random i just did a clone of rakudo 30min ago
16:18 Random and compiled with --gen-parrot
16:18 Random is that too old already?
16:18 pmichaud Random: we don't do much in the way of optimization.
16:19 pmichaud (yet)
16:19 pmichaud whereas perl5 is heavily optimized.
16:19 Random it can't be that the perl6 world is spinning that fast ;-)
16:19 pmichaud So it's not entirely an apples-to-apples comparison.
16:19 bacek Random: 30 minutes? It's like 18th century!
16:19 pmichaud bacek: keep in mind that Rakudo is still using a much older parrot
16:19 Random not optimzed is ok, but 1.4G of RAM and still running after 60seconds... is a little too extreme for me
16:20 Random ah ok, so I should also checkout parrot from svn and compile
16:20 Random and use that with my 30min old rakudo, right?
16:20 pmichaud well, "much older" in this case means about 12 hours
16:20 pmichaud Random: even with the more recent parrot it's still likely to take a while.
16:21 pmichaud There's just a *lot* going on behind the scenes that we haven't tried to work out yet.
16:21 bacek hm... It actually consumes too much memory
16:21 dalek parrot: r39277 | Infinoid++ | branches/io_rewiring/src/pmc (2 files):
16:21 dalek parrot: [io] Clean up Pipe and PipeHandle some more.
16:21 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39277/
16:21 dalek parrot: r39278 | Infinoid++ | branches/io_rewiring/t/pmc/pipe.t:
16:21 dalek parrot: [io] Add a test for the Pipe and PipeHandle PMCs.  It only checks object creation and accessor methods for now.
16:21 dalek parrot: There's a test which spawns a thread to do a write/read test, but spawning the thread crashes parrot, so that's disabled for now.
16:21 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39278/
16:21 dalek parrot: r39279 | Infinoid++ | branches/io_rewiring/PBC_COMPAT:
16:21 dalek parrot: [io] Bump PBC_COMPAT for the new PMCs we've added to this branch.
16:21 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39279/
16:21 dalek parrot: r39280 | Infinoid++ | branches/io_rewiring/t/native_pbc (4 files):
16:21 dalek parrot: Regenerate PBC files after to PBC_COMPAT bump.
16:21 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39280/
16:21 pmichaud although I am surprised about using 1.4G of RAM, that sounds like a serious memory leak
16:22 Random well it definitly is
16:22 Infinoid I see that much usage here, too
16:22 Random I'll retest with the latest parrot now
16:22 pmichaud but the biggest reason for the loop being slow is that it's resulting in several million Parrot subroutine calls
16:22 Infinoid I suspect it might be more extreme on x86-64 than on x86-32
16:22 pmichaud and Parrot subroutine calls are known to be especially slow.
16:22 pmichaud (those several million Parrot subroutine calls in turn are translating to several-several-several-million C function calls)
16:23 Random I'm running on amd64
16:24 Random still in perl5 it's optimized, so it's probably 1mill perl incr for 1mill c calls
16:24 bacek pmichaud: (and "< 1000000" creates new Integer on every loop)
16:24 Random but even if parrot is not optimized yet and has calling overhead that would be too extreme
16:25 * bacek wishes for SSA and CSE for PBC...
16:25 Random the code actually was: my $i = 0; while ( $i < 1000000) { $i++ }
16:25 Random so the $i should be instanced once
16:26 Random or is there something wrong with the comparison on a constant?
16:26 * Infinoid wishes his brain had hardware acceleration for decoding TLAs
16:26 bacek afk # .zzz
16:26 Infinoid sleep well bacek
16:26 pmichaud $i < 1000000   becomes    a call to   'infix:<'($i, 1000000)
16:26 pmichaud so that's 1 million subroutine calls right there.
16:27 Random ok, and these are extremly slow and leak mem
16:27 pmichaud yes.
16:27 pmichaud then $i++ becomes    'postfix:++'($i)
16:27 bacek pmichaud: not quite true...
16:27 bacek loop32_test:
16:27 bacek find_lex $P23, "$i"
16:27 bacek new $P24, "Int"
16:27 bacek assign $P24, 1000000
16:27 bacek $P22 = "infix:<"($P23, $P24)
16:28 bacek and Parrot bails out...
16:28 Random bacek: how did you generate that?
16:28 pmichaud bacek: the fact that the 1000000 becomes a PMC is in fact irrelevant to the timing
16:29 pmichaud (unless we decide to do a loop optimization to factor it out of the loop)
16:29 bacek pmichaud: it relevant. GC isn't bloody fast.
16:29 pmichaud bacek: I'm saying that even if we generated
16:29 Random When is optmization planned for the basic loops and functioncalls?
16:29 pmichaud $P22 = "infix:<"($P23, 1000000)
16:29 pmichaud we'd _still_ end up converting that 1000000 to a PMC on each call.
16:30 pmichaud (it would just be done by the calling conventions instead of explicitly in PIR)
16:30 Random *puh*, I guess svn checkout is faster then svn up of a month old parrot ;-)
16:30 pmichaud bacek: so the fact that the 1000000 becomes a PMC is true in both cases.
16:31 bacek pmichaud: we should not.
16:31 pmichaud bacek: depends on how you define "should not"
16:31 bacek Proper way is optimise it on PIR/PBC level
16:31 pmichaud if you're saying that we should factor the constant out of the loop, I likely agree.  But that's not code that is particularly important at the moment.
16:31 Random is there a way to print how many PMCs were generated when terminating the program?
16:32 pmichaud Random: chromatic has a way of doing it -- I'm not sure what he does.  I think there might be an option to Parrot to do it, though.
16:32 pmichaud You can always run perl6 as   parrot/parrot perl6.pbc   instead of "./perl6"
16:32 pmichaud and then parrot options can be placed before the "perl6.pbc"
16:32 * bacek felt asleep
16:32 pmichaud bacek: I'm fine with optimizing it on a PIR/PBC level.  But we don't do that yet.
16:32 Random ah ok
16:34 Infinoid Parrot's -D option should report that, but it doesn't seem to work at the moment (parrot->pdb is NULL)
16:45 Random trying to compile the latest parrot r39280 fails for me
16:46 Random In file included from src/string/api.c:29:
16:46 Random src/string/private_cstring.h:31:6: error: invalid digit "9" in octal constant
16:49 Random 31:    {0759094, "__get_string"}
16:49 Random guess that should not be a octal, although it starts with a zero
16:50 Random correcting it by removing the leading zero in the file fixes that, but the header of the file says it's autogenerated, so I guess that should be fixed somehwere else
16:51 Infinoid yeah, it's generated by c2str.pl
16:51 Random and now ./miniparrot gives me a segmenation fault :-(
16:51 Random 6293 Segmentation fault      ./miniparrot config_lib.pasm > runtime/parrot/include/config.fpmc
16:52 Random guess miniparrot did not like my corrections
16:52 Infinoid I'm not surprised.  That first number is supposed to be the string's *length*.
16:52 Infinoid So your value looks a little bit large
16:53 Random hehe, well then I guess I should correct it to the length
16:53 Infinoid that fix won't last, I'd rather find out why it was wrong
16:53 * Infinoid tries a trunk build
16:54 Random yes, just saw that *alot* of lengths are wrong for the strings
16:54 Random {631, "get_number_keyed_str"},
16:54 Random should be that way either probably
16:55 Infinoid sounds pretty broken
16:55 Infinoid You mentioned being on amd64, I think?  Linux?
16:56 Random yepp
16:56 Infinoid same here.  (gentoo)
16:56 Random Linux einstein 2.6.26-1-amd64 #1 SMP Sat Jan 10 17:57:00 UTC 2009 x86_64 GNU/Linux
16:56 Random It's debian testing + a little unstable
16:56 Infinoid I just did a fresh trunk build and it worked fine
16:57 Random c2str,pl did then something odd, right?
16:57 Infinoid Can you try "make realclean", reconfigure, rebuild and all of that, to see if the problem persists?
16:57 Random of course
16:58 Random I used that working copy beofre for a build... so maybe... make realclean && perl Configure.pl --prefix /opt/parrot && make
16:58 Random let's see it helps
16:58 Random *if it helps
16:58 Infinoid Oh, you mentioned having updated from last month's parrot, right?
16:58 Infinoid Did you do a realclean/reconfigure as part of that process?
16:59 Infinoid I made some changes to c2str on Apr 25 which modified its file formats and that would have required a realclean.  (But realclean is a good policy every time you update, regardless)
17:01 Infinoid c2str used to generate a "hash" value that wasn't used by anything, and it may have been parsing that from the .str files thinking it was the length.  That got removed about a month ago, but the old .str files would have confused the new c2str
17:03 Random yepp, it IS as good policy
17:03 Random forgot that today
17:03 Random so it's built
17:03 Infinoid no miniparrot segfault?
17:04 Random do I need to rebuild rakudo with the newer parrot now?
17:05 Random nope, everything fine now, you did a good job ;-)
17:05 Infinoid you should reconfigure and rebuild rakudo when you rebuilt the parrot it depends on, yes
17:05 Random is it enough if parrot is in the path?
17:06 Infinoid there's a --parrot-config option to tell rakudo where to find parrot
17:07 Random and where is that parrot-config file?
17:07 Random i did a make install to /opt/parrot
17:08 Random ah typo, found it
17:08 Random let's see to improvements for my simple benchmarks ;-)
17:10 dalek parrot: r39281 | Infinoid++ | trunk/config/auto/icu.pm:
17:10 dalek parrot: Apply patch from ingmar++ in TT #723:
17:10 dalek parrot: Subject: [PATCH] [config] Fix ICU detection for ICU 4.2
17:10 dalek parrot: ICU 4.2's 'icu-config --prefix' outputs an extra newline, strip it.
17:10 dalek parrot: Otherwise this breaks as follows:
17:11 dalek parrot: icuheaders:  captured /usr
17:11 dalek parrot: Unsuccessful stat on filename containing newline at config/auto/icu.pm line 332.
17:11 dalek parrot: Unsuccessful stat on filename containing newline at config/auto/icu.pm line 337.
17:11 dalek parrot: For icuheaders, found /usr
17:11 dalek parrot: /include and 1
17:11 dalek parrot: review: https://trac.parrot.org/parrot/changeset/39281/
17:12 dalek TT #723 closed by Infinoid++: [PATCH] Parrot doesn't find ICU 4.2
17:15 Infinoid http://feather.perl6.nl/~a​zawawi/padre-perl6-exe.png is shiny!
17:20 Random well I guess, the 100000 incr running in a loop are still very unperformant... 57sec and roughly 1.3GB of RAM for 100000 increments of  variable.
17:20 Random time ./perl6 -e 'my Int $i = 0; while ( $i < 100000 ) { $i++; }'
17:20 Infinoid Patches welcome. :)
17:21 Random so still the same with latest rakudo and parrot, is there some kind of continious benchmark run with the smoke tests?
17:21 Random to see when performance is increased (or decreased)
17:21 Random hehe, guess that is a little too big a job for me
17:22 Infinoid I'm not sure.  There is a t/benchmark/benchmarks.t but I don't think it's run as part of normal "make test", and I'm also not sure it outputs any stats in its TAP output
17:24 Random guess it should be a seperate run like make benchtest or something
17:25 Random as it's output would need to be gathered and compared to previous versions
17:25 nopaste "infinoid" at 24.182.55.77 pasted "t/pmc/packfileannotations.t failures on linux/amd64" (27 lines) at http://nopaste.snit.ch/16729
17:26 nopaste "infinoid" at 24.182.55.77 pasted "t/pmc/packfileannotations.t failures on linux/amd64 (with stderr too, this time)" (23 lines) at http://nopaste.snit.ch/16730
17:26 Infinoid wait, what?  intermittant failure...
17:27 nopaste "Infinoid" at 24.182.55.77 pasted "Try again..." (29 lines) at http://nopaste.snit.ch/16731
17:28 Infinoid msg bacek Any idea why I'd be getting an intermittant failure here?  http://nopaste.snit.ch/16731  (your branch merge is the only recent thing I could see in the logs)
17:28 purl Message for bacek stored.
17:29 Random does anyone know which runcores are currently supported for parrot?
17:29 Random fast is the default I guess, and jit needs a jit-enabled core
17:30 Random most of the ones specified in the parrot -h output are not available
17:30 Infinoid it varies depending on your platform.
17:30 muixirt Random,  there is tools/benchmark.pl
17:31 Infinoid trace, slow, fast are typical, cg and cgp are available on (most/all) gcc architectures, jit is x86-32 only by default (I think)
17:33 Random when switching runcores, parrot has to be run for the build directory, otherwise it says it cant find PCT.pbc
17:33 muixirt how is the status of jit - it worked better in the past, if i remember correctly
17:34 Random well cgp is 1 second faster... as fast
17:34 Random but at least there is room for optimization
17:34 Random let's see about that when chromatics changes land
17:35 Infinoid jit is very difficult to keep maintained, at this point most devs are leaning toward replacing it with llvm or libjit, or reimplementing it on top of a very small (RISC-like) set of instructions
17:35 Random that would definitly be a good idea
17:35 muixirt Infinoid, it jitted only one opcode, right?
17:35 Random as those other implementation get also attention outside of the parrot world
17:36 Infinoid no, it jits a few.  But still a rather small subset of the whole
17:36 Random left #parrot
17:37 Infinoid the fact that we have more than a thousand ops doesn't help
17:39 muixirt but another vm (llvm) under the hood of the parrot vm sounds ... well ...
17:39 Infinoid We'd be inclined to prefer a solution that gives us the easiest jit access with little to no extra baggage
17:41 muixirt i like the v8 javascript vm solution: compiler -> machine code :-)
17:44 Infinoid "libJIT has some advantages (just a JIT, arguably better C bindings, more standalone, perhaps less JIT startup disadvantages (no clang to get some JIT for free, fewer developers)." -- Chromatic, http://groups.google.com/group/parrot-dev/browse​_thread/thread/e394c35085252934/73c882ba5a096172
17:44 Infinoid oops, I somehow missed a "time, closer to our existing JIT but more cross-platform) and some " in the middle of that quote
17:45 Coke bacek: (not know tcl)  it's mostly PIR. You can just run one of the examples and clean up the resulting parrot mess. =)
17:48 Coke (printing # of pmcs) I have a macro for that.
17:49 Coke http://code.google.com/p/partcl/sou​rce/browse/trunk/src/macros.pir#103
17:52 muixirt Infinoid, the problem is that you lose information with every layer between source code (Perl6) and the CPU, and that might be bad for optimization
17:52 Infinoid Right.  Hence jit.
17:55 muixirt Infinoid, erm i meant the chain perl6 -> pir -> pbc -> parrot -> llvm/libjit won't give you much speed
17:56 Infinoid Ok.  We do have very good annotations support which I'm hoping optimization can make use of, but we do very little optimization in general
17:56 Infinoid (at the moment)
17:57 * Infinoid is running callgrind on Random's perl6 while-loop
17:59 muixirt but this data has to be (at least partl) processed in each layer, that consumes memory and cpu cycles
17:59 muixirt *partly
18:01 Infinoid That's why we want to separate as much of the overhead as possible from runtime, and do it at compile tile
18:01 Infinoid time
18:02 Infinoid typically the runtime chain will be: pbc -> parrot -> jit
18:15 Infinoid In that simple while/increment loop, we do indeed spend a lot of time creating and destroying things.
18:16 Infinoid 13.19% spent in pmc_new (and functions it calls)
18:16 Infinoid 10.51% in free_pmc_in_pool
18:16 Infinoid 9.83% in Parrot_Hash_init
18:17 Infinoid the next biggest thing after object creation/destruction is hash access; 8.64% in parrot_hash_get_bucket
18:19 Infinoid but since we're calling it 6.2 million times (for a loop of 10000), that's probably not its fault.
18:20 muixirt cool
18:22 muixirt i imagine that this loop would run on my first pc (486sx-33) ... it would be better counting for myself ;-)
18:23 * muixirt praises Intel for the speedups of their cpus
18:27 Coke hurm. make -j 2 install-dev fails.
18:28 iblechbot joined #parrot
18:28 muixirt Infinoid, but the problem with the loop isn't parrots fault, i think
18:29 Infinoid No, but if we can make parrot faster for this workload, a lot of other similar workloads will benefit
18:30 * Coke tries to eliminate TclObject
18:31 Coke bah, failure.
18:31 muixirt Infinoid, true, but can you?
18:32 Infinoid At the very least, I can make a few tweaks and see what happens... if it improves things by more than a couple of percent, I'll check it in
18:34 Infinoid (but I doubt it, as chromatic and others have already been over this stuff)
18:34 muixirt i believe making any random combinations of opcodes (maybe produced by a rakudo compiler gone wild) run fast is *very* hard
18:35 Infinoid Right now I'm just looking at saving a few cycles in parrot_hash_get_bucket
18:35 Coke that's tcl's #1 c function, as I recall.
18:36 pmichaud actually, Rakudo doesn't produce a lot of opcodes.
18:36 pmichaud that's true for most compilers in general, fwiw -- compilers tend to use very small numbers of opcodes.
18:37 pmichaud it's easy to check.
18:41 pmichaud http://nopaste.snit.ch/16732
18:41 pmichaud the ones that begin with a '$' are in fact function calls
18:42 pmichaud oops, and some of them are calls to root_new
18:44 * muixirt looks at the generated pir code
18:46 muixirt looking at mips asm code would be easier ;-)
18:47 nopaste "pmichaud" at 72.181.176.220 pasted "generated pir for loop" (136 lines) at http://nopaste.snit.ch/16733
18:48 pmichaud _block14 has the main loop
18:48 ewilhelm joined #parrot
18:48 pmichaud the section called loop32_test is the loop itself
18:49 pmichaud you can see that's pretty tight already.
18:49 muixirt sub "_block25"  :anon :subid("12_1243709245") looks suspicous, it does the incement?
18:49 pmichaud yes, it does the increment
18:52 muixirt and allocating these registers is so costly?
18:52 pmichaud the costly part is the parrot subroutine calls
18:53 pmichaud for this loop, we have the following calls
18:53 pmichaud "infix:<"
18:53 pmichaud $P26()   ( the call to _block25 )
18:53 pmichaud "postfix:++"
18:54 pmichaud those are expensive.
18:54 pmichaud at least, that's the only thing that I see here that can explain why the loop takes so long.
18:55 tetragon joined #parrot
18:55 pmichaud that tells me that it's parrot that is slow, not Rakudo.  I'm hard-pressed to see how Rakudo could generate much tighter code (in the general case) than what it's generating now.
18:55 Coke the PCC are expensive.
18:57 muixirt but why the $P26() ? it seems so superfluous
18:57 pmichaud you mean, why make the call to another parrot sub?
18:57 pmichaud inlining it would end up being an optimization, yes. But that optimization is worthless if we have instead
18:57 muixirt yes, just for incrementing $i
18:57 pmichaud my $i = 0;  while ($i < 1000) { my $j; $i++; }
18:57 pmichaud because we need another block in which to create the $j
18:58 Coke it would be interesting to time it inlined and compare.
18:58 Coke see if figuring out how to do that optimization now is worth it.
18:58 muixirt $j ?
18:58 pmichaud the body of the while loop has its own local $j lexical
18:58 pmichaud parrot lexical scopes have to be separate subs
18:59 Infinoid I suppose you could cache the .lex lookup for $i, and you could cache the 1000 constant pmc
18:59 Coke muixirt: the current sample doesn't have $j - this is just an example of why we need it in the general case.
18:59 Infinoid But the loop is indeed pretty tight overall
18:59 Coke need it; it == the sub.
18:59 pmichaud Infinoid: I can cache the .lex lookup for $i *as long as* the inner loop doesn't rebind it to a different PMC
18:59 Whiteknight joined #parrot
18:59 Infinoid If the inner loop is a different scope, does the outer loop inherit that change?
19:00 Infinoid (I guess I don't fully understand binding.)
19:00 pmichaud Infinoid:    my $i = 0; while ($i < 1000) {  $i := 1000; }
19:00 Infinoid uh, /me =~ s/loop/scope/
19:00 pmichaud that causes the symbol $i to point to an entirely different PMC from the one that held the zero
19:01 pmichaud so if the while loop has a register that continues to point to the zero, we get an infinite loop.
19:01 Infinoid oh, ok.  How expensive is the .lex lookup, anyway?
19:01 pmichaud it shouldn't be that expensive.
19:01 jonathan It's a lot pricier than it need be.
19:02 Infinoid I won't worry about it then
19:02 pmichaud it's pricier than it ought to be, yes, but it's not huge.
19:02 pmichaud but I can do some benchmarks, just a sec.
19:02 Infinoid if you had to search a big stack of lexpads, or whatever, you could always have a revision number that got bumped every time you do a bind, and invalidate your cache if the rev was increased
19:02 jonathan Given that we statically know to look down the outer chain one and then statically know which register number to look in.
19:02 pmichaud Infinoid: afaik, there's not a way to know that something is rebound
19:02 Infinoid But I'm not a big fan of making things more complicated like that, generally
19:02 jonathan But instead, we do two hash indexes.
19:03 jonathan Which (surprise!) costs more than a pointer deref and an array index.
19:03 pmichaud and then instead of doing find_lex, we'd always be checking the cache, which is probably the same thing.
19:03 Infinoid Rebinding calls LexPad.set_pmc_keyed_str(), I'm guessing?
19:03 Infinoid That's all we need to detect
19:04 pmichaud Infinoid: and in the outer loop...?
19:04 jonathan I don't know that we need to cache it, so much as use the information we statically have available to not have to do lexical lookups.
19:04 jonathan As in, to not have to do them by name every time.
19:04 pmichaud jonathan: you're talking about something different than we are.
19:04 pmichaud Infinoid is talking about moving the  find_lex $P23, "$i"  outside of the loop
19:05 jonathan Oh. How would that work?
19:05 pmichaud that's my point, it wouldn't.
19:05 jonathan Ah.
19:05 jonathan :-)
19:05 nopaste "pmichaud" at 72.181.176.220 pasted "factor out find_lex" (13 lines) at http://nopaste.snit.ch/16734
19:05 * muixirt stubbornly persists, rakudo should have optimized that loop away ;-)
19:06 pmichaud Infinoid: the problem is, if something rebinds the '$i' lexical, how do we reflect that change in $P23 ?
19:06 jonathan muixirt: Well, if you want to start hacking on a Rakudo optimizer... ;-)
19:06 pmichaud i.e., how do we know that the PMC in $P23 is no longer the one for $i ?
19:06 Infinoid Right, you need a (somehow faster than .lex) way of detecting that case
19:07 pmichaud if we put in additional PIR instructions to detect that case, then we're probably better of having just kept the 1-instruction "find_lex" inside the loop.
19:07 jonathan *nod*
19:07 pmichaud *better off
19:07 Infinoid So my (completely ugly and hackish) thought was, you keep a handle on the current PMC for $i, as well as the LexPad that holds it and the old revision of that LexPad... then you bump the LexPad revision whenever you overwrite anything in that lexpad.  So you check LexPad.get_revision() or whatever against your previous version
19:08 pmichaud Infinoid: but *where* do we do that check?!?
19:08 Infinoid you do that check once per loop
19:08 pmichaud then why not just do the find_lex ?
19:08 Infinoid The goal is to make that check less expensive than just doing .lex directly
19:08 Infinoid or find_lex, sorry
19:08 pmichaud find_lex can be made much faster
19:08 Infinoid Yeah, that would be better for all parties involved :)
19:09 pmichaud and find_lex could potentially be the thing that does the check
19:09 pmichaud either way, that's a *parrot* issue and not a Rakudo code-gen one.
19:09 jonathan Ideally, we'd only call $P0 = find_lex '$name' occasionally.
19:09 pmichaud ideally, once per thread-of-execution
19:09 jonathan And an optimizer during the PIR -> PBC phase would re-write a lot of them as $P0 = find_lex 1, 2 or similar.
19:10 Infinoid Is it possible for rakudo to detect whether any binding will occur in the inner block, and simplify the generated loop?
19:10 pmichaud Infinoid: no.
19:10 jonathan "1 outer down, register number 2"
19:10 pmichaud because rebinding could also occur in any function that the inner block happens to call.
19:10 jonathan pmichaud: Only if it were marked "is context" though, no?
19:10 Infinoid Right, you could only detect it in the case that the inner block doesn't *call* any functions
19:11 pmichaud Infinoid: and doesn't call any operators, either.
19:11 pmichaud jonathan: no, even without "is context"
19:11 jonathan Infinoid: The problem is that $x++ *is* a call.
19:11 jonathan pmichaud: From Perl 6?
19:11 jonathan pmichaud: How?
19:11 pmichaud my $i = 0;   sub foo() { $i := 1000; };   while ($i < 1000) { foo(); }
19:11 jonathan oh
19:11 jonathan Yes, that's harder
19:12 jonathan However, also lexically known.
19:12 pmichaud oh?
19:12 Infinoid So a potential optimizer would have to inspect the contents of any functions called, and detect the case where any functions would be overwritten at runtime, in order to make this determination
19:12 pmichaud my $i = 0;   multi sub foo(...) { $i := 1000; };   while ($i < 1000) { foo(); }
19:12 pmichaud it's not just functions called, but also all possible functions they could call.
19:12 Infinoid yeah
19:12 pmichaud which can dynamically change
19:12 jonathan pmichaud: Yes, true.
19:12 pmichaud because we have virtual methods
19:13 pmichaud so we _can't_ know at compile time.
19:13 jonathan Aye.
19:13 pmichaud (except if we can somehow determine that every operation involved is static and hasn't been monkeypatched and ....)
19:13 jonathan So basically Perl 6 is unoptimizable.
19:13 Coke kind of like tcl! =-)
19:14 pmichaud oh, there are optimizations to be had, yes -- but avoiding lexical fetches isn't one of them in our current model (that is somewhat imposed upon us by Parrot)
19:14 Infinoid beautiful.  I'm looking forward to seeing what kinds of optimizers we can build on parrot
19:14 jonathan pmichaud: OK. But that doesn't change my suggested approach to lexical lookups that we statically known the position of.
19:14 pmichaud jonathan: agreed.
19:14 pmichaud The answer here is to make find_lex fast.
19:14 jonathan (A Parrot-level op rather than a Rakudo-level one though.)
19:15 pmichaud or to give Parrot a smarter overall lexical system
19:15 pmichaud however, let's do a few benchmarks....
19:16 pmichaud argggh, I can't run the PIR code anymore from the command line.  :-(
19:16 jonathan Going back to the while loop example that kicked all of this off though - it'd be *really* helpful to know where/how that leaks so much memory.
19:16 pmichaud oh, I can force a loadlib
19:17 pmichaud vi nopastes coming
19:18 nopaste "pmichaud" at 72.181.176.220 pasted "basic benchmark, no changes" (145 lines) at http://nopaste.snit.ch/16735
19:20 nopaste "pmichaud" at 72.181.176.220 pasted "avoiding the call to the nested block" (138 lines) at http://nopaste.snit.ch/16736
19:21 pmichaud actually, that's a little over-optimistic
19:22 nopaste "pmichaud" at 72.181.176.220 pasted "avoiding the call to the nested block, corrected" (140 lines) at http://nopaste.snit.ch/16737
19:22 pmichaud so, the 10,000 calls to the nested block cost us .140s
19:23 pmichaud factoring out the constant....
19:25 nopaste "pmichaud" at 72.181.176.220 pasted "factoring the 10000 constant out of the loop (6.903s)" (140 lines) at http://nopaste.snit.ch/16738
19:25 pmichaud purl:  7.011-6.903
19:25 purl 0.108
19:26 pmichaud creating the 10000 Ints cost us .108
19:26 pmichaud hmmmm, I'm curious.
19:27 muixirt so whats left (i cant really follow)
19:27 pmichaud I don't know, I'm a bit surprised myself.
19:27 pmichaud maybe it's the postfix:++ that is expensive?
19:28 pmichaud it would be really nice if we had a pir-level profiler for this st..... oh, right.  :-P
19:30 * Infinoid tries adding some branch prediction annotations to parrot
19:30 jan joined #parrot
19:30 pmichaud to do one loop (instead of 10000)    takes  0.728s
19:31 pmichaud I wonder if the postfix:++ is the costly part, as it has to ultimately go through the vtable interface to work
19:32 muixirt i once coded such a loop in asm for a direct threaded code model and it took 46 secs!
19:33 pmichaud a-ha!
19:33 muixirt well it counted to 1000000000 :-)
19:34 Zak joined #parrot
19:34 nopaste "pmichaud" at 72.181.176.220 pasted "original PIR, replacing "postfix:++"($P30) with inc $P30 (jonathan++ notice!)" (142 lines) at http://nopaste.snit.ch/16739
19:35 pmichaud so, it's the call to postfix:++ that is super-expensive.
19:36 pmichaud ...and looking at the postfix:++ call, I think it's no longer accurate.
19:36 pmichaud I thought I had fixed this already. :-(
19:37 jonathan pmichaud: So adding this:
19:37 jonathan .loadlib 'perl6_group'
19:37 jonathan .loadlib 'perl6_ops'
19:37 jonathan Was what helped?
19:37 pmichaud to run the PIR directly from parrot, yes.
19:37 pmichaud not for the speed improvement.
19:37 pmichaud the speed improvement comes from
19:37 jonathan pmichaud: I'm a little surprised we need them at this point, but OK.
19:38 pmichaud jonathan: we shouldn't need them -- it's a known Parrot bug.
19:39 pmichaud TT #150
19:39 jonathan ah, ok
19:39 pmichaud the speed improvement comes from
19:40 pmichaud $ diff x.pir x4.pir
19:40 pmichaud 104c104
19:40 pmichaud <     $P31 = "postfix:++"($P30)
19:40 pmichaud ---
19:40 pmichaud >     inc $P30
19:40 pmichaud $
19:40 pmichaud ...which makes me suspicious of vtables and multidispatch
19:40 jonathan pmichaud: Right, but in the general case that isn't going to fly?
19:40 pmichaud right, we really shouldn't generate "inc" here
19:40 pmichaud my point is imply that something about "postfix:++" makes it expensive.
19:41 pmichaud *simply
19:41 jonathan Oh, we are generating inc and not "postfix:++"( )?
19:41 pmichaud no, we generate "postfix:++"()
19:41 jonathan OK, that's right...but yes, I can appreciate that may be costly.
19:41 pmichaud when we do that, my benchmark takes 7.15 seconds
19:41 jonathan That's because postfix:++ does...
19:41 jonathan 1) Call .succ
19:41 pmichaud if I replace the "postfix:++"() with "inc", the benchmark takes 1.x  seconds
19:42 jonathan 2) Do an assignment
19:42 jonathan IIRC.
19:42 pmichaud it also makes a clone
19:42 pmichaud (to return the original value to the caller)
19:42 jonathan right, so it can...right.
19:43 pmichaud however, for some reason we're cloning it twice.
19:43 jonathan pmichaud: copy op in assignment.
19:43 pmichaud no, I mean in postfix:++ itself.
19:43 pmichaud there are two calls to clone
19:44 jonathan so I see
19:44 pmichaud I should revisit this.
19:44 jonathan Aye, there's almost certainly a way to make that more efficient.
19:44 pmichaud but something in there is responsible for 80% of the execution time
19:44 Austin_Hastings left #parrot
19:45 pmichaud and we could probably optimize the postfix:<++>(Int) case
19:47 jonathan *nod*
19:47 jonathan pmichaud: We gotta be a tad careful of this case:
19:48 jonathan my Even $x = 4; $x++; # should die
19:48 jonathan (provided you had defiend your subset Even)
19:48 pmichaud oh, I'll still do the assignment.
19:48 jonathan Sure.
19:49 pmichaud but instead of clone + call inc, I'll probably just add one and assign that.
19:49 jonathan *nod*
19:49 pmichaud that would *also* help with our BigInt boundary conditions, I suspect :-)
19:50 pmichaud well, I need a break for a while.  I'll come back to this later today or tomorrow.
19:51 pmichaud also need to figure out how to get the ucs2 encoding to work. There are a lot of the spectests that have non-breaking spaces in the comments, and thus run slow.
19:51 pmichaud s/run slow/parse slow/
19:52 jonathan OK, have a good break.
19:52 jonathan I need to track down a segfault next, so will probably break for the day too...
20:05 * jonathan looks at something and boggles
20:05 jonathan In Parrot's src/call/pcc.c
20:05 jonathan set_retval(PARROT_INTERP, int sig_ret, ARGIN(Parrot_Context *ctx))
20:05 jonathan Inside here do do this:
20:05 jonathan call_state st;
20:05 jonathan We pass a pointer to it...
20:05 jonathan if (set_retval_util(interp, "P", ctx, &st)) {
20:06 jonathan set_retval_util does
20:06 jonathan Parrot_init_arg_op(interp, ctx, src_pc, &st->src);
20:07 jonathan ...hang on, my head is exploding...
20:07 jonathan So at this point we've passed what woulda I guess been st.src in the original - well, rather, a pointer to it.
20:08 jonathan hmm...it may be OK. Ignore me. :-S
20:21 pmichaud ("did someone say something...?")
20:51 dalek TT #724 created by pmichaud++: [bug]  Parrot fails numeric conversion of ucs2 strings
21:10 cognominal joined #parrot
22:10 Infinoid oh, wow.
22:10 Infinoid I sped up that rakudo loop by 1 second by putting (C-level, gcc-specific) branch prediction annotations in a few key places, and then sped it up another 4 by playing with the Hash internals allocation a bit
22:12 Infinoid I need to re-analyze the branch prediction stuff.  The profile I had based it on was without any kind of gcc optimization at all, and it looks like the code generated in --optimize mode is quite a bit different
22:12 pmichaud Infinoid++
22:12 cotto It'd be pretty cool to get a pgo build using the data from the test suite.
22:12 Infinoid Either way, expect a patch from me in the near future to introduce (linux kernel-like) likely() and unlikely() macros
22:12 cotto I was thinking the same thing.
22:12 Infinoid (which do nothing on non-gcc, for now)
22:14 Infinoid I'm going to add both of these patches as tickets, rather than applying them directly, as neither of them are particularly pretty
22:28 cotto That's what you get for imitating kernel code.
22:30 dalek TT #725 created by Infinoid++: [PATCH] Add c-level branch prediction annotations
22:31 rg1 joined #parrot
22:37 Infinoid The kernel does it pretty well.  I just need to redo the callgrind profile to find useful places to use it
22:37 Infinoid it only saves about half a second as-is
22:37 Infinoid (most of that is probably in PARROT_ASSERT and ASSERT_ARGS)
22:41 Infinoid #726 is the one that could use some more eyeballs (sped up that rakudo benchmark by around 10%)
22:44 dalek TT #726 created by Infinoid++: [PATCH] Reduce calls to mem_sys_allocate() when creating a Hash
22:44 jonathan Infinoid: More notably, hashes are used heavily all over the place.
22:44 Infinoid Yeah, that's why I focused on it
22:44 jonathan Infinoid: So making hashes faster likely has a real impact on a lot of code.
22:45 jonathan Infinoid++
22:45 Infinoid Note: I haven't even looked at why the rakudo benchmark uses up so much memory yet, that's likely going to make a bigger impact
22:45 jonathan Yeah, agree.
22:45 jonathan I'd turn on context debugging and see if we're leaking a bunch of those.
22:46 Infinoid Oh, we still don't have context PMCs, do we?
22:46 jonathan No
22:46 Infinoid Ouch.
22:46 jonathan But my other concern is...
22:46 jonathan I think somebody pointed out huge GC overhead.
22:46 jonathan If we're somehow leaking contexts but leaving the PMCs they pointed to reachable somehow, that could give huge GC pressure.
22:47 Infinoid hmm.  If that were true, I think I'd see more GC activity in callgrind
22:47 Infinoid One sec, rerunning callgrind
22:47 jonathan Ah, I thought there was a lot?
22:48 Infinoid my previous callgrind invocations were for a non-optimized parrot, but there didn't seem to be a ton
22:48 Infinoid allocations and frees were more or less proportional
22:49 jonathan Ah, OK.
22:49 Infinoid jonathan: http://irclog.perlgeek.de/p​arrot/2009-05-30#i_1194677
22:49 Infinoid sadly, I can't get text output from kcachegrind.  I can post a screenshot when this one finishes tho
22:49 Austin_Hastings joined #parrot
22:51 jonathan 13.19% spent in pmc_new (and functions it calls) # this is a lot of time, given it's essentially just initializing stuff rather than doing useful work...
22:52 jonathan If you add it up, we spend a quarter of our execution time managing creating/freeing PMCs.
22:52 Infinoid yes, but then again, it is called over 6 million times
22:53 jonathan 6000000 / 10000
22:53 purl 600
22:53 jonathan Average 600 PMCs per loop?
22:53 jonathan OK, we'd need to subtract what we cost just starting up etc...
22:55 Infinoid yeah, I can work on that next
22:55 Infinoid I got the stats slightly wrong... 2.4M calls to pmc_new(), 6M calls to parrot_hash_get_bucket()
22:55 Infinoid Please see https://quack.glines.org/upload/infinoi​d/callgrind-rakudo-while-i-le-10000.png
22:56 jonathan wow, pretty!
22:56 Infinoid parrot_create_hash shrank a bit from the last time, but it's still a fairly big block on that chart
22:57 jonathan Hmm. It spends a lot of its time inside malloc.
22:57 Infinoid parrot_gc_sweep_pool weighs in at around 3%
22:57 jonathan Parrot_Class_isa?
22:58 jonathan That seems to have some percent too
22:58 jonathan I can't quite tell
22:58 jonathan Can only se Parrot_Class_i on the diagram...
22:58 Infinoid about 2% in Parrot_Class_isa and Parrot_Class_isa_pmc in total
22:59 Infinoid there's a Parrot_Class_isa_pmc and a Parrot_Class_isa_pmc'2, I dunno what the difference is (I'm pretty new to this tool)
22:59 Infinoid I think it's the same thing but from a different call chain
22:59 jonathan ok
23:00 jonathan Parrot_new_str_COW is also the other one that looks sizable.
23:00 Infinoid overall, I think our overhead is pretty well spread out.
23:00 jonathan Yes, for the most part it seems so.
23:01 jonathan I guess chromatic++ has got a lot of the hot spots tracked already.
23:01 Infinoid yeah, and he's been advocating this tool for some time now
23:01 Infinoid I can see why.
23:02 jonathan yeah, me too after that screenshot.
23:02 Infinoid Parrot_new_str_COW is sizable mostly because it's called more than 3 million times
23:02 Infinoid which results in more than 2 million calls to Parrot_gc_new_string_header
23:03 Infinoid oh, it calls it 3 million times, just from 2 different places
23:03 jonathan Oh?
23:04 Infinoid yeah, there's an if/else based on whether the passed in string was constant or not
23:04 jonathan ah, ok
23:05 Infinoid So, since Hashes are so incredibly high-profile, what do you think about the TODO on src/hash.c line 981 regarding optimizing for lots of small hashes?
23:05 * jonathan would love to have output in this style at PIR level
23:05 Infinoid uh, that's line 981 in my build, but I have a couple of patches applied here
23:06 Infinoid 972 in trunk
23:06 jonathan yeah, I see th eone
23:08 jonathan Trying to work out exactly what it means
23:08 jonathan I think what it's driving at is trying to do 1 malloc not 2.
23:09 Infinoid ah, so it's pretty much what I did in TT #726
23:09 Infinoid But I didn't stick it in the struct, I just tacked it onto the end of the buffer
23:10 jonathan No, I think it might be meaning "don't allocate a separate bucket store"
23:11 Infinoid Yeah, that's what I changed, but they're suggesting reducing the bucket store size as well
23:11 jonathan The comment below that leads me to wonder if Hash generally is doing stuff to make OrderedHash easier.
23:11 Infinoid hmmm.
23:11 * Infinoid tweaks the number of buckets and redoes the benchmark
23:13 jonathan I'm no great data structures guy but part of me wonders...
23:13 jonathan HashBucket *bs;             /* store of buckets */
23:13 jonathan HashBucket **bi;            /* list of Bucket pointers */
23:13 jonathan From what I can see, we're manintaining an array of the buckets
23:13 jonathan And also maintaining a linked list.
23:14 Infinoid yes.  After my patch, *bs is a pointer to the memory directly following the Hash struct itself
23:14 Infinoid they're both allocated in one go
23:14 Coke I wonder again why we're not just using a hash library. =-)
23:14 jonathan Coke: Because concaine libraries are so much better!
23:14 Infinoid *bs ends up pointing somewhere else when we expand the hash, but initially, they're both in the same buffer
23:14 jonathan OK
23:15 Infinoid By the way, changing INITIAL_BUCKETS from 16 to 8 speeds us up by another 3 seconds (another 7% or so)
23:15 snarkyboojum joined #parrot
23:15 jonathan I think you're not the first person to observe this.
23:15 jonathan But not sure why the change didn't make it in before...
23:16 jonathan In reality most hashes seem to be quite small.
23:16 Coke isn't that going to depend on the benchmark you're running?
23:16 Infinoid eh, 2 seconds on average
23:16 Infinoid Yes, absolutely
23:16 jonathan Coke: Yes, it is.
23:18 jonathan Infinoid: How icky is it to lazily allocate the bucket store?
23:18 jonathan That is, not allocate it until we actually store something?
23:19 nopaste "Infinoid" at 24.182.55.77 pasted "Ooh pretty numbers!" (18 lines) at http://nopaste.snit.ch/16741
23:19 jonathan I ask because a lot of the time, e.g. whenever there's a :named :slurpy, there's nothing put into it.
23:21 cotto I like laziness because it reflects the way I work.
23:21 Infinoid jonathan: That sounds a bit more involved.  We don't really pay anything extra now that it's allocated in the same buffer as the Hash
23:22 Infinoid If we were to introduce lazy allocation, I don't think the NULL checks would slow things down *that* much, but I don't think we'd gain anything either
23:22 dalek partcl: r400 | coke++ | trunk/runtime/tcllib.pir:
23:22 dalek partcl: the NS referenced here is long gone.
23:22 dalek partcl: review: http://code.google.com/p/p​artcl/source/detail?r=400
23:22 Infinoid It would help memory usage and hurt performance slightly.
23:23 Infinoid performance is probably a wash
23:23 Infinoid jonathan: Didja look at my pretty shiny numbers?
23:23 jonathan OK. It's just that it'd save us a malloc and the initialization work (though you already made that cheaper though).
23:24 jonathan And afaik PGE heavily uses the empty-slurpy-hash pattern.
23:24 Infinoid it won't save us a malloc any more, we'd just be allocating slightly less
23:24 jonathan Your shiny numbers are interesting.
23:24 jonathan Oh, OK. In that case, then probably not wroth it then.
23:24 jonathan *worth
23:25 jonathan I think it may be worth running some other comparative benchmarks.
23:25 jonathan Rather than just optimizing for this one.
23:25 jonathan But in Rakudo generally, small hashes is most likely the trend.
23:26 Infinoid Yeah, absolutely
23:26 jonathan (Not least because of the PGE has a lot of empty ones issue.)
23:26 jonathan (And Rakudo is about to get %_ on all methods...)
23:27 Infinoid what's %_?  argument hash?
23:28 jonathan By S12, all Perl 6 methods are meant to get a default slurpy hash parameter.
23:29 jonathan (Which is normally not so interesting, but gets more interesting now we are about to have deference.)
23:30 * jonathan loves how "Windows" doesn't even appear on the valgrind platforms to port to page.
23:30 Infinoid hah
23:30 Infinoid valgrind emulates the hardware and the OS, and porting it is quite a lot of work, so they're pretty selective
23:31 jonathan Aye, understandable.
23:32 Infinoid I didn't see much difference between 16 buckets and 4, from examples/benchmarks/primes.pasm, after bumping its loop limit up to 20000.  4 buckets was slightly faster, but no more than 3%
23:33 * Infinoid looks for more benchmarks
23:33 jonathan Sure. Performance regressions are probably the more interesting issue.
23:34 Infinoid Any suggestions on how to find those?
23:34 Infinoid Lots of big hashes would probably be the losing performers here
23:35 Infinoid fairly evenly distributed datasets, forcing lots of calls to expand_hash
23:35 jonathan *nod*
23:36 jonathan Not sure where I'd look for them.
23:36 jonathan Did you try benchmarking say Rakudo make test?
23:37 Infinoid I'll do that now
23:46 Infinoid between trunk and trunk+TT725+TT726, rakudo "make test" got about 200ms faster.  It got another 100ms faster from changing INITIAL_BUCKETS from 16 to 4.
23:46 Infinoid I don't trust those numbers tho, they're less than the margin of error
23:46 * Infinoid does an average of 10 runs each
23:47 jonathan not a regression but not a noticable win either.
23:48 Infinoid yeah, the effects were much more noticable in the while loop case
23:48 Infinoid actually, based on my initial test, TT725 may have regressed it slightly and TT726 brought it back, I'll know in a few minutes
23:53 particle joined #parrot

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced