Camelia, the Perl 6 bug

IRC log for #parrot, 2010-09-15

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 * cotto takes $3.0 from whiteknight
00:00 whiteknight done
00:00 bacek_at_work cotto, did you miss "M" after $3.0?
00:00 chromatic msg atrodo What do we need to do to improve JONSLORITO ?
00:00 purl Message for atrodo stored.
00:00 aloha OK. I'll deliver the message.
00:00 * whiteknight is fast. Like a ninja
00:01 cotto I'll start small.
00:03 dalek parrot: r49005 | luben++ | trunk/src/call/args.c:
00:03 dalek parrot: Use plain hashes, not Hash PMC for internal named args processing
00:03 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49005/
00:03 dalek parrot: r49006 | whiteknight++ | trunk/docs/project/release_manager_guide.pod:
00:03 dalek parrot: +cotto
00:03 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49006/
00:05 whiteknight cotto: What's the status of the gsoc_instrument branch?
00:06 whiteknight I can say that it fails in parallel make. Regular make works fine
00:08 NotFound There are a bunch of throws in that function, and the flow is not enough clear to know if some is never reached with named_used_list. The safe side is to check all until lne 1107
00:08 chromatic Turn on the Coke-signal and let him check the dependencies.
00:09 whiteknight na-na-na-na-na-na-na-na Coke MAN!
00:09 whiteknight Coke MAN!
00:10 whiteknight feel lucky you can't hear the singing
00:11 chromatic I already GCd it.
00:12 whiteknight :)
00:12 NotFound If some day someone is able to rewrite that function in a cleaner and shorter way, I'll proclam he or she King or Queen of the parrots.
00:13 cotto whiteknight, safe to nuke
00:13 chromatic Is there a cash reward?
00:13 whiteknight cotto: nuke? I want to love it and merge it
00:14 whiteknight NotFound: which function?
00:14 NotFound chromatic: you can put taxes on your subdits.
00:14 whiteknight oh, named_used_list
00:14 luben Done, I am freeing the strunct on all exceptions
00:14 NotFound whiteknight: fill_params
00:14 cotto The code lives on github atm, though I don't think khairul's done much with it.
00:14 cotto It's definitely an awesome piece of code.
00:15 cotto I love having a PMC that's a runloop.
00:15 cotto s/loop/core/
00:15 whiteknight Ah, okay. I misunderstood you. So the branch can go, but the code lives on github. Gotcha
00:15 cotto yup
00:15 chromatic Why is the code on Github and not merged?
00:15 cotto http://github.com/khairulsyamil/parrot-instrument
00:16 cotto It's more like an external project than a core feature.
00:16 cotto Do you think it'd be better in core?
00:16 chromatic It'll break less in core.
00:16 chromatic Also get used more.
00:16 davidfetter +1 for core, fwiw
00:16 cotto this is true
00:17 cotto seen khairul
00:17 purl khairul was last seen on #parrot 22 days, 19 hours, 55 minutes and 37 seconds ago, saying: hio cotto. i'm slowly moving the code over to here, http://github.com/khairulsyamil/parrot-instrument  [Aug 23 04:21:36 2010]
00:17 aloha Sorry, I haven't seen khairul.
00:18 whiteknight I thought the code was pretty deeply integrated into libparrot
00:18 whiteknight maybe thts a datd understanding of it
00:19 cotto yr mssng sm vwls
00:20 dalek parrot: r49007 | luben++ | trunk/src/call/args.c:
00:20 dalek parrot: free internal hash struct on exceptions
00:20 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49007/
00:20 NotFound Another fine project: add sms encoding to parrot strings
00:20 NotFound Extra point for dcoding.
00:20 davidfetter /m lrnd hrbrw t n rly g, hnc hs n sr tm rdng thngs wtht vwls
00:21 whiteknight yeah, I think my keyboard is going all 'tarded
00:22 whiteknight ok. I see what this project does. Different from what I thought
00:22 whiteknight still cool though
00:23 nopaste "luben" at 192.168.1.3 pasted "Hash sizes stats" (13 lines) at http://nopaste.snit.ch/23343
00:25 whiteknight 40k hashes of size 0? FAIL
00:27 cotto That's unexpected.
00:28 patspam left #parrot
00:29 luben yes,
00:31 whiteknight luben: can we figure out where those wasted hashes are coming from?
00:31 luben they come as Hash PMC
00:31 luben most probably
00:32 whiteknight can we make Hash PMC more lazy?
00:32 chromatic 80% of hashes have 0-2 keys?
00:32 plobsing luben: what are those stats for? perl6 startup?
00:33 luben I hava made hash struct allocation lazy
00:33 luben plobsing, yes
00:33 whiteknight luben: in trunk?
00:33 luben yes
00:33 whiteknight so you've already made it lazy, and we're still seeing these numbers?
00:33 luben index/buckets area is allocated in lazy manner
00:34 luben hash struct is not allocated lazy
00:34 whiteknight ok
00:35 luben may be I could look in Hash PMC and see if it could allocate hash struct more lazy
00:35 chromatic Easily.
00:36 NotFound Maybe will be more productive to know what parts are creating lots of unused hashes.
00:40 chromatic Also true.
00:40 luben T think that's rakudo that creates them
00:40 chromatic Empty namespaces?
00:40 plobsing empty lexpads?
00:40 chromatic That could do it.
00:40 NotFound Sound likely
00:41 chromatic Sub's invoke() seems to create them only when necessary though.
00:42 luben args processing also creates hashes only when needed
00:43 chromatic Running cachegrind to see.
00:43 plobsing cachegrind can get you that info?
00:44 chromatic It can show the call paths that create hashes.
00:44 plobsing I suppose when 80% of hashes are near-empty, the largest allocator of hashes is very likely a large allocator of near-empty hashes.
00:45 chromatic Lots of LexInfos thawed.
00:45 luben chromatic, I am exploring callpaths with kcachegring also
00:47 cognominal left #parrot
00:47 chromatic Could be the thaw of Subs.
00:47 plobsing yes, but why do subs have lexinfos frozen in? seems like that's a problem in IMCC.
00:48 chromatic I'm not sure they do, but I don't understand the freeze/thaw in Sub very well.
00:48 plobsing VISIT_PMC_ATTR(INTERP, info, SELF, Sub, lex_info);
00:48 plobsing if lex_info is null, it will be stored and thawed as null
00:49 cognominal joined #parrot
00:52 chromatic Ah.
00:52 chromatic CallContext's exists_keyed*() VTABLEs.
00:52 chromatic They shouldn't call get_hash().
00:52 chromatic ... at least, they shouldn't autovivify the hash.
00:54 NotFound There is a ticket about deprecation of autovivifys. Time to ge rid of them?
00:55 NotFound They can probably create a lot of empty unused hashes
00:56 davidfetter left #parrot
00:56 NotFound The probability of breaking things is not negiglible, though.
00:59 NotFound Big problem: named_used_list is freed in line 1139 but is used in 1182
01:00 luben NotFound, looking
01:03 pmichaud empty hashes:  my guess would be named slurpy parameters where no named arguments were passed.
01:09 luben NotFound, shipped a fix
01:10 NotFound luben: looks good
01:10 dalek parrot: r49008 | whiteknight++ | branches/hash_allocator:
01:10 dalek parrot: luben++ committed some things to trunk that this branch was working on. Useless now
01:10 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49008/
01:10 dalek parrot: r49009 | luben++ | trunk/src/call/args.c:
01:10 dalek parrot: fix using a hash after it is freed
01:10 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49009/
01:10 luben thanks for catching the bug
01:10 cotto How comfortable is tcurtis with git?  It looks like he's on the schedule for the first release manager after the git move.
01:12 whiteknight cotto: trial by fire? There will be plenty of support personel
01:12 cotto If he's not solid, I'd try to sucker dukeleto for it.
01:13 kid51 left #parrot
01:13 whiteknight do you really think some ninja-level git-fu is going to be required for a release?
01:13 whiteknight Most of the release process is filling out metadata and running fulltest
01:13 cotto not in general, just for the first one after the move
01:14 cotto in case we missed something
01:14 whiteknight If it involves more than making and pushing a tag, I'll be surprised
01:14 pmichaud fwiw, one huge advantage of git is that anyone can try out the release process in full.
01:14 pmichaud (at any time)
01:14 whiteknight true
01:14 cotto excellent point
01:14 Patterner left #parrot
01:14 pmichaud we found that out with rakudo; essentially, anyone can clone the rakudo repo and walk through the entire release process to see if there are any problems.
01:15 pmichaud without having to affect the canonical repo
01:15 cotto dukeleto (or anyone who cares) can do a dry run before 2.10 comes out to make sure there aren't any omissions
01:15 pmichaud I would highly recommend that.
01:15 cotto git++
01:15 whiteknight maybe we'll all drop the ball in a major way, and we won't be on git for 2.10
01:15 pmichaud also might be worth trying it out with the 2.9 release
01:16 pmichaud i.e., clone a git repo of parrot and pretend like one was doing the 2.9 release using git.  update the docs from there.
01:17 cotto we might even consider moving to got before 2.9 if all the pieces are in place
01:17 whiteknight or, if we do it in the present-tense, we can move to git by then
01:18 whiteknight :)
01:18 cotto either way, we're not moving until all the pieces are in place
01:23 rurban joined #parrot
01:32 esskar left #parrot
01:33 esskar joined #parrot
01:37 plobsing it appears that anything with an :outer gets a lexinfo
01:38 whiteknight yeah, I suspected something like that would be the case
01:38 whiteknight NQP is pretty good about not creating :outer subs if lexinfo isn't needed
01:38 cotto seen mj41
01:38 purl mj41 was last seen on #parrot 49 days, 8 hours, 8 minutes and 41 seconds ago, saying: Lost connection to server irc.perl.org.  [Jul 27 17:30:16 2010]
01:38 aloha mj41 was last seen in #perl6 1 days 9 hours ago joining the channel.
01:40 luben I have made callcontext to not create hashes when not needed (in get_X_keyed_str vtables)
01:40 plobsing whiteknight: !metaclass_method_forwarder gets an outer but has no .lex vars. why does it need a lexinfo?
01:40 luben empty hashes are down to 28k
01:41 whiteknight plobsing: I'm not familiar with that function
01:41 luben now I'll test it and ship
01:42 whiteknight blarg. parrot-instrument does't build against trunk
01:43 cotto khairul was working on using distutils when I last checked.  I think the effort was incomplete.
01:44 whiteknight plobsing: ping
01:44 whiteknight cotto: haven't heard from him in a while?
01:45 cotto nope
01:45 whiteknight that's disappointing
01:45 cotto he's probably busy with school
01:45 plobsing whiteknight: pong
01:45 whiteknight plobsing: parrot-instrument is having some build issues involving the interp struct. Do you know what happened with interp->save_func_table?
01:46 cotto He didn't give me a commit bit, but forking is easy
01:46 s1n left #parrot
01:46 whiteknight cotto: I already forked it
01:46 cotto commit bit plz?
01:46 plobsing whiteknight: interp->save_func_table got moved to code_segment->save_func_table
01:46 plobsing after dynops mapping, each cs has its own func table (to accomodate different dynop load orderings)
01:47 whiteknight cotto: done
01:48 luben When I should use GET_ATTR_attrname and when GETATTR_PmcName_attrname ?
01:48 whiteknight plobsing: can you take a quick look at this function? http://github.com/Whiteknight/parrot-instrument/b​lob/master/src/dynpmc/instrumentruncore.pmc#L704
01:48 whiteknight luben: the former is for private access inside the PMC itself
01:48 whiteknight the later is for general access everywhere else
01:48 luben OK, thanks
01:48 plobsing GET_ATTR_attrname could be ambiguous outside the .pmc file so isn't accessible
01:49 cotto the short form is mangled into the long form by pmc2c for VTABLE functions
01:49 cotto and I think for METHODS too
01:50 hercynium left #parrot
01:52 plobsing whiteknight: detecting whether a dynoplib has been added has become easier - interp.n_libs will have incremented
01:52 whiteknight plobsing: okay, I figured things had changed there. I haven't been keeping up with your oplibs work like I should have
01:52 whiteknight I think I need to fix the setup.pir code here first to make this crap build, then I can go about fixing/debugging the C
01:52 plobsing whiteknight: bus number is dangerously low there.
01:53 plobsing more eyes welcome
01:53 whiteknight plobsing: I'll do my best to familiarize myself with it. Do you have any docs or summaries to point at, or just the POD in the files?
01:54 dalek tracwiki: v31 | cotto++ | GitMigration
01:54 dalek tracwiki: give git migration more solid dates, add tentatively responsible parties
01:54 dalek tracwiki: http://trac.parrot.org/parrot/wiki/Gi​tMigration?version=31&action=diff
01:54 plobsing whiteknight: there's not a lot of doc. I've added functions and put a little POD in where appropriate, but there really isn't anywhere obvious to put documentation for this
01:55 whiteknight okay, that's fine. I'll meander around and see what I see
01:55 whiteknight but it's late now, so I'm heading to bed. Laterz
01:56 whiteknight left #parrot
01:58 s1n joined #parrot
01:59 dukeleto hola
01:59 purl niihau, dukeleto.
02:00 * dukeleto gives purl a penny
02:01 dalek parrot: r49010 | luben++ | trunk/src/pmc/callcontext.pmc:
02:01 dalek parrot: Do not create empty hashes in get_X_keyed_str vtables
02:01 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49010/
02:02 cotto dukeleto, how does the git migration schedule look as far as stuff I put you down for?
02:05 nopaste "luben" at 192.168.1.3 pasted "updated hash usage stats" (13 lines) at http://nopaste.snit.ch/23346
02:09 * cotto thinks of a better approach
02:10 chromatic parrot_hash_clone(INTERP, hash, get_hash(INTERP, dest)) ?
02:10 chromatic ah, that's fine
02:10 luben just the formating is not ok...
02:11 luben I missed a tab
02:13 chromatic I was thinking something else, but I was wrong.
02:15 chromatic Hm, 3.18% faster Rakudo startup in the past few days.  Very nice.
02:19 plobsing luben: how are you instrumenting the hashes?
02:20 luben just printing to stderr in hash_destroy and counting
02:24 chromatic left #parrot
02:25 cognominal left #parrot
02:35 janus left #parrot
02:35 chromatic joined #parrot
02:35 janus joined #parrot
02:42 chromatic luben, I only see 33,053 hashes created for Rakudo startup.
02:42 chromatic ... but 94,201 destroyed.
02:43 atrodo chromatic> well, I'm working on a default lookup method.  after that, it should be ready to start playing with writing other pmcs
02:43 atrodo and how does it do that?
02:44 cotto that's very ... destructive
02:46 chromatic Sounds good.
02:46 luben In my callgrind profile parrot_create_hash was called 93486 times. hash_destroy 93152 times
02:47 chromatic Strange.
02:47 purl But true.
02:49 cotto Isn't there a way to instrument functions so that valgrind will treat them as a malloc/free pair and track any that leak?
02:50 cotto or would that be redundant, since hashes have a chunk of memory associated with them
02:52 plobsing cotto: isn't that what PARROT_MALLOC is supposed to do somehow?
02:58 plobsing if a sub doesn't have any .lex declarations, does it need a lexinfo (it would be empty in this case)?
03:00 davidfetter joined #parrot
03:00 chromatic It shouldn't, no.
03:01 plobsing luben: can you recount empty hash creation after r49011 (requires rakudo rebuild to take effect)?
03:02 luben ok
03:03 Patterner joined #parrot
03:07 luben plobsing, 25294
03:07 plobsing huh, not as much as I had hoped
03:08 dalek parrot: r49011 | plobsing++ | trunk/compilers/imcc/pbc.c:
03:08 dalek parrot: avoid creating unnecessary lexinfos for subs with :outer s
03:08 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49011/
03:11 nopaste "luben" at 192.168.1.3 pasted "updated hash stats" (13 lines) at http://nopaste.snit.ch/23348
03:19 dalek tracwiki: v32 | cotto++ | GitMigration
03:19 dalek tracwiki: http://trac.parrot.org/parrot/wiki/Gi​tMigration?version=32&action=diff
03:30 chromatic Still an improvement.
03:30 purl an improvement is external to the program contruct.. knuth may be saying there is no advantage going multithread inside the program?
03:33 plobsing what made all the other hash stats drop?
03:33 plobsing other being non-empty
03:33 chromatic Smartening up CallContext.
03:35 davidfetter left #parrot
03:41 luben how do I regenerate core_ops.c ?
03:41 plobsing ./ops2c --core
03:41 cotto nice response time
03:42 luben yes, plobsing++
03:42 plobsing muscle memory
03:42 purl i heard muscle memory was sometimes cruel
03:57 luben now, how do I regenerate vtable.h ?
03:57 plobsing that one I've never had to do
03:58 luben I am ripping deprecated logical operation vtables
03:58 plobsing woohoo! good ridance!
04:00 chromatic Modifying vtable.tbl should suffice.
04:00 luben I think I found how :) tool/build/vtable_h.pl
04:01 luben oo, yes, it is automatically rebuild
04:06 nwellnhof_ joined #parrot
04:08 hercynium joined #parrot
04:09 nwellnhof left #parrot
04:09 nwellnhof_ is now known as nwellnhof
04:40 hercynium left #parrot
04:49 tcurtis joined #parrot
05:13 rurban_ joined #parrot
05:15 rurban left #parrot
05:15 rurban_ is now known as rurban
05:30 theory left #parrot
05:35 dalek tracwiki: v2 | cotto++ | GitHubTracPluginTests
05:35 dalek tracwiki: reorganize a bit, add another test case
05:35 dalek tracwiki: http://trac.parrot.org/parrot/wiki/GitHub​TracPluginTests?version=2&action=diff
05:40 dalek parrot: r49012 | luben++ | trunk (10 files):
05:40 dalek parrot: Ripped out deprecated logical VTABLES
05:40 dalek parrot: Parrot nead realclean and all depending code should be rebuild.
05:40 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49012/
05:40 dalek parrot: r49013 | luben++ | trunk/src/pmc/callcontext.pmc:
05:40 dalek parrot: fix indentation
05:40 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49013/
05:41 luben When removing deprecated struff, does it needs to be removed from DEPRECATED.pod?
05:46 chromatic yes
05:46 plobsing luben: yes. also put it on the wiki.
05:47 plobsing follow the steps at "How To Deprecate"
05:47 luben plobsing++
05:48 cotto Ideally there'll already be a notification on the wiki.  Don't assume it though.
05:48 cotto luben++ for ripping out some cruft
05:48 luben I am reading the thicket again. As I understand it, it suggest also removing logical ops pn PMC
05:48 cotto did you check that it doesn't make Rakudo cry?
05:49 luben cotto, yes
05:49 cotto good for you
05:49 luben I have converted logical ops with PMC args to continue to work, they use VTABLE_get_bool internally
05:53 luben should I remove them?
05:58 * cotto is glad his dayjob mostly didn't involve manual testing
06:04 dukeleto cotto: yeah
06:08 dukeleto we really need to remove the SVK info from parrot.org
06:09 dukeleto updating parrot.org/download is another thing for the migration
06:09 cotto I'd probably be using svk if it'd had a release in the recent past.
06:09 cotto +1
06:09 purl 1
06:09 cotto botkick
06:10 dalek tracwiki: v3 | cotto++ | GitHubTracPluginTests
06:10 dalek tracwiki: flesh out most of the tests
06:10 cotto dukeleto, are you adding it?
06:10 dalek tracwiki: http://trac.parrot.org/parrot/wiki/GitHub​TracPluginTests?version=3&action=diff
06:12 dukeleto cotto: just did
06:12 cotto axesome
06:13 dukeleto cotto: nice work on the github trac plugin tests
06:13 cotto thanks
06:16 * dukeleto worked with OSU+particle to get smolder working, but it is still broken
06:17 cotto dukeleto, who'd be the best person to take point for the various bits of the git migration that'll need osuosl coordination?
06:18 dukeleto cotto: not sure. hanging out in #osuosl on freenode helps
06:18 * cotto joins
06:21 nwellnhof left #parrot
06:26 dukeleto i just gave one of the OSUOSL guys the info for the user we need created
06:26 dukeleto we will help him fix it tomorrow, hopefully
06:27 dalek tracwiki: v33 | dukeleto++ | GitMigration
06:27 dalek tracwiki: http://trac.parrot.org/parrot/wiki/Gi​tMigration?version=33&action=diff
06:31 dalek parrot: r49014 | luben++ | trunk (2 files):
06:31 dalek parrot: Updated DEPRECATED.pod, bumped PBC_COMPAT
06:31 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49014/
06:33 uniejo joined #parrot
06:42 bacek_at_work left #parrot
06:45 bacek_at_work joined #parrot
06:46 bacek_at_work left #parrot
06:51 bacek_at_work joined #parrot
07:02 jsut_ joined #parrot
07:07 jsut left #parrot
07:14 chromatic left #parrot
07:31 luben_work joined #parrot
07:32 luben_work left #parrot
07:32 luben_work joined #parrot
07:33 tadzik joined #parrot
08:05 dalek rakudo: 3f6392d | patrickas++ | src/core/operators.pm:
08:05 dalek rakudo: Avoid copying all of the lhs since we only need  elements off of it
08:05 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/3​f6392dd0b59f64dda5d8b421db3a8b2f8a29cb6
08:25 bacek aloha, humans
08:36 tcurtis left #parrot
09:54 dalek parrot: r49015 | bacek++ | branches/gc_massacre/src/gc/gc_ms2.c:
09:54 dalek parrot: Rework triggering of GC MS2 by rough counting of allocated memory since last collect
09:54 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49015/
09:55 sorear bacek++ picking up the GC
10:32 bacek ooookey...
10:32 bacek We do need properly encapsulate "String GC". Without "shared buffer" Rakudo consumes too much memory for small substrings.
10:33 bacek msg whiteknight We do need properly encapsulate "String GC". Without "shared buffer" Rakudo consumes too much memory for small substrings.
10:33 purl Message for whiteknight stored.
10:33 aloha OK. I'll deliver the message.
10:44 dalek parrot: r49016 | bacek++ | branches/gc_massacre/src/gc/gc_ms2.c:
10:44 dalek parrot: Count PMC Attributes in used memory.
10:44 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49016/
10:51 wow hello
11:20 contingencyplan left #parrot
11:45 s1n left #parrot
12:05 tadzik left #parrot
12:33 bluescreen joined #parrot
12:37 patspam joined #parrot
12:57 sjn joined #parrot
12:58 sjn hey folks
12:58 sjn can anyone here tell me if Allison is on IRC? (and if so, what's her nick?)
12:59 sjn !seen allison
12:59 sjn seen allison
12:59 purl allison was last seen on #parrot 16 hours, 32 minutes and 39 seconds ago, saying: dukeleto: great (on porting)
12:59 aloha allison was last seen in #parrot 4 days 1 hours ago saying "msg kid51 commented on TT #905".
12:59 sjn there
12:59 sjn thanks, purl! :D
13:10 pmichaud can anyone think of a way to remove an exception handler (not necessarily the last) from the current list of active exception handlers?
13:12 rurban_ joined #parrot
13:15 rurban left #parrot
13:15 rurban_ is now known as rurban
13:18 Patterner left #parrot
13:18 Psyche^ joined #parrot
13:18 Psyche^ is now known as Patterner
13:23 whiteknight joined #parrot
13:26 whiteknight good morning, #parrot
13:27 whiteknight yay! messages!
13:30 atrodo It's always exciting to discover that someone wants you
13:39 luben_work plobsing, ping
13:49 dalek roast: d94794a | moritz++ | S32-io/note.t:
13:49 dalek roast: unfudge .note test for rakudo
13:49 dalek roast: review: http://github.com/perl6/roast/commit/d9​4794aee2ae01a1f1b5ff372ae59c366f34efef
13:50 moritz are you interested in the commits from roast? if not I can probably teach dalek to only report them to #perl6
13:50 dalek rakudo: a15b2b1 | moritz++ | src/core/Bool.pm:
13:50 dalek rakudo: implement Bool.pick
13:50 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/a​15b2b1c6f6911305f101de639f61da8d8fb77d5
13:50 dalek rakudo: 52bf6f3 | moritz++ | docs/ChangeLog:
13:50 dalek rakudo: update ChangeLog
13:50 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/5​2bf6f376d498565be353d657c23ed599a22eb7a
14:01 fperrad joined #parrot
14:03 dalek rakudo: 15c3f75 | moritz++ | src/core/Bool.pm:
14:03 dalek rakudo: remove Cool type constraint from Bool.pick
14:03 dalek rakudo:
14:03 dalek rakudo: It doesn't allow a Whatever-star, so it's certainly wrong. It now let the
14:03 dalek rakudo: re-dispatch to Parcel.pick do the type check or coercion, if any.
14:03 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/1​5c3f75925542a5d6e9eb4bcf7a0e75e8af49406
14:08 aloha left #parrot
14:09 bacek left #parrot
14:09 uniejo left #parrot
14:10 bluescreen left #parrot
14:11 NotFound pmichaud: ping
14:11 pmichaud NotFound: pong
14:11 aloha joined #parrot
14:11 NotFound pmichaud: I've ruminated several times if that removal will be useful. There is no current way, AFAIK
14:12 pmichaud NotFound: okay.
14:12 pmichaud I've since decided it's not the right way to go for what I'm working on.
14:12 NotFound Solved my doubt, then ;)
14:14 plobsing luben_work: pong
14:18 NotFound t/pmc/boolean.t Failed test:  -- 23 not ok 23 - 0xor1 == 1 for Booleans
14:18 dalek roast: d71f4c3 | moritz++ | S02-builtin_data_types/bool.t:
14:18 dalek roast: Bool.pick
14:18 dalek roast: review: http://github.com/perl6/roast/commit/d7​1f4c3086d509ed7d616bbae88a1616e092c734
14:20 bacek joined #parrot
14:23 fperrad left #parrot
14:24 fperrad joined #parrot
14:24 bluescreen joined #parrot
14:25 dalek parrot: r49017 | gerd++ | trunk/NEWS:
14:25 dalek parrot: some news for 2.8.0
14:25 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49017/
14:26 pmichaud what I'm looking for is a way to invoke a block from an exception handler with the current handler disabled for the duration of that block.
14:26 pmichaud oops, I'll rephrase into Parrot terms
14:27 pmichaud what I'm looking for is a way for an exception handler to invoke a Parrot Sub such that any exceptions thrown by that Sub won't be caught by the handler
14:27 NotFound pmichaud: let it propagate to the next handler?
14:27 contingencyplan joined #parrot
14:28 pmichaud I don't understand that phrase
14:28 NotFound pmichaud: if an exception happens, let the next handler in the chain, if any, handle it.
14:28 pmichaud how do I do that?
14:28 NotFound It was a question.
14:28 NotFound Is that the behaviour you want?
14:29 pmichaud if an exception happens for the sub the handler invoked, then yes, let it simply proceed to the next handler.
14:29 NotFound Let me look at it...
14:32 luben_work plobsing, I am working on deprecations of logical vtables
14:32 NotFound Hackish way that may be useful for testing: set min_severity on the EH to EXCEPT_exit + 1
14:33 Andy joined #parrot
14:33 pmichaud ...and then restore it when we get back from the Sub?
14:33 NotFound pmichaud: yeah
14:33 plobsing luben_work: cool. anything you need help with?
14:33 luben_work TT #1655. Yes
14:34 pmichaud I'm wondering if that means it's possible for it to not be restored, though.
14:34 luben_work I have removed vtables and have updated logical ops on PMCs to use get_bool/set_bool
14:34 NotFound pmichaud: the looking of the code is not much sphisticated, I think this simple approach should work well enough to test the idea.
14:35 luben_work the ticket suggests that we should remove the ops also. But I have not found a way to implement same functionality in pure PIR
14:35 pmichaud yeah, it might be sufficient to at least see what else might break.
14:36 NotFound pmichaud: if it serves your needs, we can add a 'disabled' attribute or something like that.
14:36 luben_work as s start, I do not how to call VTABLE_get_bool on any PMC (regardles of its type)
14:36 pmichaud there's already an unused 'invoked' attribute.
14:36 plobsing luben_work: it should be possible. don't we have some kind of get_bool op?
14:36 luben_work plobsing, no
14:36 luben_work I have double checked\
14:36 pmichaud (fossil from previous mechanism used to disable exception handlers)
14:36 NotFound luben_work: if
14:36 pmichaud istrue
14:36 pmichaud $I0 = istrue $P0
14:37 pmichaud (invokes the get_bool on $P0)
14:37 NotFound What we don't have is a way to call VTABLE_set_bool from pir
14:38 luben_work NotFound,  yes, that also is a problem
14:38 luben_work istrue will work for get_bool
14:39 NotFound pmichaud: the one in Continuation, you mean?
14:39 pmichaud NotFound: yes.  ExceptionHandler overloads get_integer and set_integer to twiddle that flag.
14:40 timbunce joined #parrot
14:40 plobsing set_bool should be added as it seems to be an omission
14:40 pmichaud but I think those overloads exist only to preserve backwards compability somewhere, not because they're actually ued.
14:40 pmichaud *used
14:42 luben_work plobsing, ok, I will work in that direction
14:42 NotFound plobsing: You mean an opcode?
14:43 timbunce left #parrot
14:43 plobsing NotFound: yes. if we have istrue, it is only logical that we be able to set it.
14:43 timbunce joined #parrot
14:44 timbunce what's the url for the parrot testing smolder instance?
14:44 NotFound plobsing: other than ortogonality, there is some reason that pays the price of an specific opcode?
14:45 moritz aloha: smolder?
14:45 purl smolder is http://sourceforge.net/projects/smolder or web-based smoke test aggregator used by developers and testers to upload (automated or manually) and view smoke/regression tests using the Test Anything Protocol (TAP). or http://smolder.plusthree.com/app​/public_projects/smoke_reports/8
14:45 fperrad left #parrot
14:45 aloha moritz: I have no idea.
14:45 pmichaud ...is there a way to get the pmc of the current exception handler from within the handler itself?
14:46 plobsing NotFound: there doesn't seem to be any other way of calling set_bool vtable from PIR
14:46 fperrad joined #parrot
14:46 plobsing seems like something PIR should be able to do
14:47 NotFound pmichaud: last time I looked at it, no. finalize accepts both the excepcion and the handler because of that.
14:48 NotFound plobsing: looks like people has survived a log time without tha hability. Just for testing purposes is not enough, IMO.
14:48 pmichaud I agree.
14:49 NotFound I don't like walking in circles, and I'm almost sure that in short time someone will suggest deletring it bcacues it's useless.
14:49 pmichaud and the only pmcs that appear to implement set_bool vtable are Integer, Float, and String.
14:49 pmichaud (and boolean)
14:49 pmichaud although it does get called from a variety of places.
14:54 theory joined #parrot
14:56 NotFound pmichaud: ah, yes, the handler_iter attribute in the Exception
14:56 pmichaud NotFound: Perfect.
14:56 purl perfect is the enemy of good enough.
14:56 pmichaud NotFound++
14:57 NotFound Look a t finalize in experimetal.ops
14:58 pmichaud eh = VTABLE_get_pmc_keyed_int(interp, iter, -1);
14:58 pmichaud ...is that safe?  ;-)
14:59 pmichaud i.e., I can be guaranteed that iter[-1] is always the previous thing iterated?
14:59 NotFound Safe enough to make finalize work
14:59 pmichaud I mean in the sense of "is this a valid public API I can rely on?"
14:59 NotFound Given that finalize is experimental, hardly a public API.
15:00 NotFound But if we want it to be a public API we just need to declare it as such and write some tests.
15:01 pmichaud I'd rather have a method on Exception to return its current handler
15:01 NotFound That feature, I mean, not finalize.
15:01 pmichaud right.
15:02 pmichaud put another way, perhaps  the current handler should be an attr of Exception PMC
15:03 NotFound pmichaud: I think a method is better. Public attributes exposes implementation details that maybe we need to change later.
15:03 pmichaud I'd be okay with that... although methods are currently very slow.
15:04 pmichaud I only recommend attribute because the other attributes are already exeposed in Exception PMC
15:04 NotFound The usage you have in mind is speed critical?
15:04 pmichaud no, not terribly.
15:04 pmichaud which is why I'd be okay with a method. :)
15:05 NotFound Then we can go for the method, and provide a faster way later with usage data if we need to.
15:06 NotFound s/with/based on
15:11 timbunce left #parrot
15:11 nopaste "plobsing" at 192.168.1.3 pasted "sources of empty hashes in rakduo" (16 lines) at http://nopaste.snit.ch/23357
15:11 NotFound Uh, pod in Exception.pmc is severily outdated.
15:11 masak joined #parrot
15:12 NotFound "When an exception handler is called, the exception object is passed as as the first argument, the message as the second argument of the call"
15:12 plobsing where are all of those hashes in bytecode comming from?
15:13 masak s/as as/as/
15:15 NotFound Someone has worked on http://trac.parrot.org/parrot/ticket/1406 or is just a good wish?
15:16 SingAlong joined #parrot
15:17 NotFound plobsing: I bet ParrotInterpreter and Namespace provide a bunch of them.
15:18 plobsing I dind't think namespaces were frozen to vtables
15:18 plobsing s/vtables/packfiles/
15:18 SingAlong k. just got back to punch thru the PCT Tutorial again and reproduce my earlier problem
15:18 Coke joined #parrot
15:18 NotFound plobsing: I never bet that parts of parrot don't do irreasonable things.
15:18 Coke msg bacek IWBNI aloha knew about private messages for seen.
15:18 purl Message for bacek stored.
15:18 aloha OK. I'll deliver the message.
15:19 Coke left #parrot
15:19 pmichaud std: rand()
15:19 p6eval std : OUTPUT«[31m===[0mSORRY![31m===[0m␤Unsupported use of rand(); in Perl 6 please use rand at /tmp/j38e0CHiZO line 1:␤------> [32mrand[33m⏏[31m()[0m␤Parse failed␤FAILED 00:01 114m␤»
15:21 nwellnhof joined #parrot
15:23 dmalcolm joined #parrot
15:23 nwellnhof t/pmc/boolean.t Failed test:  23
15:34 dalek parrot: r49018 | nwellnhof++ | trunk (2 files):
15:34 dalek parrot: [str] Introduce growable strings
15:34 dalek parrot: This greatly speeds up the concatenation of a long and a short string.
15:34 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49018/
15:34 dalek parrot: r49019 | gerd++ | trunk/NEWS:
15:34 dalek parrot: add and change NEWS for 2.8.0 release
15:34 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49019/
15:34 luben_work nwellnhof, I will look at boolean.t failures
15:35 dalek rakudo: 5507cf6 | masak++ | src/core/IO.pm:
15:35 dalek rakudo: [core/IO] created/modified/accessed/changed methods
15:35 dalek rakudo:
15:35 dalek rakudo: Not spec yet, but the spec sounded very non-committal, and I need those
15:35 dalek rakudo: methods now. :)
15:35 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/5​507cf6e7c61c58d559dfeffc05b0ff3f104a304
15:45 luben_work concern about logical ops on PMC deprecation: If we remove them we could not make logical operations on Bool PMCs
15:47 dalek rakudo: 323a672 | masak++ | docs/ChangeLog:
15:47 chromatic joined #parrot
15:47 dalek rakudo: [ChangeLog] new IO methods
15:47 dalek rakudo: review: http://github.com/rakudo/rakudo/commit/3​23a672ffe4d7cd54977d7dc8e9be25bde29a6b1
15:50 NotFound "Closed a lot of tickets" is a NEWS item?
15:51 NotFound Following that way, we can add an item: "Worked a lot"
15:51 whiteknight NotFound: we don't have a number of tickets closed yet, because we're not done closing them
15:52 whiteknight once we have a number, we can make NEWS more specific
15:53 NotFound whiteknight: I don't think number of a tickets, opened, closed or watever, is an interesting NEWS item.
15:54 NotFound I think that r49018 is a way of saying: "We said strings are immutable? Hohoho."
15:56 whiteknight Yes, r49018 is a little suspect, but I think it can be made to work correctly
15:57 whiteknight After all, immutable strings only guarantees that string headers point to buffers which appear to always be immutable. What we do to the underlying buffer can be more tricky
15:57 NotFound whiteknight: How on earth we can correctly mutate immutable things?
15:58 whiteknight think about it this way. We have a string A, which points to buffer "foo"
15:58 whiteknight if we grow that buffer and add more stuff to it, we have buffer "foobar", but string A still points to the same place and has the same length, so string A still is "foo"
15:59 whiteknight but now we can have string B pointing to the same buffer with length 6, and it says "foobar"
16:00 NotFound whiteknight: maybe that idea can be made to work, but at the cost of putting one more problem against multithreading.
16:00 whiteknight now think about the lower-level, inside the GC. we allocate a large pool and cut it up into buffers. any string might be directly next to any other string inside the pool
16:01 whiteknight NotFound: that's the thing, the buffers still never change contents once they are allocated. But we can play a trick inside the allocator, and write new bytes at an area directly next to the buffer, and treat that as a new, larger buffer without ever changing the first string
16:01 NotFound I think you are just missing the times of several heisenbugs a week.
16:01 chromatic Suppose the initial growable string came from a Class's get_string and represents the name of the class.
16:02 whiteknight I don't like the term "growable" here
16:02 chromatic Suppose the initial makeablewrongable string came from a Class's get_string and represents the name of the class.
16:02 whiteknight we have a buffer in memory with fixed contents, which never moves.
16:03 whiteknight if other strings choose to overlap their storage with that existing buffer, without ever modifying that "window", there's no change in semantics
16:04 chromatic ... until you want to ensure that the contents of a string never changes, at which point you sprinkle tests and copy operations throughout the system like holy water.
16:04 whiteknight when we concatinate, we can pretend to copy the original contents by simply pointing a new header at the same buffer, giving it a longer length, and writing more data in memory directly after the first buffer
16:05 whiteknight I disagree. There are no tests or holywater necessary here
16:05 whiteknight our strings are still immutable.
16:05 whiteknight and immutable strings don't need to be copied around
16:05 chromatic If the contents of a string change, that string has mutated.
16:05 whiteknight I'm saying specifically that the contents do not change
16:06 whiteknight A string is a header with a pointer and a length. If the pointer never changes, the length never changes, and the contents in that window never change, the string is immutable
16:06 nwellnhof The content never changes, there's only stuff appended to a buffer.
16:07 whiteknight if we happen to write new things in memory directly next to that original buffer, the original string is none the wiser
16:07 nwellnhof The only tricky thing is to figure out who is allowed to append to a buffer
16:07 whiteknight or how we guarantee that we have free space next to the buffer to append into
16:08 chromatic This relies on shared buffers?
16:08 nwellnhof yes, at least for the concat_s_s_s opcode
16:08 whiteknight what's wrong with shared buffers, if we guarantee that the contents are immutable?
16:08 nwellnhof i like shared buffers, too
16:09 whiteknight if I have two strings, "bar" and "foobarbaz", there's no reason we can't compact that down to 9 characters of total allocation
16:09 chromatic They have GC implications, but that's merely an implementation detail.
16:09 chromatic If this patch (which I'm starting to understand) merely shuffles buffer contents to avoid extra memory copying instead of changing the contents of strings, that's very different from my initial reading.
16:10 whiteknight I don't know what that patch does. I haven't read it closely enough. In theory though, if we end up sharing buffer space between overlapping immutable strings, that's not a problem
16:10 whiteknight it's like the idea of a cheap substring, but we create them in a different order
16:11 whiteknight if that patch doesn't do that kind of thing, it's probably wrong
16:11 NotFound Other than opinions about the idea, there is a proble with that code. You are modifying a, which is cost.
16:11 NotFound Yes, using a cast, but that just hides the problem.
16:11 NotFound You are lying to the compiler.
16:12 nwellnhof we have to use the const_cast thing with strings in many places.
16:12 NotFound If we are going that way we need to drop the const qualifier in all functions that take STRING* parameters.
16:12 NotFound nwellnhof: that is the problem, but incrementing it is not a solution.
16:14 NotFound We used to have that problem with PMC at one time. The times of lots of heisenbug everywhere.
16:14 chromatic We still have that problem with PMC and STRINGs because of the GC flags in their headers.
16:15 nwellnhof if we don't want the const casts, there's no other way than to change the function headers.
16:17 NotFound nwellnhof: at the risk of losing optimizations done by the compiler in favor of manual optimizations.
16:17 nwellnhof i have no problem with changing headers.
16:18 chromatic If this optimization works, it's probably more powerful than compiler optimizations.
16:18 NotFound chromatic: most string functions don't need to change PObj flags, unless we play tricks. The more tricks, the more the probem.
16:18 chromatic True.
16:18 NotFound Changing headers is a public API change.
16:19 NotFound Thus subject to depecation policy.
16:22 nwellnhof most of the other const cast ugliness is related to hash tables.
16:23 chromatic Lazily calculating hash values, right?
16:23 nwellnhof yes, that's one thing.
16:23 nwellnhof the other is that hash_put doesn't accept const values.
16:24 NotFound hashes are less problematic because direct acess to its internals is localized. But direct usage of the STRING content is everywhere, and we are providing macors to make it even more frequent.
16:26 NotFound The hashval of the string is another problem, yes.
16:30 NotFound My view in short: we can go the way of being creative with the string buffers, or we can take benefit of the imutability by using const evrywhere. But not bot things at the same time.
16:30 masak left #parrot
16:33 chromatic The biggest problem of string concatenation seems to be the recycling of unused STRING headers.
16:33 chromatic That's a GC problem.
16:34 NotFound If that is the case, attempts of optimizations in the buffers will have low impact.
16:35 chromatic They could have some.
16:43 tcurtis joined #parrot
16:45 whiteknight is there anything that we can do to avoid the churn of string headers like that?
16:45 fperrad left #parrot
16:45 chromatic Rewrite into StringBuilder.
16:46 whiteknight yeah, that's what I figured you would say
16:46 whiteknight which is something I'm in favor of
16:46 chromatic If we had working escape analysis, we could recycle garbage more frequently.
16:47 nwellnhof we can't force every HLL to use StringBuilder.
16:47 whiteknight I personally would probably like to see STRINGs disappear entirely as a separate low-level type, and do everything through either String or StringBuilder
16:47 whiteknight in fact, we would only need StringBuilder, and rename it "String"
16:47 NotFound I don't think we must optimize for cases that programmers will avoid an even compilers can handle.
16:48 pmichaud 16:33 <chromatic> The biggest problem of string concatenation seems to be the recycling of unused STRING headers.
16:48 pmichaud 16:33 <chromatic> That's a GC problem.
16:48 pmichaud ...given my test that showed that disabling GC made the string concatenation take longer overall, is it still really a 'gc' problem?
16:48 pmichaud or is "memory allocation" being included as part of "GC"?
16:49 whiteknight pmichaud: that's likely due to the performance of the string compactor, which is decidedly part of the GC
16:49 chromatic Not just the string compactor, but marking every live GCable in the program repeatedly.
16:49 pmichaud if GC is disabled, we still mark?
16:49 chromatic Then sweeping every live GCable in the program repeatedly.
16:49 whiteknight chromatic: GC is off
16:49 chromatic No, but we do page.
16:50 chromatic and swap
16:50 chromatic and swap
16:50 chromatic and swap
16:50 NotFound I think this is one more argument against tricks with string buffers. If each string buffer is owned by just one string, deallocating it is trivial and does not need gc.
16:50 pmichaud the resulting string in my example isn't that big -- only 140K or so
16:50 whiteknight when the string pool fills up, we allocate a new, larger pool and copy all existing contents over
16:50 pmichaud I'm pretty sure I never hit swap
16:50 chromatic How many concatenations does that example perform?
16:50 pmichaud 25,000
16:50 whiteknight without GC to trim that pool down, we're carrying around a lot of cruft for each copy
16:51 pmichaud well, 50,000 I guess
16:51 chromatic Let's cut that 140k in half then multiply it by 25,000.
16:51 pmichaud rakudo:  70000*25000
16:51 purl 1750000000
16:51 p6eval rakudo 15c3f7:  ( no output )
16:51 whiteknight ...and swap
16:52 pmichaud yeah, that's 1.7 gb there
16:52 chromatic All of those calls to malloc() add overhead that skew the benchmark.
16:53 NotFound With unshared buffers, one run of the gc can free all buffers owned by all temprary string headres discarded.
16:54 pmichaud interestingly, when I run no-gc and load Rakudo into the interpreter, it takes 10 seconds less than with gc enabled
16:54 pmichaud basic program, no rakudo, with gc:  3 seconds
16:54 pmichaud basic program, no rakudo, gc disabled:  14 seconds
16:54 chromatic There's nothing for the GC to free when loading Rakudo.
16:54 pmichaud basic program, with rakudo, gc disabled:  21 seconds
16:54 pmichaud basic program, with rakudo, gc enabled:  30 seconds
16:55 pmichaud right, understood that the 'mark' phase adds a lot of overhead when rakudo is loaded
16:55 nwellnhof pmichaud: i never got your test case in #1789 to run in anything near 3 seconds.
16:55 chromatic Not necessarily mark.
16:55 nwellnhof it always takes at least 15 seconds.
16:56 NotFound nwellnhof: buy a faster machine ;)
16:58 pmichaud anyway, I agree with the earlier discussion about r49018 -- it looks to me like it just reintroduced copy-on-write semantics.
17:00 nwellnhof i don't know the old COW string implementation, but i guess that r49018 reintroduces some of it.
17:01 NotFound Againg, I don't like walking in circles,
17:04 pmichaud perhaps I have a very biased view, but to me, it seems that the most common (unique) operations on strings tend to be concatenation, substring, and substitution.  Those have to be fundamentally fast in Parrot.
17:05 NotFound pmichaud: but not necesarily for worst case scenarios unlikely to happen.
17:05 pmichaud repeated concatenations to a string are hardly "unlikely to happen"
17:06 NotFound pmichaud: systems with immutable strings have stringbuilders ot the like for some reason.
17:06 pmichaud NotFound: but not every language has a stringbuilder.
17:06 chromatic Parrot has a StringBuilder.
17:06 pmichaud you can say that Parrot only efficiently supports languages that do things in terms of stringbuilding, but that knocks out Perl 6.
17:07 NotFound pmichaud: is someone knows a way of having the best of both worlds, I'm all ears.
17:08 pmichaud NotFound: if immutable strings are fundamentally incompatible with Parrot's target languages, then I think immutable strings have to go.
17:08 pmichaud I'm not saying they are incompatible.
17:08 whiteknight pmichaud: if you're using a String PMC, and that String PMC does efficient string building internally, does that matter to you?
17:09 pmichaud I'm saying that if it turns out that they are, then Parrot gets to pick which to support.
17:09 NotFound A possible solution, but costly, is to make the String PMC more complicated.
17:09 pmichaud whiteknight: Strings are largely opaque to me.  I don't really care if they're in PMCs or string registers (more)
17:09 whiteknight that's my point exactly. Strings as a low-level type are not tenable really
17:09 whiteknight Strings as PMCs allow us to have low-level details and abstract them away from the user
17:10 pmichaud the only reason I use string registers now is because nearly all of the string opcodes require them to be there.
17:10 pmichaud certainly all of Rakudo and NQP's strings that get stored in lexicals are in String PMCs
17:10 pmichaud (or stored as attributes)
17:10 * dukeleto waves hello
17:10 chromatic Well yeah, that's the only way you can store them right now.
17:11 pmichaud Most of NQP and Rakudo's choice about how to handle strings is completely dictated by the API parrot provides, not by language design choice.
17:11 whiteknight if we ever want to have reliably good performance across the board, we can't be exposing our low-level string types to the user
17:12 whiteknight strings are too much of an icky implementation detail for anybody outside parrot core to have to worry about
17:12 NotFound whiteknight: winxed uses native strings without any problem.
17:12 pmichaud NotFound: can winxed duplicate the sample PIR program I gave?
17:13 whiteknight NotFound: and if I write a Winxed program that concatinates lots of strings, your performance will go down too
17:13 whiteknight and that's a stupid thing to be telling our end users
17:13 pmichaud whiteknight: yes, but that's not a function of the existence of string registers.
17:13 chromatic "If you write code that's slow, your code will be slow."
17:13 whiteknight "we provide concat, but if you're stupid enough to use it, everything goes to hell"
17:13 chromatic That's a fine thing to tell end users.
17:13 pmichaud It just means that the STRING type may be insufficiently advanced.
17:14 whiteknight STRING isn't really a type. It's a very low-level form of storing text
17:14 NotFound pmichaud: the one if the ticket 1789?
17:14 pmichaud NotFound: Yes.
17:14 whiteknight a PMC is the kind of object with an interface and an abstraction layer that people should be using
17:14 NotFound Mmm.. just tell me how to use rand for pir.
17:15 chromatic Hypothetical: if we deprecated every user-facing use of a STRING and replaced it with a String PMC, would that improve performance?
17:15 pmichaud $I0 = rand 10000   # get a random integer up to 10000
17:15 chromatic Bonus question: if not, why?
17:15 NotFound Oh, I was looking to the perl, sorry.
17:15 chromatic Super bonus question: if not, what's the *real* problem?
17:15 whiteknight chromatic: I believe it would, overall
17:15 pmichaud chromatic: if we simply replaced String register with String PMC, I wouldn't expect any performance improvement.
17:15 chromatic I invite you to do some profiling then, because I believe it would not.
17:15 pmichaud I can even prove it.
17:15 whiteknight because internally, String can act a hell of a lot like String builder, and concat costs drop way down
17:15 chromatic Not only in the program, but every PIR-facing use of STRING.
17:16 pmichaud whiteknight: STRING can act like String builder too.
17:16 pmichaud whiteknight: if you can do it for a PMC, you can do it for String also.
17:16 whiteknight pmichaud: it really cant. STRING is too low-level
17:16 pmichaud *for STRING also
17:16 whiteknight STRING isn't an object type. Doesn't have methods. Has limited storage. Isn't subclassable
17:17 pmichaud I'm not claiming that one can replace String with STRING, though.
17:17 pmichaud I don't see how that applies here.
17:17 * dukeleto attempts to help OSUOSL folks figure out the smolder issue
17:18 cotto dukeleto++
17:18 * whiteknight waves a "dukeleto is #1 flag"
17:18 whiteknight chromatic: If we got rid of externally-facing STRINGs, there's no real reason for String PMC to contain a STRING header as an attribute
17:19 whiteknight we can store a pointer to the buffer directly. That's one less GCable per String PMC
17:19 chromatic Is the content of this ubiquitous String PMC mutable?
17:19 dalek roast: 8e4955e | KodiB++ | S02-builtin_data_types/ (2 files):
17:19 dalek roast: Added tests for Sets and Bags.
17:19 dalek roast: review: http://github.com/perl6/roast/commit/8e​4955e60e6ea9da5dde0d1130e21fcac352dc2e
17:19 whiteknight The buffer itself is immutable. But we can repoint our pointer at different buffers if we need to
17:20 whiteknight or we can make the PMC itself immutable if somebody wants that. Or we can have multiple String types that do whatever people want their strings to do
17:20 chromatic From the user point of view, if I have a String PMC in $P0 and do something with that PMC, say pass it to a function, will the contents of that String PMC remain the same *by guarantee enforced by Parrot itself* after the call?
17:20 whiteknight we do have (a lousy implementation of) readonly PMCs
17:21 whiteknight chromatic: that's an implementation detail of the PMC type itself
17:21 chromatic Then get used to cloning PMCs.
17:21 whiteknight we can have *many* String-like PMC types
17:21 whiteknight The caller can click the read-only flag on the PMC before passing it
17:22 chromatic Yeah, I fixed a handful of bugs like that with mutable STRINGs.
17:22 chromatic Do you know one reason why immutable STRINGs were so much faster than mutable STRINGs?
17:22 whiteknight I'm not engaging in a design discussion based on the fact that we've had bugs in previous implementations of things
17:22 NotFound real    0m2.289s
17:22 NotFound user    0m2.264s
17:22 NotFound sys     0m0.024s
17:22 whiteknight or that software occasionally contains bugs, regardless of the algorithm
17:23 chromatic You have to face the bugs inherent in a design decision.
17:23 whiteknight chromatic: it's sounding like, to me, that immutable strings are *slower* in some cases
17:23 chromatic Yes, and faster in many cases.
17:23 whiteknight if our users want to do a shittonne of concats, that's a very real performance consideration
17:23 chromatic If you are confident that you can audit the whole of your program as well as all of the C code that might possibly manipulate any of the values of your program, you can get away with flipping that flag only in the cases where you need it.
17:24 chromatic (if it is indeed as cheap as flipping a flag)
17:24 NotFound pmichaud: the time looks aproximate to the pir program. More testing is pointeless because the winxed generated code is almost identical to the pir.
17:24 pmichaud NotFound: right.  Now try the winxed program after loading 'perl6.pbc'  :-)
17:24 pmichaud or, tell me how you would write the winxed program differently to avoid the repeated concats.
17:25 atrodo chromatic> That always seems odd to me.  Every time any language has touted immutable strings (java/c#/javascript) people run funny hacks and extra magic to have reasonable speed for often changing strings
17:25 NotFound pmichaud: using StringBuilder
17:25 pmichaud (and then tell me how the p5 programmer should rewrite his program to be faster)
17:25 atrodo but I think that's implementation after seeing parrot did it
17:25 whiteknight immutable strings are great until somebody wants to mutate a string. We can't act all surprised and say "ur doin it wrong" every time somebody calls concat_s_s_
17:26 chromatic If you refuse to see the design tradeoff, you're going to come up with a lousy design.
17:26 whiteknight and if we don't want people using concat_s_s_s, take it off the table
17:26 whiteknight I see the design tradeoff
17:26 chromatic I don't think you do.
17:26 pmichaud (and then tell me how the php programmer should rewrite his program to be faster)
17:26 NotFound That's my point. We can't switch to mutable to immutable to mutable... every time one problem of the current approach hits someone.
17:26 whiteknight if we want to fix the concat problem, we take concat_s_s_s away and replace it with concat_p_s_s, which always returns a StringBuilder
17:26 atrodo pmichaud> (use perl?)
17:27 pmichaud atrodo: :-)
17:27 pmichaud atrodo: that is exactly what the bulk of users would end up doing :)
17:27 whiteknight and we have a concat_p_p_p, which always returns a StringBuilder
17:27 chromatic Okay, now someone answer the *root* question, which is "Why is SB faster than repeated concat_s_s_s?"
17:28 atrodo pmichaud> or if you're facebook, write a php->c translator...
17:28 whiteknight because StringBuilder isn't reallocating and copying data around on every operation
17:28 chromatic No.
17:28 nwellnhof SB saves some string headers
17:28 chromatic YES
17:28 whiteknight because StringBuilder contains magical fairies and unicorns
17:29 whiteknight chromatic: we can get rid of STRING headers *entirely*
17:29 chromatic No.
17:29 NotFound Magical pony-sized unicorns.
17:29 whiteknight yes
17:29 pmichaud Can I concatenate a string with a StringBuilder?
17:29 pmichaud if no, why not?
17:29 pmichaud oh, it's inplace.
17:29 pmichaud :(
17:29 chromatic You can remove the STRINGs from pmichaud's benchmark and demonstrate the same problem *without concat*.
17:30 whiteknight not to the same degree. concat is a special kind of drain
17:30 chromatic Try it.
17:30 * pmichaud tries it.
17:30 whiteknight there are problems with strings throughout, but concat is especially lousy
17:30 chromatic Try it.
17:31 pmichaud removing the concats in my version makes the program run in under a second.
17:31 pmichaud both with and without perl6.pbc loaded.
17:31 chromatic Replace all of the STRING manipulation with the single line $P0 = box 1.
17:31 chromatic Run it with and without loading perl6.pbc.
17:32 pmichaud without loading perl6.pbc, 0.030 sec
17:32 whiteknight so you're saying that because we have other performance problems, that string handling and concat in particular are not far slower than they should be?
17:32 pmichaud with loading perl6.pbc 0.730 sec
17:32 pmichaud I must be misunderstanding the thing I'm supposed to change.
17:32 * pmichaud nopastes
17:33 chromatic I'm saying that concat exposes the real problem.
17:33 chromatic concat is not the real problem.
17:33 pmichaud http://gist.github.com/581102  # replacing concats with 'box 1'
17:33 whiteknight concat is *a* real problem
17:33 whiteknight we have many real problems
17:33 chromatic Run the profile, whiteknight.
17:33 chromatic Look at it.
17:34 chromatic The performance characteristics are the same *without concat* in this example.
17:34 pmichaud chromatic: that's not what I'm seeing.
17:34 nopaste "chromatic" at 192.168.1.3 pasted "no concat; same characteristics" (14 lines) at http://nopaste.snit.ch/23361
17:35 chromatic Real time without loading perl6.pbc: 0m0.017s
17:35 chromatic Real time with loading perl6.pbc: 0m1.834s
17:35 pmichaud yes, but how long does it take to load perl6.pbc ?
17:35 pmichaud in the string concat example, it's negligable relative to the overall performance.  In this example, it dominates.
17:36 whiteknight because in the concat example, the concats increase overall cost by several seconds, not several hundredths of a second
17:36 whiteknight the PMC-only example is significantly cheaper than the string concat example
17:37 whiteknight because string concat is expensive
17:37 chromatic Fine, whatever.
17:37 chromatic Remove STRINGs.
17:38 whiteknight We don't even need to remove strings. I would like to in the long run but that isn't really necessary for this one example
17:38 whiteknight removing concat_s_s_s and replacing it with concat_p_s_s is sufficient here
17:38 whiteknight changing String PMC so it contains a pointer to the buffer directly, instead of wrapping a STRING removes the need for one GCable per String PMC
17:39 pmichaud http://gist.github.com/581115   # reworking chromatic's example
17:40 pmichaud there's another possibility, fwiw
17:40 chromatic I overwrite the previous string concat example.
17:40 Coke joined #parrot
17:40 pmichaud well, perhaps not.
17:41 Coke is there a failure in t/pmc/boolean.t?
17:41 pmichaud I was thinking that if StringBuilder PMC would support the  concat p_p_sc  opcode, then Rakudo could simply switch to using StringBuilder instead of String as its base string class
17:41 NotFound winxed version: string with concat: 0m2.183s StringBuilder with append_format: 0m0.124s
17:41 luben_work Coke, yes, I am wirking on it
17:42 NotFound Even with method call overhead and format parsing
17:42 pmichaud NotFound: I don't dispute that using StringBuilder is faster than String.
17:42 Coke luben_work: ok.
17:42 pmichaud NotFound: I'm saying that it's not a trivial task to get language XYZ to be able to optimize to use StringBuilder instead of String.
17:42 whiteknight if StringBuilder does not support the concat_p_p_sc opcode yet, it certainly should soon
17:43 pmichaud and we can't just tell all of the Perl/PHP/Python programmers out there that they have to rewrite their source to make it possible for the compiler to use StringBuilder instead of String.
17:43 pmichaud whiteknight: I'm not sure that will help.
17:43 whiteknight This is why I really think String and StringBuilder should be merged together. The end user shouldn't have to care about performance characteristics so long as the interface stays the same
17:43 chromatic -1
17:43 purl -1
17:43 pmichaud Because with   $P0 = concat $P1, $S2,    $P0 has to be a new StringBuilder PMC -- we can't reuse the one in $P1
17:44 whiteknight pmichaud: it can be a new String PMC instead.
17:44 pmichaud whiteknight: and if I want to concatenate again, what do I do?
17:44 pmichaud move it to a new StringBuilder PMC?
17:44 NotFound Is even possible to return a StringBuilder from the conctenation operation on String
17:44 Coke left #parrot
17:44 whiteknight the three-argument form always does create a new PMC for the result
17:44 whiteknight the two-argument form can do it in place
17:45 whiteknight that's nothing new
17:45 pmichaud I'm only using the three argument form, period.
17:45 pmichaud Forget the two argument form exists -- it's completely irrelevant to Rakudo.
17:45 purl pmichaud, I didn't have anything matching two argument form exists -- it's completely irrelevant to rakudo
17:45 whiteknight what's the current behavior of the three-arg concat? it returns a new String, or returns the same String?
17:45 pmichaud returns a new one.
17:45 pmichaud it has to.
17:45 whiteknight right
17:46 NotFound Sometimes purl seems a magical fairy
17:46 whiteknight if we get rid of StringBuilder, and add magical cheap concat to String, we can pass the string to concat and get back a new String from it
17:46 whiteknight just like we do now, but internally it goes faster
17:47 pmichaud whiteknight: my point is that we can verify this without that work.
17:47 pmichaud if you add magical cheap concat to StringBuilder (which we don't have now), then I can benchmark it without having to do anything with String.
17:48 pmichaud if it works, then we can consider migrating StringBuilder's internals into String
17:48 whiteknight I can fix that tonight so StringBuilder works nicely with concat_p_p_sc
17:49 chromatic What happens when it doesn't run (much) faster?
17:49 pmichaud I can have the test program that uses StringBuilder in just a couple of minutes, once that's available.
17:49 pmichaud how does StringBuilder do its concatenation operations now, ooc?
17:49 chromatic pmichaud, can you paste your concat benchmark again?
17:49 pmichaud it's in the ticket :)
17:49 pmichaud but sure, I can paste.
17:50 chromatic Which ticket?
17:50 purl Which ticket are you referring to?
17:50 pmichaud 1789, I believe
17:50 whiteknight pmichaud: StringBuilder stores the STRINGS in an array, keeping a running tally of total size and best-fit encoding/charset
17:50 dalek parrot: r49020 | coke++ | trunk (3 files):
17:50 chromatic Thanks.
17:50 pmichaud whiteknight: I meant what's the API for concatnenation
17:50 NotFound pmichaud: it grows a buffer by more or less doubling it each time gets full
17:50 dalek parrot: Fixup some makefile dependencies for chromatic++
17:50 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49020/
17:50 dalek parrot: r49021 | Paul C. Anagnostopoulos++ | trunk (4 files):
17:50 dalek parrot: First round of improvements to Parrot debugger
17:50 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49021/
17:50 pmichaud not "how does it do it internally"
17:50 whiteknight pmichaud: I don't know, I have to look at it.
17:50 whiteknight I don't know why it wouldn't work already
17:51 cognominal joined #parrot
17:51 NotFound pmichaud: AFAIK it only knows push_string and the append_format method
17:52 NotFound Uh, no, it also has concatenate_str
17:52 pmichaud http://gist.github.com/581137  # what I get when trying to concat to a StringBuilder PMC
17:52 NotFound i_concatenate_str and i_concatenate
17:52 pmichaud those are the inplace (2-arg) forms
17:52 pmichaud I need a 3-arg form.
17:53 whiteknight great. MMD problem. we'll get that sorted out tonight
17:53 pmichaud it's not really an MMD problem, it's a NYI problem I think  :)
17:53 NotFound A three-arg form tha does not do a lot of copying will be tricky.
17:54 pmichaud correct -- the three-arg form still requires copying (which is why it's worth benchmarking before undertaking whiteknight's proposal)
17:55 pmichaud however, in this case it should copy the array of STRINGs instead of the STRINGS themselves.
17:55 pmichaud that could be faster.
17:55 NotFound Is not using an array.
17:55 NotFound Maybe it should.
17:55 pmichaud oh.  (I was going by whiteknight's comment above that it did)
17:56 pmichaud right, if it's a buffer.... we're back to copying.
17:56 whiteknight it may be a tree
17:56 whiteknight I am not looking at the code right now
17:56 NotFound I think they talked about using an array of string during its implementation, but later switched.
17:56 whiteknight the exact details aren't really significant
17:57 pmichaud anyway, in all of this keep in mind that Perl 6's Str type is immutable
17:57 pmichaud i.e., Perl 6 doesn't need mutable strings.
17:57 chromatic Has anyone else profiled this code?
17:58 pmichaud (other languages might, but neither Perl 6 nor NQP are requiring mutable string capabilities internally-- they tend to model strings as immutable also)
17:58 pmichaud chromatic: I don't know if others have.  I'd welcome confirmation or refutation of the timings on the ticket.
17:59 pmichaud but so far it's been highly repeatable for me.
18:00 chromatic memcopy is 2% of runtime
18:00 chromatic (at least the memcopy within Parrot_str_concat)
18:00 chromatic Parrot_str_new_noinit() is a third of runtime.
18:00 chromatic (at least all of the calls from within Parrot_str_concat)
18:01 chromatic We have a few options here.
18:01 chromatic Let's discard what NotFound rightly characterized as a flipflop between mutable and immutable.
18:02 chromatic One option is to say "STRING concat is too slow" and try to fix that.
18:02 whiteknight I agree. We need to stick to our guns and keep immutable strings
18:02 chromatic One suboption is to try to fix it in place, such that users never have to rewrite the code that seems obvious.
18:02 chromatic Another option is to try to give them a fairly obvious idiom to which to rewrite their code using PMCs or SB or something.
18:02 theory left #parrot
18:03 NotFound I think we should test the approach of never sharing string buffers and freeing the buffer when the string header gets collected.
18:03 chromatic My preference is the third option, which addresses what I believe the profile shows is the real problem.
18:03 pmichaud NotFound: I think that's unimportant (more)
18:03 whiteknight I do not think Parrot should be in the business of providing or enforcing idioms. We are trying to support too many separate languages and projects to do that.
18:04 pmichaud NotFound: one of the advantages of immutability is that it's okay to share data.
18:04 whiteknight people will reject Parrot on the basis that it uses foreign-looking idioms
18:04 NotFound Don't forget that if a language has particular needs for its string type it can just HLLmap whatever pleases it.
18:05 whiteknight that's one saving-grace that I see. If perl6 wants immutable strings, they can implement that. If Python wants mutable strings, they can hll_map those in
18:05 atrodo chromatic> I must have missed it, what is the underlying problem?
18:05 whiteknight or, mutable-looking ones (if internally the buffers are all immutable)
18:06 NotFound pmichaud: it's okay to share the strings, but not necesarily parts of strings.
18:06 whiteknight but chromatic is right on the one point: our performance woes are much larger than the problems with strings and concat
18:06 pmichaud NotFound: if substrings are going to become expensive, then we need a lot more work on Parrot's overall API.
18:06 chromatic The underlying problem is that Parrot_str_new_noinit causes the GC to run, which takes up 32.73% of runtime.
18:06 atrodo wow
18:06 pmichaud chromatic: is that problem still true when GC is disabled, ooc?
18:07 pmichaud i.e., Parrot_str_new_noinit runs the GC even when it's disabled?
18:07 whiteknight no, it shouldnt
18:07 pmichaud (honest question, not rhetorical)
18:07 whiteknight but when GC is off, you run into several other performance-draining problems
18:07 chromatic Everything between Parrot_str_noinit (inclusive) and the GC is 0.28% of runtime.
18:07 dalek parrot: r49022 | luben++ | trunk/src/ops (2 files):
18:07 dalek parrot: fix bug in logical not on PMCs when dest == src
18:07 dalek parrot: Introduced in logical vtables removal
18:07 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49022/
18:08 whiteknight when GC is off, all that strain ends up on the string memory allocator, which is not sufficiently smart to operate without the GC in a performance-reasonable manner
18:08 pmichaud whiteknight: is the strain on the allocator itself, or on system resources?
18:08 whiteknight pmichaud: likely the allocator itself. It allocates large blocks which wouldn't cause too much fragmentation in the system
18:09 whiteknight but you do end up then with lots of swapping and thrashing, which is a system issue
18:09 whiteknight depends how you look at it
18:09 pmichaud well, I'm not hitting swap on my system.
18:09 whiteknight I haven't really looked at the performance of the allocator with GC turned off
18:10 whiteknight but I can tell you it was certainly never designed for that situation
18:10 pmichaud I'm wondering if the allocator gets worse as there are more objects allocated.
18:10 whiteknight in terms of strings? yes
18:10 pmichaud if that's the case, then it's not just concatenation that is a problem.
18:10 whiteknight fixed-size objects like headers are allocated slab-style, so they should weather it a little better
18:11 pmichaud it's anything that reads or creates strings
18:11 whiteknight chromatic: are you looking at kcachegrind now?
18:11 chromatic yes
18:11 whiteknight what chunk of the performance pie does the string compactor eat up?
18:11 pmichaud I'm guessing from what chromatic reported earlier (0.28%) that the problem is really more in GC than allocation, though.  not sure how that's being sliced.
18:11 pmichaud afk, lunch
18:11 dukeleto can someone do "make smoke" with r49023 and tell me what happens?
18:12 chromatic Hard to say what the string compactor does, because I'm using my GC patch.
18:12 dukeleto i am getting stupid errors when trying to run "make smoke"
18:12 chromatic With -G, memcopy becomes the most expensive part of Parrot_str_concat() -- 20% of runtime.
18:13 whiteknight If I remember correctly, the string allocator allocates a single large pool for all strings. When that pool fills up it allocates an even larger pool, copies everything from the first to the second, and frees the first
18:13 whiteknight that may be an old and simplified explanation
18:14 NotFound Scary
18:14 whiteknight if the GC never runs, the pool fills up with all the little string parts, and also all the intermediate concat results
18:14 whiteknight all that memory storage, plus the constant copying would create the memcopy numbers chromatic is seeing
18:15 NotFound Is really that bad?
18:15 chromatic I'm only reporting the memcopy directly within Parrot_str_concat.
18:15 whiteknight I have to look at the code.
18:16 whiteknight I don't know that the string allocator uses memcopy, since it's expecting the pool to be fragmented after a GC run
18:17 chromatic Without loading perl6.pbc, the memcopy within Parrot_str_concat is 87.96% of runtime.
18:17 chromatic (with GC disabled)
18:17 dukeleto no ICU lib loaded
18:17 dukeleto make: *** [t/op/testlib/test_strings.pbc] Error 1
18:18 dukeleto anybody know what is up with that?
18:18 nwellnhof dukeleto: which subtest exactly?
18:19 dukeleto nwellnhof: not sure, i did a realclean and trying again. will let you know
18:20 dukeleto nwellnhof: ./parrot -o t/op/testlib/test_strings.pbc t/op/testlib/test_strings.pir
18:22 luben_work left #parrot
18:23 chromatic I wonder what would happen if we switched small STRING content allocations to come from slabs, not that compactable mess.
18:23 whiteknight chromatic: I would love to see all string allocation happen in a more organized, slab-like manner
18:23 whiteknight anything smaller than half a page should come from slabs
18:24 chromatic There's clearly a GC problem specific to STRINGs, but I still maintain the big problem here isn't specific to STRINGs.
18:24 dalek parrot: r49023 | dukeleto++ | trunk/lib/Parrot/Harness/Smoke.pm:
18:24 dalek parrot: [harness] Update smolder submission info
18:24 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49023/
18:24 whiteknight not to harp on the point, but we do have many problems. Calling any one the "big problem" sort of underplays the amoung of work we need to do
18:25 nwellnhof dukeleto: i'm on it
18:25 NotFound In the meantime, can you please revert r49018 ?
18:25 chromatic Any benchmark which creates and throws away a lot of garbage immediately will expose one really big problem.
18:25 * dukeleto begs people to run "make smoke" again
18:25 whiteknight we have certainly never completely capitalized on the promise that immutable strings was supposed to deliver
18:25 dukeleto on trunk
18:26 chromatic Sure, but that's because we haven't finished improving the GC for immutable STRINGs.
18:26 luben_work joined #parrot
18:27 whiteknight encapsulating the string portion of the GC is the next item on bacek's gc_massacre tasklist
18:27 cotto dukeleto, smoking now
18:27 whiteknight once that encapsulation is complete and merged, we should be free to reimplement as necessary
18:27 chromatic That has to be our focus to land as soon after 2.8 as possible then.
18:27 NotFound We surely still have functions that return copies or clones "just in case"
18:27 whiteknight strong agreement
18:27 chromatic I removed the biggest just in case copiers.
18:28 chromatic The worst culprit was register copying in PCC.
18:29 NotFound cotto: me smoking and drinking
18:30 dukeleto 500 on "make smoke". arg.
18:31 luben_work left #parrot
18:31 nwellnhof dukeleto: the tests add in r48987 only work with ICU
18:32 dukeleto nwellnhof: can you fix that?
18:32 dukeleto nwellnhof: or skip them properly if icu is not present ?
18:32 nwellnhof dukeleto: yes, should be easy.
18:33 dukeleto this concerns me: t/tools/parrot_debugger.t ................... skipped: parrot_debugger changes have rendered these tests obsolete.
18:33 NotFound dukeleto: blame the greek and me
18:33 dukeleto what the hell is the point of having a test file which is unconditionally skipped?
18:34 NotFound dukeleto: easy: if we don't have it, people will complain.
18:34 cotto no luck submitting the smolder report
18:35 cotto 500 error from the server
18:35 cotto oh.  You saw that already.
18:36 dukeleto cotto: thanks for playing. you get the "home" version of the game: a bag of coal.
18:37 cotto yay?
18:38 NotFound How do you make a link that points to a line in the irclog?
18:39 moritz NotFound: click on the timestamp on the left of the line
18:39 NotFound O, the time, I was looking for line number or something
18:40 moritz rakudo on latest parrot fails some Unicode tests
18:40 moritz for example t/spec/S02-whitespace_and_comments/unicode-whitespace.t
18:44 NotFound moritz: after r49018?
18:44 moritz NotFound: I tested on r49023
18:44 moritz it's bad there
18:45 NotFound moritz: and between charset massacre and r49018 worked?
18:45 moritz no idea
18:45 * moritz really wished that parrot developers tested branch merges with rakudo
18:45 NotFound I guess ByteBuffer get_string change is the culprit.
18:46 moritz it's a much more complete test suite than parrot's test suite itself
18:49 dukeleto http://smolder.parrot.org/ap​p/projects/report_details/3
18:49 dukeleto SMOLDER IS BACK!
18:50 * NotFound plays the Imperial March
18:50 dukeleto moritz: i have plans to test branch merges. we want to, we just need the infrastructure
18:50 * dukeleto thinks it would be nice if people joined #osuosl on freenode and say thanks (the channel requires a registered nick)
18:52 moritz r49017 already shows the Unicode problems
18:54 nwellnhof dukeleto: the t/op/testlib issue is fixed
18:57 NotFound moritz: no need to perl suite, parrot tests are also failing
18:57 NotFound # src/gc/gc_ms.c:1284: failed assertion '!(*Buffer_bufflagsptr(str) & Buffer_shared_FLAG)'
18:57 dukeleto nwellnhof++
18:57 nwellnhof http://smolder.parrot.org/a​pp/projects/tap_stream/3/72
18:59 dalek parrot: r49024 | nwellnhof++ | trunk (2 files):
18:59 dalek parrot: Skip tests from r48987 without ICU
18:59 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49024/
18:59 cotto Could we get a buildbot hosted somewhere where the bus number is greater than 1?
19:01 cotto ttbot is great, but it's been a while since mj41 has been around and we'll eventually want the bot to pull from git instead of svn.
19:01 nwellnhof the gc_ms.c assertion is caused by r49018
19:02 NotFound nwellnhof: I was almost sure, but checking r49017 just in case
19:02 NotFound Yes, it pass with r49017
19:03 nwellnhof Parrot_str_escape_truncate reallocates a string from Parrot_str_concat
19:03 nwellnhof which might be shared
19:03 moritz r49005 is good
19:03 NotFound Urgh. That function still exists and is used?
19:07 NotFound Ah, no, I was thinking that it worked in place... but still ugly.
19:09 whiteknight dukeleto++
19:10 * whiteknight is going to have to run a lot of smolder reports tonight, to make up for all the ones I've missed with the server down
19:10 NotFound set S0, utf8:"\u2001\u2002\u2003\u2004\x{e01ef}\u0114"
19:10 NotFound \u2001\u2002\u2003\u2004\x{e01ef}\u0114
19:10 NotFound WTF?
19:13 whiteknight dukeleto: Are we able to add other projects in there?
19:13 dukeleto whiteknight: i can try. what do you want?
19:13 whiteknight dukeleto: first thing that comes to my mind is PLA. I would love to be able to push smoke reports somewhere for PLA
19:14 whiteknight I think any other ecosystem project which is actively maintained and developed could make good use of the service
19:15 whiteknight plumage might also be a nice one. partcl
19:15 whiteknight (does rakudo have their own test reporting service?)
19:15 dukeleto whiteknight: rakudo does not, to my knowledge
19:16 moritz they have
19:16 moritz had
19:16 moritz http://tinyurl.com/rakudosmoke
19:16 NotFound Oh, shit, is right.
19:17 whiteknight I don't know what the limitations are for our server from OSUOSL, so we might not be able to handle reports from high-bandwidth projects like rakudo
19:17 whiteknight (if we can, I'm happy to offer them the option)
19:19 dukeleto whiteknight: http://smolder.parrot.org/a​pp/projects/smoke_reports/2
19:19 dukeleto whiteknight: see Parrot::Harness::Smoke for how to submit to it
19:22 whiteknight thanks!
19:27 NotFound cmp.ops 1000:inline op xor(invar PMC, invar PMC, invar PMC) :base_core {
19:27 NotFound Shouldn't the first arg be out?
19:28 pmichaud 18:25 <whiteknight> we have certainly never completely capitalized on the promise that immutable strings was supposed to deliver
19:28 pmichaud fwiw, I did a benchmark run of x1.pir against older versions of parrot.  The newer versions are significantly faster in this respect.  :)
19:29 pmichaud so, not disagreeing with whiteknight++, but wanted to note that strings *have* gotten a lot better over time.
19:30 NotFound Until now?
19:30 dukeleto anybody want smolder projects?
19:30 whiteknight pmichaud: I'm not happy with some improvement. we really need ground-breaking, jaw-dropping performance improvements
19:30 whiteknight dukeleto++
19:30 dukeleto i can make one right now, for whoever wants one.
19:30 dukeleto i made one for plumage and parrot-linear-algebra, so far
19:31 dukeleto and plparrot
19:31 purl plparrot is the postgres+parrot integration project or http://github.com/leto/plparrot
19:31 pmichaud whiteknight: I'm definitely in favor of ground-breaking, jaw-dropping performance improvements.  :)
19:32 tcurtis /me runs "svn up; make reconfig && make && make smoke"
19:32 cotto but if they were easy...
19:32 NotFound pmichaud: that's easy, just insert an abort in imcc_main
19:32 NotFound An all benchmarks can run in less than a milisecond.
19:33 pmichaud "We fail a lot, but we do it faster than anyone else!"
19:33 dukeleto tcurtis: i do make realclean before, to be sure. reconfig doesn't work all the time
19:33 moritz "getting the wrong result in O(1) is easy" -- MJD
19:33 NotFound pmichaud: good slogan
19:34 cotto mjd++
19:34 mj41 cotto: Good evening from Czech republic.
19:34 tcurtis dukeleto: reconfig does realclean first.
19:35 nopaste "chromatic" at 192.168.1.3 pasted "significant improvement in STRING benchmark" (49 lines) at http://nopaste.snit.ch/23363
19:35 dukeleto tcurtis: hmm. well then.
19:35 moritz ouch. My bisect identified r49014 as causing the Unicode problems in Rakudo.
19:35 cotto hi mj41
19:36 cotto our wiki seems to be down
19:36 moritz dukeleto: could we please have a smolder target for rakudo?
19:36 dukeleto moritz: of course! that should be an interesting stress test....
19:36 moritz dukeleto: I thought ours also worked, but it turns out it didn't
19:36 NotFound Guru Meditation:
19:36 NotFound XID: 441151254
19:36 whiteknight did trac go down?
19:36 NotFound parrot.org runs in a Commodore Amiga?
19:37 NotFound I mean, trac.parrot.org
19:37 cotto mj41, what would it take to get ttbot to pull from a github repo instead of svn?
19:37 cotto same for parrot.org
19:37 whiteknight smolder is also down
19:37 purl okay, whiteknight.
19:37 whiteknight damnit purl
19:37 purl i think damnit i am a bot!!!
19:38 tcurtis smolder?
19:38 purl hmmm... smolder is http://sourceforge.net/projects/smolder or web-based smoke test aggregator used by developers and testers to upload (automated or manually) and view smoke/regression tests using the Test Anything Protocol (TAP). or http://smolder.plusthree.com/app​/public_projects/smoke_reports/8 or down
19:38 NotFound Use some better machine, like an Atari ST for example.
19:38 * dukeleto is talking to OSUOSL
19:38 whiteknight stress test preliminary results: FAIL
19:39 dukeleto BACK
19:39 whiteknight yes it is. Why did it go down?
19:39 dukeleto i was asking the OSUOSL folks to try and get https for smolder, so our smolder admin pass wasn't going over plaintext
19:39 GeJ Bonjour everyone.
19:39 whiteknight hello GeJ
19:40 dukeleto perhaps https for smolder is not worth it
19:40 mj41 cotto: taptinder clients are driven by taptinder server, which has database data copied updated from svn repos. So it is not so easy. It's first item on my todo list, since bacek make donation to taptinder git support.
19:41 GeJ Hi whiteknight.
19:42 NotFound Isn't PBC_COMPAT invalid?
19:42 NotFound # please insert tab separated entries at the top of the list
19:42 whiteknight Figuring out how to make PLA's setup.npq and t/harness compatible with smolder submissions is a different issue entirely
19:42 NotFound Followed by a non-tab separated entry
19:43 NotFound Don't we have a test for that?
19:43 tcurtis left #parrot
19:44 NotFound (I guess the "please" is enough to grant correcteness)
19:45 whiteknight dukeleto++
19:47 dukeleto evidently only one vhost per server can have ssl, and trac has it. so no ssl for smolder. oh well.
19:47 dukeleto they are looking into it, but it is not high priority
19:48 dukeleto moritz: http://smolder.parrot.org/a​pp/projects/smoke_reports/5
19:49 moritz dukeleto: thanks. What's the submission URL?
19:50 dukeleto moritz: all the details you need are in Parrot::Harness::Smoke
19:50 dukeleto moritz: you just need to change your project_id to 5
19:50 * moritz tries
19:50 dalek parrot: r49025 | mikehh++ | trunk/PBC_COMPAT:
19:50 dalek parrot: PBC_COMPAT requires tab separators not spaces
19:50 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49025/
19:50 dalek parrot: r49026 | mikehh++ | trunk/src/debug.c:
19:50 dalek parrot: fix codetest failure - trailing spaces
19:50 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49026/
19:50 dalek parrot: r49027 | mikehh++ | trunk/src/debug.c:
19:50 dalek parrot: fix codetest failure - ASSERT_ARGS does not have a ; after and
19:50 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49027/
19:52 mikehh damnit there was supposed to be a fix c_function docs after the and :-{
19:52 mikehh on a second line but oh well
19:55 kid51 joined #parrot
19:56 kid51 dukeleto++ for getting our Smolder server up
19:56 kid51 particle++ for getting our Smolder server up
19:56 kid51 And my first submission to the new Smolder, http://smolder.parrot.org/ap​p/projects/report_details/5, is a FAIL!
19:57 kid51 t/op/string_cs.t test 49
19:57 * kid51 goes back to $job
19:59 dukeleto we need one of the irc bots to look for fails on smolder and post them here. that shouldn't be hard
20:00 TimToady phone
20:02 whiteknight left #parrot
20:08 dalek parrot: r49028 | nwellnhof++ | trunk/src/string/api.c:
20:08 dalek parrot: [str] Don't use str_concat in Parrot_str_escape_truncate
20:08 dalek parrot: It interferes with Parrot_gc_reallocate_string_storage
20:08 dalek parrot: review: http://trac.parrot.org/parrot/changeset/49028/
20:26 tadzik joined #parrot
20:29 theory joined #parrot
20:29 Andy left #parrot
20:44 dalek roast: ea8f32d | KodiB++ | S02-builtin_data_types/bag.t:
20:44 dalek roast: [bag.t] Use .roll instead of .pick with :replace.
20:44 dalek roast: review: http://github.com/perl6/roast/commit/ea​8f32d7a1df64bdfb9a8bb190cb3efd5a816001
20:44 mikehh All tests PASS (pre/post-config, make corevm/make coretest, smoke (#8), fulltest) at r49028 - Ubuntu 10.10 beta {g++-4.5 with --optimize)
20:45 mikehh the test t/op/t/op/string_cs.t test 49
20:45 PacoLinux left #parrot
20:46 mikehh try again - the test t/op/string_cs.t test 49 is failing an assert args which does not happen with --optimized builds
20:47 mikehh so with an --optimized build it does not do the assert args, but get the appropriate result and passes
20:48 PacoLinux joined #parrot
20:53 mikehh s/get/gets/
20:53 patspam left #parrot
20:55 whiteknight joined #parrot
21:00 kid51 mikehh: But we have to get that test to pass either way.  I do not generally build with '--optimized', nor should I be required to do so.
21:03 mikehh kid51: yes I agree, just explaining why it passed for me - am looking at it
21:03 davidfetter joined #parrot
21:04 nwellnhof t/op/string_cs.t should be fixed in r49028
21:04 mikehh kid51: I usually do an optimized build when I am going to test rakudo and others
21:07 bacek good morning, humans
21:07 chromatic bacek
21:07 bacek chromatic, hi
21:08 chromatic Have you left the high tech world of the future for the sheep-farming world of Australia yet?
21:09 bacek chromatic, even worse... I was quite busy at $dayjob and didn't have any energy to do coding.
21:09 bacek "Powerpoint Architect", sigh...
21:09 cotto yech
21:09 chromatic I've rounded up several suspects willing to enforce your will on gc_massacre to get it to a mergeable state after 2.8.
21:10 chromatic If you let us know what to do, we'll do it.
21:10 bacek chromatic, it's blocked on proper String GC encapsulation.
21:10 chromatic Do you have a list of what we need to do there?
21:10 chromatic or at least a sketch of what you'd like to see?
21:10 bacek 1. Decouple string pools from old GC.
21:11 bacek 2. ...
21:11 bacek 3. Profit
21:11 bacek Check compact_pool.
21:11 bacek It's poking into header_pools of old GC.
21:11 chromatic Does it need its own pools?
21:11 bacek Nope.
21:12 bacek It need something like GC_Subsystem->iterate_over_live_strings(callback)
21:12 chromatic Oh, it hardcodes the pools.
21:12 bacek Aye.
21:12 rurban_ joined #parrot
21:13 chromatic What if we moved all of the "You may compact string storage pools now!" logic into a single function pointer each GC can provide?
21:13 bacek I don't like it.
21:13 chromatic GC_Subsystem->sweep_pmc_pool
21:14 bacek I prefer "String GC" as separate subsystem.
21:14 chromatic GC_Subsystem->sweep_string_pool
21:14 bacek chromatic, it's other way round
21:14 sorear bacek: FWIW, if you were to reimplement utf32 and utf16 portably, NQP-rx would stop depending on shared buffers
21:14 bacek For compacting we have to iterate over _live_ objects
21:15 cotto Why are strings special wrt gc?
21:15 chromatic Sure, and if we keep the PObj_lives flag in STRING headers, we have to iterate over live objects too.
21:15 rurban left #parrot
21:15 bacek chromatic, we are compacting live strings.
21:15 rurban_ is now known as rurban
21:16 bacek or we can introduce refcount for string buffers.
21:16 dduncan joined #parrot
21:16 chromatic I agree; I don't see any disagreement.
21:17 kid51 left #parrot
21:17 bacek Then after sweep we can just iterate of buffers.
21:17 NotFound sorear: shared buffers for utf? Why?
21:17 dduncan left #parrot
21:17 mikehh nwellnhof, kid51: t/op/string_cs.t passes at r49028 with out --optimize on amd64 gcc-4.5
21:17 bacek No, we can't. Variable_Size_Pool doesn't store buffer boundaries.
21:17 chromatic Separate sweep and compact stages?
21:18 bacek chromatic, they are pretty much separated already. Just abstraction leak between "Headers GC" and "String Buffers GC"
21:18 bacek ... which have to be fixed.
21:18 chromatic I don't see how to separate "sweep string headers" from "compact buffers".
21:19 bacek After "sweep" we have only live string headers.
21:19 bacek GC can provide iterate_string_headers(callback) which will be used by compact_pool
21:20 theory left #parrot
21:20 chromatic How do we iterate over only the live headers?
21:20 bacek In GC MS2 they are stored in simple list.
21:20 chromatic Are there any gaps in the list?
21:20 bacek sweep just remove dead objects from this list
21:20 bacek chromatic, nope
21:21 chromatic Okay, that was the part I didn't understand.
21:21 bacek gc_ms2_sweep_pool
21:21 theory joined #parrot
21:21 bacek line 977, 1000 in src/gc/gc_ms2.c
21:21 theory left #parrot
21:21 chromatic You need someone to add that function and make it work.
21:22 chromatic Not just for GC MS2 but for MS1
21:22 bacek Yes.
21:22 bacek It can be done.
21:22 chromatic ping luben, nwellnhof, whiteknight
21:22 whiteknight pong
21:23 chromatic We have a next task for gc_massacre.
21:24 cotto smells like progress
21:25 chromatic That's going to make GC MS1 look worse from a performance POV.
21:25 bacek cotto, can you retest gc_massacre? I changed gc triggering logic and it shouldn't eat all memory now.
21:26 bacek chromatic, why?
21:27 cotto I'll do that.
21:27 chromatic GC MS1's compact_pool walks the string header pools to flip the PObj_live flag and compact at the same time, right?
21:27 bacek It's just one more undirected call.
21:27 bacek chromatic, nope. It's just compacting.
21:27 chromatic It's already making a second trip through the header pools?
21:27 bacek chromatic, yes.
21:27 bacek all of them actually
21:27 chromatic That's not the design of a genius.
21:27 bacek not only string/buffers
21:28 chromatic PMC header pools too
21:28 chromatic ?
21:28 bluescreen left #parrot
21:28 bacek for (j = (INTVAL)mem_pools->num_sized - 1; j >= 0; --j) {
21:28 bacek Fixed_Size_Pool * const header_pool = mem_pools->sized_header_pools[j];
21:28 bacek _all_ of them
21:29 chromatic The good news is that fixing that should be a cheap way to get a couple of percentage points of improvement.
21:29 chromatic The bad news is that I need to repaint my office wall.
21:31 cotto build looks good; testing now
21:33 sorear NotFound: NQP-rx uses substr($orig, $pos, infty) to implement string cursors
21:34 sorear NotFound: NQP-rx needs string cursors because Parrot Win32 doesn't have any Unicode string encoding with O(1) indexind
21:34 sorear if Parrot gained a portable implemententation of utf32, or Parrot went back to supporting ICU unconditionally, NQP wouldn't need the substr hack
21:34 NotFound string cursors?
21:35 sorear iterating over a string in order to match bits of it against a regex
21:35 NotFound sorear: can't it use a StringIterator?
21:36 sorear that sounds like a question for pmichaud
21:36 sorear probably it avoids stringiterator because of pmc overhead
21:37 sorear it looks like our StringIterator uses indexing inside :(
21:38 cotto bacek, some test failures but the run completes without killing my machine
21:38 sorear so it won't do
21:38 bacek cotto, test failure are expected.
21:38 NotFound sorear: ATTR String_iter  iter;      /* String iterator */
21:38 NotFound StringIterator uses a String_iter
21:38 cotto ok.  I won't bother nopasting them then.
21:39 sorear My copy of StringIterator doesn't have that line.  *svn up*
21:39 sorear the interesting case is when parsing large files, say core.pm, which is 157KB and has ~0.1% double-byte UTF8 characters
21:39 sorear indexing near the end of that hurts
21:39 NotFound sorear: I think it changed in the charset masacre
21:41 whiteknight Can anybody enlighten me about how to post a smolder report using distutils?
21:41 whiteknight the magic incantations escape me
21:42 sorear NotFound: definitely a pmichaud question, then
21:42 whiteknight dukeleto: ping
21:42 * sorear will probably try this at some point
21:43 bacek $dayjob time. C U
21:48 pmichaud here's why StringIterator doesn't work.
21:48 pmichaud Suppose I have a 30,000 character source code file.
21:48 dukeleto whiteknight: pong
21:48 pmichaud I've parsed through the first 29,000 characters.
21:48 pmichaud I need to know if the next sequence of characters is the word 'whle'
21:48 pmichaud 'while'
21:48 pmichaud ...how do I do that?
21:48 whiteknight dukeleto: I'm having a hell of a time seding smolder reports
21:49 pmichaud (in PIR, please :)
21:49 dukeleto whiteknight: what is the problem?
21:49 purl hmmm... the problem is that [% user.comments.count %] makes TT call $user->comments in list context, which returns returns $user->comments_rs->all
21:49 hercynium joined #parrot
21:49 purl was kicked by pmichaud: purl
21:49 purl joined #parrot
21:49 whiteknight all the projects besides parrot dont allow anonymous uploads
21:50 nopaste "mikehh" at 192.168.1.3 pasted "test failures from make test in gc_massacre branch" (22 lines) at http://nopaste.snit.ch/23367
21:50 cotto forget the problem
21:50 purl cotto: I forgot problem
21:51 whiteknight it gives me the error about no anonymous uploads even when I am logged in as parrot-autobot
21:52 luben chromatic, pong
21:52 whiteknight when logged in as parrot-autobot, only the project "parrot" appears under the list of "my projects"
21:52 whiteknight I suspect that user is not associated with all the rest of the new projects
21:59 chromatic luben, bacek gave some guidelines as to what we can do with gc_massacre.
22:00 chromatic pmichaud, is that due to a lack of ops which work on PMCs as opposed to STRINGs?
22:00 tadzik left #parrot
22:01 pmichaud chromatic: that's likely the biggest part of it, yes.
22:01 pmichaud if we had string ops that could understand a StringIterator as an argument then it might be much better
22:01 pmichaud but note that NQP/regex also needs the ability to backtrack
22:01 chromatic Right.
22:01 pmichaud i.e., if I iterate forward, I also have to have the ability to get back to where I was before
22:01 pmichaud so the iterator can't just discard things
22:02 NotFound pmichaud: clone it
22:02 chromatic You almost need a PMC which can contain a STRING and a position within the STRING.
22:02 pmichaud NotFound: cloning gets *real* expensive very quickly.
22:02 sorear StringIterator has a pretty cheap .clone, it doesn't touch the underlying string
22:02 pmichaud that would be far more expensive than what we have now.
22:02 sorear (although it does make a new PMC)
22:02 pmichaud I'm pretty sure that substr is currently much cheaper than a PMC clone.
22:03 chromatic If iterating through the STRING via the PMC doesn't have to perform substrs, I can imagine that being much cheaper.
22:03 pmichaud well, if we want to implement all of the various string opcodes on our String-ish PMCs, that can work.
22:04 pmichaud that would need to include find_cclass, find_not_cclass, is_cclass, etc.
22:04 NotFound You don't need ICU for that?
22:04 pmichaud we do need ICU for parts of it
22:05 sorear I think the ultimate fix would be to allow String_iter to be stored (opaquely) in an IREG, and then have ops working on that
22:05 sorear (in practice it would be a byte offset)
22:05 NotFound sorear: And what black magic will distinguishit from a proper INTVAL?
22:06 dukeleto whiteknight: let me see if I can fix that
22:07 Paul_the_Greek joined #parrot
22:07 whiteknight dukeleto: on the parrot project, parrot-autobot is listed as a "Developer"
22:07 whiteknight not so on the other projects
22:07 Paul_the_Greek Howdy, boys and girls.
22:07 sorear NotFound: PAST::Compiler type safety and a dash of Hungarian
22:07 whiteknight howdy dr nick
22:07 Paul_the_Greek ping nwellnhof
22:07 sorear NotFound: it's not like this is a memory address, it doesn't need to be interpreted by the GC's frame walker
22:07 dukeleto whiteknight: try now
22:08 nwellnhof paul: pong
22:08 whiteknight dukeleto: seems to work. I have access to the upload form no
22:08 whiteknight now
22:08 whiteknight I'm going to try posting a report
22:08 NotFound sorear: I think that way is too black magic and even more exposure of internals of a subsystem to completely unrelated parts.
22:08 Paul_the_Greek nwellnhof: Just a silly point: I presume that string catenation short-circuits the cases where one of the strings is the empty string.
22:09 dukeleto whiteknight: i just fixed that issue for all the smolder projects. thanks for noticing! ;)
22:09 nwellnhof yes, Parrot_str_concat does
22:09 Paul_the_Greek Just checking.
22:09 NotFound Looks like no.
22:09 purl hmmm... Looks like no. is there any reason I should not apply it?
22:10 Paul_the_Greek Concerning substring: Have you thought about preallocating all the ASCII 1-byte strings?
22:10 luben backloging now... I will look at the code tomorrow - it's after midnight here
22:10 Paul_the_Greek luben: Go to bed.
22:10 luben Indeed
22:11 NotFound Parrot_str_concat checks for STRINGNULL but I don't see any check for empty strings.
22:11 Paul_the_Greek What is STRINGNULL?
22:11 NotFound Paul_the_Greek: a null STRING
22:11 purl a null string is a hold over from C where a string is a char* and the addr that pointer references is 0
22:11 Paul_the_Greek That's what I meant by empty: ""
22:12 NotFound Paul_the_Greek: is not.
22:12 whiteknight dukeleto: Well, I just tried a smolder report again and it failed, but I suppose the error is on my side
22:12 Paul_the_Greek Is not?
22:12 purl Is too.
22:12 plobsing "" != STRINGNULL
22:12 Paul_the_Greek Now I'm confused.
22:13 dukeleto whiteknight: what error did you get?
22:13 NotFound Paul_the_Greek: we even have two kinds of null strings: STRINGNULL and (STRING*)NULL
22:13 theory joined #parrot
22:13 whiteknight dukeleto: no error. distutils only tells me "200 OK"
22:13 Paul_the_Greek What the hell? You have a preallocated empty (null) string, right?
22:14 plobsing kNotFound: not quite right. AFAIK, STRINGNULL = NULL for optimized builds.
22:14 Paul_the_Greek You only need one of them, after all.
22:14 NotFound Paul_the_Greek: yes, but in a lot of cases a NULL arg is allowed, so STRING_IS_NULL checks both
22:14 Paul_the_Greek Right, I can understand that, but what is STRINGNULL?
22:14 sorear A null pointer
22:14 NotFound Paul_the_Greek: an opaque object that means that the string is null.
22:14 chromatic It's a valid pointer.
22:14 plobsing a sometimes-valid pointer
22:15 sorear But fiddled to make it cheaper to test
22:15 Paul_the_Greek I'm roaring here. I'm completely confused.
22:15 Paul_the_Greek More empty strings makes it easier to check?
22:15 NotFound Paul_the_Greek: welcome to parrot
22:15 sorear Paul_the_Greek: string->vtable->do_stuff
22:15 chromatic Are you familiar with the null object pattern?
22:15 sorear Paul_the_Greek: STRINGNULL has vtables that throw exceptions
22:15 NotFound If you aren't confused by parrot internals, you haven't seen enough of them yet.
22:15 sorear Paul_the_Greek: NULL is like STRINGNULL but instead of throwing exceptions it segfaults
22:16 Paul_the_Greek chromatic: No.
22:16 chromatic Okay, let me find a link.  In the meantime, ignore everything sorear has written about this.
22:16 Paul_the_Greek So STRINGNULL represents the empty string and there are no normal strings of zero length?
22:16 NotFound STRINGNULL is a valid value of a S register, NULL is not.
22:16 dukeleto whiteknight: ug. perhaps try modifying Parrot::Harness::Smoke in parrot to submit to your project, just to see if it work?
22:16 dukeleto works, rather
22:16 dukeleto whiteknight: or perhaps try curl
22:16 Paul_the_Greek But why not just have a normal string of zero length?
22:16 chromatic Paul_the_Greek, see http://c2.com/cgi/wiki?NullObject
22:17 sorear Paul_the_Greek: STRINGNULL is not a string of any length
22:17 dukeleto whiteknight: i am not sure if the bug is distutils or smolder, at this point
22:17 NotFound Paul_the_Greek: STRINGNULL is a null string, not a zero length string.
22:17 plobsing Paul_the_Greek: we can have many 0-lengthed strings (one for each encoding) on top of stringnull
22:17 whiteknight dukeleto
22:17 purl dukeleto is probably mentoring a few peeps. can't remember everyone. sure. or jonathan at leeeeeeeeeeeeto dot net or hacking on git docs at http://github.com/parrot/parrot/tree/leto/git_docs
22:17 Paul_the_Greek What is the difference between a null string and a string of zero length?
22:17 whiteknight :I'll play with it. dinner time now
22:17 chromatic There is only one STRINGNULL in the system.
22:18 Paul_the_Greek plobsing: Yes, one for each encoding makes sense.
22:18 NotFound Paul_the_Greek: Have you used SQL?
22:18 chromatic It's a valid STRING pointer which means "This is not a valid STRING value."
22:18 sorear Paul_the_Greek: do you understand the difference between "" and (char*)NULL in C?
22:18 Paul_the_Greek Oh. Why do I want a string pointer that says it's not a valid string?
22:18 Paul_the_Greek sorear: Yes.
22:19 Paul_the_Greek Why not just use a universal NIL object for this?
22:19 Paul_the_Greek Maybe we don't have a universal NIL object.
22:19 chromatic Because we want our C code to tell the difference between "A STRING pointer we know is not a valid STRING" and "Not a pointer."
22:19 NotFound We have several universals, we live in a multiverse %-)
22:20 Paul_the_Greek Okay, I guess I can understand that. I'll look for cases where NULLSTRING is used.
22:20 chromatic See also PMCNULL.
22:20 Paul_the_Greek Meanwhile, does it make sense to have one preallocated empty string?
22:20 Paul_the_Greek PMCNULL I understand.
22:20 Paul_the_Greek Oh wait, and so now I understand STRINGNULL!
22:20 NotFound Paul_the_Greek: we have: CONST_STRING(interp, "")
22:21 plobsing but that can only cover the ascii case
22:21 Paul_the_Greek And does every string function return that string when the result is empty?
22:21 NotFound He asked for one.
22:21 chromatic Meanwhile we should excise all uses of the NULL pointer as equivalent to STRINGNULL.
22:21 Paul_the_Greek Now, how about all the 1-character ASCII strings?
22:21 chromatic We don't perform any STRING coalescing.
22:21 Paul_the_Greek There immutable now, right?
22:22 Paul_the_Greek s/There/They're/
22:22 chromatic Right.
22:22 Paul_the_Greek So substr(, , 1) could return one of the preallocated strings.
22:23 NotFound Not a bad idea, provided we can have a way to obtain it from the interpreter quickly,
22:23 Paul_the_Greek Let's see ... 1,135 substr operations in *.pir
22:24 Paul_the_Greek 315 cases of substr , , 1
22:25 NotFound A lot less of string header cretaed is a lot of usages.
22:25 NotFound s/is/in
22:25 Paul_the_Greek Possibly 256 1-character ASCII strings and 128 1-character Unicode strings.
22:25 Paul_the_Greek Or maybe just the first 128 ASCII characters.
22:26 NotFound Paul_the_Greek: there are no 256 ascii characters.
22:26 Paul_the_Greek There aren't?
22:27 plobsing Paul_the_Greek: do it in a branch. show that it is a worthwhile complication.
22:27 NotFound Wen we say ascii we mean ascii, not ascii-extended.
22:27 Paul_the_Greek Oh, 7-bit ASCII. Right.
22:27 Paul_the_Greek plobsing: Not a bad idea.
22:27 Paul_the_Greek Say, is there INTEGERNULL?
22:28 chromatic No, because integer and floats are value types.
22:28 NotFound Paul_the_Greek: is in the roadmap togethet with negative NaN
22:28 Paul_the_Greek What makes them value types and not strings?
22:28 dukeleto aloha, msg whiteknight bug is in distutils, i just submitted a PLA smoke report: http://smolder.parrot.org/a​pp/projects/smoke_reports/2
22:28 aloha dukeleto: OK. I'll deliver the message.
22:28 aloha dukeleto: Okay.
22:29 chromatic They're not pointers.
22:29 Paul_the_Greek Oh, they are immediate types instead of heaped types. Got it.
22:29 chromatic Do you have a Lisp background?
22:29 Paul_the_Greek Yes.
22:30 chromatic Okay, that makes it easier to explain then.
22:30 Paul_the_Greek I've worked on a few dynamic languages and I've never seen STRINGNULL, so that's why I was confused.
22:31 NotFound I think I'm starting to understand lexicals... should I be worried?
22:31 Paul_the_Greek NotFound: Closures? Be very afraid.
22:31 chromatic It's a little like NIL, except we don't have a singly-rooted hierarchy for heaped types.
22:31 NotFound Paul_the_Greek: lexical usage in pir, I mean
22:31 Paul_the_Greek Right, in Lisp we just used NIL everywhere we didn't have any other value to store.
22:32 Paul_the_Greek NotFound: I've ignored it so far.
22:33 NotFound Paul_the_Greek: I can't, I need to make closures work in winxed.
22:33 Paul_the_Greek I'll buy you a beer once you're finished.
22:34 NotFound I have a test that works. Only for one level, but is a start.
22:37 nopaste "NotFound" at 192.168.1.3 pasted "Closures in winxed, first test" (31 lines) at http://nopaste.snit.ch/23369
22:38 whiteknight dukeleto: what was the bug?
22:38 whiteknight dukeleto: and have you committed a fix?
22:38 dukeleto whiteknight: no bug. I just know that submitting reports for PLA works from Parrot::Harness::Smoke
22:38 whiteknight dukeleto: ah, okay
22:38 dukeleto whiteknight: either you are doing something wrong, or there is a distutils bug
22:39 dukeleto whiteknight: sorry I wasn't clear about that
22:39 whiteknight that's okay
22:46 NotFound We got distracted from the initial point: str_concat do not special case zero length strings. Should do it, ignoring encoding?
22:46 chromatic Seems like a potential optimization.
22:47 NotFound I think at a time did it.
22:47 NotFound Not sure.
22:48 Paul_the_Greek It's a pretty simple optimization.
22:48 chromatic Easy to benchmark.
22:48 Paul_the_Greek Does fixed8_substr() still do ASCII substrings?
22:49 NotFound Paul_the_Greek: don't assume fixed8 is ascii.
22:49 Paul_the_Greek I've been trying to drill down to the actual substring function for ASCII and I'm going around in circles.
22:50 NotFound Before charset masacre fixed8 can be ascii, binary or iso-8859-1
22:50 Paul_the_Greek And after?
22:50 purl it has been said that after is like a BUILD by name, it seems.
22:50 NotFound Paul_the_Greek: after, it should not exist.
22:51 Paul_the_Greek What file contains substring in the massacre branch?
22:52 Paul_the_Greek I think I have a catenate routine somewhere that cases on the length of the first operand and then, for each case, cases on the length of the second operand.
22:52 Paul_the_Greek Does all sorts of cleverness.
22:52 NotFound Looks like fixed8_substr now is a function shared bt the appropiate encodings
22:52 Paul_the_Greek It copies the entire source string and then sets the offset and length.
22:53 NotFound So you can't use it unless you are in string internals and know what you are doing.
22:56 NotFound Paul_the_Greek: sounds difficult to adapt to a multiencoding world.
22:57 kid51 joined #parrot
22:57 Paul_the_Greek I think all encodings could special-case empty strings in catenation. ASCII could special-case 1-character strings in substring.
22:58 Paul_the_Greek Oh, and empty strings, too.
22:58 Paul_the_Greek I've also seen empty strings represented by an immediate value. STRINGEMPTY?
22:58 Paul_the_Greek Probably not worth the special cases everywhere.
22:59 chromatic Probably not, as you have to be careful of encodings.
22:59 Paul_the_Greek Easier back when everyone agreed on only one encoding.
22:59 whiteknight msg fperrad Parrot has a new smolder server (http://smolder.parrot.org) and I can't find an option combination to make distutils send reports there. Any ideas?
22:59 purl Message for fperrad stored.
22:59 aloha OK. I'll deliver the message.
23:01 Paul_the_Greek Don't have to preallocate all the 1-character strings, but can allocate the single copy lazily.
23:01 chromatic I like that idea better.
23:02 Paul_the_Greek Vector of 128 pointers, lazily allocate, return allocated one.
23:02 Paul_the_Greek Separate slot for pointer to empty string.
23:04 Paul_the_Greek Have we considered allocating string headers and data together?
23:05 chromatic Makes walking the pools difficult.
23:06 Paul_the_Greek Allocate them in the fixed-size pools, for strings up to some size limit.
23:06 dukeleto RSS for failed smolder reports for parrot: http://smolder.parrot.org/​app/projects/feed/1/failed
23:07 chromatic We still have to be able to identify which pool items are STRINGs.
23:07 NotFound Paul_the_Greek: very hard for a now, several parts of parrot still abuse string internals horribly.
23:07 Paul_the_Greek The could be in there own pile of fixed-sized pools, like PMCs are.
23:07 Paul_the_Greek s/there/their/
23:08 NotFound Paul_the_Greek: take a look for example at windows specific functions in src/library,c
23:09 dalek roast: ea0bb44 | KodiB++ | S02-builtin_data_types/ (2 files):
23:09 dalek roast: Revamped keyhash.t.
23:09 dalek roast: review: http://github.com/perl6/roast/commit/ea​0bb443bab8795b58381b19653bd3e3b7c4d618
23:19 esskar left #parrot
23:42 bacek_at_work ~~
23:44 Paul_the_Greek left #parrot
23:46 nwellnhof left #parrot
23:53 whiteknight blah. I have no idea why distutils is not posting my smolder reports
23:57 SingAlong_ joined #parrot

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced