Camelia, the Perl 6 bug

IRC log for #darcs, 2010-08-11

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:06 lispy1 Yes it is.
00:20 lispy1 mornfall: are you working on mapPrimFL or something else?
00:28 mornfall No, file path API.
00:33 lispy1 mornfall: I think to further optimize my parser I will have to use darcs as a library, generate some synthetic patches/inventories and feed them directly into the parser.  There is just too much noise in the profiles when I run all of darcs
00:33 lispy1 mornfall: do you have any machinery for synthesizing patches/inventories?
00:35 mornfall lispy1: Not really. You should probably look at the unit tests, they have.
00:36 mornfall (There are Arbitrary instances for a lot of stuff.)
00:36 lispy1 Hmm
00:36 lispy1 Those generate pretty small things, IIRC
00:36 lispy1 But maybe I can crib from them
00:36 lispy1 concatenating the GHC (or darcs) inventories might give me a good benchmark for the patch info parser
00:49 gbeshers joined #darcs
00:51 lispy1 joined #darcs
01:05 Igloo lispy1: Surely you can generate large input to parse by writing a 20 line hunk, say, and then copying it n times
01:06 lispy1 Igloo: yeah, that will work.  Similar to the idea I had with the patch infos
01:06 lispy1 Igloo: I also wanted a test where I had 10000s of patches, all tiny and in one file
01:07 lispy1 like 1 line hunks and lots of adds/rms
01:07 Igloo But that's easy to generate, surely? They don't have to /sensible/ patches, as you're only going to parse them
01:08 lispy1 right, and I can use the test suite to assume the parser is correct
01:08 lispy1 Yeah, seems simple now
02:18 [1]intripoon joined #darcs
02:23 exlevan joined #darcs
02:40 abuiles joined #darcs
02:46 sshc_ joined #darcs
02:53 sshc_ joined #darcs
02:58 sshc_ joined #darcs
03:06 sshc joined #darcs
03:11 sshc_ joined #darcs
04:04 gwern joined #darcs
04:06 bos joined #darcs
04:22 modest_nick__ joined #darcs
04:29 lispy ?tell kowey if you want to watch the ticket: http://code.google.com/p/ndm​itchell/issues/detail?id=330
04:29 lambdabot Consider it noted.
05:06 twb joined #darcs
05:31 twb joined #darcs
05:32 sshc_ joined #darcs
05:38 Quiark joined #darcs
06:07 JaffaCake joined #darcs
06:13 lispy mornfall: I think I found/fixed the bug in the patch bundle I sent before.  doing a full test now
06:13 lispy mornfall: when I left-factored the white space lexing, I missed one spot.
06:13 lispy mornfall: readMerger.  Which is why it failed in a conflict test for darcs1 repos
06:25 abuiles left #darcs
07:02 darcscommitbot joined #darcs
07:03 darcswikibot joined #darcs
07:17 * lispy hooks up readPatch to criterion
07:23 mornfall :)
07:26 lispy hrm.  I can't get much output from criterion.  It can only generate csv on my platform and gnuplot won't install
07:55 mornfall Hm. 65.9kg... this hacking frenzy is certainly taking its toll. :D (Although so far it is actually a bonus...)
08:53 exlevan joined #darcs
09:30 kowey joined #darcs
09:30 kowey good morning
09:30 lambdabot kowey: You have 1 new message. '/msg lambdabot @messages' to read it.
09:31 kowey lispy: thanks for filing the ticket
09:32 kowey (hopefully will see this in scrollback, otherwise will say hi later)
09:48 mornfall Re.
10:02 exlevan joined #darcs
10:02 exlevan morning
10:10 mornfall Hi exlevan.
10:44 mornfall And I'm seriously out of battery. Be back in a bit, from home.
11:16 kowey hmm, hopefully this will fix our anchor tags problem in the latex2html manual: http://tug.org/mailman/htdig/late​x2html/2003-September/002433.html
11:16 kowey (which I realise is going away in the long term, but since we don't know how long, maybe we can go for a bit more stability)
11:22 mornfall ii/me points eric at http://buildbot.darcs.net/builders/6.12.1%​20Vista/builds/21/steps/shell_1/logs/stdio
11:25 mornfall There seems to be a substantial penalty for just executing programs on Windows.
11:39 kowey anything in particular I should have spotted?
11:40 mornfall It's windows.
11:40 mornfall darcs-HEAD.*bat*
11:40 mornfall (It also seems that full get on ghc repo is ... taking a *long* time...)
11:41 kowey oh, you're saying you have to wrap it in a batch file to make avoid some sort of strange performance penalty?
11:42 mornfall No.
11:43 mornfall I am just trying to point out that we are now running benchmarks on the windows slave...
11:44 kowey ah!
11:45 kowey hmm, if only we could work out a good visualisation for these benchmarks
11:46 mornfall Well, first we'll see if the full get of ghc even finishes. :)
11:46 mornfall Then we can think about that. :P
11:49 * kowey can never remember what we're supposed to do with doc duplication
11:50 mornfall kowey: All it takes is a darcs-benchmark report that can produce better output. I have already added binary name disambiguation and I believe it should be sorted by date as well.
11:50 kowey we have darcs help environment and the user manual now...
11:50 kowey identifying binaries by build date?
11:51 kowey I guess what we may see in the future is some sort of timeline/scatter
11:51 kowey for each benchmark: x axis is build date and y axis is time (3 points per day)
11:55 mornfall kowey: http://wiki.darcs.net/Benchmarks/BuildBot -- to see what's there
11:56 mornfall kowey: The problem is that the tables/graphs don't seem to use any particular sorting for columns
11:56 mornfall But the list of binaries is OK.
11:57 mornfall Another downside is that the TRY binaries are mixed up with the HEAD binaries. But I'll eventually fix that as well.
11:57 mornfall I just need to script may way out of it in master.cfg.
11:59 kowey man those google chart URLs must get pretty massive when you have a lot of data
11:59 mornfall They can't. There's a limit on URL size.
11:59 kowey a limit as in the definition of a URL?
11:59 kowey or maybe I should be training myself to say URI
12:00 mornfall Hm. Maybe not defined.
12:00 mornfall Well, it's probably quite impractical to be plotting couple of years of history using gcharts.
12:27 abuiles joined #darcs
12:28 abuiles good morning/afternoon
12:30 abuiles hi kowey, thanks for the review I will amend and send later on, about the set_time_out, I was thinking and I'm doing too much high level stuff in C, I think I will move some things to haskell, and just call the function I need of libcurl from Haskell, in that way, we can have the default timeout specified in URL.hs or HTTP.hs, and do user notification easily.
12:32 kowey higher level code does sound generally better
12:37 abuiles kowey, I'll be in touch later on, bye ( gotta run to do some Uni readings )
12:37 abuiles left #darcs
12:42 Igloo kowey: Shouldn't mail from the bug tracker have headers so that "reply" will only send mail to bugs@darcs.net?
12:43 Igloo Hmm, and it doesn't have a link to the ticket, either
12:44 Igloo Interestingly, the darcs-devel copy of the mail does have a reply-to header
12:49 kowey Igloo: it's not easy for me to figure out the best practices in this sort of thing
12:49 kowey because there's always some sort of subtlety or unforeseen consequence of mucking around with these things
12:49 kowey I don't mean technically, but from an interaction/workflow standpoint
12:50 kowey ie. if we change this, then user Bob will do $some_obvious_thing which breaks in some amusing way, and not because of any technical issues
12:50 kowey (sorry for my knee-jerk fearful reply! the right answer is probably "maybe!")
12:55 mornfall Wow. I just segfaulted darcs!
12:56 kowey nice one
12:56 burp that must be the work of unsafePerform* :/
12:56 kowey ticket?
12:57 mornfall kowey: I am halfway through patching it up, so hard call. Probably not.
12:57 mornfall I am ripping through the filepath handling code.
13:02 kowey Igloo: fwiw, I think roundup *does* set reply-to to bugs@darcs.net
13:03 Igloo kowey: It didn't in the mail to me
13:06 kowey Igloo: you're on roundup's blacklist, aren't you?
13:06 kowey I'm a bit confused about why it's sending mail to you
13:07 kowey are you sure it's not the original sender that's sending it?
13:07 kowey I guess you would be because the From line wouldn't say bugs@darcs.net
13:08 kowey (one of my todo items is to patch roundup so that its mail function accepts an optional blacklist flag, so that I can implement vacation mode properly)
13:08 kowey (vacation mode is a feature that would allow users to be nosy without receiving messages)
13:08 Igloo kowey: Original sender was you
13:09 kowey so Eric Kow <kowey@...>, right?
13:09 Igloo kowey: Don't spend too much time on this just for my sake, incidentally
13:09 kowey :-) thanks (am still interested in nailing down vacation mode, though, as I've had other complaint)
13:10 kowey if the mail was from Eric Kow <bugs@...> then I'm confused, if it was from kowey@ then I'm not confused; it just mean the mail came from me and not roundup
13:10 Igloo Yes, "Eric Kow <kowey@d.n>" was the from address
13:10 kowey phew! universe appears to be in order then
13:10 Igloo For both the message with "Reply-To: bugs@d.n" and the one without
13:14 mornfall I think darcs-devel@ is mangling reply-to on its own.
13:23 kowey I notice that I also cc'ed darcs-devel@ in my message
13:24 kowey the only confusion left is why it's in the cc list
13:24 mornfall kowey: The segfault is my fault, likely.
13:25 mornfall So no worries.
13:25 kowey (I guess it's not so confusing. http://bugs.darcs.net/issue1676.  Message is initially bcc'ed to darcs-devel, somebody replies to all, roundup sees darcs-devel in cc list and adds it to nosy)
13:26 kowey you can actually configure roundup not to add to/cc people to nosy...
13:26 kowey actually the default is to only add them on new tickets, but not on replies; somebody changed it to always adding them
13:37 _ilbot2 joined #darcs
13:37 Topic for #darcsis now can't talk? see -> | http://wiki.darcs.net/IRC | http://darcs.net/ | latest is 2.4.4
14:10 * mornfall feels like an idiot.
14:10 mornfall Maybe I should really write proper tests.
14:21 mornfall kowey: The mean + sdev numbers are unusable. :(
14:25 mornfall Even with 100 runs the data is jittery like hell.
14:25 kowey :-/
14:25 kowey and it's not because of darcs jitter
14:25 mornfall And a single outlier in 100 runs can distort it, if most of the runs are in range of 10ms.
14:26 mornfall If the OS decides to flush a page out of disk cache, it can screw the whole benchmark it seems.
14:26 kowey I think the criterion blog post talked about detecting outliers http://www.serpentine.com/blog/2009/09/29/crit​erion-a-new-benchmarking-library-for-haskell/
14:29 kowey huh, the chart stuff seems to be self-contained
14:29 kowey maybe divorcing criterion from criterion-chart is actually really easy
14:29 kowey could be one of those 2 minute action deals I keep getting bitten by
14:36 mornfall Uh oh. I benchmarked a binary with traces in it. So much for being disappointed by the results.
14:37 mornfall I can *see* the outliers.
14:37 mornfall I mean, the progress dots just hiccup.
14:37 mornfall When they hiccup, the mean goes through the roof.
14:37 mornfall I mean, the stddev.
14:38 kowey hmm, so I'm hoping that this works (criterion divorce)
14:38 kowey it should mean that you can just drop Criterion.Plot.Chart in place of Criterion.Plot
14:46 kowey I do hope cabal learns to suggest package deps one day...
14:53 mornfall 301.5ms (223.4ms)
14:53 kowey some jitter
14:53 kowey I will cc you on the criterion divorce patches
14:55 mornfall Can we maybe also report medians?
14:55 mornfall I know it's primitive... but.
14:56 mornfall It can be expected to be infinitely more stable than the mean.
14:56 mornfall (Under given conditions...)
14:56 * kowey shrugs
14:57 kowey it could be worth trying
14:57 kowey I expect the best thing is either to learn the stats or outsource it to somebody who's thought about it more than we have
14:58 mornfall If we outsource them, we need to be also able to apply the solution.
14:59 kowey I think my university may have a statistics service that I could ask if you want
14:59 kowey I could book an appt with a Real Statistician and present to them what we're trying to do
15:00 mornfall Well, I think the first step would be to get the data in a processable way.
15:00 mornfall What we have now looks more like a random mishmash of datastructures.
15:01 kowey in the meantime, my hope is that if I can get criterion-chart to compile, I can just send what I have to Bryan
15:01 kowey without actually trying it...
15:01 mornfall Well, not important anyway. I'll go back to working on h-s 0.6.
15:02 kowey whatever "bootstrapping with 100000 resamples" means, I somehow suspect criterionised darcs-benchmark would not be saying that
15:07 mornfall Oh, it could.
15:07 mornfall It's offline computation.
15:07 mornfall You get the resulting data and take random subsamplings and do stuff with them.
15:08 mornfall We don't have too much data to not do that number.
15:08 mornfall It'll take couple seconds I guess.
15:08 kowey not knowing what it means, it sounds like output like "found 11 outliers among 100 samples (11.0%)" is what we're looking for
15:09 kowey and "variance introduced by outliers: 7.997%" and "variance is slightly inflated by outliers"
15:09 mornfall Yes, that's what we are looking for.
15:09 mornfall Among other things.
15:09 mornfall We (I) would also like the ability to pinpoint these outliers and kick them out.
15:10 mornfall And recompute the results without the outliers.
15:10 mornfall It may be optimistic, but we need something more stable than what we have now.
15:10 kowey my big delay here is gtk2hs taking its sweet time to compile
15:11 mornfall And we also have to account for things like disk access having very wild latency distribution.
15:11 mornfall (Which criterion probably doesn't care about.)
15:13 exlevan joined #darcs
15:17 kowey criterion could be a good candidate for cmdlib btw
15:17 kowey it's just using raw getopt
15:18 Igloo What are the advantages of cmdlib?
15:20 burp mornfall: didn't you suggest yourself to run the benchmarks on a ramdisk?
15:23 mornfall Not me, IIRC.
15:23 mornfall I am not very fond of that, although it could certainly help with these issues.
15:23 mornfall But having 3 copies of the GHC repo in a ramdisk, well. It could get a bit painful.
15:23 mornfall Having a ramdisk doesn't help if you are swapping. :D
15:24 burp hm yes, disable swap ;)
15:24 burp how big is the ghc repository?
15:24 kowey cmdlib could give shorter, easier to maintain code
15:24 kowey (or cmdargs)
15:26 mornfall burp: 170M
15:26 mornfall cca
15:26 mornfall It's not *so* bad I guess these days.
15:26 burp RAM is cheap these days ;)
15:26 mornfall Most machines probably have 2G.
15:26 burp I have a 1.5GB ramdisk as /tmp/ in my laptop ;)
15:27 mornfall Is that a ramdisk or tmpfs?
15:27 mornfall 4G of RAM have you?
15:27 burp sadly just 3G, thing doesn't support 4 :/
15:28 burp and it's indeed tmpfs
15:28 mornfall So it's not eating that 1.5G most of the time is it... :))
15:28 burp I think not
15:28 mornfall I could use a SSD drive.
15:29 burp hm, or effective cpu time used, and something like IO seeks/reads?
15:29 Igloo burp: Is that better than an extra 1.5G of disk cache?
15:29 burp that in details would be of interest
15:31 burp Igloo: hm I don't see it like that, but I think having /tmp in a tmpfs on RAM is especially useful
15:32 burp also when compiling, unpacking/packing some "temporary" stuff
15:32 kowey argh, this code divorce will be a bit trickier than I thought
15:32 Igloo burp: Have you timed it, or are you guessing?
15:33 burp well, it doesn't get written to disk in any case
15:34 burp where as else, it might get written to disk when the disk cache is full
15:34 mornfall Igloo: Not hitting disk can be important when you are on a laptop and your disk is not spinning.
15:34 mornfall And writes always hit the disk.
15:34 burp yes, another aspect
15:34 mornfall Sooner or later.
15:43 balor joined #darcs
15:47 bos joined #darcs
16:04 tux_rocker joined #darcs
16:08 tux_rocker @tell mornfall did you notice that issue 1909 is now amend-requested? test trouble again...
16:08 lambdabot Consider it noted.
16:10 mornfall -ENOTIME ...
16:10 lambdabot mornfall: You have 2 new messages. '/msg lambdabot @messages' to read them.
16:16 kowey criterion/criterion-chart divorce appears to be successful if clumsy
16:25 exlevan joined #darcs
16:29 mornfall Hi exlevan. How goes?
16:30 * mornfall tries to remember, what was the NewFormat/OldFormat difference? With patch paths?
16:30 mornfall Hm, double utf8 I guess.
16:32 kowey http://lists.osuosl.org/pipermail​/darcs-users/2010-May/024073.html in case it helps
16:33 kowey also http://irclog.perlgeek.de/​darcs/2010-05-31#i_2387317
16:36 mornfall Yeah.
16:36 mornfall I figured a way out it seems.
16:47 mornfall Hm.
16:48 kowey doesn't darcs send generate unified diffs?
16:48 kowey i mean, with its usual darcs-high-level representation
16:48 kowey but for hunks, doesn't it provide context?
16:49 mornfall --unified
16:50 kowey it's not in darcs help send
16:50 kowey ah! droundy baked it in in 2008
16:56 exlevan hi mornfall! I'm fightin here with typechecker, trying to convince it that my code is correct :) Gonna make a smart pull tonight
17:06 mornfall :)
17:06 mornfall O-K.
18:04 drksd joined #darcs
18:38 abuiles joined #darcs
18:39 arjanb joined #darcs
18:45 mornfall darcs-HEAD pull 100 [GHC opt pris]: !..............               ?3.6s (0.9s) 22.0M 4x
18:45 mornfall darcs-newhs pull 100 [GHC opt pris]: ..................              ~2.6s (0.2s) 23.0M 5x
18:45 mornfall \o/
18:45 mornfall Hm. Although.
18:46 mornfall Means aren't nearly as exciting.
18:46 mornfall 2.7 v 2.4
18:46 * mornfall sighs
18:47 Igloo sighs?
18:47 Igloo Do you mean those are medians?
18:48 mornfall No those are means.
18:48 Igloo 30% faster sounds good?
18:49 mornfall Well, that's more like 10 and even that could just be noise.
18:49 mornfall Dear lord,      10 copyFileUsingCache             Darcs.Repository.Cache  36.3    0.1   1910   1897179
18:49 mornfall That function must die.
18:49 mornfall It's a horrible mess anyway.
18:50 mornfall isGZFile 13.7%? Something's must be wrong with the profiler...
18:53 Igloo How many times is it called? And how many patches are being pulled?
18:56 mornfall Is the RTS using walltime for profiling?
18:56 mornfall (If it's counting in waiting for IO then that's plausible.)
18:59 mornfall Ok, comparing check with pull is instructive.
18:59 mornfall (Turns out checking a repository takes 26s, while pulling it 44s, with profiling)
19:01 Igloo I think it's wall time, yeah
19:07 * mornfall wonders if darcs really applies the whole patch sequence twice... (pristine, working)
19:08 mornfall It would probably make sense to store away unrecorded changes, apply to pristine, copy out changed files from pristine and apply the unrecorded patch on top.
19:10 Igloo Sounds like a lot of complexity for little reward
19:11 Igloo You also have to worry about files in working that aren't in pristine, but are added by a patch you're pulling, for example
19:11 mornfall Well, it`s about 40 %.
19:11 mornfall You have to do that anyway.
19:11 Igloo I would be surprised if it was a significant amount of time in a normal workflow
19:13 mornfall Maybe I should rework the benchmarks. :))
19:14 mornfall Since get, pull and annotate are basically the only things that aren't in the millisecond range.
19:14 mornfall Pushing milliseconds further down probably doesn't make sense.
19:15 mornfall (Since we are faster than mercurial already anyway, for these operations... or at least that's how it seems, maybe I just use mercurial the wrong way)
19:24 mornfall 2m30.0s (0.5s) -> 2m9.6s (0.5s) ... these numbers I like more.
19:25 mornfall So applying stuff to pristine did get more efficient. Probably time to rip out more of the path cruft.
19:39 drksd joined #darcs
19:44 zooko joined #darcs
20:21 bos_ joined #darcs
20:33 bos joined #darcs
21:02 sshc joined #darcs
21:03 zooko joined #darcs
21:30 abuiles left #darcs
22:04 iago joined #darcs
22:05 darcswikibot joined #darcs
22:10 balor joined #darcs

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary