IRC log for #darcs, 2008-08-14

← Previous day | Channels | #darcs index | Today | Next day → | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time	S	Nick	Message
00:00		mornfall	lispy: Which hash?
00:00		lispy	SamB: well, IIRC, a context exists between the most recent tag and the current pristine state
00:00		lispy	mornfall: I'm not sure..you said something about commute and hashes changes I thought
00:00		SamB	lispy: oh
00:00		lispy	mornfall: but I'm sort of lazy to check my scroll back :)
00:00		mornfall	lispy: The HashedIO hashes change when you commute. The "patch hashes" don't.
00:01		mornfall	lispy: (The patch hashes are hashes of date, author and patch name.)
00:01		SamB	lispy: I thought it logically included all the things the TAG depended on?
00:01		lispy	mornfall: what is the difference?  hashio is a security measure over the payload and the other is a unique identifier?
00:01		mornfall	lispy: Yes, patch hashes are IDs.
00:02		mornfall	lispy: HashedIO uses hashes to address files based on content, for lazy repositories, and for security.
00:02		lispy	SamB: You can split the inventory at tags because a tag depends on all the patches recorded before it.  So Ithink what ou said about the context is right
00:04		lispy	mornfall: okay, so we have hashed PatchInfos and we have hashedIO that corresponds to the exact contents of a particular patch as it's currently represented
00:04		dcoutts	mornfall: in cabal? yes, but not very well, you can make the package not buildable conditional on the ghc version, but the error message to users is pretty uninformative
00:04		SamB	the trouble with my idea is that I can't imagine how you'd commute the darcs changes with a change of base git commit ...
00:05		lispy	mornfall: BTW, is HashedIO really the name?  That's almost as undescriptive as slurpy
00:05		mornfall	lispy: HashedIO is a module...
00:05		SamB	lispy: I don't think so
00:06		mornfall	lispy: Slurpy ain't that bad...
00:06		lispy	To me a slurpy is sometihng that I drive to 7-11 to get on a hot day
00:06		SamB	lispy: what what?
00:07		mornfall	: - )
00:07		SamB	I thought a slurpy was a ... cold dessert item
00:07		lispy	http://images.google.com/image[…]UTF-8&sa=N&tab=wi
00:07		lispy	slurpee is the product
00:07		lispy	icee is another name
00:08		SamB	I don't think I've actually seen slurpee brand
00:08		lispy	In general, I thought the names picked in the darcs source are odd
00:08		lispy	For example, Non
00:08		lispy	What is Non?
00:08		SamB	but why do you want it on a hotdog?
00:08		lispy	hot day
00:09		lispy	SamB: the scroll back says I didn't even typo that time :)
00:09		SamB	oh!
00:09		lispy	The only comment in Non.lhs is, \section{``NonPatch'' patches}
00:09		SamB	hot day and hot dog look a lot alike I guess
00:10		lispy	nods
00:10		lispy	and 7-11 sells hotdogs
00:10		lispy	but you probably didn't know i'm vegetarian
00:10		SamB	indeed
00:10		lispy	So, it's unlikely I would buy a hotdog :)
00:10		SamB	and actually I would have been working from my knowledge of what they sell at wawa ;-)
00:11		lispy	I probably would have called it a LazyDirectoryTree
00:11		lispy	LDT
00:11		SamB	is that what Slurpy is?
00:11		lispy	yeah
00:12		dcoutts	ohh, I've got one of those
00:12		lispy	so that says it should be in a library :)
00:12		SamB	I think HashedIO is a *bit* more clear of a name
00:13		SamB	like, "oh, that probably has to do with all these files in _darcs with hexadecimal names!"
00:13		lelit	rtyler: hi
00:14		lispy	Pristine is also a bit iffy.  But then again if we want to make a technical term, then ine
00:14		sm	+1 for LazyDirectoryTree, if that's what it is
00:14		SamB	pristine is a pretty good name I think
00:14		lispy	SlurpDirectory is intended to give a nice lazy way of traversing directory
00:14		lispy	trees.
00:14		mornfall	It's not a pristine.
00:14		mornfall	lispy: I don't really have a problem with Slurpy as a name.
00:14		sm	thinks HashedIO is information-free
00:15		SamB	isn't HashedIO about more than that ?
00:16		lispy	is not convinced that we gain anything in the case of large repositories by using a lazy representation of the directory tree
00:16		lispy	I think streaming + chunking is more effective
00:16		lispy	Not as elegant though
00:18		lispy	I think languagse like C and C++ continue to demonstrate the efficiency of buffer based IO
00:18		lispy	and I think darcs continues to demonstrate that what we do does not scale for large datasets
00:19		mornfall	lispy: Oh. You gain a functionaly view on the directory tree.
00:19		mornfall	lispy: Don't forget that the Slurpy is a non-monadic entity.
00:19		mornfall	lispy: It's a value. That's very valuable.
00:19		mornfall	... ; - )
00:19		lispy	Hmm...if it's a value then it must have some IO hidden inside?
00:19		mornfall	s/functionaly/functional/
00:20		mornfall	lispy: The hidden IO is through the laziness only.
00:20		lispy	we import this:  unsafeInterleaveIO
00:20		mornfall	lispy: It has unsafeInterleaveIO's hidden inside, indeed.
00:20		SamB	I was under the impression that Slurpy was mostly a valuable source of performance bugs?
00:20		mornfall	Oh come on.
00:20		lispy	No actually I think SamB is more right than we'd like to admit
00:21		mornfall	It's not like rewriting everything that touches files in an imperative style would really help anything.
00:21		SamB	I thought I'd heard that from a darcs developer?
00:21		lispy	I think our usage of mmap is WORSE, but I'm not convinced of the performance virtues of Slurpies
00:21		mornfall	Well, maybe we should write it in C++, afterall.
00:21		SamB	that would be almost as stupid as using INTERCAL
00:21		mornfall	Well, I have worked with slurpies. They are actually fairly good.
00:22		mornfall	Especially given the alternatives.
00:22		mornfall	(Like dragging everything and their mother into the IO monad.)
00:22		zookofamtime	I suspect that there are virtues to Haskell other than side-effect-freeness/laziness.
00:22		zookofamtime	At least, I hope so.
00:22		SamB	zookofamtime: yes! they are called typeclasses
00:22		zookofamtime	Because filesystem interaction is not a good place for side-effect-freeness/laziness.
00:23		mornfall	zookofamtime: Well, it might be surprising, but it actually is. And GIT is a good showcase of functional filesystem treatment.
00:23		lispy	I don't think we're really complaining about Haskell
00:23		SamB	I don't remember git using laziness per-se
00:23		mornfall	SamB: No. But you might remember that they don't rewrite files.
00:24		lispy	I'm complaining about the IO architecture of darcs not being ammenable to performance in, (extreme?), many cases
00:24		SamB	I wish I could be the first to admit that git is in fact a wonderful example of functional storage ;-)
00:24		mornfall	lispy: Can you name at least one?
00:24		lispy	mornfall: yes
00:24		lispy	mornfall: try to record a file of 6 GB
00:24		SamB	well, what if I run darcs record foo ?
00:24		lispy	mornfall: anything over about 3GB should fail on a 32 bit bus
00:25		mornfall	lispy: Nothing to do with IO architecture, really. It wouldn't work in C++ either.
00:25		SamB	does that still involve pulling in the pristine for the entire tree?
00:25		lispy	mornfall: the way we do it wouldn't work in any language no.
00:25		mornfall	lispy: It's also a very poor usecase for darcs.
00:25		SamB	and comparing with my entire working directory?
00:25		SamB	and THEN filtering only the changes that actually have anything to do with foo?
00:26		lispy	mornfall: another example, then.  We have had bugs in the past, that forced us to use strict hacks, when adding too many files at once because we run out of mmap handles in our lazy reading
00:26		mornfall	SamB: Not a problem of IO architecture, either.
00:26		SamB	mornfall: well where the heck did it come from?
00:26		mornfall	SamB: From the fact we first do a smart_diff and then only look at which changes we want? Instead of telling smart_diff to only look at foo and bar.
00:26		lispy	mornfall: if we bounded our mmap usage, we could record files of arbitrary size by streaming the chunks
00:27		mornfall	lispy: Once you use mmap, you can never handle anything >3G on 32bit arches.
00:27		SamB	mornfall: well, I was assuming that the IO architecture somehow encouraged this
00:27		SamB	mornfall: heh
00:27		SamB	untrue
00:27		mornfall	SamB: No, it doesn't.
00:27		lispy	mornfall: once you use mmap to mmap *entire files* you mean
00:28		SamB	doesn't think git can record such large files either though
00:28		mornfall	lispy: Ah, well. A friend of mine tried to get partial mmaping to work once. (In a C++ app, if that helps any.) It took I think months to debug.
00:28		mornfall	lispy: The syscall interface is more than brittle.
00:29		mornfall	But yes, it's fine if you want to optimize darcs for multi-gigabyte files.
00:29		lispy	Okay, here is another point.  We should be able to only ever have 4 patches in memory while doing a commute.  Even better, we could stream them to/from disk during the commute and bound the total memory to the length of a line (or maybe even do better than that)
00:29		mornfall	I'd say there are *way* more pressing problems.
00:29		SamB	hmm, well, they're saying that the practical limit on git is actually a lot lower than 4GB
00:29		SamB	lispy: it might not be optimal to actually only have 4 patches all the time ;-)
00:29		mornfall	lispy: Okey, that is a little more problematic and finally something that relates to the IO architecture.
00:30		lispy	SamB: sure
00:30		SamB	lispy: also, a lot of these patches would likely be temporary
00:30		SamB	so you'd want to put them seperate from the other patches at first
00:31		mornfall	lispy: But then, noone says we have to use slurpies to read in patches.
00:31		lispy	mornfall: well, if we're sequentially mmap parts of the file just to read the whole thing, I bet we'd find it not too difficult.  I wouldn't be surprised if you're friend was implement a database or someting with records and having problems with that logic?
00:31		mornfall	lispy: A tile swapper, really.
00:31		lispy	(I don't want to make it sound like mmap is trivial, I know it can be a PITA)
00:32		mornfall	Don't forget there's no mmap on windows.
00:32		mornfall	And that typical file size is 10k.
00:32		lispy	I'm happy to forget mmap completely for now :)
00:32		SamB	mornfall: sure there is an mmap on windows
00:32		mornfall	SamB: "an mmap" maybe... not mmap though.
00:32		lispy	mornfall: yeah, for the buffer size I was thinking we'd do at least 1 buffer of 4k
00:33		lispy	Ram is cheap
00:33		SamB	well, yeah. iirc, Python on windows has a function of the same name but different arguments as the *nix Python ...
00:33		lispy	We could do a 10MB bufer
00:33		dcoutts	SamB: windows has something very much like mmap
00:33		mornfall	lispy: ByteString.Lazy? I have tried that when I was working on whatsnew for the file reading case.
00:33		mornfall	lispy: It was always slower than gzReadFile whatever there was before.
00:34		mornfall	And much more memory hungry.
00:34		mornfall	gzReadFilePS
00:34		lispy	mornfall: I don't trust lazyiness when we're dealing with large cases.  Strictness doesn't always compose in obvious ways
00:34		dcoutts	mornfall: ByteString.Lazy was slow? I thought it was the last word in IO performance :-)
00:34		dcoutts	mornfall: my zlib binding + lazy bytestring is the same speed as the C version, I've benchmarked it.
00:35		mornfall	dcoutts: Hm, not so in darcs, for some reason.
00:35		SamB	mornfall: more of the NIH darcs suffers so much from ?
00:35		lispy	mornfall: I'm utterly convinced that to get the extreme cases down you really do need strict streaming + chunking.  And I think the contined lack of performance reputation in darcs speaks to that (obviously darcs isn't completely IO bound and the algos need to change too)
00:36		mornfall	lispy: Oh. Darcs is almost never IO bound. It may be memory bound though.
00:36		dcoutts	mornfall: if you have a test case I'd like to see. lazy bytestring are not doing anything crazy, they should be able to read/write very quickly, so long as the chunk size is big enough.
00:36		mornfall	dcoutts: Well, I tried replacing c_gzread with Codecs.Compression.Zlib and some bytestring magic to get a single strict ByteString...
00:36		lispy	okay, maybe I was too cavalier in my terminology :)
00:37		dcoutts	mornfall: ohh, you don't want a single strict bytestring
00:37		dcoutts	mornfall: that'll be slow
00:37		mornfall	dcoutts: Well, the type system doesn't let me to use anything else without rewriting half of darcs.
00:37		dcoutts	mornfall: when you convert to a strict bytestring you have to copy the *whole* thing, twice on average.
00:37		SamB	hmm, what is with the numbers to the left of the patches in _darcs/patches?
00:37		SamB	er. of the hashes.
00:37		mornfall	dcoutts: Probably true.
00:37		lispy	SamB: file size in hex I thnk?
00:38		dcoutts	mornfall: ok, if you have to use strict bytestrings internally then yes, reading with lazy and converting is not going to be a win
00:38		SamB	lispy: it doesn't look like hex to me
00:38		dcoutts	mornfall: if you can do things in a streaming style without converting the whole thing to a single strict chunk then lazy bytestrings are very very fast
00:38		mornfall	SamB: Filesizes.
00:39		mornfall	dcoutts: I'd believe that, I suppose. And I probably also want to convert darcs to use them. But it doesn't seem to make sense to do gradually. Everything has to use them.
00:39		mornfall	dcoutts: Because the conversions are expensive (as you point out).
00:39		dcoutts	mornfall: right
00:40		lispy	Hmm..I wonder what that would do to speak leakage if all of them became lazy
00:40		dcoutts	mornfall: well the good thing at least is that it's easy to experiment
00:40		lispy	And this discussion of rnf, we have lots of places in darcs where we need to make something strict to add the GC, but I've never seen rnf used.  We should probably adopt that.
00:40		dcoutts	mornfall: if you've got it using ordinary strict bytestring then you just switch the imports and experiment
00:41		mornfall	dcoutts: I first need to get the few foreignptr bits that directly peek at the strict stuff removed.
00:41		lispy	mornfall: hmm...does that exist even when not using FPS?
00:41		lispy	mornfall: oh, you're thnking of has_funky?
00:41		mornfall	lispy: You mean the ByteString.Internal.toForeignPtr uses? Yes.
00:41		mornfall	fromPS2Hex, eg.
00:42		twb	If somebody said something to me here, last night after I slept, it passed my scrollback
00:42		mornfall	dcoutts: I suppose all uses of ByteString.Internal need to be removed first.
00:42		mornfall	Then mmap needs to be.
00:42		mornfall	And the C bindings to zlib.
00:43		lispy	Is there a good reason why we have things like conv_to_hex written in C?
00:43		mornfall	lispy: Historical?
00:44		lispy	dcoutts: is it bad performance or memory wise to do an inplace update of a lazy bytestring?
00:44		lispy	I guess that's why we do it?  to avoid making a copy?
00:44		mornfall	:q
00:44		mornfall	Err, sorry.
00:44		mornfall	lispy: Might be premature optimisation.
00:44		mornfall	That code is probably very old.
00:45		twb	lispy: fix it, then profile before and after
00:45		mornfall	All these things take huge amounts of time.
00:46		SamB	lispy: I don't think it makes any sense to do an in-place update of a lazy bytestring
00:46		lispy	SamB: I kinda figured
00:47		mornfall	There's really no sane way to.
00:47		mornfall	Anyway.
00:47		SamB	I think it makes marginally more sense with a strict one
00:47		SamB	but, you know, if *Python* doesn't allow it, it might be a bad idea!
00:48		zookofamtime	<mornfall> twb: I'll possibly have to smuggle those Dmitry's patches into
00:48		zookofamtime	Lenny. Not looking forward to that work...
00:48		lispy	Seriously not sure I get this.  We take and shift the char right 4 places and output a hex character
00:49		lispy	Then we take the first 4 bits and do the same with it
00:49		lispy	Ah, well that is 0 - 15
00:49		mornfall	lispy: It's a normal hex converison?
00:49		twb	zookofamtime: thanks
00:49		gwern	lispy: well, the c bits aren't always sane. vide one of the things I was involved with, checking for binary chars - it checkde the entire thing
00:49		twb	Now I gotta stagger to work
00:50		gwern	wonders why I've used 'vide' for the 3rd time today
00:50		twb	gwern: hubris?
00:51		gwern	yes! my overweening vocabulary shall be my undoing!
00:51		lispy	gwern: I recall that very well
00:51		mornfall	Hmh. I'm not extremely happy about today's revolutionary mood, to be honest...
00:51		lispy	conv_to_hex is really unsafe
00:51		gwern	wanders into the inner city and complains about the niggardliness of the shops. wait, no, it has nothing to do with 'niggers' AIEEE
00:51		mornfall	(I might have myself some guilt in that...)
00:52		lispy	Oh, wait, it makse a copy
00:53		mornfall	Nevertheless, I'll prolly have to sleep. Good luck with the wisdomful revolution of sorts. Hopefully world still spins when I wake up.
00:54		gwern	but if we stop the world, we could get off
00:54		lispy	mornfall: g'night
00:55		mornfall	Well, don't forget to not overdo it. Talk is dangerous. Too much talk about future puts people like me off. I have too many times been in this situation, and it never happened that I wouldn't be disappointed soon afterwards.
00:55		lispy	mornfall: well we don't do an inplace replace, we copy the damn thing to a new string that is twice as big, surely using lazy + map wouldn't hurt us
00:55		mornfall	(The initial hooray fades away quickly.)
00:55		sm	agreed agreed.. sustained delivery of working code trumps all talk
00:56		lispy	I should be working on other things but suddenly a rewrite to lazy byte string and a removal of FPS strikes me as interesting
00:57		mornfall	So, not completely free of worry, I crash to bed. Highlight me if you have something that I should read by the morning. Or maybe mail. Whatever. Night.
00:57		sm	night mornfall  :)
00:57		lispy	g'night!
00:57		lispy	sleep well
00:57		lispy	I'll probably just read papers tonight
00:57		gwern	lispy: how hard could it be? just go to NewFPS.hs, and swap the imports right?
00:57		gwern	basically?
00:57		lispy	I need to unwird
00:57		sm	lispy, you are crazy man.. do it
00:57		lispy	gwern: I dunno, never realy putz'd in here
00:57		sm	wait, which is unwinding, coding or reading papers ?
00:58		lispy	sm: I'm not sure
00:58		lispy	sm: I feel exhausted
00:58		sm	probably not coding, then
00:58		lispy	It's either more coffee now, or go to bed in an hour
00:58		gwern	dunno. reading papers can be awfully tough, and coding can be quite fun
00:58		gwern	epsecially haskell-related papres
00:59		sm	take a bath, go to bed early, get up early and do two hours dawn hacking
00:59		lispy	cabal install failed to install bytestring :(
00:59		sm	that's what I never do but will one of these days
01:00		lispy	Saving old package config file... ghc-pkg.bin: /Users/dagit/.ghc/i386-darwin-6.8.2/package.conf: renameFile: permission denied (Permission denied)
01:00		SamB	if in doubt, play videogames ;-)
01:00		sm	hmm not true, I did it this morning
01:00		SamB	"this morning" is a strange definition of "never"
01:00		sm	oh, I've got one for you..
01:00		sm	yes SamB, it's so new I didn't recognise it
01:01		sm	ok, I've got the perfect game here.. do I unleash it on the channel and cripple darcs progress ? hmm :/
01:01		lispy	okkay, i changed the perms and it installed
01:01		gwern	I played some Defcon this morning for the first time in ages. and then wargames was on tonight
01:01		gwern	spookynchronous!
01:01		lispy	gwern: so what was the next step? I have bytestring, I need to configure?
01:02		gwern	lispy: what, to install bytesting itself?
01:02		lispy	no, darcs
01:02		lispy	./configure --enable-bytestring
01:02		sm	well, http://puppygames.net -> droid assault, or any of them, but fixing darcs will probably win out. later all
01:03		gwern	I forget
01:03		gwern	I think it autodetects?
01:03		lispy	gwern: now you said NewFPS?
01:03		gwern	no no, I was distinguishing OldFastPackedString.hs from (new) FastPackedString.hs
01:05		lispy	oh man, removing the dependency on ByteString internal is probably beyond me
01:06		lispy	like how would I replace Data.ByteString.Internal.unsafeCreate ?
01:07		gwern	dunno. does the internal stuff not work with lazy bytestrings?
01:07		lispy	Probably not
01:08		lispy	It probably makes them strict?
01:08		gwern	I can't imagine why. aren't both strict and lazy bytestrings constructed out of stuff from internal?
01:08		lispy	oh wait, I see lazy internal
01:09		lispy	oh but they have totally different internals
01:10		dcoutts	lispy: if you need help with lazy bytestring internals, ask me, I wrote it
01:10		lispy	dcoutts: hey
01:10		lispy	thanks
01:10		lispy	So...D.BS.Internal has mallocByteString
01:10		lispy	But, lazy does not
01:10		dcoutts	pulls up the code
01:11		lispy	and I'm guessing for a good reason
01:11		dcoutts	lispy: a lazy one is just a chunk of strict ones
01:11		dcoutts	erm
01:11		dcoutts	a list of strict ones
01:11		dcoutts	so if you want a single chunk lazy bytestring then you can make one
01:12		dcoutts	lispy: eg Lazy.fromChunks [Strict.mallocByteString bla blah]
01:12		lispy	Ah
01:13		dcoutts	lispy: so that's why it's not there internally
01:13		dcoutts	mallocByteString is really pretty low level
01:13		dcoutts	it's just a wrapper around mallocPlainForeignPtrBytes
01:14		dcoutts	the more interesting things are create, createAndTrim etc
01:14		lispy	Which is ironic, since the darcs is just renaming BI.mallocByteString to mallocForeignPtr :)
01:14		dcoutts	hah
01:15		dcoutts	lispy: it's a tad faster than mallocForeignPtrBytes
01:15		dcoutts	since we call mallocPlainForeignPtrBytes when using ghc 6.6 and later
01:15		dcoutts	which avoids allocating space for finalisers
01:15		lispy	BS.I.unsafeCreate, should we be using this?
01:15		dcoutts	lispy: so what are you trying to do, what's the use case?
01:15		lispy	Hmm...
01:16		lispy	This right now is a first pass at changing from strict BS to lazy BS
01:16		lispy	So, I can't say how this function gets used
01:16		dcoutts	but what source data format do you have that you want to make into a ByteString (strict or lazy)
01:17		lispy	I'm not sure
01:17		dcoutts	lispy: remember doing strict -> lazy conversions are trivial, since you just get a lazy bytestring with a single chunk. Conversions the other way are expensive (if there's more than one chunk)
01:18		lispy	Well, I want to see if I can make all the byte strings lazy
01:18		lispy	Just totally forget about having strict ones
01:18		lispy	Is that stupid?
01:18		dcoutts	depends on the use case
01:19		dcoutts	there are sometimes advantages to having just a single chunk, but in general having a list of them is a win
01:19		lispy	what is a known instance of the former?
01:19		dcoutts	well for one thing it guarantees you never have fragmentation
01:20		dcoutts	some operations are slightly simpler (ie faster)
01:20		lispy	fragmentation leads to space leaks?
01:20		dcoutts	no, poor performance
01:20		lispy	lowering the memory foot print is a nice goal
01:21		dcoutts	if you can arrange that the operations you're doing are lazy/streaming then you can have a big win for larger quantities of data
01:21		dcoutts	you can always drop down to the explicit list of strict chunks view if necessary
01:21		lispy	no spanEnd in D.BS.L.C?
01:21		dcoutts	ah, so that's a good example
01:22		lispy	dcoutts: cool!
01:22		dcoutts	working from the back is hard in a list representation but trivial when you know you've only got a single chunk
01:22		dcoutts	you have to think of it more like a list again and less like an array
01:23		dcoutts	though in practise we try and use pretty large chunks, eg 4-64 k
01:23		lispy	I should try to move all the C bits into our strings module, that would reduce the need for the low level stuff
01:24		lispy	is mapping over a BS fast?
01:24		dcoutts	lispy: yep
01:24		dcoutts	if you've got lots of C code for manipulating strings then that's hard to adapt to a chunked representation
01:24		dcoutts	C string handling assumes contiguous representation
01:25		lispy	right, so I should just nuke the C stuff I would think
01:25		dcoutts	what kind of C ops have you got?
01:25		lispy	most ofit is just maps
01:25		lispy	Ithink
01:25		lispy	like this, utf8_to_ints
01:26		lispy	int utf8_to_ints(HsInt *pwc, const unsigned char *s, int n) {
01:26		lispy	/* returns number of unicode chars in the output.  The output array is
01:26		lispy	assumed to have the same number of elements as the input array, which
01:26		lispy	is n. */
01:26		dcoutts	lispy: it's all in fpsstring.c ?
01:26		dcoutts	inspects
01:26		lispy	yeah, I think so
01:26		lispy	One tricky case will be the PS2Hex
01:26		dcoutts	hmm, utf8 length
01:26		lispy	you write two bytes per input byte
01:27		lispy	it's ord
01:27		lispy	But, for unicode
01:28		lispy	"returns number of" means, "returns int value of"
01:28		dcoutts	that would work with strict ByteString in almost exactly the same way as it works now with the packedstring
01:29		dcoutts	lispy: if it's performance sensitive then leave it using strict bytestring in C
01:29		lispy	Shouldn't I really just be able to call ord?
01:29		lispy	map BSL.ord mystring
01:29		dcoutts	ord is slow :-)
01:29		lispy	hmm
01:29		dcoutts	no, map is Byte -> Byte
01:29		dcoutts	you're doubling the length
01:30		lispy	doubling the length?
01:30		lispy	still talking about utf8_to_ints riht?
01:30		dcoutts	oh, I meant PS2Hex
01:30		dcoutts	for utf8 you've actually got to do the decoding
01:31		lispy	dcoutts: but here is the thing, if we use utf8_to_ints and it still requires a Strict bytestring, why convert any of it?
01:31		dcoutts	lispy: utf8_to_ints could be adapted to use a chunked representation if you needed to
01:31		dcoutts	it's relatively simple
01:32		dcoutts	all it needs is to return the state at the end of the chunk
01:32		dcoutts	so you can pass that when you call the function again on the next chunk
01:32		lispy	fold don the chunks
01:32		dcoutts	eg if there is a 2 or 3 byte utf8 char that was split over a chunk boundary
01:32		dcoutts	exactly, a left fold on the chunks
01:33		dcoutts	suming as we go and also carrying the slop state over chunk boundaries
01:33		dcoutts	so the C code would be slightly more complex
01:33		lispy	Although, that might be hard to do in terms of modifying utf8_to_ints
01:34		dcoutts	lispy: so it's a non-trivial amount of work, you'd want to be sure it was worthi
01:34		dcoutts	worth it
01:34		dcoutts	lispy: just switching to Data.ByteString should help
01:34		dcoutts	since more people know that api and we've pretty heavily benchmarked it
01:34		dcoutts	and it'd let you use all the same C tricks as you've got now
01:34		lispy	I would love to toss FPS, but we're not at that point yet.
01:35		dcoutts	and that'd be a starting point for sseing if there are places where lazy bytestrings might be a win
01:35		lispy	We can build with BS, but it's optional if you know what I mean...I also don't think it's the default, let me check
01:35		dcoutts	lispy: what's the minimum ghc version you accept atm?
01:36		dcoutts	6.6?
01:36		lispy	dcoutts: 6.4, iirc
01:36		dcoutts	oh, not if you use OPTIONS_GHC ;-)
01:36		lispy	interesting
01:36		dcoutts	lispy: well Data.ByteString works with 6.4
01:36		lispy	okay, then we have lapsed into not supporting 6.4.  I'm happy about hat.
01:37		dcoutts	lispy: in the worst case, just bundle the whole of Data.ByteString
01:37		lispy	do you know if the BS that shipped with 6.6 had the readFile problem?
01:37		dcoutts	lispy: which readFile problem? not closing the handle at the end?
01:37		lispy	yeah
01:37		dcoutts	that was only lazy bytestring
01:37		lispy	ah
01:38		lispy	looks like we could probably remove this FPS cruft at this point
01:38		dcoutts	and it's only fixed in the bytestring that comes with 6.8.3
01:38		lispy	gotcha
01:38		dcoutts	though people could upgrade
01:38		dcoutts	lispy: yeah, I'd just bundle the whole lib and then not touch it
01:38		lispy	I'm on 6.8.2 and I installed jsut now via cabal so I think I'm oka'y
01:39		lispy	you mean, bundle the latest BS release instead of FPS?
01:39		dcoutts	yes
01:39		dcoutts	though use the standard one if it's available in a suitable version
01:39		dcoutts	then at least you've got a single modern api
01:39		lispy	6.6 didn't ship with BS did it?
01:39		dcoutts	it did, yes
01:39		dcoutts	it was in base back then
01:40		lispy	Any reason to not use that one?
01:40		dcoutts	well, if you bump into any bugs
01:40		dcoutts	there's not much we can do to fix deployed versions
01:40		dcoutts	we know the current version is pretty solid
01:41		dcoutts	we've not had serious bug reports for over a year
01:41		lispy	Bah, I'd like to just support 6.8.3 and newer (6.8.2 has a buggy getSymbolicLinkStatus)
01:41		lispy	it's not my decision at the moment yet though
01:42		dcoutts	lispy: I know the feeling, but if people want to support 6.6, you can do it without too much pain
01:42		dcoutts	because although it's code in your repo, it's not code you have to maintain
01:42		dcoutts	indeed you should make it a rule not to touch it
01:43		dcoutts	and if you're using the released version in a default configuration then there's little temptation to modify or extend it
01:43		dcoutts	with the bundled just for compat
01:44		lispy	This is a proposal on the table actually...that as we depend more and more on standard libs that at least for a while we provide a bundled copy in the stable releases
01:44		lispy	The dev version wouldn't come with anything
01:44		twb	Is the BTS down?
01:44		lispy	twb: I think so
01:44		dcoutts	lispy: that sounds like a sensible, practical plan.
01:45		lispy	dcoutts: we should be able to factor out a file downloader, an LCS algo, a file differ, and mabe a slurpy library, really
01:45		dcoutts	lispy: it gives all the advantages of a common known api, with the ability to build on old platforms
01:45		dcoutts	lispy: you use libcurl right?
01:45		lispy	it's an option yeah
01:46		dcoutts	it does http keepalive and pipelining right?
01:46		twb	dcoutts: pipelining is a ./configure switch.
01:46		lispy	no idea about keepalive, pipelining might work?  I guess pipelining is hard to trigger and test
01:46		dcoutts	it can be a big perf win
01:46		lispy	yeah, I know david tried to set it up
01:46		dcoutts	much like your system of using a single ssh connection
01:47		lispy	But, yeah know, from a blackbox point of view, darcs get is basically just a recursive download tool.  And yet, it's orders of magnitude slower
01:47		dcoutts	all http-1.1 servers grok it, though not all enable it by default
01:48		dcoutts	aye, if you know which files you need then you should be able to pipeline lots of requests
01:48		lispy	I was hoping we could analyze what darcs get is doing that makes it so slow and make it faster..make it closer to just a download tool
01:48		dcoutts	and not have to incur the round trip latency between each request
01:48		dcoutts	there are some free http protocol analysers
01:49		lispy	dcoutts: it would also be nice if we could compare notes about the lazy directory tree, and make a common library
01:49		dcoutts	and check out any libcurl docs on pipelining
01:49		dcoutts	lispy: oh I've only got something pretty trivial
01:49		lispy	Well, in that case you may just want to switch to what we have...but either way I bet a shared lib would get more testing
01:50		dcoutts	I don't use it for anything serious, but lots of script like things tend to want it
01:51		lispy	Well, I give up on the BS refactor for right now.  Just stick with Data.BS for now.  But thanks for your help.
01:51		lispy	?version
01:51		lambdabot	lambdabot 4.1
01:51		lambdabot	darcs get http://code.haskell.org/lambdabot
01:51		lispy	yay!
01:51		lispy	?vixen I missed you!
01:51		lambdabot	what's worth more a flush or a full house?
01:51		dcoutts	lispy: yeah, converting to lazy would be much easier after moving completely to the standard Data.ByteString
01:52		lispy	dcoutts: Well, we don't need FPS at all
01:52		lispy	it's just that we keep it around, so I'm not sure I follow
01:53		dcoutts	lispy: I mean you don't need to keep it around
01:54		lispy	making a new patch format would help more than optimizing our strings I just remembere
01:55		lispy	in fact, I bet we could make commutes trivial with the right patch formats
01:55		lispy	trivial in the sense of how much memory is involved
02:00		dcoutts	lispy: sure, the main advantage is maintainability not performance
02:01		dcoutts	of course if we were doing it for performance we'd start by profiling rather than guessing :-)
02:01		lispy	you saw the roadmap right?
02:02		dcoutts	yep
02:02		lispy	big fat performance regression suit
02:02		lispy	suite*
02:02		lispy	if all we did was to compare to previous stable releases that would be a huge improvement in profiling
02:03		dcoutts	aye
02:03		lispy	But, I know something that could significantly improve the memory usage anyway
02:03		lispy	I don't think commute really needs to hold the contents of patches in memory
02:03		lispy	I see that invert does
02:03		lispy	er doesn't
02:04		lispy	inverts just swap new with old
02:04		lispy	So, if we just held a reference to the new and old it wouldn't cost anything
02:05		lispy	But, then I dunno, maybe data sharing in the RTS makes this not an issue
02:05		twb	Do you know who droundy reminds me of sometimes?
02:05		twb	Olin Shivers.
02:06		lispy	twb: http://www.ccs.neu.edu/home/shivers/ ??
02:06		twb	lispy: probably.
02:06		lambdabot	Title: Olin Shivers
02:06		twb	I'm specifically thinking on droundy's post <117f2cc80808131505k7219d13ejc8afccba2b56e4abmail.gmail.com>
02:07		lispy	I have no idea who Olin Shivers is or how to figure out which post you mean :(
02:08		twb	lispy: eh, you won't get any value from knowing then.
02:09		ChilliX	darcs 2.0.2 on MacOS does often just hang in what seems like a network operation when I pull from darcs.haskell.org
02:09		ChilliX	The same doesn't seem to happen with 1.0.9
02:09		ChilliX	Does anybody have an idea what the problemc ould be?
02:09		twb	ChilliX: not without more information, I think.
02:10		zooko	I've heard people complaining that darcs.haskell.org is realllly slow recently.
02:10		zooko	Try darcs get http://allmydata.org/source/ta[…]runk-hashedformat for comparison.  :-)
02:10		zooko	And you can add -v and --timing for lots of detail.
02:10		SamB	it might actually be due to using git over http
02:10		ChilliX	zooko: that doesn't explain that it works with 1.0.9
02:11		ChilliX	its not a new problem
02:11		zooko	ChilliX that's interesting.
02:11		dcoutts	ChilliX: over http in both cases?
02:11		ChilliX	I think it is usually http subrepos where it happens
02:11		SamB	ChilliX: true
02:11		ChilliX	in any case, both versions of darcs would use the same method as they run on th same repo
02:12		lispy	ChilliX: do you happen to know what http library you built with?
02:12		lispy	ChilliX: probably libcurl?
02:12		SamB	maybe your darcs doesn't have pipelining?
02:13		lispy	heads home
02:13		ChilliX	lispy: its darcs 2.0.2 straight from MacPorts (which uses libcurl afaik)
02:13		ChilliX	yes it does
02:13		ChilliX	Library Dependencies: readline, curl, gmp
02:14		ChilliX	(uses curl 7.18.2 - I was wondering whether it could be that)
02:15		ChilliX	SamB: Why would it hang without pipelining
02:16		ChilliX	its not just slow, it hangs until I ctrl-c it
02:16		zooko	Oh, this is the one that I diagnosed earlier tonight.
02:16		ChilliX	zooko: you did?
02:16		zooko	I couldn't ticket it because bugs.darcs.net wasn't working for me.
02:16		zooko	Well, I determined that it was a problem with darcs-2.0.2 and libcurl-7.18.2, but that upgrading to the current trunk of darcs fixed it, which means that Dmitri's recent libcurl patches fix it.
02:17		zooko	I asked kowey to add it to the issue tracker since he was able to access the issue tracker, but he didn't say whether he did so.
02:17		dcoutts	ChilliX: perhaps as a workaround try building against the same version of libcurl that you're using with darcs-1.0.9
02:18		ChilliX	dcoutts: that's hard, as I just got a 1.0.9 binary somewhere off the net (can't build the old version through macports, I think)
02:18		dcoutts	hmm, right
02:18		ChilliX	When will the next darcs release be out
02:19		ChilliX	with the abovementioned patch
02:19		zooko	ChilliX: we talked about that, too.  The action that was taken was that kowey created the "2.0.3" category on the tracker.
02:19		zooko	I don't know when the next version will come out.
02:19		zooko	For what it is worth, I urged the others to move it along.  :-)
02:20		ChilliX	ok
02:20		zooko	There is another major issue, a performance regression, that they are working on.
02:20		zooko	I think once they have that one wrapped up then they are going to proceed with 2.0.3 which is just a few valuable fixes over 2.0.2.
02:20		zooko	Meanwhile, it is pretty easy to build it yourself if you want.
02:20		zooko	I've done it on Mac OS X many times.
02:21		twb	OS X: just another Slackware
02:21		twb	ducks
02:22		zooko	;-)
02:22		zooko	Say, is anyone familiar with the terminology and research literature of Byzantine systems?
02:22		ChilliX	zooko: Yeah, sure I could build it myself, but so far I get by with using 1.0.9 half of the time
02:23		zooko	I'm trying to argue in this paper that the Tahoe Least-Authority Filesystem is dramatically better than the best known results in the theoretical literature because it leverages public key cryptography.
02:23		zooko	And I don't want to say anything stupid...
02:23		zooko	ChilliX: okay, you're good until 2.0.3 then?  Thanks for the bug report.
02:23		ChilliX	Anyway, thanks, guys, for the information
02:23		zooko	If you can log into bugs.darcs.net, and if kowey didn't already open a bug.
02:23		zooko	Oh, he posted a URL to this channell...
02:23		ChilliX	zooko: If it annoys me too much, I'll build my own version; otherwise, I'll wait.
02:23		zooko	<kowey> zooko, is it http://bugs.darcs.net/issue876 ?
02:24		zooko	So you might look at that URL, and if it matches your observations or not you could add a note to the ticket.
02:24		zooko	Thanks.
02:25		ChilliX	the bug tracker gives me an error message
02:25		ChilliX	Can I file a bug for the bug tracker? ;)
02:25		zooko	sighs.
02:25		zooko	Yes.
02:25		zooko	You can.
02:25		zooko	Write to darcs-usersdarcs.net .
02:25		zooko	Saying that the tracker is broken.
02:26		ChilliX	Ah, but I think you said earlier that it is broken, so I assume people know about it
02:28		zooko	Yeah, it isn't necessary to post about it.
02:28		twb	zooko: the Byzantine empire had computers?
02:28		zooko	Now it is Read With Wife Time, here in Boulder, Colorado.
02:28		zooko	:-)
02:29		twb	zooko: I thought there weren't even human ones until India and Ireland.
02:29		zooko	twb: it makes me slightly uncomfortable to use that terminology.
02:29		zooko	What if some Orthodox Christians think that I am making fun of them?
02:29		zooko	Okay, by for now.
02:29		twb	stares blankly
02:30		zookofamtime	http://zooko.com/lafs.pdf
02:33		twb	ERROR: cannot verify zooko.com's certificate, issued by `/C=GB/ST=Some-State/CN=zooko.com/emailAddress=zookozooko.com'
02:33		Riastradh	zookofamtime, are you aware that your certificate expired a year ago?
02:35		gwern	it
02:35		gwern	's been mentioned
02:54		lispy	find irony in zookofamtime being such a crytpo buff and having an expired cert
02:55		lispy	dcoutts, ChilliX: FWIW, if the fix is dmitry's patch then the problem is that to improve error reporting we now build darcs in a threaded way...and so linking against the old libcurl (which probably doesn't allow pipelining, which you want) is unlikely to help
02:56		lispy	oh sorry, that probably wasn't 100% clear. The halted download was some sort of race condition it seems
02:56		lispy	I haven't followed any of that, but that's the gist I got through ossmossis
02:57		ChilliX	lispy: I see.  The the problem is that libcurl is not thread-safe?
02:58		lispy	ChilliX: that's a likely scenario, but I really don't know
02:58		ChilliX	ok
02:58		lispy	otoh, it semes to be fixed with dmitry's patch
02:59		lispy	and I build darcs on osx with the default configure and it works great
03:00		ChilliX	so maybe, MacPorts should do that, too (not sure who the maintainer of that is)
03:06		lispy	Igloo: running the test suite from an account with a global cache is foolish isn't it?
03:06		lispy	Igloo: because wouldn't each test always produce the same entries in the global cache, and hence hide some bugs?
07:51		dons	beautiful response to Claus, kowey, lispy.
07:51		twb	mornfall: do you have a server I can get sh access on to test some changes to the darcs debian integration?
07:52		dons	that's the kind of reponsive, open project leadership we like to see :)
07:52		kowey	dons: :-) teamwork!
07:52		dons	go team!
07:52		lispy	tags kowey into the ring
07:52		lispy	I'm going to bed for real? this time?
07:52		kowey	runs around like a headless chicken
07:52		dons	sjanssen and i learnt that lesson: two is better than one for getting things done.
07:52		dons	you can back each other up, and always have someone on call.
07:53		lispy	dons: makes sense.  I have tihs book from SoC about managing FOSS projects...i should damn well finish reading it
07:53		dons	i wish they'd send that to the mentors.
07:53		dons	you know, the ones doing the managing.
07:53		kowey	what's the title?
07:53		lispy	just a sec, where i did i put it...
07:54		lispy	"producing open source software" by karl fogel
07:54		dons	that's it.
07:54		kowey	thanks! good night, lispy :-)
07:55		lispy	g'night
07:56		lispy	heads for bed
08:02		dons	kowey, lispy, it'll be fun to have the xmonad and darcs teams have a hackathon together :)
08:02		dons	good press too
08:50		kowey	sounds like a fun idea indeed, dons
08:50		kowey	are you thinking in the near future?
09:01		mornfall	*yawn*
09:04		mornfall	twb: Re.
09:04		mornfall	Now that's gonna be tricky.
09:06		twb	mornfall: eh, don't worry about it for now, then
09:07		twb	I should get off my arse and become a DM or DD
09:08		mornfall	I only have one private machine at home, all the rest of my dev systems belong to either university or Red Hat... And I apparently can't give out access to those : - (
09:08		mornfall	And home = crap uplink and it's not even powered on most of the time.
09:09		twb	Yeah, I got rid of my home machine
09:10		twb	Thinking "oh yeah, we have a vmware server at work"
09:10		twb	Except that it locks up for ten minutes every time you try to do I/O
09:10		twb	So dpkg-source -b ends up taking (literally!) four hours.
09:10		mornfall	Icky.
09:11		mornfall	twb: I've pushed your patches at least.
09:12		twb	I wanted to fix the index.html/darcs.html bug, but I need to do a build to be confident my fix will be happy
13:02		mornfall	zookofamtime: Ping me when you appear, please.
14:05		dcoutts	mornfall: so how would I tell if I'm suffering the lstat bug with darcs2?
14:05		dcoutts	I'm using 6.8.2 but on 64bit linux
14:05		dcoutts	strace shows me that darcs2 is stating each pristine.hashed file twice and then opening it
14:06		mornfall	dcoutts: That's likely to be lstat bug. You can do darcs revert to try syncing mtimes, if someone touched the  files.
14:06		mornfall	dcoutts: How do the pristine filenames look like? If the have no dash in filenames, then it's the wrong-repo-format bug.
14:07		dcoutts	mornfall: I just converted in fresh now
14:07		dcoutts	0000001750-26cc6a293470d41582c28354615​c1296a1d9a844e05d1c1045001f863fdb6d52
14:07		mornfall	Right. So not that one.
14:07		mornfall	dcoutts: Do you have working ghci?
14:07		dcoutts	yep
14:07		mornfall	dcoutts: Try :m + System.Posix
14:07		mornfall	dcoutts: fileSize `fmap` getSymbolicLinkStatus "something"
14:08		mornfall	dcoutts: If it gives nonsensical size, you are affected by the bug.
14:08		dcoutts	907
14:08		mornfall	Sounds good.
14:08		mornfall	Hmm.
14:08		dcoutts	for the above patch
14:08		dcoutts	I though the file size was encoded in the patch name?
14:08		mornfall	dcoutts: Yes, uncompressed size though. It's gzip'd.
14:09		dcoutts	ah, 1750 vs 907 would make sense then
14:09		mornfall	Indeed.
14:09		mornfall	Can you try reverting then?
14:09		mornfall	(And. strace -c is your friend: if you see reads dominate the statistic, you have a problem with the mtime optimisation...)
14:10		dcoutts	revert doesn't change the timings
14:10		dcoutts	just trying with strace again...
14:10		mornfall	zooko: Hi.
14:10		dcoutts	9926 lstat, 1533 open
14:11		dcoutts	hmm futex seems to be a high one
14:12		dcoutts	mornfall: so strace -c gives us % of system times per syscall right?
14:12		dcoutts	% time     seconds  usecs/call     calls    errors syscall
14:12		dcoutts	------ ----------- ----------- --------- --------- ----------------
14:12		dcoutts	90.99    0.008410           5      1601        83 futex
14:14		dcoutts	mornfall: still, the most time seems to be user time, not sys time, 1.04s vs 0.16s for user vs sys
14:14		dcoutts	I guess that must be because it's actually reading the patches after opening the files
14:15		mornfall	zooko: Do you happen to know if it's possible to make buildbot grab eg a directory of html files produced by the build and publish them along the rest?
14:16		mornfall	dcoutts: The actual timings are not interesting really -- just the number of calls.
14:16		mornfall	dcoutts: reads meaning that smart_diff opens the actual files...
14:16		mornfall	dcoutts: Which is wrong.
14:16		dcoutts	mornfall: right
14:17		mornfall	dcoutts: What's the actual runtime of darcs wh?
14:17		dcoutts	mornfall: 1s on the ghc testsuite repo
14:17		dcoutts	vs 0.2s for the v1 equivalent
14:17		mornfall	Hm.
14:17		zooko	Good morning folks!
14:17		dcoutts	that's 0.2s with darcs 1 or 2, on the v1 format repo
14:17		zooko	Reading backlog.
14:17		zooko	Just got to the part where mornfall said to ping him...
14:17		mornfall	dcoutts: That might be realistic...
14:18		dcoutts	checks strace on the v1 format
14:18		mornfall	dcoutts: the HashedIO slows down even the mtime checks substantially...
14:19		dcoutts	mornfall: yeah, strace darcs2 whatsnew on the v1 format repo is just doing a load of lstat calls, no opens
14:19		mornfall	dcoutts: I'd say a second is realistic for over 5000 files in working/pristine.
14:19		mornfall	Hmm.
14:19		mornfall	dcoutts: Do you build your own darcs?
14:19		dcoutts	mornfall: yep, gentoo
14:20		zooko	mornfall: sure, you can definitely do that with buildbot.  Either write some Python code that grabs a directory of files and transfers them somewhere, or
14:20		dcoutts	mornfall: want me to check what configure options we're using?
14:20		zooko	have the Python code invoke a shell script or a Makefile target, which does the same thing.
14:20		zooko	I'll get an example...
14:21		mornfall	zooko: I have been meaning to publish them in the waterfall, together with stdio and such.
14:21		dcoutts	mornfall: see one second is not so bad for a local disk, but when Simon does this over nfs, darcs v1 format is still really really quick with just stating, but opening makes the whole thing slow to a crawl (4 seconds real time).
14:22		mornfall	dcoutts: Can you grab the source from  -- dump that into microbench.hs, remove -Werrohttp://rafb.net/p/qSEAdI64.htmlr from GHCFLAGS in GNUmakefile and make microbench -- then run that in your repository with -m
14:22		mornfall	Aaah.
14:23		mornfall	Screwed paste, sorry.
14:23		mornfall	http://rafb.net/p/qSEAdI64.html and remove -Werror
14:23		lambdabot	Title: Nopaste - No description
14:23		dcoutts	mornfall: 'k
14:24		mornfall	Should dump a diff on mtimes+filesizes.
14:33		mornfall	*sigh*
14:35		mornfall	It might be bug to import HashedRepo, but I want to fix bugs not refactor code that should really be scrapped.
14:38		dcoutts	mornfall: hmm make micorbench doesn't seem to call ghc
14:39		mornfall	dcoutts: It doesn't? Duh?
14:39		mornfall	Make it microbench
14:39		mornfall	(Instead of micorbench -- typo.)
15:03		zooko	Gotta do some family work (financial planning using open office spreadsheets stored in darcs).
15:04		zooko	I opened a ticket on bugs.darcs.net to hold example code for building and uploading packages from buildbot.
15:37		lispy	dons_: yeah, that will be some nice synergistic foo (the xmonad + darcs hackathingy)
15:41		dcoutts	ssh hates me, I can ssh to all my ordinary hosts, but scp/sftp hangs every time for larger files
15:41		dcoutts	mornfall: sorry, I cannot upload the microbench output anywhere, unless you know of any pastebins that accept ~64k
15:42		mornfall	dcoutts: Paste just one file line maybe.
15:42		dcoutts	oh right, which sort of line(s) do you want
15:43		dcoutts	work file, cur file? both?
15:43		mornfall	dcoutts: Ah, well, the pair of them.
15:43		mornfall	dcoutts: And file, yes. Dirs are harmless.
15:44		dcoutts	work file: fp2fn "read021.hs", length: 0, mtime: 1218722467
15:44		dcoutts	cur file: fp2fn "read021.hs", length: 0, mtime: 1218722468
15:45		dcoutts	hmm, that file really is 0, let me find another one
15:45		mornfall	dcoutts: Oh. Is that over nfs?
15:45		dcoutts	local
15:45		mornfall	The one-second timestamp difference is puzzling.
15:46		mornfall	And you say darcs revert and then microbench again doesn't change anything?
15:46		sm	morning all
15:46		kowey	morning, sm! how's bugtracker bugtracking?
15:46		sm	-devel is seeing comments.. that's good
15:46		sm	http://bugs.darcs.net seems broken in safari.. not good
15:46		dcoutts	mornfall: here's another one:
15:46		dcoutts	work file: fp2fn "Setup.hs", length: 87, mtime: 1218722466
15:46		dcoutts	cur file: fp2fn "Setup.hs", length: 87, mtime: 1218722467
15:46		mornfall	(The revert should say "syncing pristine" for a while before exiting...)
15:46		mornfall	dcoutts: Yes, a one-second diff everywhere, I suppose?
15:47		mornfall	Sounds like a somewhat intricate bug.
15:47		sm	I see it working in firefox 3, I'd appreciate any other browser reports
15:47		kowey	works in Camino, but I guess that's no news
15:47		dcoutts	mornfall: so revert does as you say
15:47		dcoutts	mornfall: makes no difference to time of darcs whatsnew, lemme check the output of your tool again
15:49		dcoutts	mornfall: output of the tool is identical after doing darcs revert
15:49		mornfall	Interesting indeed.
15:49		mornfall	dcoutts: Did revert say anything about syncing pristine?
15:49		mornfall	(After saying there's nothing to revert etc.)
15:50		sm	kowey, lispy, good posts lately but to my ears they sound a bit "markety"
15:50		dcoutts	mornfall: it may have displayed something for a second, but nothing that remains
15:50		kowey	sm: thanks! (you mean to claus?)
15:50		dcoutts	mornfall: the second time, darcs revert was very quick
15:50		sm	yeah
15:50		kowey	sm: sincerity above everything ... thanks for keeping me honest
15:51		sm	it could be just me. I know that's part of the goal after all
15:51		mornfall	Hmh.
15:51		mornfall	I'm puzzled.
15:53		sm	kowey, it sounds like mark stosberg could help with access.. is he ever on irc ?
15:53		kowey	sm: we should go through droundy first
15:54		sm	ok, I mailed him directly yesterday. So I'll do what I can
15:54		sm	have to do client work today anyway. But this safari breakage really needs fixing
15:54		sm	stares at traceback
15:54		kowey	are you sure it's browser related, and not just I-have-the-cookie-related?
15:54		sm	not at all, it could be just that
15:55		kowey	i would test, but I kinda want to continue looking at the bugtracker :-)
15:56		sm	thanks, no problem, I'm on it
15:56		sm	nice work on bugs
16:01		batmanLabError	http://bugs.darcs.net is not working for me as well
16:01		batmanLabError	iceweasel 3.0.1
16:04		sm	opens http://bugs.darcs.net/issue998
16:07		lispy	sm: oh is roundup back?
16:07		sm	only if you're logged in :(
16:08		lispy	sm: Can you estimate how long it will take to get it working again?
16:09		sm	not yet
16:09		lispy	sm: good luck!
16:10		dcoutts	lispy: thanks for the cabal feedback, I've sent a (hopefully) detailed reply
16:10		lispy	dcoutts: oh cool
16:11		lispy	dcoutts: and hopefully a workaround?
16:11		dcoutts	yep
16:11		dcoutts	ghc-pkg --user unregister bytestring-0.9.1.0 && cabal install bytestring --enable-library-profiling
16:12		lispy	dcoutts: did you include me as a direct TO ?
16:12		lispy	dcoutts: if so the mail didn't arrive yet, I'll check my spam bin
16:12		dcoutts	lispy: grr, it's still sending. Oh for my ADSL is made of poo!
16:16		dcoutts	lispy: is that working?
16:16		lispy	dcoutts: yes, although I always seem to get this:
16:16		lispy	Writing new package config file... done.
16:16		lispy	cabal: /Users/dagit/.cabal/logs/bytestring-0.9.1.0.log: openFile: does not exist (No such file or directory)
16:16		mornfall	dcoutts: I'll try to look into that 1-sec difference bug later, maybe tomorrow, as I have other stuff to do today. : - (
16:16		lispy	It seems ignorable though
16:16		mornfall	See you all around. Gotta run home.
16:16		lispy	dcoutts: I've given the right perms, I think, to ~/.cabal/logs but it fails to create that file
16:18		dcoutts	lispy: sounds like you're using the dev version, try the released version it doesn't have half-completed features :-)
16:18		dcoutts	or if you're brave, update to the very latest darcs version of cabal-install
16:25		lispy	hmm
16:25		lispy	I thouht I installed from hackagedb
16:27		jnaimard	is it me, or bugs.darcs.net is half down ? I can't log in.
16:27		lispy	jnaimard: we have someone working on it right now
16:27		lispy	jnaimard: you're correct though
16:28		malcolmw	does anyone know if there is a darcs2 binary available on darcs.haskell.org?
16:28		jnaimard	ok, it would be nice to put it in the channel topic
16:28		Igloo	malcolmw: I'll instal it if you like
16:29		malcolmw	Igloo: Ta.  There is a ghc-darcs2 repo, so there must be a binary somewhere...
16:29		sm	phew.. tracker fixed
16:30		lispy	jnaimard: we'll do that next time, good idea!
16:30		Igloo	Could be in SM's ~ or something
16:30		Igloo	I'll install it anyway, as then community, www and monk will all look the same
16:30		malcolmw	good
16:38		lispy	dcoutts: FYI, it still didn't arrive...
16:38		dcoutts	lispy: I finally managed to cut it down so that I could actually send it
16:38		dcoutts	there's something very wrong with my adsl
16:38		dcoutts	lispy: so it should arrive shortly
16:39		malcolmw	dcoutts: I've seen it on the cabal-devel list
16:39		dcoutts	malcolmw: good
16:49		sm	oops, my mistake kowey - david took care of that access already. forgot my private mail
16:51		Igloo	malcolmw: igloomonk:~$ darcs2 --version
16:51		Igloo	2.0.2 (release)
16:53		malcolmw	Igloo: ta
16:53		Igloo	Oh, hmm, putting it on www.haskell is going to be non-trivial, isn't it
16:54		malcolmw	Igloo: are there any repos on www. in any case?
16:54		Igloo	About 20
16:54		Igloo	Mostly websites, I think
16:55		gwern	'
16:55		Igloo	I think I'll just ignore it
16:55		gwern	'Patch's been rejected because of violating some sort of unwritten rule about
16:55		gwern	module imports. I have no intention to refactor the surrounding code to appease
16:55		gwern	these rules just to get the fix through -> unassigning, kthxbai
16:56		kowey	we can fix it!
16:57		gwern	WE HAVE THE TECHNOLOGY
16:58		kowey	droundy was just a bit direct... no big deal, no real offense taken, right?
16:58		kowey	just some annoyance about having to amend-record (sigh) yet again...
16:58		gwern	you don't usually use 'kthxbai' when you are unoffended
16:58		gwern	that's generally a least a little sarcastic in this kind of context
16:59		kowey	nod
17:00		lispy	Ah, I had frustrations when iwas new to darcs too
17:00		lispy	gwern: who wrote that?
17:00		lispy	how can I contact the author
17:01		gwern	mornfall, 'twould seem
17:01		gwern	just pulled it off the ml
17:04		sm	who cares.. he'll be back, or someone will fix and resend
17:04		sm	PATCH ON
17:05		kowey	it's inevitable too... also means he's starting to be part of the family
17:06		kowey	hugs mornfall
17:06		kowey	and retreats to a safe manly distance
17:07		gwern	I just don't get how people have whatsnew problems
17:07		gwern	I mean, 'darcs whatsnew' on ghc with darcs-2 takes like .01 seconds here
17:07		gwern	with no changes
17:08		kowey	are you following issue973?
17:08		gwern	dunno
17:08		gwern	am I?
17:08		mornfall	gwern: oldfashioned repo, for sure?
17:09		gwern	yeah, I didn't convert my ghc copy
17:09		mornfall	gwern: That's it, then. The problem is with hashed repos.
17:09		lispy	mornfall: good morning
17:09		kowey	http://bugs.darcs.net/issue973
17:09		lambdabot	Title: Issue 973: darcs whatsnew significantly slower on hashed format - Darcs bug trac ...
17:10		gwern	hm
17:10		gwern	I'd better convert and see how that changes timings
17:10		kowey	ooh, lambdabot returns
17:10		mornfall	lispy: Hi.
17:10		lispy	kowey: yeah, I offered server space to Cale...turns out c.h.o didn't like that lambdabot is leaky
17:10		kowey	thanks! lispy++
17:11		lispy	mornfall: I understand that our patch acceptance policy is frustrating.  Hang in there, kowey and I are trying to come up with a submission policy that everyone likes and gives us a looser commite model so we can get things into the code base and then refine them
17:14		mornfall	lispy: I think this one was a little overboard. What I read there is, "we have no intention to fix bugs, not even those marked urgent on the tracker (or fixing tracker priorities), and we consider it a deluxe service that we integrate fixes from our users...". More or less.
17:14		lispy	mornfall: heh
17:14		lispy	mornfall: that would be a bad policy
17:15		lispy	hopes we don't adopt that any time soon
17:15		kowey	shelves away the plans and mutters
17:15		lispy	mornfall: I do see what david meant though
17:16		gwern	deluxe service? oh noes, I can barely afford to keep up as is
17:16		lispy	mornfall: IIUC, he intends for anything below src/Darcs/Repository/* to be "internal" but Haskell doesn't have a notion of private
17:16		lispy	mornfall: I think this speaks to the libdarcs api
17:29		lispy	but I'm not sure, maybe this is still possible even if you factor it out
17:29		mornfall	lispy: Enforcing policies on newcomers that the code already breaks is a rather silly thing to do.
17:29		mornfall	lispy: I have enough of that at work, thank you.
17:29		lispy	nods
17:29		mornfall	If I had to make all bad code in darcs good when I touch it, I would never do anything else.
17:29		mornfall	Yes, there is good code and there is bad code. Fixing bad code to appease rules is wasted time.
17:29		gwern	lispy: we could enforce privacy if we had done libdarcs...
17:29		gwern	(if a package doesn't export a module....)
17:29		mornfall	(If it remains bad or even worse afterwards, anyway.)
17:29		lispy	mornfall: What's the plan?
17:29		mornfall	No idea.
17:29		lispy	let's work on one then
17:29		lispy	I'm going to stare at your patch for a bit to see if I can understand the situation better
17:29		mornfall	(You can rest assured that I'd have fixed it the right way if the library code in question wasn't one big YAGNI violation and a complete mess at that.)
17:29		gwern	http://producingoss.com/ <-- looks like an interesting read
17:29		lambdabot	Title: Producing Open Source Software
17:29		lispy	mornfall: YAGNI?
17:29		lispy	goes to look it up
17:29		kowey	haha, I was on the wikipedia page as you said that
17:29		gwern	you aren't going to need it
17:29		gwern	ie. don't write any code you don't need right this second for something
17:29		mornfall	lispy: That basically means that there's this all-general-acting function called writePatchSet that has exactly two users: the function a few lines below it, patchSetToRepository, and Repair.lhs.
17:29		lispy	mornfall: re: perfect code only; We agree.  kowey and I just identified that this morning.  We need to get the unstable branch back perhaps.
17:29		lispy	mornfall: hmm
17:29		mornfall	lispy: Then there is that patchSetToRepository which is only used by Get.lhs.
17:29		gwern	dang it, I still don't see. darcs whatsnew on a darcs-2 repo of GHC is still very fast
17:29		gwern	gwerncraft:16589~/bin/ghc/ghc_0>time =darcs whatsnew                                                                 [ 1:24PM]
17:29		gwern	diffing dir...
17:29		gwern	binary ./WindowsInstaller/Glasgow\​32\Haskell\32\Compiler.ism
17:29		gwern	rmfile ./WindowsInstaller/Glasgow\​32\Haskell\32\Compiler.ism
17:29		gwern	=darcs whatsnew  0.09s user 0.02s system 73% cpu 0.153 total
17:29		mornfall	lispy: Which means neither of them are very general-purpose and are more or less a trap into the library code to appease the non-import rules.
17:29		kowey	gwern: did you compile with 6.8.2 or 6.8.3?
17:29		gwern	.2
17:29		kowey	hmm!
17:29		mornfall	lispy: Out of the 2 users, one gets it wrong, trying to writePatchSet to a repository that doesn't exist.
17:29		lispy	gwern: maybe we need to be testing on windows?
17:29		lispy	mornfall: oh
17:29		Igloo	gwern: But you are on a differen FS to the people reporting problems, I assume. The questino is, does the code do something silly, like opening all the files?
17:29		mornfall	lispy: Well, doesn't exist. It exists halfway-through and no code there tries to make it complete.
17:29		kowey	mornfall: does this mean that darcs init cannot be implemented with writePatchSet?
17:29		lispy	mornfall: so your patch is really short
17:29		mornfall	lispy: So I added the code to "finish" creating the repository. Yes, it's not an extremely good fix. But doing this *right* would be quite invasive.
17:29		gwern	Igloo: I am on ubuntu hardy, ext3, intel 64-bit
17:29		lispy	gwern: ah, it's a 32bit problem also
17:29		lispy	gwern: I don't recall where I learned that...talking to SimonM?
17:29		gwern	oh
17:29		gwern	bleh
17:29		Igloo	Whereas the problem was reported on Windows, NFS or SMB, 32bit, I think
17:29		gwern	secretly wishes everyone would just move to 64bit already
17:29		gwern	better add my experience to the bug report then
17:29		SamB	gwern: great! buy me a new computer!
17:29		SamB	or two
17:29		gwern	SamB: go buy your own! I don't have any money
17:29		SamB	neither do I
17:29		kowey	gwern - the bug that mornfall found: http://hackage.haskell.org/trac/ghc/ticket/2093
17:29		lambdabot	Title: #2093 (getSymbolicLinkStatus (and possibly other functions) broken on systems wi ...
17:29		mornfall	lispy: It's short on purpose. Minimal changes lead to minimal breakage. That's something you could hardly deny. And it's easier to validate.
17:29		lispy	mornfall: ah, I wasn't criticising, just trying to follow what it does
17:29		mornfall	lispy: Ah, there's another software engineering rule. Don't mix refactorings with functional changes.
17:30		lispy	mornfall: so you've added a "wher blah $ do HashedRepo.foo; HashedRepo.bar" to Get, and also added HashedRepo.bar
17:30		gwern	SamB: I suggest doing tricks with 10 chicks
17:30		gwern	or maybe 1 really fat chick
17:30		gwern	what's the command to get info about a repo?
17:30		kowey	show repo
17:31		lispy	mornfall: sure.  BTW, droundy is hyper intelligent, but he's not trained as a software enginner, so we can't assume he'll automaticaly grok the SE practices
17:31		mornfall	lispy: when "we are in hashed repo", "create tentative_pristine in there" (which seems to be required for the rest of stuff to work, and "add the repo we are getting from to the sources of the new repo, otherwise we die on inability to find patches".
17:31		mornfall	+/- paren balance
17:33		mornfall	is not trained either way... and generally I don't like what "software engineering" is about. But there are some good ideas there. Worth sticking to.
17:33		lispy	I may have to apply this bundle to understand it, the diff is just not helping me
17:33		mornfall	(Just when we are waving all around with rules...)
17:34		mornfall	lispy: Read the problem analysis on the tracker, that could help a little.
17:34		mornfall	At least with the revert_tentative_changes.
17:34		lispy	mornfall: yes, and it seems as thought we give the law of demeter a thorough thrashing in places and fixing it might lead to some better code too
17:34		mornfall	The add_source or whatever it's called is the other part of the fix.
17:35		gwern	I am amused - darcs query repo takes longer than whatsnew
17:36		kowey	it's interesting, people seem to like the query command better than the show one
17:36		gwern	show?
17:36		kowey	query was renamed to show
17:37		kowey	e.g. darcs show contents somefile
17:39		mornfall	(Talking of which, darcs show is becoming a junkyard of incohesive subcommands, is it not?)
17:39		mornfall	Or, maybe not, looking at help.
17:39		mornfall	It's not that bad as I imagined.
17:40		lispy	mornfall: I see a different way to fix this problem that droundy might prfer
17:41		mornfall	Great, go ahead...
17:41		lispy	er wait
17:42		lispy	mornfall: so what was the reason to not write that pristine file in the function writePatchSet?
17:42		lispy	mornfall: just that we didn't want to couple those actions?
17:43		mornfall	lispy: Because writePatchSet doesn't have the right inputs?
17:43		mornfall	lispy: Ah, for pristine.
17:43		mornfall	lispy: It'd need to be renamed then.
17:43		mornfall	writePatchSet doesn't hint me that it would revert existing tentative changes right away.
17:43		lispy	okay
17:43		lispy	So what about doing it one level up?
17:44		lispy	in patchSetToRepository?
17:44		mornfall	Same problem.
17:44		mornfall	Ah. Maybe not.
17:44		lispy	well, writePatchSet returns the repo
17:44		mornfall	It's just that it takes the wrong parameters.
17:45		mornfall	Or maybe. Dunno.
17:45		lispy	We want to write it into the repo returned by writePatchSet, correct?
17:45		mornfall	I had a reason not to do it.
17:45		lispy	nods
17:45		lispy	but what was it...
17:45		mornfall	lispy: No, you  have to do it *before* writePatchSet.
17:45		mornfall	That                HashedRepo.finalize_tentative_changes repo opts
17:45		mornfall	will break if you haven't called that revert by that time.
17:46		mornfall	And indeed, you don't have the repo in patchSetToRepository before writePatchSet.
17:47		mornfall	See, it's a total waste of time. There is a fix that is perfectly reasonable given the circumstances. There's no use in finding a way to escape that rule. You either fix it correctly or you work around it. If doing the latter, it doesn't make sense to spend hours trying to make the workaround look more like a correct fix (it's not going to be any more correct anyway... it's just disguise).
17:47		lispy	hmm
17:48		mornfall	(And correct fix probably involves scrapping the functionality that allows an incomplete repo to exist outside of Repository.lhs, anyway.)
17:48		mornfall	(Or maybe fixing the way tentative_pristine is expected to exist when it's allowed to not exist outside Repository. Whatever pleases you.)
17:48		lispy	that does seem bad, intuitively, partial repos
17:49		lispy	mornfall: so, if finalize_tentative_changes expects that file, how can writePatchSet ever do it's job correctly?
17:49		mornfall	lispy: Because tentative_pristine almost always seems to exist.
17:50		mornfall	lispy: It doesn't exist if no tentative action ever took place, it seems.
17:50		mornfall	Don't forget you have another problem with that add_source.
17:51		lispy	see what you mean about the comment around patchSetToRepository doesn't match the code...no call to error
17:51		mornfall	Right.
17:52		mornfall	I'd suggest not throwing any more time on trying to invent a fix that'd get through. It might make sense to refactor the code instead, then a fix could be possible without workarounds.
17:52		lispy	mornfall: another idea, what if finalize_tentative_changes did nothing when there is no tenative pristine?
17:53		mornfall	lispy: I had that too. I have no idea how far-reaching that is.
17:53		mornfall	lispy: So I don't change it, since it seems to change behaviour of farily consistency-critical function that's invoked a lot.
17:53		mornfall	Might be over-cautious.
17:54		lispy	might be worth running by David
17:54		mornfall	(If I was the maintainer, I wouldn't be able to tell right away, I guess.)
17:54		lispy	I'm just greping for it at the moment
17:55		mornfall	But you are running into a risk of regressions. We have no idea about test coverage. It seems the code is not exactly over-tested.
17:55		mornfall	(Well, we might have, but I don't.)
17:55		lispy	mornfall: so writePatchSet claims to not do anything to pristine cache, but in fact, finalize_tentative_changes does write to the pristine cache
17:55		mornfall	lispy: True, that.
17:56		lispy	mornfall: on ghc-6.8.3 you can run the test suite HPC enabled...there were patches for it, I think they were accepted.  We could examine coverage
17:56		mornfall	lispy: But it's a noop in the case that no tentative changes to pristine happened before calling it.
17:56		lispy	exactyl, is that going to be a bug still?
17:56		mornfall	lispy: I don't have .3... Well I do, but I haven't installed it, since it'll likely break all libraries?
17:56		lispy	get cabal-install and just reinstall them
17:56		kowey	in fact, if you've already got cabal-install
17:57		mornfall	lispy: It's an API bug. Harmless now, since the function is basically unused.
17:57		kowey	and you upgrade, your cabal-install will still work
17:57		lispy	cabal-install knows how to install them for your specific ghc, I have 6.6.1, 6.8.2 and 6.8.3 no problems
17:57		dcoutts	fixes one of lispy's cabal-install complaints...
17:57		mornfall	lispy: I am running from debian packages of most of the libs, though.
17:57		lispy	dcoutts: \o/
17:57		mornfall	And I have built ghc 6.8.3 as a .deb, too.
17:58		lispy	mornfall: in that case, you can make it work, but I'm not as familiar with the details
17:58		mornfall	It might be time to roll a full ghc install into /usr/local and get rid of the Debian one.
17:58		mornfall	But, I've spent much more time on all this today already than I had.
17:58		lispy	k
17:58		lispy	mornfall: thanks!
17:58		dcoutts	lispy: it now does:
17:58		dcoutts	cabal install bytestring
17:58		dcoutts	Resolving dependencies...
17:58		dcoutts	No packages to be installed.All the requested packages are already installed.
17:58		mornfall	Seems I'll have to postpone that release I have pending. Pchm.
17:58		lispy	dcoutts: kwel
17:59		lispy	er kewl
17:59		dcoutts	(erm, but with a space between the sentences)
17:59		mornfall	(Oh. And see darcs-users@.)
18:00		lispy	mornfall: okay
18:01		lispy	needs to break for lunch
18:01		lispy	mornfall: let's not let this fall away
18:01		lispy	mornfall: but I need to stop for now too
18:04		mornfall	Let me think for a bit.
18:04		mornfall	kowey: Any ideas?
18:05		kowey	mornfall: that went by too fast... can I have some time to look at the analysis and the discussion?
18:05		mornfall	kowey: No, no not about that, sorry.
18:06		kowey	oh sorry
18:06		mornfall	kowey: Just darcs-users@.
18:06		mornfall	I'm wondering if I can keep a public branch with fixes.
18:06		mornfall	It's likely to diverge fast.
18:07		mornfall	Since that lures me into dropping autoconf and stripping the legacy cruft.
18:07		kowey	and just supporting one ghc, for example
18:07		mornfall	I already went halfway through with that just as an experiment. Getting a cabal-built version with a very minimal configure.
18:08		kowey	well what I was hoping we could suggest with David
18:08		mornfall	kowey: Possibly. I have no problem with having it build with other versions. But working around issues with ancient GHC is not among my hobbies.
18:08		kowey	is to re-instate unstable... but as a really really unstable branch
18:09		kowey	meaning that it tends to err more on acceptance than quality
18:10		kowey	either (a) david pulls from unstable regularly or (b) me and lispy send nice bundles from time to time
18:10		mornfall	kowey: I think we should go for pull model.
18:10		mornfall	kowey: And feature branches.
18:11		kowey	feature branches sounds interesting... possibility of just experimenting with stuff in a blessed fashion
18:11		kowey	it also sounds like it might be confusing... hmm
18:12		mornfall	Well, I'm willing to work on the code, *if* I get enough freedom to actually work on it. That probably means having my own branch and merging with whatever other branches there are.
18:12		mornfall	The micromanagement of getting each patch through review is killing me, even before I actually started to *do* anything.
18:12		kowey	when juliusz was with us, he used to do that (on his own server)
18:12		kowey	for darcs-git
18:13		mornfall	Okey, makes sense.
18:13		kowey	and then he'd send a big bundle
18:13		kowey	that said, the svn folks have said to avoid "powerplants"
18:14		kowey	as a question of project maintainance - big bundles of stuff are inherently harder to review than incremental changes
18:14		mornfall	So first thing I do is branch off for thorough cabalisation, cutting as much rotten flesh as I can without breaking the testsuite.
18:14		mornfall	We can worry about merging later.
18:14		kowey	nods
18:15		kowey	we hope to be talking to david Real Soon Now
18:15		mornfall	Oh. And now we have a veto on darcs-devel nosy on all.
18:15		mornfall	Marvelous.
18:17		kowey	R
18:17		kowey	+
18:17		kowey	err
18:17		kowey	?
18:17		mornfall	kowey: darcs-users@, Simon's mail.
18:18		dons	for what its worth, i think we considered david a power plant dev when working on xmonad.
18:18		dons	big bundles that were essentially unmaintainable. it made things difficult.
18:18		kowey	hmm... physicist code
18:19		kowey	actually has no idea how physicists code in general
18:19		kowey	although they seem fond of Fortran
18:19		dons	that and the exponential algorithms for laying out tiles. ah well
18:19		Igloo	In general, physicists code badly in antique languages
18:19		Igloo	Then again, so do most other classes of people, in general  :-)
18:21		Igloo	(well, not so antique for most classes)
18:28		dons	"I gave you a simple way to
18:28		dons	tell when this is happening, and now you're going to leave in a huff?
18:28		dons	If you don't care about the quality and maintainability of darcs'
18:28		dons	code, then we're better off without you. "
18:28		dons	isn't good! :(
18:28		dons	NOT GOOD.
18:29		mornfall	Well, that was directed at me. Although I worry more about other people reading that.
18:29		sm	but mornfall, all, let's not assume because a patch was made it should be applied right away. This discussion and refining of patches is exactly what we need to grow the pool of skilled darcs devs as well as keep code quality high
18:30		sm	linux doesn't accept patches that aren't right, and ignorance of undocumented rules doesn't matter
18:31		mornfall	sm: Assuming there is only "right" and "wrong" and that the reviewers never mistake.
18:31		sm	of course, but that's the great thing about a public discussion
18:31		mornfall	sm: Also, I don't contribute to Linux for a reason. The code is atrocious and the community about same.
18:33		sm	I guess our goal is to make this community work. Like xmonad's, or haskell in general
18:34		mornfall	For some values of "our", I'm sure.
18:34		sm	I mean, that's the way to make a darcs that rocks
18:35		kowey	droundy is trying to avoid the risk of breaking darcs... but I guess that doesn't mesh well with having only a single repo
18:36		kowey	because conservatism about accepting patches isn't always encouraging for the new darcs hacker
18:36		mornfall	kowey: That doesn't relate to breaking darcs, though.
18:36		sm	beyond a certain size, a great app takes a great high-functioning tolerant community. I think.
18:36		mornfall	This is about right and wrong.
18:37		sm	what stops people making high-churn public repos ? That would be great
18:37		sm	david is maintaining darcs-stable
18:37		mornfall	sm: The prospect of ending up being a fork?
18:37		sm	a branch. Darcs is good at that :)
18:37		mornfall	That's a great motivation to run your own branch, to get a "no thanks, we won't be merging with you" response.
18:44		sm	you didn't get one of those yet
18:44		sm	it just seems to me some hidden practices have usefully been flushed out, this patch needs a bit of refinement and then it will go in no problem - ie not a huge deal, DR's discouragingly-terse review style aside
18:45		mornfall	sm: Have you seen the discussion with lispy?
18:45		sm	I think I've seen all public discussion ? not last night's chatlog
18:45		mornfall	sm: I mean today, a hour or so back.
18:46		sm	I did skim all that, I don't know what specifically you mean
18:46		mornfall	sm: I mean that "all that" was a failed attempt at fixing the patch to meet inclusion criteria.
18:46		mornfall	So sorry, but unless lispy throws a few more hours at making the patch *look* better (without actually getting better), it won't get in.
18:47		sm	so you're making the point that fixing it is not easy ?
18:47		mornfall	If he has a bunch more time, he might *make* it better, too.
18:47		mornfall	sm: I already explained all this to lispy. I have better things on my hands to do that explain over and over all the problems with the code surrounding that patch.
18:47		mornfall	than*
18:47		sm	I know nothing about the technical issue. Sorry your excellent work is uphill :/
18:48		sm	<- back to work
19:00		lispy	mornfall: I'm sorry you're having a bad first experience collaborating with us.  You're right that the process needs to change.
19:13		rtyler	lelit: ping
19:14		SamB	it sounds like the main issues are political?
19:28		gwern	hm. does anyone know whether the debian lenny darcs-2 incorporated my binary optimization?
19:29		lispy	gwern: no idea, I don't even know who uploads those packages anymore
19:29		kowey	SamB: the main issue is that droundy is doing what he can with limited time (in my opinion)
19:30		kowey	and the sooner we find a solution to relieving the time pressure, the better
19:51		mornfall	--> darcs-users@ ...
19:51		mornfall	And sm has tracked down the bounce issue it seems. Great.
19:51		sm	nice post! just read it
19:51		kowey	mornfall++
19:51		sm	oops, sent my test before fixing the problem. maybe kowey or lispy can edit the mailman subscribers ?
19:51		kowey	lispy?
19:53		lispy	kowey: yo
19:54		kowey	didja see what sm did with the bugtracker?
19:54		lispy	no, what's up?
19:54		kowey	just needs a mailman tweak (see list or bug report)
19:54		kowey	I think he may have found the root of the double-bug trouble
19:54		sm	http://bugs.darcs.net/issue636
19:54		lambdabot	Title: Issue 636: Replying to bugsdarcs.net generates a bounce / new bug reports arriv ...
19:54		lispy	k, just a sec, knee deep in email to duncan
19:54		dcoutts	heh heh
19:55		dcoutts	lispy: I've also implemented the --reinstall option
19:55		lispy	dcoutts: col
19:55		sm	oh no.. there goes halg the email..
19:55		sm	half
19:55		lispy	dcoutts: I think I figured out my misunderstanding
19:55		dcoutts	cabal install --reinstall -p bytestring
19:56		lispy	dcoutts: the idea that the --enable-foo toggles are orthogonal didn't fully hit me
19:58		lispy	dcoutts: and I'm not 100% sure you guys should have to fix that
19:58		dcoutts	aye, being able to enable/disable them all independently is pretty useful
19:58		dcoutts	eg for debian packages where they split profiling and non-profiling
19:58		lispy	ah right, debian should fix that :)
19:58		dcoutts	so they build twice, once with enable one and disable the other
19:58		dcoutts	and then again, the other way around
19:58		mornfall	dcoutts: When we have you here. Do you have an idea how it is with ./configure'd cabal packages and windows? I guess that doesn't work "out of the box"..?
19:58		gwern	what irks me is the lack of profiled ghc
19:58		gwern	so annoying!
19:58		sm	perhaps should have remembered to obscure the email addresses in issue comment :(
19:58		dcoutts	mornfall: right, if you use configure then users need sh and the rest of the junk
19:59		dcoutts	mornfall: it's possible to implement most of this stuff in Haskell in the Setup.hs, though it's perhaps less convenient
19:59		mornfall	So I better do without configure. Makes sense.
20:00		dcoutts	mornfall: if you want to claim it's for the benefit of windows users, yes.
20:01		mornfall	Well, I need to get endianity of the system, or better get rid of SHA1.lhs. But I can't find anything to replace it, sadly. (At least not on Hackage. The Crypto lib is all Stringy and not ByteStringy, it seems.)
20:02		dcoutts	mm, endianness
20:03		dcoutts	and you currently use a configure test
20:03		mornfall	Yes...
20:03		dcoutts	in principle of course you can just implement that test in Haskell
20:03		dcoutts	even if it's calling out to external progs etc
20:03		mornfall	AC_C_BIGENDIAN
20:03		lispy	kowey: so what action is required of me?
20:04		dcoutts	right, where as autoconf already provides the feature test built-in
20:04		sm	lispy: see that link for exactly instructions
20:04		kowey	re the bugtracker? remove darcs-bugsdarcsvm.osuosl.org from the list of subscribers to -devel
20:04		dcoutts	mornfall: btw, while you're looking at this, keep a list and file it as a feature request in the hackage bug tracker. So we know what kinds of feature tests are common and we could help with.
20:05		lispy	sm: unsub this? darcs-bugsdarcsvm.osuosl.org
20:05		lispy	Hmm... in -devel?
20:05		lispy	I see it in bugs
20:07		lispy	sm: okay, I removed darcs-bugsdarcsvm.osuosl.org from the BUGS list
20:07		lispy	sm: I didn't see it on the darcs-devel list
20:07		lispy	sm: I hope that works
20:08		sm	whats the bugs list ?
20:08		sm	another mail list ?
20:08		kowey	it's the list that droundy uses to moderate new bugtracker submissions, I think
20:08		kowey	spam-prevention device
20:08		kowey	we used to have a problem of people submitting spammy bugs
20:08		sm	interesting, I didn't know about that one
20:09		sm	thank you, perhaps that will take care of it...
20:10		mornfall	Wow, the autoconf endianness checks are scary.
20:13		sm	looks good :)
20:14		kowey	hooray! no more Failed tracker submission
20:14		dcoutts	mornfall: I've got some c code to do the test in c2hs
20:14		dcoutts	mornfall: if you're looking for something simpler
20:15		mornfall	dcoutts: It might be haskell-doable, even.
20:16		dcoutts	mornfall: oh, actually, my mistake, we've got code for bitfield directions, padding, if int is signed and alignment, but not endian :-)
20:16		mornfall	But it would require some pointer fiddling.
20:17		mornfall	Well, one creative way to do that check is to define big_endian, compute some sha1 and check if it is right for the given input. If yes, fine, if no, go little_endian and try again.
20:17		mornfall	; - )
20:26		lispy	mornfall: good reply to david on the list
20:52		mornfall	Hm. All SHA1s are really slow compared to the darcs one. Grrmh.
20:56		sm	golden opportunity to file a special bug: the next will be #1000
20:56		sm	do we have anything worthy of that honour ?
20:56		mornfall	sm: Yeah. "World is a sucky place." -> wont-fix
21:00		sm	oops, sent one
21:00		sm	too scattered.. lunchtime, later all
21:11		kowey	good night, everybody! peaceful hacking!
21:21		mornfall	Now I just need a bigendian box.
21:22		mornfall	With ghc on it...
21:26		mornfall	Found something. Hooray.
21:27		mornfall	Hell YEAH.
21:30		dcoutts	mornfall: I've got a sparc box if it helps you at all
21:30		lambdabot	dcoutts: You have 1 new message. '/msg lambdabot @messages' to read it.
21:30		mornfall	dcoutts: I have already found a working mips.
21:30		mornfall	dcoutts: But thanks. :)
21:30		dcoutts	oo, mips
21:30		dcoutts	and it runs ghc? gosh
21:30		mornfall	dcoutts: Debian.
21:31		dcoutts	right
21:31		mornfall	Its L2 cache is smoked tho, so it's pretty slow.
21:45		mornfall	dcoutts: Now the question is, how do you do checks in Setup.hs? Is there some docs I have missed?
21:45		dcoutts	mornfall: it's not well documented, but...
21:45		lelit	rtyler: here I'm
21:46		dcoutts	mornfall: http://haskell.org/cabal/relea[…]ution-Simple.html
21:46		lambdabot	Title: Distribution.Simple, http://tinyurl.com/5db2sr
21:46		dcoutts	mornfall: basically you get to play with these UserHooks
21:46		dcoutts	mornfall: for each action there's a pre, a do it, and a post action
21:46		dcoutts	mornfall: like preConf, confHook, postConf
21:47		mornfall	dcoutts: Hmm. The output seems to be LocalBuildInfo though, and I don't see cppflags or flags or anything there?
21:47		mornfall	Ideally, I'd just flip some cabal flags and have the conditionals in .cabal to take care of stuff?
21:48		dcoutts	mornfall: at the moment there is no way in Setup.hs to set flags
21:48		mornfall	Oh.
21:48		mornfall	I can set cppOptions though?
21:48		rtyler	lelit: howdy, are you the Tailor guy :D
21:48		dcoutts	mornfall: but you get to add additional BuildInfo in the Setup.hs
21:48		lelit	yes
21:48		dcoutts	mornfall: certainly
21:48		mornfall	That should do it.
21:49		rtyler	lelit: mind help me figure out why Tailor is croaking on SVN renames? :/
21:49		dcoutts	mornfall: if you can do it before the rest of the configuration then that's easier and you can use preConf
21:49		mornfall	dcoutts: I suppose I can't have cabal generate a bunch of foo = True (foo = False) declarations under autogen and should use CPP for that?
21:49		rtyler	lelit: http://monoport.com/36806
21:49		lambdabot	Title: mono pastebin
21:50		dcoutts	mornfall: you could do if you did something in the preBuild hook
21:50		mornfall	dcoutts: preConf should do -- it's using Foreign.*, is that an issue for Setup?
21:50		dcoutts	mornfall: you can import anything from the base package
21:50		mornfall	It seems to work fine with runghc...
21:50		dcoutts	or from Cabal or it's deps
21:50		mornfall	Should be in the small base even. Okey, thanks.
21:51		dcoutts	mornfall: if you can avoid importing things from Distribution.Simple.* then you win greater api stability
21:51		lelit	rtyler: I see. Is that a public repo?
21:51		dcoutts	mornfall: Distribution.* is relatively stable, Distribution.Simple.* is not at all stable.
21:51		rtyler	lelit: unfortunately not, our internal company repository
21:52		rtyler	lelit: every time there is a rename in one of the revisions, the subsequent git add call fails
21:52		lelit	oh
21:52		lelit	can you paste me at least the output of svn log -v -r xxx?
21:53		mornfall	dcoutts: Any way to get at preConf without Simple?
21:53		rtyler	lelit: sure thing
21:54		mornfall	Well, I currently could care less about API issues.
21:55		dcoutts	mornfall: yes, it's exported from Distribution.Simple
21:55		rtyler	lelit: http://monoport.com/36858
21:55		lambdabot	Title: mono pastebin
21:55		dcoutts	mornfall: Distribution.Simple itself is stable, it's the modules inside/underneath
21:55		mornfall	dcoutts: Ah. Great, ta.
21:56		lelit	uhm, nothing strange. The git backend needs some love
21:57		rtyler	gives the git backend a hug
21:59		_keturn	is darcsum no longer functional?  I just did darcsum-whatsnew and it didn't do anything user-visible.
21:59		lelit	I use that now and then, and it works for me...
22:04		rtyler	lelit: any idea what that is/what could be changed easily :)
22:04		rtyler	Tailor is part of a huge master plan to switch off of SVN
22:04		lelit	I can imagine :)
22:06		_keturn	it does invoke darcs, but I don't know what it does with the output.  :-/
22:06		lelit	do you have latest darcsum? and darcs2?
22:07		gwern	darcsum is, AFAIK, at http://chneukirchen.org/repos/darcsum
22:07		lambdabot	Title: Index of /repos/darcsum/
22:11		mornfall	Eh.
22:14		_keturn	that is the repo I'm pulling from, yes
22:14		_keturn	and yes, with darcs2
22:15		mornfall	dcoutts: Pretty please, could you hint me as to what is preConf supposed to return? It gets called just fine, and I return (Just $ emptyBuildInfo { cppOptions = ... }, [] ) -- but probably that's not the right thing...
22:15		dcoutts	mornfall: that's exactly the right thing.
22:15		mornfall	Hmh.
22:15		dcoutts	mornfall: the build info gets merged with that from the .cabal file
22:15		mornfall	dcoutts: Maybe I check in the wrong place then?
22:15		mornfall	Where would the data end up?
22:16		dcoutts	in the call to ghc
22:16		dcoutts	it's also stashed in the dist/setup-config
22:16		mornfall	dcoutts: That's what I'd expect, but it doesn't seem to...
22:16		dcoutts	mornfall: you're using build-type: Custom now right?
22:17		mornfall	Indeed.
22:17		dcoutts	tries a quick experiment
22:18		mornfall	Let's try building.
22:19		mornfall	src/SHA1.lhs:41:0:  error: #error "No endianness?"
22:19		mornfall	It doesn't seem to get through. : - (
22:19		mornfall	from cabal configure, I get: endian: -DLITTLE_ENDIAN
22:20		mornfall	(that's a putStrLn in my preConf hook...)
22:20		mornfall	http://hpaste.org/9647 is in SHA1.hs
22:24		Jaak	#elifdef maybe?
22:26		Jaak	or #elif defined ...
22:29		mornfall	dcoutts: (Hardcoding the -D into cpp-options in .cabal works, though.)
22:31		mornfall	dcoutts: Moving to preBuild doesn't help, eitehr.
22:31		mornfall	either*
22:31		mornfall	(Get's called, but no effect.)
22:34		dcoutts	mornfall: mm, yes, seems it doesn't get saved
22:35		rtyler	lelit: Tailor is all Python right? can you just point me to where I might be able to fix this?
22:35		rtyler	our tree is 90k revisions, so we have a few renames here and there ;)
22:35		dcoutts	mornfall: so the way the Configure build type works is to override all the other hooks and to read the .buildinfo file that configure wrote.
22:35		lelit	ouch
22:35		mornfall	dcoutts: Well, it doesn't even get propagated I think, since preBuild shouldn't go through disk files...
22:36		dcoutts	mornfall: it gets passed into configure, but configure discards it
22:36		dcoutts	tested and looked at the code
22:36		dcoutts	mornfall: the hooks part of cabal is the worst bit
22:37		dcoutts	mornfall: so take a look at what autoconfUserHooks does
22:38		lelit	rtyler: sorry, I'm trying to finish a thing before the upcoming thunderstorm...
22:38		dcoutts	mornfall: while obviously it should work by just passing some info into Cabal and Cabal saving it in it's own config, that's not how it works. The Setup.hs is responsible for writing the .buildinfo file and restoring it for each hook that needs it. Yeah, crappy. I didn't design that bit.
22:39		rtyler	lelit: okie doke
22:40		dcoutts	mornfall: it was designed with ./configure scripts in mind, where they'd fill in a .buildinfo file (from a .in) and then each command would have to load and merge that info in. And nobody thought to make something for just ordinary Haskell code to pass the info in an ordinary way and have it work.
22:40		lelit	the strange thing seems that "git add" on the removed file
22:42		dcoutts	mornfall: so yeah, override the autoconfUserHooks, make sure you override postConf or it'll go looking for ./configure. Then instead, use writeHookedBuildInfo
22:42		dcoutts	mornfall: which you can get from Distribution.PackageDescription
22:43		mornfall	Sorry my cable modem went silent.
22:44		mornfall	Let's look then.
22:46		dcoutts	mornfall: I'm afraid I can only apologise for the steaming pile that is the hooks api, since I took over as maintainer I've tried as hard as possible to avoid it, and to make it's use less and less necessary. But that doesn't help you here.
22:46		dcoutts	mornfall: So don't worry about pestering me when you get stuck.
22:50		mornfall	dcoutts: Gotcha. It has to run from preBuild, and it needs to           return $ ( Nothing, [("darcs", emptyBuildInfo { cppOptions = [edef] })] )
22:50		dcoutts	mornfall: oh, yes, right, because it's the exe, not a lib. I forgot
22:50		mornfall	dcoutts: The Maybe BuildInfo goes with "a library" and the list goes as exes.
22:50		mornfall	Right.
22:50		dcoutts	yep
22:50		mornfall	A little icky, but alas, works.
23:02		_keturn	maybe it's the progress messages that throw off darcsum
23:13		_keturn	yeah, if I -q to whatsnew so it suppresses the "reading pristine" messages, it works better
23:13		_keturn	this is why parsing stdout is not a replacement for an API :-/

← Previous day | Channels | #darcs index | Today | Next day → | Search | Google Search | Plain-Text | plain, newest first | summary

Powered by ilbot, built with the Perl programming language. Provided by Moritz Lenz. Please direct suggestions and bug reports to moritz on irc.freenode.net or moritz 'at' faui2k3 dot org.

Imprint