Camelia, the Perl 6 bug

IRC log for #darcs, 2009-07-12

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:00 darcscommitbot joined #darcs
01:08 copumpkin joined #darcs
01:34 amuck joined #darcs
02:05 Jerri joined #darcs
05:27 HaPK joined #darcs
07:00 darcscommitbot joined #darcs
07:09 gour joined #darcs
07:27 lelit joined #darcs
07:33 balor joined #darcs
07:46 copumpkin joined #darcs
07:48 HaPK joined #darcs
08:46 mornfall lelit: Hi! How do you get those --match 'hash ...' bits for show contents in tracdarcs?
08:56 lelit hi mornfall
08:57 mornfall lelit: Hi! (See also mail, I guess.)
08:58 lelit tracdarcs keeps an internal repr of the repository, slurping "changes --xml"
08:59 mornfall lelit: Right, but how does it call show contents?
09:00 mornfall If it uses the last hash to modify a given file to get its last revision, then it's incurring a lot of overhead that would be quite unneccessary.
09:00 mornfall I.e. you should probably not query for first revision to have given file unchanged, but for last.
09:01 lelit uhm, sorry, but I do not understand... whenever Trac needs the content of a file (be it to compute a diff, or to show it), it asks the backend for that specific file at the requested revision
09:01 mornfall The file content is same in some interval [a,b] of revisions (inclusive).
09:01 lelit yes, it knows that
09:01 mornfall If you ask for a, that is potentially much more expensive than asking for b.
09:01 mornfall Even though the content is the same.
09:01 * lelit ponders
09:02 mornfall (Because darcs has to go much farther in history that way, since [a,b] can be quite a big interval -- in zooko's case, a year worth of patches, apparently...)
09:03 mornfall If there's a way to cache the b for each a, that would likely help tracdarcs performance *a lot*.
09:03 lelit wait: tracdarcs knows at which revision a file has been touched
09:03 mornfall Right.
09:05 lelit and it computes the most-recent possible, if I understand, that is: a repo has four patches, A thru D; the file "file_a" is introduced by A, and subsequently changed by C
09:06 mornfall Now will it ask for D when looking for [C,D] or for C?
09:06 lelit when trac asks for content of "file_a" at revision B, it asks darcs to compute the state at patch A... ah! is that what you are saying, that I could just ask darcs for state at B?
09:06 mornfall Yes!
09:07 mornfall That would possibly help a lot. With 2.4, this could be a factor of 40 or 50 speedup, I reckon.
09:07 mornfall At least for the most common case.
09:07 lelit uhm, I'll try to code that...
09:07 mornfall That'd be great.
09:08 mornfall (Darcs 2.4... with 2.3 and earlier, there's a factor of --match always reading the pristine, which cannot be eliminated by this approach, but it'll still be a nontrivial improvement, I believe.)
09:10 lelit ok, thanks for the suggestion: I dunno yet if it's possible, but I'll surely try
09:11 mornfall Thanks. Please keep me informed (and zooko, probably).
09:14 lelit sure
09:22 lelit the problem is determining the upper bound of the interval
09:23 lelit that is, in the context above, tracdarcs knows that "file_a" was modified by patch A and patch C
09:24 lelit your suggestion means that it should determine "C-1" (that is B)...
09:24 lelit but should be doable
09:39 mornfall Yes, you basically need C-1. Not sure how hard that is, but if you work with full darcs changes output, you may be able to pick that from there.
11:08 balor joined #darcs
11:19 gour joined #darcs
11:38 gour joined #darcs
12:54 waern joined #darcs
13:00 darcscommitbot joined #darcs
13:18 zooko joined #darcs
13:18 kpreid_ joined #darcs
13:26 gour_ joined #darcs
13:32 gour_ left #darcs
13:33 HaPK joined #darcs
14:37 zooko Riastradh: got nice new darcs on NetBSD?
14:48 arjanb joined #darcs
15:10 kowey joined #darcs
15:31 Riastradh Not yet, zooko.  Still working on GHC.
15:32 zooko Heh heh.
15:41 Riastradh Fortunately this is so far just a matter of making GHC figure out where libraries are located, and then iterating the build.
15:45 * kowey begins the darcs.net upgrade
15:45 lambdabot kowey: You have 1 new message. '/msg lambdabot @messages' to read it.
15:45 Riastradh I got 6.6.1 to bootstrap; now I'm building 6.10.3.
15:47 zooko :-)
15:47 zooko Howdy, kowey!
15:47 kowey hiya zooko
15:48 kowey thanks for restarting that build master
15:48 zooko Sure thing.  I had agreed to do something for the windows buildslave too, but that seems to have fallen on the floor.
16:51 mornfall Hi kowey!
16:51 kowey hey mornfall
16:56 HaPK left #darcs
16:57 mornfall kowey: I just posted a (late) progress report.
17:00 mornfall kowey: Also, Neil mailed me that hlint 1.6 is out, with all the features we asked for, so I'll try to check that out.
17:08 kowey hmm... darcs.net has been sitting there doing nothing for a while, the last thing it told me being "Fetched 570MB in 11min48s (805kB/s)"
17:09 kowey mornfall
17:09 kowey oops
17:10 kowey mornfall: thanks for the update (and for the freezer-friendly doc)
17:10 mornfall kowey: Actually, I don't intend to put anything in the freezer, it seems, at least not after the conclusion I reached while working on the doc.
17:11 kowey oh... (I'm still up to "hatcheries")
17:12 kowey I wonder if there are any git people we can bring in to comment on this
17:14 copumpkin joined #darcs
17:14 mornfall kowey: Don't panic!
17:14 kowey a potentially good example of "too small tags" might be David's auto-generated "tested by machineX" tags
17:15 mornfall kowey: Yes, I had that in mind, sort of. >:-)
17:15 kowey don't panic about darcs.net?
17:17 mornfall Yes.
17:20 mornfall It still pings.
17:22 kowey no web, no ssh, though
17:22 mornfall Yeah. Is your ssh session completely dead as well?
17:23 mornfall It might have been a good idea to kill gitit first, I guess you hadn't?
17:23 mornfall (The machine may just be swapping its way through the upgrade...)
17:24 mornfall Pings don't trip to userspace, so a swapped machine usually still pings, but nothing much else.
17:24 mornfall I'd probably give it some 10-15 minutes and then restart the VM.
17:24 kowey I didn't think to kill gitit, no
17:25 mornfall Well, just don't hit on the machine if that can be avoided, so that the services don't get swapped in unneccessarily.
17:29 kowey hmm
17:29 kowey oh well, at least I successfully upgraded aptitude on that machine
17:40 mornfall kowey: Any luck?
17:42 kowey still no response
17:43 kowey sigh... I did this wrong in hindsight... should have made sure I had one of the osuosl guys on hand in case anything went wrong
17:43 Igloo Do you have a serial console for it?
17:44 kowey what's that? and how do I know if I have one?
17:44 kowey :-)
17:46 mornfall Well, you would usually need an account on the physical host, and some sort of minicom or such thing to access the serial port of the VM.
17:47 kowey I see, I'm not aware of having one
17:47 kowey hmm
18:02 amuck joined #darcs
19:00 darcscommitbot joined #darcs
19:11 balor kowey: Have you used your last quarter well?
19:11 kowey eh
19:12 kowey balor: I messed up in upgrading darcs.net... the mistake here being not making sure that our hosting people were around to help out in case anything went wrong
19:12 balor ah
19:12 kowey which, inevitably....
19:18 Heffalump still gone, then?
19:21 mornfall Well, you can probably announce the fact on the mailing list (I believe this is running from elsewhere than our osusl VM, right?)
19:21 mornfall We can definitely survive till tomorrow without darcs.net I guess.
19:39 waern_ joined #darcs
19:43 kowey doesn't the bit that figures out that darcs-users@ and bugs@ go to different machine still live on darcs.net?
19:43 kowey (I'm just "reasoning" by analogy with the fact that my kowey@ redirection to gmail doohickey is also affected by this)
19:44 Heffalump I'd expect it to be affected, yes
19:50 amuck joined #darcs
19:50 kowey I'm going to try @lists.osuosl.org
19:52 Heffalump that ought to work
20:02 mornfall Hmh. darcs.net mail is handled by 5 smtp.osuosl.org.
20:02 mornfall But it could go to the VM and then back to osusl, I guess.
20:04 kowey do we still believe that this may be something to related to not having shut gitit down?
20:04 Heffalump I'd have expected the machine to have started responding by now if it was just swap death
20:11 mornfall Hm, I have cabal install darcs-beta running in Windows XP (in a VirtualBox).
20:11 mornfall Wondering what happens next...
20:11 mornfall So far it was matter of installing windows xp, installing haskell platform and msys and typing "cabal update" and "cabal install darcs-beta -f-curl"...
20:11 Heffalump nice
20:14 SamB mornfall: slipstreamed?
20:15 mornfall SamB: ?
20:15 SamB the XP
20:15 SamB or did you just leave the service packs out?
20:17 idnar maybe someone should slipstream the haskell platform :)
20:17 SamB the trouble with slipstreamed disks is you're not allowed to distribute them
20:18 SamB that's the problem with having to buy a license to your OS :-(
20:19 idnar SamB: I get my slipstreamed disks from MS :P
20:20 idnar (of course, they don't have the Haskell Platform yet)
20:20 * SamB has a sudden urge to obtain DOS 7.0 SP2
20:20 mornfall SamB: I guess no, whatever slipstreaming is. I just grabbed a MSDN AA copy.
20:21 SamB mornfall: was it of XP+SP2?
20:21 SamB or just XP?
20:21 SamB also what does AA mean ?
20:21 mornfall Academic Aliance
20:21 mornfall XP+SP2
20:21 mornfall I believe.
20:22 SamB that would be slipstreamed
20:22 mornfall Ah.
20:22 mornfall Hm, darcs doesn't particularly work.
20:22 SamB don't ask me why making disks like that is called slipstreaming, except of course it sounds fastish
20:22 mornfall Doing a remote get is like super-duper slow.
20:23 SamB mornfall: and other remote access isn't ?
20:23 idnar SamB: they're "slipped" into the "stream" of the install :P
20:23 mornfall Well, cabal install works just fine.
20:23 SamB I mean, using other tools?
20:23 SamB well, cabal install downloads a lot less files, doesn't it ?
20:24 SamB it just downloads each tarball
20:24 SamB not every single patch
20:24 mornfall Yes, but doing --verbose --debug on darcs shows that it downloads about 4 files, then after a few seconds fifth and then stalls.
20:24 SamB ah
20:24 mornfall I'll try a smaller repo than darcs itself in a sec.
20:24 SamB okay, now that's clearly not *just* some virtualization-based latency
20:25 mornfall Hm, the hashed-repo flies by.
20:25 mornfall hashed-storage repo
20:26 mornfall Maybe there are some hidden references to http://darcs.net in my repos.mornfall.net branch-2.3, so it won't download...
20:26 SamB mornfall: hmm?
20:26 SamB your darcs can't hit darcs.net?
20:27 * SamB suddenly wonders if Amigas can do IPv6
20:27 * kowey broke darcs.net
20:27 SamB kowey: ah
20:27 SamB that would be bad
20:28 kowey http://lists.osuosl.org/pipermail/​darcs-users/2009-July/020391.html
20:32 mornfall Anyway, my darcs sort-of works, it's failing a bunch of tests and the index stuff is not working at all.
20:32 mornfall Failing in mmap somehow.
20:32 mornfall Anyway, I'll look into that later.
20:32 mornfall Gotta sleep, goodnight folks.
20:33 kowey good night
20:33 mornfall kowey: If you have any feedback on the storage stuff or any other recent SoC bits, feel free to e-mail me or such. As for that Nicolas-mail, I have had a quick look some time ago, I'll look in more detail later hopefully.
20:33 kowey will do
20:36 WorldMaker joined #darcs
20:36 WorldMaker Hey y'all!
20:37 WorldMaker I was thinking a little bit just now about 'in-repo branching'...
20:39 kowey hey! haven't seen you on IRC in a while
20:40 WorldMaker I never got into the habit of leaving IRC up at all times...
20:41 Topic for #darcsis now Latest is 2.2.0, latest unstable is 2.3 beta 2 | darcs.net down http://lists.osuosl.org/pipermail/​darcs-users/2009-July/020391.html
20:42 WorldMaker Anyway, I was thinking, kowey, that in-repo branching might be a great use case for working with the darcs library and hashed-storage...
20:43 WorldMaker I think it would be a appropriate to build it as an add-on/separate tool and I'm betting that hashed-storage alone might be all that is needed to accomplish something interesting.
20:44 kowey in-repo branching as a separate tool?
20:44 * kowey imagines symlinks flying around
20:45 WorldMaker Yes, a simple darcs-branch tool I think would be sufficient. I also would find it a good irony that most of the things that we like built into darcs are git addons, and something so crucial to git (cheap in-repo branching) might be a useful darcs addon.
20:46 kowey don't you need some darcs patch stuff to deal with swapping things in and out of the working directory?
20:46 kowey by the way, I saw on the wiki (if you have an offline copy somewhere) an old request for a darcs switch command, way back in 2003
20:46 WorldMaker You could 'merely' store multiple pristines.
20:47 kowey and inventories
20:48 kowey would this darcs-branch tool then invoke things like darcs pull / darcs obliterate?
20:48 kowey or would it really be independent from darcs?
20:48 WorldMaker Yeah, just invoke darcs for patch operations, otherwise you are working on git-like references to various pristine/inventory objects in hashed-storage.
20:49 WorldMaker It's sort of the reverse of the way patch queues work in git/hg.
20:50 kowey woo! osuosl are around! somebody's rebooted darcs.net
20:51 Heffalump cool
20:51 WorldMaker Since you probably want to use hashed-storage directly you probably want to use darcs as a library, but theoretically you could get away with shell scripting it... as long as your pristine/inventory-swapping is fast enough.
20:52 kowey they say it OOMed and was taking everything with it
20:52 * kowey tries the upgrade again, this time without gitit running
20:53 kowey WorldMaker: I might stop responding, but hopefully #darcs will have nice comments :-)
20:53 Heffalump WorldMaker: that sounds nice, I think
20:53 Heffalump actually, does anything preclude doing that already, with hashed repos?
20:54 Heffalump you just need to switch the current pristine/inventory pointers
20:54 WorldMaker No, I bet it could be done today with hashed-storage if you found someone willing to build it.
20:55 Heffalump I don't understand what hashed-storage has to do with it, that's all. Darcs already uses hashes, hashed-storage is just improving on that in a separate library
20:55 Heffalump (unless I'm confused, which I might be)
20:57 kowey oh yay, apt/aptitude is clever enough not to re-fetch packages
20:57 WorldMaker Mostly it's just the nice fact that hashed-storage, from what I've seen, provides a clean third-party usable API for everything. Whereas, darcs' hashed pristine code probably isn't that friendly to reuse.
20:57 Heffalump yes, fair enough
20:57 Heffalump kowey: it caches them in /var/cache/apt/archives
20:59 WorldMaker Also, by building it upon hashed-storage you can help Petr prove that hashed-storage is precisely the sort of library that needed to be spun out of darcs. I think darcs-branch is a more interesting usage scenario for hashed-storage than camp is.
20:59 kowey yeah, was just thinking that this would be a good way of keeping hashed-storage clean
21:00 kowey hmm... darcs-branch list, darcs-branch switch, etc
21:00 Heffalump does darcs normally cleanup old hashed stuff?
21:00 WorldMaker Yes, and that's what Petr's last blog post was on and what partly got me thinking about this.
21:01 WorldMaker Petr's debating GC strategies for hashed-storage moving forward (and incorporating packs/archives)...
21:01 Heffalump ok, I skimmed that but didn't pick up that detail
21:03 WorldMaker He made an explicit point that hashed pristine is very like git's object store at this point, with the big difference being that darcs can quickly GC old pristine objects because darcs uses the patch objects.
21:04 Heffalump ok, so stop darcs doing the GC, and darcs-switch is nearly there
21:06 WorldMaker Basically. It would probably make it more useful to think in terms of 'freezing' (and naming) useful pristine state pointers.
21:07 WorldMaker Darcs produces a lot of junk transient pristine states; you don't want to save "everything".
21:07 Heffalump well, I meant that the new command would do the GC instead
21:07 Heffalump it may well make sense to just put the concept of special roots in to darcs itself
21:08 Heffalump you need to keep inventory pointers too, right?
21:08 WorldMaker It wouldn't even need to bother with doing its own GC, it just needs to add a 'named pointer' into the pristine collection.
21:08 WorldMaker I think Petr already thinking down that line in hashed-storage that you might need multiple root pointers.
21:09 kowey is there any link between this discussion and that on zooko's short secure fast version identifiers? http://bugs.darcs.net/msg5797
21:10 mornfall There are links everywhere (yeah, I had trouble falling asleep...)
21:10 mornfall In-repo branching and version hashes are related sort of.
21:11 mornfall Although I guess that's a little distant relation.
21:11 mornfall Your "branch pointer" could be a hash of the current hashed_pristine, eg.
21:12 mornfall If you kept such pointers for other revisions than branch tips, you get version identifiers.
21:12 mornfall Assuming you would also keep the corresponding inventories.
21:13 Heffalump the inventories are what matter
21:13 Heffalump aren't they?
21:13 Heffalump keeping the pristine is just an optimisation
21:13 mornfall Yes of course.
21:13 mornfall Inventory = set of patches = repository.
21:13 mornfall Tag = version. Tag after every patch and you have your secure identifiers.
21:13 mornfall :)
21:14 kowey I remember back in the darcs-git days, that David had an idea called "tagging like crazy"
21:14 mornfall Yes.
21:14 mornfall That's exactly it. For darcs, a tag is what git calls a commit.
21:14 mornfall Or at least, there's a good correspondence.
21:15 Heffalump right, tags serialise development in the same way that git does by default
21:16 mornfall Interestingly enough, as things are, if you tag like crazy, you'll get an inventory file per patch, and this will have a hash that exactly identifies that revision.
21:16 mornfall (Since each inventory has a hash of the previous inventory in it, and of its patches, this is also quite secure.)
21:17 mornfall It's also very inefficient. :)
21:17 WorldMaker More usefully meaningful would be tagging after every pull/push/apply and then you have something equivalent to git merge states.
21:17 mornfall WorldMaker: Also before.
21:17 mornfall Unless there's already a tag.
21:17 WorldMaker Right.
21:18 mornfall But then, you would have to obliterate lots of tags just to get rid of a patch that's buried a little.
21:19 WorldMaker Which is why you sort of need "softer" tags to do such a thing...
21:19 Igloo kowey: Is darcs.net supposed to have gone again?
21:19 kowey I'm doing the dist-upgrade now
21:19 kowey I saw it restart apache
21:19 Igloo OK
21:20 kowey this is going much more smoothly
21:20 WorldMaker ...the question becomes, could hashed-storage make it more efficient to store/name lots of little inventories/contexts than tagging like crazy?
22:25 kowey OK, I'm now going to reboot darcs.net
22:25 kowey I'm just naively following http://www.debian.org/releases/stable/i​386/release-notes/ch-upgrading.en.html and have installed a kernel metapackage
22:37 Topic for #darcsis now http://darcs.net | http://wiki.darcs.net/irc | Latest is 2.2.0, latest unstable is 2.3 beta 2
22:47 zooko joined #darcs
23:23 kowey OK, so most of the darcs.net upgrade is complete
23:24 kowey I wouldn't be suprised if there was some breakage in our build process, etc, but I can look at that later
23:24 kowey also I have to work out how to upgrade from PostgreSQL 8.1 to 8.3 (just following the default instructions)
23:24 kowey good night!
23:24 gwern what are we using postgresql for?
23:24 kowey roundup
23:25 gwern ah
23:25 kowey can you help?
23:25 kowey I'd rather stay up and just finish the job if I had somebody who knew what they were doing
23:25 * gwern knows nothing about sql
23:26 kowey oh well... for what it's worth (maybe somebody here knows)
23:26 kowey I'm trying to do the most standard boring possible thing, which I gather consists of
23:27 kowey installing 8.3, then pg_dropcluster --stop 8.3 main [to get rid of the default cluster], followed by pg_upgradecluster 8.1 main
23:28 kowey but the last step fails with could not bind IPv4 socket: Cannot assign requested address
23:30 kowey ugh... looks like I have to stay up anyway
23:34 zooko "Cannot assign requested address" ?
23:34 amuck joined #darcs
23:34 zooko Do you know what IPv4 address you are trying to bind to?
23:35 kowey 5433
23:35 kowey oh sorry
23:37 kowey erm, no actually... presumably 127.0.0.1?
23:37 WorldMaker I thought that the 8.1 to 8.3 upgrade is handled automatically.
23:37 WorldMaker I've only ever needed pg_upgradecluster between major versions (ie, 7.x to 8.y)
23:38 kowey I got this little box http://hpaste.org/fastcgi/hpaste.fcgi/view?id=6911
23:39 kowey anyway, another issue that's creeped up is that roundup won't let us search or edit stuff from the web... it looks like it's unrelated

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary