Perl 6 - the future is here, just unevenly distributed

IRC log for #darcs, 2017-04-30

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:03 Riastradh joined #darcs
00:15 mizu_no_oto joined #darcs
02:17 byorgey joined #darcs
02:19 vikraman joined #darcs
03:37 mizu_no_oto joined #darcs
04:06 Riastradh joined #darcs
04:16 pointfree Heffalump: It may be necessary to expose the darcs post-hooks of commands through the Haskell API after all.
04:28 pointfree Primarily for getting the list of patches just applied without being limited by the maximum size of the environmental variable, but there's also a bunch of other darcs commands that would benefit from a post-hook through Haskell for darcshub.
09:18 Heffalump pointfree: yeah, I can imagine. I guess we want some kind of higher-order API where we pass in the hook as a parameter?
12:15 bfrk joined #darcs
12:15 bfrk Hi, still trying to get darcs to work with hub.darcs.net
12:16 bfrk I have just compiled darcs-2.10.2 from source with curl enabled and http disabled and it still fails. While /usr/bin/darcs (which is also 2.10.2) succeeds. WTF?
12:22 Heffalump sorry, I haven't been following the details. What fails?
12:23 Heffalump I think "http disabled" is actually a bit of a funny configuration that we (I) dropped in HEAD
12:27 bfrk The problem is: I cannot access repos on hub.darcs.net via (self-compiled) darcs. I get: darcs failed:  Not a repository: http://hub.darcs.net/kowey/tabular (Problem with the SSL CA cert (path? access rights?))
12:28 bfrk But, strangely, it works with the /usr/bin/darcs from my distro (ubuntu 16 LTS), which is darcs-2.10.2
12:28 bfrk I can also look at the repos with a browser and curl also suceeds
12:30 bfrk I talked about this with sm and pointfree yesterday, they said it could be related to the use of letsencrypt on hub.darcs.net
12:30 bfrk This is all really not my specialty!
12:32 bfrk The reason I disabled http is to be as near as possible to what /usr/bin/darcs --exact-version said
12:36 Heffalump I see, maybe the flags do that automatically. The old flags were a bit confusing.
12:40 ggherdov` joined #darcs
12:41 carter joined #darcs
12:46 bfrk Since /usr/bin/darcs worked and said it is 2.10.2 I compiled 2.10.2 from source but that failed with the same SSL CA message
12:50 Heffalump it could be to do with curl configuration, maybe the one you're building against is looking at a different certificate list. I have no idea about any of this either though.
12:58 pointfree bfrk: Are /usr/bin/darcs and your recently built darcs linked against the same libraries (ssl in particular)? Compare the output of:
12:58 pointfree ldd /usr/bin/darcs
12:58 pointfree ldd your/other/darcs
13:09 bfrk Hm, my self-compiled darcs has very few run-time deps with neither curl nor ssl appearing. Whereas /usr/bin/darcs has many, among them gnutls and curl
13:09 bfrk ldd /usr/bin/darcs | grep tls says:
13:09 bfrk libcurl-gnutls.so.4 => /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007f991a927000)
13:09 bfrk libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f99187a4000)
13:11 bfrk I am using cabal new-build. How can I make the result have these run-time dependencies?
13:15 bfrk I am now trying to build using the regular cabal commands. Will take a while...
13:18 pointfree bfrk what about ssl? Btw, you can also check which version of ssl it's actually using when you try to clone that repo with:
13:18 pointfree strace /usr/bin/darcs clone --lazy http://hub.darcs.net/kowey/tabular 2>&1 | grep ssl
13:18 pointfree strace ~/.local/bin/darcs clone --lazy http://hub.darcs.net/kowey/tabular 2>&1 | grep ssl
13:27 Heffalump bfrk: do you have the dev packages for curl installed?
13:27 Heffalump it's very odd for you to have a darcs that doesn't link against curl. Does it link against some dynamically compiled Haskell libraries that themselves link against curl?
13:57 bfrk Heffalump: Probably, and perhaps this is due to cabal new-build. I have just compiled using the regular cabal install --only-dependencies && cabal build and that works just fine
14:00 bfrk pointfree: have to grep for tls, not ssl (I am using gnutls)
14:01 Heffalump bfrk: oh, interesting
14:01 bfrk Correction, it has nothing to do with cabal.
14:02 bfrk Sorry for the confusion, tried too many things at once. It now succeds with cabal new-build, too.
14:03 bfrk I think I may have had the wrong libcurl stuff installed.
14:06 bfrk my distro offers me: libcurl4-gnutls-dev libcurl4-nss-dev libcurl4-openssl-dev of these, building against libcurl4-gnutls-dev seems to work and and I think I had one of the other two installed previously
14:07 Heffalump ahah
14:07 Heffalump SSL does seem like a bit of a swamp :-(
14:22 leg joined #darcs
15:45 sm pointfree++, ldd & strace were a good idea
15:46 sm bfrk: just read chat log.. did you get it working now ?
16:07 bfrk sm: yes it works now and I can't even seem to be able to reproduce it =:|
16:08 bfrk prehaps just re-installing the libcurl...dev stuff fixed it
16:08 bfrk thanks for all the help
16:11 Heffalump pointfree: any comments on what this hypothetical hook api should look like?
16:46 sm bfrk: ha.. good!
16:49 pointfree Heffalump: Yes, I was just thinking about it. Passing the post-hook in as a higher order parameter through the Haskell API sounds like a good idea. Then I could pass in addNewsItem for the darcsden feed.
16:49 pointfree Somewhere inside addNewsItem I can have it get the information into the newsitem document from some data structure consistent across darcs commands (apply, init, pull, etc) as well as other information from darcshub.
16:49 pointfree Also, just a thought: the full contents of the patch is not provided by the patches xml.
16:49 pointfree Thinking ahead, it could be a good idea to store the patches themselves in structured couchdb documents for easier querying, for not having multiple copies of the same thing, and for a search/filter feature (next after the feed and mobile css).
16:49 pointfree Storing the patches themselves in couchdb requires a bit more thought.
16:49 pointfree I tried using "apply" from src/Darcs/UI/Commands/Apply.hs in darcsden-ssh. The interactive patch selection thing works but the patches won't apply. Maybe there's some other flags and things I need to add.
16:52 pointfree Interesting: https://hub.darcs.net/pointfree/darcsden-feed-couchdb/browse/src/DarcsDen/Handlers/RepoHandlerUtils/Forks.hs#166
16:56 sm I find couchdb complicated and kind of wish we could get rid of it
17:11 pointfree sm: If we want to go that route, I think the den/darcsden-local stuff would be a good starting point.
17:11 pointfree The transactional stuff that couchdb does could be handled by darcs. Issues could be stored in the darcs repo and be carried with the version of the repo in question...or as just newsfeed-item files.
17:11 pointfree Feed/issue subscription stuff could be done with (hard|sym)links or just plain files.
17:11 pointfree Store keys stored in the normal filesystem way: .ssh/
17:11 pointfree Future repo search could be done from a xapian index (?)
17:11 pointfree Perhaps it's not a great idea to increase our dependency on couchdb.
17:11 pointfree CouchDB is interesting, but for a supposedly big data database it hasn't performed as well as advertised. It's tricky to make the couchdb views perform well.
18:09 bfrk joined #darcs
18:33 bfrk I know it's a can of worms... anyway: can we fix the encoding business for file paths in darcs? It shouldn't be too hard, it's just tedious.
18:34 bfrk My plan: (1) throw out the code in main that disables what ghc does (since a decade or so)
18:35 bfrk (1a) that includes setting stdin and stdout to binary (remove that)
18:35 bfrk (2) find all places where we treat String (aka FilePath) as if it were [Char8]
18:36 bfrk (3) internally, encode paths using utf-8 when using ByteString
18:37 bfrk Would that work?
18:38 bfrk Re (1): what ghc does by default is to interpret everything that comes from the user as being encoded according to the user's locale and convert it properly to [Char], where Char is unicode points
18:39 bfrk and similar for output
18:41 bfrk The idea of the plan is to maintain a few simple invariants. A char is a Char, not a byte. A byte is a byte, not a Char.
18:43 bfrk The contents of files should be treated as it is now: as uninterpreted blob of bytes; when presenting file content to the user, encode according to current locale, perhaps let user override with the usual escaping env vars
18:46 bfrk Some of the types used internally can go away e.g. FileName (newtype for FilePath).
19:02 Heffalump bfrk: I was thinking we should decorate the code with newtypes to track the encoding as it flows through. Otherwise we'd have no hope of finding/fixing all the bugs reliably
19:12 mal`` joined #darcs
19:46 byorgey joined #darcs
20:13 bfrk Heffalump: I think this was already tried more than once. See FileName, the code is littered with fn2ps and fn2fp. See also the printer API.
20:15 bfrk What helps is clear rules. String/FilePath is always unicode, basta. Communication with user (command line, file paths) is via the user's encoding. ByteString is either raw bytes (file content) or utf-8 encoded path names or meta data.
20:17 bfrk The only interesting question is how to display file content.
20:19 bfrk For instance AnchoredPath. It uses ByteString, so by the rules above these must be utf8 encoded.
20:22 bfrk If we use String as it is meant to be used, I see no need to track encoding, except perhaps for file content vs. file names.
20:25 bfrk But I think it is pretty clear in most contexts whether a certain piece of ByteString is file content or file path.
20:27 bfrk In any case, the first thing to do is remove the old hack that twists ghc/base into giving us (and expecting from us) Strings that are in fact raw bytes.
20:31 bfrk Prior to this post I have tried in vain to get darcs to accept file paths with german umlaute in them. It just doesn't work no matter what I tried.
20:43 bfrk I see that we also use an ad-hoc whitespace codec for file names. Can someone please tell me what good this does?
20:44 Heffalump for putting them in patches?
20:44 bfrk (I am referring to Darcs.Util.Path.en/decodeWhite which is used indirectly in HashedIO)
20:46 bfrk Does the patch format prescribe that whitespace in filenames must be encoded in this way? Wh?
20:46 bfrk I mean: why?
20:46 Heffalump no idea, I'm just guessing; but if it is done that way then by definition it's prescribed, right? Not necessarily for any good reason now but for backwards compatibility.
20:48 bfrk The code has no, repeat no single comment about what the whitespace encoding is used/needed for. Are you sure it is the patch format?
20:49 Heffalump no, I'm not sure. I was just guessing.
20:49 Heffalump but where else would it be used?
20:50 bfrk Ah, me too. If the patch format does indeed have this strange encoding then it must be preserved.
20:50 Heffalump someone reported a bug a few months ago about umlauts etc
20:51 Heffalump http://bugs.darcs.net/issue2524
20:52 Heffalump at least, I guess it's the same bug; I'm not implying that umlauts are the same as random accented characters in other languages.
20:52 bfrk They are.
20:53 bfrk ...at least as far as unicode is concerned.
20:53 bfrk (AFAIK)
20:53 bfrk But I am pretty sure it is the same bug.
20:57 bfrk I have tried darcs record -l. It behaves inconsistently. I can record things. But when I say darcs record again, darcs doesn't see that it has the files and offers to remove the files and their content.
20:57 Heffalump ouch
20:58 bfrk You name it.
21:00 bfrk I have experimented with the plan I proposed above. I have taken out the hack in darcs/main. I am now in the process of removing all unnecessary en/decoding so that at least whatsnew gives me consistent behavior.
21:00 Heffalump fair enough
21:01 bfrk It is not that bad actually. Whatsnew without --summary behaves correctly out of the box. It is only --summary that needs fixing.
21:25 bfrk Ahem, not true. Whatsnew --no-summary also lists the just recorded changes as to be removed.
22:10 bfrk I just verified that the white space encoding is indeed the patch format. Incredibly stupid.
23:49 bf_ joined #darcs

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary