Perl 6 - the future is here, just unevenly distributed

IRC log for #darcs, 2015-11-30

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:04 aristid joined #darcs
00:26 aristid joined #darcs
01:29 maerwald mh, GET requests seem to fail with "getCurrentDirectory: does not exist (No such file or directory)"
01:30 maerwald like the repositories are gone
01:40 aristid_ joined #darcs
01:46 notdan joined #darcs
01:46 notdan joined #darcs
01:59 vikraman joined #darcs
02:21 Big_G joined #darcs
02:39 vikraman joined #darcs
04:44 ilbot3 joined #darcs
04:44 Topic for #darcs is now http://darcs.net/ | logs: http://irclog.perlgeek.de/darcs/ | darcs 2.10.0 is out http://darcs.net/Releases/2.10
05:04 dleverton joined #darcs
05:04 dleverton joined #darcs
06:56 Heffalump maerwald: If you're still interested in getting a diff command working, try out http://hub.darcs.net/ganesh/darcs-2.10-issue2481
07:03 lelix joined #darcs
08:04 lelit joined #darcs
08:17 dleverton joined #darcs
08:17 dleverton joined #darcs
10:02 Reyu joined #darcs
10:37 lelit joined #darcs
10:40 lelix joined #darcs
11:16 maerwald Heffalump: the initial idea was to pass a repository and a list of patches, now I got DarcsFlags and SubPaths. DarcsFlag has 190 constructors
11:22 maerwald not sure I want to read all of them to figure out which ones I need...
11:30 aristid joined #darcs
11:39 carter_cloud joined #darcs
12:10 peb` joined #darcs
14:26 maerwald darcsden.cabal says "-- not -threaded until libdarcs is working directory thread safe"... but it seems if we use snap-server with ssl, then we need threaded
15:18 sm morning maerwald, all
15:19 sm morning/afternoond
15:19 sm hmm yes that might be why ssl hasn't happened yet
15:20 sm I wonder what happens with forkIO $ sendmail.. when we are running single-threaded, btw
15:22 maerwald "Using GHC without the -threaded option, all foreign calls will block all other Haskell threads in the system, although I/O operations will not."
15:23 maerwald I think we should move to threaded and put darcs IO calls behind https://hackage.haskell.org/package/base-4.8.1.0/docs/Control-Concurrent.html#v:runInBoundThread
15:25 sm I'd love to move to threaded, if we can avoid those very annoying intermittent random failures due to loss of CWD
15:26 sm re #30 whole-issue-tracker subscription - I guess there are a few details still, like how does whole-tracker and individual-issue subscription interact
15:27 sm we could disable the issue subscribe button if they are subscribed to the tracker
15:28 sm and if they are subscribed to the tracker, it should ignore any addition subscriptions they have to individual issues (ie not send them multiple copies)
15:29 maerwald well, I want to avoid introducing something like an "unsubscribed" DB entry...
15:29 maerwald github probably does that, so that you can explicitly unsubscribe and never get re-subscribed again
15:29 maerwald but I find that too complicated
15:30 sm I don't see the need for that
15:30 sm just another subscriber list on the tracker level, and when it sends notifications take the union of that and the issue subscriber list
15:31 sm and in the ui, check current user's tracker subscription status and disable the issue subscription button if appropriate
15:33 sm I guess github calls this "watch this repo"
15:34 sm and you're right they have an "unwatched" status, which I guess is to silence everything if you happen to be subscribed to individual issues. Maybe nice to have that later
15:51 pointfree joined #darcs
15:56 pointfree Timestamp anchors as permalinks: http://hub.darcs.net/simon/darcsden/issue/144 Is the timestamp format okay? Should the <space>UTC be chopped off the end?
15:59 maerwald sm: what things exactly in the darcs API are not thread safe?
15:59 byorgey joined #darcs
15:59 byorgey joined #darcs
16:00 maerwald I tested the 4 ssl patches and it works fine locally. Also note that the "baseUrl" change is non-trivial, but important
16:00 maerwald I couldn't find anything that breaks with it
16:00 maerwald http://hub.darcs.net/maerwald/darcsden-patches/patch/83dbd1d909bea2b0f4b08a6d4990990249884fc8
16:02 maerwald so I'd say the only thing left is wrapping darcs API calls with "runInBoundThread"
16:06 dleverton joined #darcs
16:06 dleverton joined #darcs
16:07 sm pointfree: just commented
16:09 sm maerwald: the problems show up intermittently only on a busy multi-user site like darcs hub (as various different errors in random operations).
16:09 maerwald sm: I cannot follow
16:10 sm what's not thread-safe is the current working directory. This is a GHC/base thing. So if two darcs operations happen at just the right time, one changes the working directory while the other is working, causing confusion and failure
16:10 sm I was looking the the issue about this, but haven't found it yet
16:10 sm does it make sense ?
16:10 maerwald sm: so I basically want to wrap all darcs operations that _change_ a repository?
16:11 maerwald so that a change to a repo locks the thing
16:11 sm it might be more operations than that.. I think pretty much everything libdarcs does requires setting the current directory
16:11 sm Heffalump may remember more about this
16:13 sm afk
17:34 jackhill joined #darcs
18:19 carter joined #darcs
18:29 siel joined #darcs
19:12 Heffalump maerwald: I guess that's roughly what a CLI equivalent API would be - I can point out the ones you need if you tell me the command-line flags you want to mimic. Or we can figure out the right thing with a list of patches
19:12 Heffalump I think even reading a repository sets the cwd, yes
19:26 dleverton joined #darcs
19:41 maerwald Heffalump: also... how does darcs even know if the list of DarcsFlags is valid? Afais this isn't even the right data structure for it. But I can imagine doing it differently would be considerable effort
19:47 Heffalump it just ignores the ones it doesn't expect
19:47 maerwald well, I can tell you the information I have which I then need to feed to darcs in one way or another: 1. repository (the directory path to be precise) and 2. hashes (either one or several)
19:47 Heffalump at least by the time you get to that function
19:47 Heffalump darcs diff doesn't take several hashes, does it?
19:47 Heffalump It either takes one, or --from-hash and --to-hash
19:47 maerwald yeah
19:47 maerwald in case of from to it "merges" them
19:47 Heffalump FWIW the options system is in the middle of being reworked to use explicit types for each command rather than a large bag of flags, but that hasn't reached darcs diff yet
19:48 maerwald ah
19:48 Heffalump what it actually does it looks at the repo state at the from hash, the repo state at the to hash, and diffs them
19:48 maerwald 190 constructors is just a bit... odd ;)
19:48 Heffalump the sequence of patches inbetween needs to be contiguous in the current repo representation
19:48 maerwald I mean... you have to pattern match it somewhere, right? Is probably not fun
19:49 Heffalump the code that handles it mainly just pattern-matches for specific things and falls through for the rest
19:49 maerwald yeah, that's why some commands of darcs just do nothing instead of giving an error
19:51 Heffalump that should be rare, the command-line parsing does know what's valid
19:51 Heffalump it's just by the time it gets to this code, that's all done
19:51 maerwald darcs diff asdsdalaösdlöasdölasdlö
19:51 maerwald does nothing
19:52 maerwald it should probably say something like "invalid argument" or "argument xy missing"
19:52 Heffalump the output is technically correct, though unhelpful: it's telling you there are no local changes to the file "asd..."
19:53 maerwald the file does not exist
19:53 Heffalump that's what I mean by unhelpful
19:53 Heffalump but that's mostly independent of the flag validation
19:54 maerwald this is what git does: fatal: ambiguous argument 'alksdlsak': unknown revision or path not in the working tree.
19:54 maerwald but it behaves simlar for "git log -- akslasdlk"
19:54 Heffalump sure, I'm just saying that the problem you're describing is specific to that particular argument to that particular darcs command. I'm sure there are other examples too. But it's not a general problem caused by inappropriate fallthrough in flag handling code.
19:54 Heffalump so anyway, what command-line flags do you want the equivalent DarcsFlags for?
19:55 maerwald I have the darcs repo path and one or two hashes. if it's to hashes, then it's --from-hash --to-hash
19:56 maerwald the darcsden getPatch function does something like "withRepositoryDirectory blahblah" and then "readRepo"
19:56 maerwald but everything following that is not what I want
19:57 Heffalump ok, OneHash for a single hash, AfterHash and UpToHash for --from-hash and --to-hash
19:57 Heffalump and for the darcs repo path I think you might need to change cwd first, I'm just checking
19:58 maerwald I'd expect that "withRepositoryDirectory" does that?
19:58 Heffalump WorkRepoDir for the repository directory
20:00 Heffalump if you want to get close to the behaviour of the CLI, then pass the things you want to do in [DarcsFlag] - if you want to go lower-level and use functions like withRepositoryDirectory I'm very happy to keep discussing the API that uses the internal structures
20:00 Heffalump though not right now as my train is just pulling in
20:07 pointfree sm: I like your idea of statically generating the feed in response to events. Easier on servers, and easier to implement with regards to updates other than patches (issues and issue comments for instance). I could hook into channelRequest from ssh-server/darcsden-ssh.hs ?
20:08 sm pointfree: you'd have to hook into a number of places in the web app too
20:09 sm or there's the other suggestion of using _darcs/prefs/defaults, if that works
20:09 sm but I was thinking really darcs needs some changes to make this easier - you should be able to configure the hook in one place, similar to .git/hooks/posthook.sh or whatever it is
20:11 sm as with git, uncommenting that script should be all that's required to get a robust hook
20:12 sm that would be much better than complexifying darcsden
20:14 sm even better, it should be possible to specify hooks when using the libdarcs api
20:16 pointfree I'd feel better about the feeds being handled on the server side but I wouldn't mind if libdarcs helped with that, darcsden imports libdarcs anyway.
20:17 pointfree Although darcs and darcsden are officially one project now.
20:20 pointfree Although... we are thinking about doing issues tracking locally.
20:20 vikraman joined #darcs
20:24 pointfree Maybe moving stuff like feed generation to darcs (on the client side) introduces upgrade and backwards compatibility troubles...
20:29 maerwald uhm, so how do I test the darcs patch with darcsden
20:30 sm pointfree: officially one project, why do you say that ?
20:31 pointfree sm: somebody else said it here on #darcs a while ago...
20:33 sm I'm not sure it's accurate or a good idea, unless I forgot some past discussion
20:33 maerwald I think strictly separating them is better for the API of both
20:35 sm re moving feed generation into darcs - no, really what I'm suggesting is improving the hooks mechanism that's already in darcs. Currently it (a) isn't accessible via the libdarcs api and (b) is unnecessarily complicated and fragile for the user to configure
20:35 pointfree maerwald: I agree. Also, separation makes it more inviting to third parties.
20:35 sm sure, they're very closely related, and darcs hub was always intended to help accelerate darcs dev
20:36 maerwald can I somehow tell my cabal sandbox to use a patched darcs version?
20:37 sm maerwald: yes, typically I would bump the version number in darcs.cabal and depend on that to make sure I'm using the right one
20:37 sm then also, cabal sandbox add-source ../darcs or something
20:37 sm nowadays, I add the local darcs to stack.yaml instead
20:38 maerwald ah, add-source it is?
20:38 * Heffalump reappears
20:38 sm however, sometimes I'll symlink Darcs under src and just build it as part of darcsden temporarily, to reduce the hassle of multi-package builds
20:39 sm this is good for ghci reload, etc.
20:39 maerwald oh, does that trick work?
20:39 maerwald will it build it locally then?
20:39 sm it might need some fiddling, but yes pretty sure I did that
20:40 maerwald Heffalump: shouldn't it be IO (Maybe Doc)? What happens if the hash does not exist?
20:42 Heffalump hmm, true
20:44 Heffalump the code just throws, and it's fairly deep down in the call stack
20:44 Heffalump though I think you'll just have to catch it
20:50 maerwald src/Darcs/UI/Commands/Diff.hs:20:46: Not in scope: ‘getDiffText’
20:58 maerwald other than that, works great!
20:59 Heffalump oops, got the export wrong :-)
21:00 maerwald getRawPatch dir patch = getDiffDoc [OneHash patch, DUI.WorkRepoDir dir] Nothing >>= (\x -> return $ renderString Standard x)
21:00 Heffalump pushed the fix - though I guess I'll merge those two patches to submit to darcs
21:06 maerwald http://hub.darcs.net/maerwald/darcsden-diff/patch/6b56bf00a202f23361dbe528651d934e71355433
21:20 sm good.. as per usual this can go into darcsden when it's in a darcs release, possibly sooner into darcs hub
21:22 sm also I think we should start providing basic performance/security analysis with any proposed new routes
21:24 sm what does it cost to serve, does it bring any new security considerations
21:24 sm that'll help get things deployed quicker
21:25 maerwald nice, range of patches works too
21:27 maerwald sm: http://hub.darcs.net/maerwald/darcsden-diff/patch/db40b78c3c04d77fe9e38f9cc93808bc14c8e515
21:27 maerwald that's pretty neat
21:27 maerwald It might be possible to do that for non-raw patches too, but I don't know how the UI will behave on that exactly
21:27 sm nice
21:29 sm what will you use it for ?
21:29 maerwald I think it's easier if you want to show someone a patch-set
21:29 maerwald as well as linking on issue trackers
21:30 maerwald that's what I miss a bit in darcsden... on github they show you the commits in addition to the "merge commit"
21:31 maerwald which gives you a better overview of the total changes
21:31 sm yes I can see it being good for seeing the overall effect of some patches (though not the names of which patches to merge)
21:31 sm nice
21:31 maerwald oh, you can see the patches you want to merge! darcs copies all patch descriptions
21:32 sm cool
21:33 sm when you get a chance, can you do some measurements of time/memory for  large repos (many files), old repos (many patches), diffing large/many files, diffing across many patches, etc.
21:34 maerwald snap has some sort of timeout right?
21:34 sm yes, though it's set fairly long for darcs hub to accomodate slow darcs operations
21:34 maerwald sm: did you deploy http://hub.darcs.net/maerwald/darcsden-mail-notify/patch/ca2318a3b17ccdad6c801c88592f64596a27e6b1 ?
21:34 sm maxRequestTime in darscden.conf
21:35 maerwald ah right
21:35 sm but just limiting things isn't enough of course, if we get a lot of those requests running for a minute it clogs up the works
21:35 sm maerwald: no
21:36 maerwald I am starting to think a front-proxy is really a good idea... you'd be able to control behavior more fine grained
21:36 sm yeah, we should be clear about what things are best handled there and what in darcsden
21:37 sm timeout needs to be on darcsden I think
21:37 sm at least
21:37 maerwald yeah, but something like ddos prevention or somesuch... I don't know if you can do that reasonably on darcsden level
21:40 sm well.. I think at least we should stay mindful of performance, try to avoid making it more vulnerable than it is now
21:42 sm and darcs hub is particularly easy to DDOS (unintentionally, even), since even if the proxy times out darcsden is still tied up running costly routes for a while
21:43 maerwald why is that so?
21:44 sm well imagine you (or a bot) requests 100 slow patch or repo-compare views in parallel
21:46 sm each of those requests will run till completion or (darcsden's) timeout
21:46 sm and we have to have a long timeout, since a lot of operations are quite slow
21:46 maerwald can we not set timeout depending on the handler?
21:46 maerwald I am pretty sure you can do that with an nginx front proxy
21:47 sm yes that might be possible and worthwhile
21:47 sm it sounds like a bit of a bandaid
21:48 sm I mean, it sounds potentially a bit fragile and maintenance-heavy, but it's an idea
21:48 maerwald mh, can I not do "add-sources" without a sandbox?
21:48 sm I'd love if darcs just became god-awful fast at everything :)
21:49 sm no you can't
21:50 sm but you can just cabal install your custom darcs
22:22 notdan joined #darcs
22:22 notdan joined #darcs
22:50 maerwald sm: timeout can be set by route it seems
22:51 maerwald in snap
23:53 dleverton joined #darcs
23:54 maerwald Heffalump: is it possible that WorkRepoDir does something different than withRepositoryDirectory? I get a "Not a repository" error on my server, but locally the rawpatch stuff works

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary