Perl 6 - the future is here, just unevenly distributed

IRC log for #darcs, 2015-07-04

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:21 sea-gull joined #darcs
01:46 ilbot3 joined #darcs
01:46 Topic for #darcs is now http://darcs.net/ | logs: http://irclog.perlgeek.de/darcs/ | darcs 2.10.0 is out http://darcs.net/Releases/2.10
03:38 f-a joined #darcs
03:56 mizu_no_oto joined #darcs
05:50 sea-gull joined #darcs
05:51 f-a joined #darcs
08:30 f-a joined #darcs
08:33 f-a joined #darcs
09:20 rieper joined #darcs
09:25 amgarching joined #darcs
14:03 sea-gull joined #darcs
15:15 mizu_no_oto joined #darcs
16:37 mizu_no_oto joined #darcs
17:04 mizu_no_oto joined #darcs
17:24 mizu_no_oto joined #darcs
17:40 f-a joined #darcs
17:53 amgarching joined #darcs
18:13 * Heffalump appears
18:37 notdan` joined #darcs
18:37 notdan` Heffalump: hi
18:37 Heffalump hi. I'm mostly around but might get dragged away briefly.
18:38 notdan` OK, sure. Let me outline for you the updates so far
18:38 Heffalump great
18:38 notdan` As usual, changes are to be pulled from http://hub.darcs.net/co-dan/darcsden-local and http://hub.darcs.net/co-dan/darcsden-cli
18:40 notdan` 1) Both darcsden and `den' (command line tool) use the same configuration file. Darcsden now looks for a) darcsden.conf in the current directory, b) .darcsden.conf in the home directory c) uses default settings
18:40 notdan` 2) The Darcsden-local settings have been moved to a separate module. No more messy code in Production.hs!
18:41 notdan` in general, I did a bit of the overhaul of the -local part, in preparation of it being merge into upstream
18:41 notdan` there still some issues regarding this I would like to discuss
18:42 notdan` 3) repository metadata updates
18:42 notdan` 4) http auth and all the other stuff I emailed you about
18:43 notdan` 5) ability to fork repositories and create new repositories
18:44 notdan` I think that's about it for the "features"
18:45 notdan` Oh, I've also implemented last updated dates for the repositories.
18:45 f-a left #darcs
18:45 notdan` The thing is -- this information is not actually used anywhere
18:45 notdan` this gracefully brings me to the part where I ask you a lot of questions :)
18:46 notdan` darcsden has different front pages for the regular version and for the hub version
18:46 notdan` I want to have a different front page for the -local version; in particular, I want dates to be featured there
18:46 notdan` The problem is, the "switching" between frontpages is kinda.. convoluted : http://hub.darcs.net/co-dan/darcsden-local/browse/src/DarcsDen/Handlers.hs#168
18:47 notdan` The way the -local frontpage is going to be handle will crucially depend on the way we introduce the -local branch into upstream.
18:47 notdan` What should it be? A compilation-time flag? A separate executable?
18:48 notdan` I was thinking about adding frong page info to the Settings
18:49 Heffalump I replied to the stuff you emailed about, right?
18:51 Heffalump I think -local should be a separate executable, it makes it easier for people to be able to choose how to use it later
18:52 notdan` Yeah, I got your reply; of course, there is still something to discuss. The listing of all repositories is still a big issue and I didnt manage to tackle it :(
18:53 notdan` Do you want to go over the questions one by one, or do you want me to write up a list of questions?
18:53 Heffalump whatever suits you, though you might need to keep track to make sure they all get answered
18:54 notdan` yeah, I have a small .org file with all of this stuff
18:55 * Heffalump looks at the switching code
18:55 notdan` re: separate executable; got it. will try to make it exactly that
18:55 Heffalump or a runtime parameter
18:55 Heffalump actually maybe that's better really, it's even less duplication, unless the code would be substantially different
18:56 notdan` hm yeah, sounds reasonable
18:56 Heffalump let's imagine that the hub hacks should also be runtime parameters, even if you don't change them right now
18:57 Heffalump sm: is it reasonable to aim to change -DHUB into a --hub runtime flag or similar?
18:57 notdan` I will also add the frontpage parameter to the Settings (something of the type [User bp] -> [Repository bp] -> DDXML)
18:57 Heffalump that makes sense
18:58 notdan` I have some draft changes that adds a bunch of other stuff to the local backend settings -- albeit they are not very important for the local darcsden, more interesting to the single-user gitweb-style server
18:58 notdan` customizing the name of the site, etc
18:58 notdan` OK, that's good
18:59 notdan` When renaming the repository, should we
18:59 notdan` a) fail with an error
18:59 notdan` b) silently ignore the renaming and just apply all the other changes
18:59 notdan` ?
18:59 notdan` currently I do (a)
19:00 Heffalump yes, I'd go for that. Is that something the API allows in general, i.e. you might see some new settings that changes multiple things at once?
19:01 notdan` Sorry, allows what in general?
19:02 notdan` erm
19:02 Heffalump I was wondering what you meant by "apply all the other changes"?
19:02 Heffalump do you get renames along with other changes all at once
19:02 notdan` oh
19:02 notdan` yeah
19:02 notdan` the way it works you get (RepositoryData bp)
19:02 notdan` together with a RepoRefKey which ponpoints the repository uniquely
19:03 notdan` so you get all of the new information there is to get
19:03 notdan` (sorry, didn't understand the question at first)
19:03 Heffalump ok, so yes, I think just error at that point, at least for now
19:03 amgarching joined #darcs
19:07 notdan` Another question that I had is how to add "unrecorded changes" button (or a link or whatever) in a non-ad-hoc way. But if you don't have any immediate thoughts I guess I will just get back to you with this when I have a PoC
19:08 Heffalump I think there'll always need to be a "darcsden is in local mode" flag
19:08 Heffalump so I'd find somewhere to put that and make that kind of button conditional on it
19:08 notdan` that's how you though we should present unrecorded changes -- as a "patch"
19:08 notdan` Heffalump: yeah, but I just don't want to drag a Boolean parameter throughout the whole code
19:09 notdan` a flag in the Settings? Perhaps
19:09 notdan` but then we could have conflicting flags
19:09 Heffalump yes, that was my initial though too
19:09 Heffalump how
19:09 Heffalump ?
19:09 notdan` and redundant options
19:09 notdan` well you can't run darcsden both in local mode and in hub mode
19:09 Heffalump we can refactor the settings type to be more of a discriminated union
19:09 Heffalump if necessary. But I think that could wait.
19:09 notdan` and if you are running darcsden in local mode it doesn't make sense to set up a couchdb information
19:10 notdan` OK, I will hack it up as a parameter in settings/config for now
19:10 notdan` it makes sense, especially if we want to add even more specific features like recording
19:13 notdan` Next thing: the `den' command line tool should start the darcsden server if it's not already running. How should we check if it is already running? Normally I would say have a lock file in /tmp, but I am not sure if it's portable e
19:14 Heffalump good question
19:14 Heffalump :-)
19:16 Heffalump I guess if one is running, the followon question is what do we need to know from it?
19:16 Heffalump at least the HTTP port to connect with
19:17 notdan` That + login information + the root directory
19:17 notdan` all of that can be read from the ~/.darcsden.conf
19:17 Heffalump doesn't the HTTP port need to be chosen dynamically in case of conflicts?
19:17 Heffalump If it's static, then just try to connect to it and see if you find darcsden there.
19:18 Heffalump though I guess there's the risk of someone else hijacking you then
19:18 notdan` yeah, the 'den' program should switch the port if the default one is already in use
19:18 notdan` but you are right, there are a lot of issues there: another program using the port, including a malicious one
19:18 notdan` damn, security is hard
19:19 Heffalump my gut feeling is that /dev/shm is the right thing on Linux, but it's highly unportable AFAIK
19:20 amgarchIn9 joined #darcs
19:20 Heffalump hmm, sounds like tmp files are recommended if you don't care about performance: http://superuser.com/questions/45342/when-should-i-use-dev-shm-and-when-should-i-use-tmp
19:20 Heffalump I think just go with tmp files for now, it seems good enough especially if you check ownership
19:22 notdan` ok
19:24 notdan` The authentication security is still issue on Linux, as you have mentioned yourself. Frankly, I don't know what to do with that.
19:25 notdan` There is an opportunity for a crazy hack: instead of opening http://user:pass@site, create an HTML file which redirects the user to that URL and open that HTML file itself
19:25 notdan` then delete it, presumably?
19:25 Heffalump cunning!
19:25 Heffalump I was thinking about something with cookies, but didn't know how to do it portably.
19:25 Heffalump you can just make the HTML file private to the user, then you don't care about deleting it
19:28 notdan` hm. makes sense
19:28 notdan` I will try to hack together a PoC for this one
19:28 notdan` Alright, I think I am almost done with the questions
19:28 amgarching joined #darcs
19:29 notdan` apart, of course, for the elephant in the room: terrible, terrible performance when there are a lot of directories in the root
19:29 Heffalump how many is lots and how bad is terrible?
19:30 notdan` the thing is, even if you don't attempt to list all of the repositories, when you access a specific repository darcsden has to gather all of the information about this repository - including the forks
19:30 notdan` without the use of caching (read: a separate "database") you can pretty much do it only by filtering all of the repositories and searching for forks
19:31 notdan` lots = having a cabal sandbox along the way; terrible = request times out
19:31 Heffalump can you measure the actual time (e.g. by running the code by hand) and count the folders? Would be good to get an rough idea of the scale of the problem.
19:34 notdan` ok, let me try that
19:35 notdan` btw, I've only managed to find one git client that does something similar to searching for repositories -- gitbox
19:35 notdan` i wonder how they do it
19:35 notdan` i mean the bottom line is that you have to do a recursive directory search
19:35 Heffalump right, but 'find' can do it pretty quickly, right?
19:36 Heffalump so it must be possible..
19:36 Heffalump hmm, maybe find isn't that fast
19:40 Heffalump ok, so it just took about 2 minutes on my machine
19:40 Heffalump but most interestingly, a second run took about 3 seconds - must be some serious caching in play
19:41 Heffalump and my home directory has 260K directories
19:45 alexei joined #darcs
19:47 alexei_ joined #darcs
19:48 Heffalump ok, so the directory listing problem is the worst issue. Apart from that, what's needed to get something mergeable?
19:50 notdan` So, those are my results: http://0bin.net/paste/Qf8fqy1MiA4DPCVT#SSfSlPuDoajOYeyC6HU++cw1U9CmSnnnHFp57oKqTgO
19:51 notdan` getAllRepositories does a pretty bad job (like twice as slower), but hey -- find was optimized for that stuff
19:51 notdan` so we have a room for imporvement here :]
19:52 Heffalump twice isn't that bad, and might be hard to improve on signficantly enough for it to matter
19:53 Heffalump hmm
19:53 Heffalump so I guess we can at least cache the answers in-process
19:53 Heffalump though of course in theory they can change while it's running
19:54 alexei_ joined #darcs
19:57 Heffalump or maybe we should just keep an explicit database :-(
19:57 Heffalump it could just be a single file with a list of repositories
19:57 Heffalump and an option to scan for them (which would then invoke the slow-ish code)
19:57 Heffalump scan for them and update the file that is
19:58 notdan` hm, I see
19:59 alexei_ joined #darcs
20:00 notdan` ok, so shall we postpone this particular issue?
20:02 Heffalump yeah, if you do the explicit file plus some way to generate it, that seems like a good enough substitute
20:02 Heffalump you could waste a lot of time trying to make directory scanning fast
20:03 alexei___ joined #darcs
20:06 amgarchIn9 joined #darcs
20:09 notdan` OK. I don't have any further questions at this point. Let me paste my short-term plan
20:09 notdan` 1. Merge the changes from upstream (oh boy, I can already imagine
20:09 notdan` sorting out the conflicts; that's what you get for not doing your
20:09 notdan` homework and merging early!)
20:09 notdan` 2. Extract darcsden-local into a runtime parameter (what about -DHUB?
20:10 notdan` let's hear from sm on that)
20:10 notdan` 3. Customize the frontpage business (add a frontpage of the type
20:10 notdan` `[User bp] -> [Repository bp] -> DDXML` to the settings)
20:10 notdan` 4. (Related to 2) Add "darcden-in-local-mode" flag to the settings or
20:10 notdan` somewhere like that. This will make sure that darcsden-local is
20:10 notdan` sort of integrated in the "core" of darcsden, but that's not a
20:10 notdan` problem. Based on this flag we will be displaying and turning
20:10 notdan` on/off certain features like unrecorded changes, repository
20:10 notdan` renaming, etc
20:10 notdan` 5. The `den` program should start the darcsden-local server if it's
20:10 notdan` not already running. Use a lock file in /tmp for that, or
20:10 notdan` something.
20:10 notdan` 6. The HTML file with redirect hack for `den`.
20:10 notdan` what do you think?
20:13 Heffalump few more possibles: (a) make the changes to read the repo list from a file and be able to generate it (b) integrate the 'den' program into the main repo instead of having a separate one (c) start showing what you've done to sm and discuss merge plans
20:13 Heffalump (c) is important because that kind of back-and-forth will always take longer than you think especially depending on his availability
20:15 notdan` Can I do (c) before (a) and (b)?
20:15 notdan` My reasoning is that it's better to get (a) right and that it would be easier to merge (b) once we merge the other changes
20:16 Heffalump sure
20:16 amgarchIn9 joined #darcs
20:17 notdan` great! :)
20:18 notdan` Hm, I don't have anything else off the top of my head
20:20 amgarching joined #darcs
20:20 Heffalump me neither
20:22 alexei joined #darcs
20:30 alexei_ joined #darcs
20:35 alexei_ joined #darcs
20:38 alexei___ joined #darcs
20:48 amgarchIn9 joined #darcs
20:51 amgarching joined #darcs
20:55 alexei joined #darcs
21:00 alexei joined #darcs
21:04 alexei_ joined #darcs
21:13 alexei___ joined #darcs
22:11 sea-gull joined #darcs
22:29 alexei___ joined #darcs
22:36 amgarchIn9 joined #darcs
22:40 amgarching joined #darcs
22:43 alexei joined #darcs
22:48 alexei_ joined #darcs
22:53 alexei___ joined #darcs
23:02 amgarchIn9 joined #darcs
23:12 amgarchIn9 joined #darcs
23:14 amgarching joined #darcs
23:17 alexei joined #darcs
23:20 alexei_ joined #darcs
23:49 alexei___ joined #darcs

| Channels | #darcs index | Today | | Search | Google Search | Plain-Text | summary