Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2014-12-11

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:42 josephwb joined #opentreeoflife
02:31 towodo joined #opentreeoflife
04:50 mtholder joined #opentreeoflife
04:52 mtholder jimallman I was about the redeploy phylesystem-api to api.opentreeoflife.org is that OK with you? (it seems broken based on joseph's report).
04:52 jimallman fine by me. i sent some email trying to track down the actual problem...
04:53 jimallman something wrong with _study2shard_map ?
04:53 mtholder i just responded. your emails were quite helpful, thanks.
04:53 jimallman ah, cool!
04:54 mtholder I thought I was careful with _study2shard_map, but perhaps not careful enough...
04:54 jimallman ah, that’s interesting (your email). i’m surprised this hasn’t bitten us before now!
04:54 mtholder It could be something weird like a "delete study" never making it to master (but removing the entry from _study2shard_map)
05:02 mtholder jimallman: I haven't deployed to production in a while. Should I wait for jar to do it? I can't remember if we decided that only he would do that?
05:03 jimallman hrmmm, it’s been awhile.
05:03 jimallman i don’t recall the current protocol, but i believe we’re trying to have at least a second pair of eyes on anything before it goes to production.
05:04 jimallman if JWB isn’t hovering over his keyboard waiting, it might be best to send an email and let towodo handle this in the morning.
05:13 mtholder yeah. I'll wait. there is a workaround (given that the data is available via git)...
05:13 jimallman any way to “goose” the API server short of pushing new code?
05:41 mtholder sorry  missed your question. was looking at the code. I think I found the issue and submitted a PR.
05:41 mtholder jimallman^
05:42 jimallman kewl
05:42 mtholder it was at the beautiful intersection of thread-safety and exception-safety...
05:42 mtholder (which would explain why we don't see it often)
05:42 jimallman :D
05:43 mtholder strange that it seems to have hosed the map for the latest studies only - but I'm not sure what to expect...
05:43 jimallman that does seem fishy...
05:44 jimallman so creating a new study triggers some kind of refresh of this map?
05:44 jimallman how / when is ot_212 first added?
05:44 mtholder non-thread safe access to a dict could result in a dangling ref to an old version of the dict.
05:44 jimallman ah, gotcha
05:44 mtholder yes. ingesting a study or deleting a study updates the map.
05:45 mtholder I should check how this interacts with "merging to master"
05:45 mtholder perhaps, i should never delete entries from the dict.
05:46 jimallman hmm. i assume the map-dict is oblivious to git’s branches (incl. WIP branches)?
05:47 mtholder yes. since studies don't move it works. but it might not return an old version of a deleted study.
05:47 jimallman since a WIP branch presupposes the existence of a study in master. but what if a study is deleted in a WIP branch, while someone else has modified the study?
05:48 jimallman i guess that’s a bigger problem, but we wouldn’t want to remove the study from the map unless/until the deletion had been merged to master
05:49 mtholder I'm pretty sure that I thought about that  - can't remember if I had a foolproof solution...
05:49 * jimallman is way down a rabbit hole of resources vs. representations vs. media types… interesting!
05:53 mtholder There may be a bug in the case that you mention - study deletion that then fails to merge to master.
05:53 jimallman i’m on fire!
05:56 mtholder indeed. thanks.
05:57 mtholder I think we just need a set of active studies and the map of id 2 filepath.
05:57 mtholder I shouldn't be using the same struct for both concepts.
05:58 * jimallman nods
05:58 jimallman so entries in id2filepath would stick around, to play it safe if their fate is uncertain?
05:58 mtholder we can always check for whether a study exists in a version of the repo using a test for the existence of the json on the filesystem.
05:58 mtholder yes.
05:58 jimallman makes sense
05:59 mtholder and we'd have an "active on master" set so we can have a quick list of studies currently in the system.
05:59 jimallman yep
05:59 mtholder I may just implement a fall back: if the study ID is not in the map, look for it where it should be.
06:00 mtholder that will mean that we don't have to retain the full history of deleted studies.
06:00 jimallman if it’s not on master, it’s not real (sorta).. but is it real for the user who owns the WIP branch? in other words, will they be locked out of a study that “exists for them”?
06:00 jimallman or vice versa?
06:01 jimallman (i can’t remember, but i thought we transparently switch to WIP branch for the person who owns it..)
06:01 jimallman if i’m mis-remembering, pls ignore
06:02 mtholder Anyone can still get that commit because we return a list of SHA's for all WIPs for each study.
06:03 mtholder I should just make sure that phylesystem does a checkout of an old commit (if someone requests an old commit) and then check for the existence of the file in that version of the repo.
06:04 mtholder instead of bailing out when the study ID is not recognized.
06:05 jimallman ooh, right
07:31 mtholder joined #opentreeoflife
10:19 mtholder joined #opentreeoflife
11:26 mtholder joined #opentreeoflife
13:30 mtholder joined #opentreeoflife
14:11 towodo joined #opentreeoflife
15:26 towodo_ joined #opentreeoflife
16:37 towodo_ joined #opentreeoflife
17:03 towodo joined #opentreeoflife
17:48 mtholder joined #opentreeoflife
19:32 towodo oops!  it’s PR time!  there are 2
19:32 towodo jimallman ^
19:32 jimallman hi.
19:33 towodo https://github.com/OpenTreeOfLife/peyotl/pull/74
19:33 jimallman (the 2 old opentree PRs are still parked, pending more work on my part)
19:33 towodo right
19:34 * jimallman is reading #74 (the single-tree study fetch) now…
19:34 towodo re peyotl #74, Mark asked KC for review, and she said “it seems safe to merge and then re-evaluate”
19:34 towodo I say merge
19:35 jimallman sounds good to me
19:35 towodo merged.
19:35 towodo https://github.com/OpenTreeOfLife/peyotl/pull/89
19:35 jimallman ah yes, from last night
19:37 jimallman reviewing the changes as a sanity check, but i’m pretty sure this was tested last night and is ready…
19:37 towodo The testing setup makes me nervous… although with Mark doing it I’m sure it’s ok… look at maintainer/ directory
19:39 towodo it would be nice if we could configure phylesystem for local tests (non github)
19:39 towodo I’ll make an issue for that
19:39 * jimallman is reading this now...
19:40 jimallman http://opentreeoflife.github.io/peyotl/maintainer/ describes testing with local cloned repos… the new branch must have changed..?
19:41 towodo https://github.com/OpenTreeOfLife/peyotl/issues/91
19:42 towodo hmm, maybe I was unclear
19:43 towodo updated comment to say ‘with a locally hosted repo (or set of repos) taking the place of the repo on Github’
19:43 jimallman i think i see what you mean (test against a local “upstream” phylesystem, versus the remote on GitHub)
19:43 jimallman so more of a general peyotl suggestion, vs about this particular PR..?
19:44 towodo yes.
19:44 towodo i think we can go ahead and merge, since his tests sound pretty careful
19:44 * jimallman nods
19:45 towodo confirmed.
19:46 towodo ok I think we’re done
19:46 jimallman yep
19:46 towodo thanks
19:47 jimallman you’re quite welcome! i’ll do some smoke testing on production in a bit..
19:47 towodo I updated it earlier today, won’t do so again until maybe saturday
19:48 jimallman ah, ok.
20:28 towodo jimallman, I plan to reset devtree/devapi to master on saturday.  all components other than ‘opentree’ are already on the master branch (at various commits).  but opentree is on ‘tree-conflict-view’.  so you will have to redeploy next week if you want to continue playing with that branch
20:29 jimallman understood. feel free to move this to master,
20:47 scrollback joined #opentreeoflife

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary