Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2015-05-20

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:09 josephwb joined #opentreeoflife
01:10 josephwb hey jimallman. any developments?
01:10 jimallman i think so, yes. there seems to be a mismatch…
01:10 jimallman between the branches of phylesystem-api and opentree.
01:10 josephwb ah
01:11 jimallman can you check your server-config file for devapi? is it the same as origin/master?
01:11 josephwb yes
01:11 jimallman also (very important) what branch of opentree did you have checked out during the push?
01:11 josephwb everything is master except oti and ot-base
01:11 josephwb development branch. i thought you said that did not matter
01:12 jimallman hm, that sounds right.
01:13 jimallman ah, i see the deal. we need to have opentree/support-for-tree-collections checked out during the push. this has the newest (more specific) placeholders for phylesystem-api config
01:14 jimallman PHYLESYSTEM_REPO_PATH instead of REPO_PATH, etc
01:14 josephwb ok
01:15 jimallman i’ll do a push of just the phylesystem-api from here.. that should leave oti and ot-base intact. but just to be safe, what branches are you pushing for each of those?
01:15 josephwb test-fix for both i believe. let me check tho
01:15 josephwb yes
01:16 josephwb same name for both, since they need to be implemented in tandem
01:16 jimallman awesome, thanks. i can use the same and try another round.
01:16 josephwb cool
01:17 josephwb so you will instigate a reindexing?
01:17 jimallman ah, right. yes, i can do it from here. OR, if you want to do it, just checkout opentree branch support-tree-collections, push the api, then re-index.
01:17 jimallman i’m ok either way, plenty to do here
01:18 jimallman i’ll do it, in case something goes haywire again.
01:18 josephwb would you mind doing it?
01:18 josephwb oh, good. thanks
01:31 jimallman josephwb: just fyi, re-indexing is looking good! i couldn’t find a log for this, so i’m tee’ing the operation as you suggested. nice!
01:31 * jimallman goes for a caffeinated reward
01:35 josephwb jimallman sweet. thanks for figuring out that mess. branch compatibility is a bitch
01:35 josephwb hopefully everything will work with the tree browser now
01:35 jimallman yeah, hopefully we’ll synchronize all the repos (master branches) very soon
01:36 josephwb i also find the mix of "v1", "v2", and "-0" to be confusing
01:37 jimallman i hear you. lots of moving parts, but good reasons for each.
01:39 jimallman we can add documentation, of course. the options span a few different dimensions (versions, compatibility, indexes and associated. docstores…) so it’s hard to know where to stop.
01:39 josephwb exactly
01:39 jimallman i suppose we could “grow” it as a deployment FAQ, starting with the things that confuse *us*  =)
01:39 jimallman … more good indexing results … finger crossed!
02:03 kcranstn joined #opentreeoflife
02:34 jimallman … still re-indexing oti on devapi …
02:58 kcranstn I am trying to add a new study, and it seems to be hanging
02:58 kcranstn production
03:01 kcranstn oh, finally finished
03:02 kcranstn took > 1 min for an import from treebase
03:02 kcranstn ouch
03:04 jimallman !
03:04 kcranstn (worrying a little about load for tomorrow’s workshop)
03:04 jimallman yeah, that’s painful. lots of trees, maybe?
03:05 kcranstn only one tree, <20 taxa
03:05 jimallman would it be completely weird to stagger the participants?
03:05 jimallman wow, that’s tiny.
03:05 kcranstn but it was a duplicate, so perhaps that took a while
03:05 kcranstn ?
03:06 jimallman checking for duplicates is pretty straightforward, just searches oti for the same DOI/URL
03:07 jimallman re-indexing studies on devapi is also taking a loooong time (hours)
03:09 kcranstn second try faster (not a dup)
03:09 kcranstn I think oti is a candiate for overhaul
03:10 jimallman yes, it seems we want more from it as well (collections, favorites …)
03:11 kcranstn upon first import from treebase, the study quality lists all OTUs as mapped, although none of them are mapped
03:13 kcranstn I’ll file an issue for that
03:13 jimallman when you say “lists all OTUs”, do you mean the proportion (for example, “97/99”) in the study’s list of trees?
03:13 jimallman if so, that should be a pretty quick fix (thanks for making an issue)
03:13 kcranstn when I select ‘show details’ for quality, OTU mapping = 100%
03:14 kcranstn but Mapped to taxa column empty
03:14 jimallman ok, probably just a shortcut or placeholder in the quality tool. i’ll make a note to review those.
03:15 kcranstn https://github.com/OpenTreeOfLife/opentree/issues/639
03:15 kcranstn thanks
03:15 kcranstn we may get a bunch of feedback tomorrow (I am giving them the link to the tracker)
03:17 jimallman cool!
03:31 kcranstn joined #opentreeoflife
04:16 josephwb how is indexing going jimallman?
04:45 josephwb you didn't clock out, did you jimallman?
04:46 josephwb please return your gun, badge, and neck beard
09:53 josephwb joined #opentreeoflife
11:13 kcranstn joined #opentreeoflife
11:47 kcranstn joined #opentreeoflife
12:44 jar286 joined #opentreeoflife
13:17 josephwb joined #opentreeoflife
13:39 josephwb kcranstn has anyone poked ruchi to read over the supp. material?
13:39 kcranstn I have, but I don’t think she did (also poked Gordon)
13:39 josephwb ok
13:39 josephwb she is here, yes?
13:41 kcranstn is she? I don’t know
13:42 josephwb she is giving a talk on "Tree Alignment Graphs"
13:42 kcranstn (going to take a look at recent changes and updates after I finish my workshop this afternoon)
13:42 josephwb C
13:42 josephwb 1:35
13:42 josephwb Tree inference
13:42 josephwb Ruchi Chaudhary
13:42 josephwb Constructing and Employing Tree Alignment Graphs for Phylogenetic Synthesis
13:42 kcranstn great that Ruchi is here - I’d like her to look over the docs
13:43 josephwb yes!
13:43 josephwb fresh eyes would be great
13:44 josephwb and she is the best one to do a once-over
13:58 kcranstn I’ve added some documentation on editing studies in advance of the workshop this afternoon: https://github.com/OpenTreeOfLife/opentree/wiki/Editing-studies
13:58 kcranstn Comments welcome
13:58 * jimallman is reviewing this..
14:02 kcranstn just fixed the missing add tree image
14:02 kcranstn (can’t seem to resize this)
14:03 jar286 joined #opentreeoflife
14:13 josephwb jimallman new ot-base is deployed on dev
14:22 jimallman josephwb: is there a new oti built on this?
14:23 jimallman kcranstn: was that a screenshot from a retina display? i usually drag the window to a normal display and capture again.
14:24 jimallman if that’s not convenient, i’ll see if i can resize this in the wiki page
14:26 jimallman kcranstn: would you mind if i make simple edits to the wiki page (typos)?
14:27 kcranstn yes, please fix typos
14:27 kcranstn (does not have non-retina display)
14:28 jimallman #firstworldproblems
14:29 josephwb jimallman no
14:29 josephwb well… i don't know
14:29 josephwb i was in screen, and things zipped by, and i cannot scroll up to check
14:29 jimallman josephwb: i would hope that we’d catch that dependency and recompile oti, but i’m not sure if that’s the case.
14:30 jimallman i can try a “clean” re-indexing (per jar’s guidelines) and see what we get…
14:30 josephwb i will do it explicitly with the "-f"
14:30 jimallman cool
14:30 josephwb oh, yeah, do that
14:30 jimallman so clean re-index first, as a test?
14:31 josephwb let me force a compile first
14:31 josephwb boom. done.
14:32 jimallman cool. i’m pushing install-db (to clear the oti data), then indexoti…
14:34 josephwb great
14:34 josephwb hope this works
14:34 josephwb so weird that it works for most studies, but not a handful...
14:34 jimallman kcranstn: just one comment, look for ‘jimA’ in the wiki page
14:35 jimallman josephwb: i suspect there’s something fishy in the way we validate “observed” versus explicit root node in each tree. that’s where the traceback sent us last time.
14:35 jimallman not sure whether that’s just one study, or more
14:35 josephwb right
14:36 kcranstn good point, jimallman. I’ll add that to the otu mapping section.
14:37 jimallman kcranstn: by the way, i don’t know the answer to that question (whether we need all tips mapped for synthesis, or there’s a threshold below which a tree is rejected)
14:40 josephwb jimallman cody reports that error messages are now returned (through curl, at least)
14:41 jimallman yes, which is great. i’m still inclined to clear the database to see if re-indexing happens more quickly. i forgot that i need the path to a remote .tgz file to run install-db… do you have your most recent filepath handy?
14:42 josephwb filepath? to what?
14:42 jimallman like this:   ./push.sh -c ../../deployed-systems/development/devapi.config install-db downloads/treemachine.db.tgz treemachine
14:43 josephwb oh
14:43 jimallman the last part is the filesystem path on devapi.. does this look like the latest?
14:43 josephwb idunno?
14:43 jimallman ok, checking on ot10 (devapi)…
14:43 josephwb i've only ever pushed a local db
14:44 jimallman right, this is the next step (install-db), which i believe expects the file to already be on the server
14:44 josephwb i;ve done "pushdb", but not "install-db"
14:44 josephwb sorry
14:45 josephwb but i think you want the taxomachine db, right?
14:45 josephwb not treemachine
14:46 josephwb ./push.sh -c ../../deployed-systems/development/devapi.config install-db downloads/taxomachine.db.tgz oti
14:46 josephwb i think that is what you want
14:47 jimallman thanks. i see that this filename matches the most recent on devapi, from Sept 8
14:47 josephwb yes, i see that too
14:47 jimallman ok, it’s chugging now…
14:48 josephwb wee
14:48 * jimallman will update here when Something Interesting Happens
14:54 jar286 joined #opentreeoflife
14:57 jimallman kcranstn: i’ve resized the Add Tree image using inline HTML (kludge, but it works)
14:58 kcranstn nice, thanks
15:20 * jimallman reports: install-db is still unpacking taxomachine.db.tgz…
15:20 jar286 oh yeah, that can take a while, I didn’t include that time in my <30 min estimate.  let me see if i recorded it
15:21 jar286 7 minutes 46 seconds
15:21 jimallman whoa
15:21 jar286 not too bad I think, given what a hog neo4j is
15:22 jar286 my shell transcript is 9 megabytes
15:22 jimallman surely we can prepend a step to the indexoti command that clears the db, right? if so, i’ll make an issue for this
15:23 jar286 hmm.
15:23 jar286 I don’t see why not
15:29 kcranstn joined #opentreeoflife
15:40 jimallman jar286: install-db is done already (lowered expectations!), starting indexoti…
15:41 jar286 it took more than 8 minutes?  the time I gave was for api, not devapi, but if anything devapi should be faster
15:44 jimallman ah, i misread your report as 8 HOURS, not minutes. it probably finished some time ago, i was working elsewhere.
15:49 josephwb links are broken here: http://opentree.wikispaces.com
15:52 jimallman hm, not sure where to file an issue for these. maybe in feedback repo? or the main opentree repo?
15:53 jimallman josephwb: new indexing is chugging along at a good clip so far. i’ve seen one NexsonParseException, with the message nicely spelled out. progress!
15:53 jimallman (error was our old friend specified-vs-observed-root!)
16:16 josephwb joined #opentreeoflife
16:22 jimallman josephwb, jar286: ok, we’re back to eight indexing failures, but with better explanations. All errors are the same, a mismatch between the tree’s specified and “observed” root nodes.
16:22 jimallman so either we’re making bad trees, or the root-node test is bogus.
16:30 jimallman details added to https://github.com/OpenTreeOfLife/oti/issues/40
16:39 josephwb joined #opentreeoflife
18:09 jar286 joined #opentreeoflife
18:36 mtholder joined #opentreeoflife
18:55 mtholder jimallman, are you seeing the stream of curation commits. That is nice to see.
18:56 jimallman yes, thanks for following up on these.
18:56 jimallman it is good stuff.
18:58 josephwb yeah, more contexts would be good for restricting mapping
18:59 josephwb would be nice if you could provide a name/id instead of picking from a list
18:59 josephwb b/c the list could get very long
19:00 jimallman sadly, i recall jar286 saying these search contexts are specially generated and not equivalent to clades.
19:01 josephwb right, i think taxomachine has these hardcoded
19:10 kcranstn joined #opentreeoflife
19:10 kcranstn FYI, no technical issues with curation app demo at workshop
19:10 kcranstn yay
19:25 mtholder joined #opentreeoflife
19:26 mtholder kcranstn, Paul reported an issue. but I'm glad that it went relatively smoothly.
19:27 kcranstn so did louise ;)
19:28 mtholder yes. hers shouldn't require serious code changes, (I hope)
19:30 kcranstn his may have to do with an incomplete reference string in treebase
19:32 mtholder yeah. he mentioned that. It looked to me that it was crashing in the way it was calling the phylesystem-api. But I have not run it on a local instance to get verbose logging.
19:32 mtholder I'm afraid that I'm not likely to have time to do that soon.
19:33 kcranstn he didn’t seem to be raging about it ;)
19:33 mtholder he's got a temper ;-)
19:37 mtholder left #opentreeoflife

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary