# IRC log for #opentreeoflife, 2015-05-04

All times shown according to UTC.

Time Nick Message
09:34 jar286 joined #opentreeoflife
10:12 mtholder joined #opentreeoflife
10:36 jar286 joined #opentreeoflife
10:57 mtholder joined #opentreeoflife
11:36 jar286 joined #opentreeoflife
12:04 jar286 joined #opentreeoflife
13:14 kcranstn joined #opentreeoflife
13:45 mtholder David Bryant pointed me to http://epubs.siam.org/doi/abs/10.1137/100811489 which could be useful for solving our supertree subproblems
13:46 mtholder That paper requires a set of consistent splits, and returns a minimally resolved supertree. Not the same as our "no unsupported groups" rule.
13:46 mtholder but related.
13:46 mtholder kcranston, jar286 ^
13:46 kcranstn we haven’t been aiming for minimally resolved, though
13:46 kcranstn (thanks for the paper, though)
13:47 kcranstn I need to do much reading
13:47 mtholder sort of. minimally resolved among the set of trees that display all of the groupings.
13:47 mtholder an alternative formulation of the "no extra groupings" rule.
13:47 kcranstn ah, I get it
13:48 kcranstn not “collapse into polytomies when there is conflict'
13:48 jar286 minimally resolved sounds right to me, but you need a counterbalance to prevent consensus
13:48 jar286 right. consensus = collapse.
13:48 jar286 IIUC.
13:48 mtholder their work requires a set of consistent splits.
13:48 mtholder soe each candidate should show all of the splits.
13:48 mtholder but not extras.
13:49 mtholder the BUILD algorithm of Aho et al sometime shows more than the minimal number of internal nodes required to display all of the splits.
13:50 jar286 a consensus tree will show all consistent splits, right? isn’t that almost the definition of consensus?
13:50 mtholder consensus is usually used when all of the inputs have the same leaf label set.
13:51 mtholder but yes a semistrict consensus will show all of the splits that aren't contested by the inputs.
13:51 mtholder (strict shows all that are common to all inputs)
13:53 jar286 if the question is what are we optimizing, I see it as a balance between two forces: information and consistency
13:53 jar286 at the consistency extreme we have consensus trees, but those are judged to not have enough information
13:54 jar286 this is not my area, but that’s how i’ve been making sense of the problem
13:54 mtholder yeah. it just takes one crazy tree to create huge polytomies.
13:54 mtholder (in strict consensus methods)
13:56 mtholder fwiw I added a section 1.3.1 and 1.4.2 to http://phylo.bio.ku.edu/ot/summarizing-taxonomy-plus-trees.pdf after having read the paper linked to above.
13:57 kcranstn wow, I didn’t realize this doc was do extensive!
13:57 kcranstn so
13:59 mtholder back when my memory worked well, I wrote less. Sadly, that is no longer the case...
13:59 kcranstn much better for us, though ;)
14:02 blackrim joined #opentreeoflife
14:08 blackrim left #opentreeoflife
15:02 kcranstn joined #opentreeoflife
15:03 mtholder left #opentreeoflife
15:10 kcranstn it feels like the “comparison to previous work” needs to include more than just other TAG-based methods but also other supertree methods
15:51 kcranstn joined #opentreeoflife
16:00 mtholder joined #opentreeoflife
16:21 kcranstn joined #opentreeoflife
16:42 josephwb joined #opentreeoflife
16:42 josephwb hey jimallman
16:42 jimallman howdy
16:43 josephwb do i push treemachine code to devapi, or devtree?
16:43 josephwb the former, right?
16:44 jimallman devapi
16:44 josephwb ok, thanks
16:44 jimallman devtree generally just gets opentree (see the default COMPONENTS listed in each server-config file)
16:44 josephwb can we rename devtree to something else?
16:44 josephwb confusing
16:45 josephwb is it okay to push now?
16:46 jimallman ok by me. i’ll need to merge some of my recent work into development, so will push again later today.
16:46 jimallman (development branch, that is)
16:46 josephwb thanks
17:14 mtholder joined #opentreeoflife
17:15 jar286 josephwb, as I said in my email, please use the development version of the opentree repo to get ‘push.sh’ and friends
17:20 josephwb jar286 i must have missed that one
17:21 jar286 I will push out the right version to master when the code has had review
17:21 jar286 I don’t think it will make a difference, but not completely sure
17:21 josephwb did i do something wrong?
17:21 jar286 see above
17:22 josephwb "development version of the opentree repo"?
17:22 josephwb a branch?
17:22 josephwb ok, i think i see it
17:22 jar286 branch, yes
17:23 jar286 just ‘git checkout development'
17:23 josephwb right
17:23 jar286 after ‘git fetch’ of course
17:23 josephwb didn't know it was there
17:23 josephwb will do
17:24 kcranstn joined #opentreeoflife
17:40 josephwb joined #opentreeoflife
17:40 jar286 josephwb, there is one thing you have to do… hang on
17:40 josephwb ok
17:41 jar286 it’s in my 2nd email in thread ‘devapi upgrade this weekend’
17:41 jar286 from May 2
17:41 josephwb i see it
17:42 josephwb refreshing deployed_systems
17:42 josephwb ?
17:42 josephwb yes, i did that
17:42 jar286 ok
17:59 kcranstn mtholder?
18:00 mtholder give me 5 minutes.
18:00 kcranstn just wanted to let you know that I moved that entire preprocessing / subproblem section up in the doc. Decided that we needed the detail. So, don’t panic if you see it is gone where you originally put it.
18:01 mtholder ok
18:31 jimallman josephwb: fyi, i just pushed some changes to deployed-systems.. some preparatory stuff for tree collections, should have no ill effects on your deployment.
18:32 jimallman but it expects to find key ~/.ssh/opentree/opentreeapi-gh.pem, so make sure you have that.
18:33 josephwb hmm, i don't think that i do
18:33 josephwb nope
18:33 josephwb only opentree.pem
18:33 jimallman ok, i’ll find where it’s deployed on devapi, and under what name… stand by...
18:34 josephwb thanks
18:35 jimallman it’s pushed to the server and saved as /home/opentree/.ssh/opentree (a terrible name, but that’s for another day)
18:36 josephwb ok
18:36 jimallman so try this to copy it from ot10 (devapi) to your local stash:scp ot10:/home/opentree/.ssh/opentree ~/.ssh/opentree/opentreeapi-gh.pem
18:36 jimallman (command starts \$ scp...)
18:38 josephwb got it
18:43 jimallman Added this one-liner as a comment on the deployed-systems commit: https://github.com/OpenTreeOfLife/deployed-systems/commit/86ade9f0b8c228eff95338d8214c1ea98cc88dd6#commitcomment-11030836
18:48 kcranstn joined #opentreeoflife
19:16 jimallman josephwb: the arguson on devtree looks the same as ever, at least in the structure of pathToRoot and sourceToMetaMap. can we deploy the JSON-ized version of this to treemachine?
19:16 josephwb i see that
19:16 josephwb i don't know what is going on
19:17 jimallman keep in mind that we deploy treemachine’s master branch by default, so whoever did the latest deployment would need to have specified another branch in devapi.config
19:17 jimallman i haven’t pushed today, fwiw
19:17 josephwb ok
19:18 josephwb there may be a branch problem. looking into it now
19:18 josephwb [it was working earlier. i may have broke it]
19:20 jimallman hm, i see that the treemachine repo on devapi shows itself as on branch ‘rootward-synth’
19:21 josephwb that is correct
19:21 josephwb the plugin code got swapped from another branch. fixing now
19:22 mtholder left #opentreeoflife
19:36 josephwb jimallman if you leave the treeID arg empty, it works.
19:36 josephwb otherwise, weirdness
19:36 jimallman ! seriously?
19:36 * jimallman is trying this now
19:36 josephwb no, there was indeed a major messup with versions
19:37 josephwb versions = code across branches
19:38 josephwb try this:
19:38 josephwb curl -X POST 'https://devapi.opentreeoflife.org/cached/treemachine/v1/getSyntheticTree' -H 'Content-Type: application/json' --data-binary '{"format":"arguson","maxDepth":"3","subtreeNodeID":"792659"}'
19:38 jimallman ah it’s the caching layer
19:38 josephwb oh
19:38 jimallman note cached/ in the URL above. omit it to see fresh (JSON-ized) arguson in all cases
19:39 josephwb crap
19:39 josephwb ok, still works
19:39 josephwb *phew*
19:39 jimallman no problem, but this should been cleared when the websrver restarted
19:39 josephwb something weird going on with id matching
19:39 jimallman ?
19:39 josephwb treeid matching (in the plugin)
19:40 josephwb i.e. why you need to leave it blank for the moment
19:40 josephwb oh, no, it works!
19:40 josephwb *i was testing against the cached version…*
19:41 josephwb that was a head-scratcher
19:41 josephwb i couldn't understand why it was returning results from the old tree name!
19:41 josephwb and new error handling was not being dsiplayed
19:42 josephwb yeesh
19:42 jimallman caching is based on incoming arguments. so any change in args will get a fresh result (or cached arguson for thos args)
19:42 * jimallman is looking now to see when/how we clear this cache…
19:43 josephwb i don't know how that made it into my curl call. copy-paste, i imagine
19:49 josephwb SO glad you noticed that jimallman
19:50 jimallman yeah, you can lose a day to something like that
19:50 josephwb will work if either 1) you provide a valid treeid or 2) leave it blank (will use most recent)
19:50 josephwb otherwise i put in an error message:
19:50 josephwb {
19:50 josephwb "error" : "Unrecognized \"tree_id\" argument. Leave blank to default to the current synthetic tree."
19:50 josephwb }
19:50 jimallman i’m going to clear the web2py (RAM) cache with an apache restart…  OK?
19:51 jimallman i like that error message, yes
19:51 josephwb yup
19:51 josephwb same message as other plugins
19:51 jimallman and behavior makes sense to me, unless others find it too magical (might prefer explicit tree id)
19:51 josephwb let me know what you want
19:51 jimallman here’s (fyi) how to restart remote apache on ot10…
19:51 jimallman \$ ssh adminot10 sudo apache2ctl graceful
19:52 jimallman new arguson, very nice!
19:52 jimallman i’ll get cracking on the client-side support for this
19:52 josephwb not too verbose, is it?
19:53 josephwb the pathToRoot bit
19:53 jimallman looks good to me. i’m leaning more toward clear/explicit ids these days…
19:53 jimallman oh, pathToRoot…
19:53 josephwb gives more than the 3 or 4 levels you need
19:53 jimallman no, this looks right to me as well.
19:54 jimallman wow, lots of supporting sources where i’m looking. i’ll keep an eye out to make sure we’re not capturing too many here.
19:55 jimallman {treeID: "opentree3.0", format: "arguson", maxDepth: "3", subtreeNodeID: "3878110"}
19:55 jimallman (it’s probably correct as-is, just being paranoid)
19:57 mtholder joined #opentreeoflife
19:57 josephwb it collects all of the supporting sources from the pathToRoot
19:57 josephwb it is correct as far as i can tell
19:58 jimallman cool, i’ll get cooking on the GUI. looks like i have everything i need, thanks!
19:59 josephwb a new db will be coming down the line very soon
19:59 jimallman no problem, but i’ll brace for (possibly) shifting data
20:00 josephwb cannot give you an ETA, but it will get pushed automatically when it is ready
20:02 josephwb are you there jar286?
20:02 jar286 I’m here
20:02 josephwb regarding the new push to dev
20:02 josephwb does that apply to dbs too?
20:03 josephwb wait, it should be ok
20:03 josephwb i started the analysis a while ago, but paths are the same so it should be fine
20:03 jar286 yes, I’m pretty sure it doesn’t matter.  if the site works then I doubt anything’s amiss (if it fails it fails big)
20:04 jimallman woohoo! total failure!!
20:04 josephwb nah, we should be good; different branch, but same file structure
20:06 jar286 jimallman, you can also restart apache with push.sh -c … apache
20:06 jimallman ah, true.
20:07 jimallman i’m trying to practice remote execusion via ssh. it’s great for tail’ing multiple log files, too!
20:08 jar286 yep.
20:10 jimallman re: deployment, i’m trying to re-impose our normal discipline and push only the ‘development’ branch of opentree. it’s less clear how to handle the other repos (no ‘development’ branch for these), but so far i’ve been able to push+test alternate branches as needed. luckily, all my recent work in these has been backward-compatible.
20:11 jar286 I’ve been counting on that - I merged my two PRs to development, and they’re mildly important (not sure how important)
20:11 jar286 I think the current practice is fine, using development only for opentree repo
20:12 jimallman sounds good. when in doubt (confusing behavior on devgtree or devapi), we should just check for currently deployed branches…
20:13 jar286 I hate to give you more things to do, but I do have two PRs that should be reviewed ASAP.  or maybe I can ask mtholder?
20:14 mtholder which ones?
20:15 mtholder oh i see them.
20:15 jar286 opentree 632 and 631
20:16 jar286 the ‘files changed’ tab doesn’t show that the three .conf files are clones of existing files
20:16 jar286 so it’s not really all that much new code
20:17 jar286 I could have managed the change better to make it easier to review
20:17 jimallman one more pair of eyes here… #631 looks pretty thorough!
20:17 jar286 to do due diligence one would need to diff apache-vhost-conf opentree.conf (and similarly for the other two)
20:18 jar286 yes, #631 is pretty simple.  the new script is not used except manually, so doesn’t really matter much
20:18 jimallman checking to see if we disable/hide the unstable repo when it’s not wanted… jar, how is this handled?
20:19 mtholder that is the one that i've been lookt at (#631)
20:19 jar286 there’s a thing called ‘update-java-alternatives’ but I couldn’t find an easy way to install it
20:19 jimallman any chance of unstable packages accidentally being used in place of stable?
20:19 jar286 basically the sources.list file has to be manipulated manually
20:20 jar286 no chance.  note prority 10, which is < 100 = installed
20:20 mtholder it looks fine to me, though I doubt that I'd spot anything...
20:21 jimallman ah, got it. (i’m more familiar with yum, sorry)
20:21 mtholder jimallman are you still looking at it, or do you OK if I merge.
20:21 jimallman fine by me!
20:21 jar286 we could do something fancy with updating sources.list I suppose… but nothing we do will be stable over time
20:21 mtholder #631 merged.
20:21 jar286 I’m hoping the unstable business will be replaced soon by stretch-backports
20:21 jar286 thanks
20:21 mtholder looking at #632
20:22 jar286 I guess I could have left a better trail
20:23 jar286 oops, I see a mistake
20:23 jar286 extraneous ‘shared’ in doc at https://github.com/OpenTreeOfLife/opentree/pull/632/files#diff-bcdcc71d7dab57472393a413b9f8293dR165
20:24 jar286 may as well fix the hyperlink while I’m at it
20:26 jar286 oh no, the -shared is not extraneous. but the hyperlink is still busted
20:26 * jimallman is confused. doesn’t “shared” belong in both places here? or have you moved the directive in question to another config file?
20:26 jimallman ok, i think we’re on the same page.
20:27 jimallman hyperlink is working for me, assuming you mean this one:
20:27 jimallman https://github.com/OpenTreeOfLife/opentree/blob/d35768d9a2233908a1982846870cf57326450525/deploy/setup/apache-config-shared#L15-L16
20:27 jar286 already got it, but thanks
20:27 jimallman ok
20:28 jimallman so we currently have two sets of apache config files, one for apache 2.2 and another for 2.4? how long should we keep both around?
20:29 jar286 yes. we flush the 2.2 versions as soon as all servers have been upgraded to jessie
20:29 jar286 then we make jessie a requirement for anyone trying to use this stuff
20:30 jimallman looks like debian jessie was released just last month. still, it seems like a reasonable requirement.
20:30 jimallman stable is stable
20:35 jar286 just pushed a little commit with doc fix & comments.
20:38 jimallman i like the detailed history of each config file. but i’ll like it more to toss the old (apache 2.2) versions… the renamed files are confusing.
20:39 jimallman #632 looks pretty good to me
20:39 jar286 I wanted the name to be the same as it appears in sites-enabled and sites-available
20:39 jar286 the sites-enabled name has to end in .conf
20:40 jar286 and the name should distinguish the conf file from that of other vhosts running on the same server
20:40 jar286 (e.g. ot10 has two other vhosts)
20:41 mtholder looks good to  me too. thanks jimallman.
20:41 jar286 so the name should have meaning outside of the opentree context i.e. should not depend on knowing that you’re dealing with opentree
20:41 jimallman i see what you mean, jar286. but to someone trying to understand or change deployment, it seemed useful to distinguish between apache config files and all the others.
20:42 jar286 hmm.  we make a subdirectory
20:42 jar286 would that work?
20:42 jar286 setup/apache/opentree.conf
20:43 jimallman yes, i like it!
20:43 jar286 I’ll make an issue and do it later…
20:43 jimallman on a similar note, i loathe our use of ~/.ssh/opentree   for one particular private key. so confusing…
20:43 jar286 totally agree. i’ve almost changed it more than once
20:44 jimallman i much prefer the original name of the current file: ~/.ssh/opentree/opentreeapi-gh.pem  , which at least hints at its origin and purpose, but could probably still be better.
20:45 * jimallman is making a ticket for this as well, in opentree...
20:45 jar286 I think ~/.ssh/opentree was the ‘original’ - a Duke legacy
20:45 jimallman yes, it’s been around for quite awhile
20:45 jimallman lots of keys now, and subtle policy reasons for each. i narrowly averted adding (many) more in adding tree collections
20:47 jar286 ok, thanks for the review
21:10 josephwb joined #opentreeoflife
22:32 jimallman josephwb: i’ve added a trigger to the push scripts to restart apache when re-deploying either treemachien or taxomachine:  https://github.com/OpenTreeOfLife/opentree/commit/4a0ffad5bfffa351e47353a8733b36deca65ef47
22:33 jimallman this should save some confusion in the future!
22:35 josephwb thanks jimallman
23:07 kcranstn joined #opentreeoflife
23:59 kcranstn joined #opentreeoflife