Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2015-05-08

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:09 jar286 joined #opentreeoflife
01:20 kcranstn joined #opentreeoflife
01:31 jar286 joined #opentreeoflife
01:46 mark-holder-kiwi joined #opentreeoflife
01:48 ilbot3 joined #opentreeoflife
01:48 Topic for #opentreeoflife is now Open Tree Of Life | opentreeoflife.org | github.com/opentreeoflife | http://irclog.perlgeek.de/opentreeoflife/today
01:48 kcranstn wheeeeee!
01:51 mark-holder-kiwi having fun?
01:52 kcranstn can you take a look at cody’s comments on step 4? (taxonomy pruning)
01:58 mark-holder-kiwi I added some comments.
02:17 mark-after-kiwii joined #opentreeoflife
03:48 kcranstn joined #opentreeoflife
08:07 snacktavish joined #opentreeoflife
08:12 snacktavish joined #opentreeoflife
09:09 snacktavish joined #opentreeoflife
11:23 kcranstn joined #opentreeoflife
11:52 snacktavish joined #opentreeoflife
12:29 mtholder joined #opentreeoflife
12:36 jar286 joined #opentreeoflife
12:53 blackrim joined #opentreeoflife
13:02 blackrim mtholder, first off, how are you doing? sounds like you are a bit sick? secondably, so are we holding off on edits to the supp for now?
13:03 mtholder I'm OK-ish. Thanks. i think that it is fine to edit it. AFAIK
13:03 mtholder I'm not 100%, but feeling well enough to code while drinking a beer on my balcony.
13:06 blackrim ok, sorry to hear that. hope you feel better
13:06 blackrim oh yeah, i have been meaning to ask you  yeah and when i was in england, I heard about derrick. crap. are things getting any better there?
13:19 blackrim sorry, moved to email.
13:52 kcranstn joined #opentreeoflife
14:00 codiferous joined #opentreeoflife
14:03 codiferous anybody there?
14:03 blackrim yeah
14:04 codiferous how can i help improve understanding on the doc? currently i am working on resolving comments and approving changes
14:13 blackrim I am working on a diagram example that is in the PNAS_revision folder. will poke when it is done.
14:17 josephwb joined #opentreeoflife
14:39 josephwb hey mtholder. not sure why travis fails on otcetera.
15:07 mtholder josephwb I've been meaning to look at that.
15:07 mtholder travis had some problems last week I think.
15:07 mtholder so I started ignoring it.
15:07 josephwb ok. my edits were just getting rid of spaces in the output
15:07 mtholder I tweaked my reconf when I updated clang. That is probably the culprit.
15:08 josephwb i find that clang is often the culprit
15:10 mtholder I think that I just didn't tell travis to install the version of clang that I use.
15:19 blackrim ok, codiferous and josephwb , you want to see if the tag_supp figure looks like you would expect?
15:19 blackrim could be prettier but just seeing if this would be helpful
15:19 codiferous ok, i have cleaned up the doc. i don't think it is that far away from being acceptable for resubmission.  i will be back on this afternoon if anybody has questions I might be able to help with
15:20 codiferous is the figure in the supp?
15:20 blackrim here is that https://drive.google.com/open?id=1NTYBTGXHZF9x0hzWh0k88KV5_pKUd9SB6j9XfoThBWY&authuser=0
15:20 codiferous yes, that looks helpful
15:22 josephwb fixed a typo. looks good.
15:37 blackrim i don't know what others think but feel free to do what you like. i can add it to the doc but perhaps it isn't helpful
15:43 codiferous joined #opentreeoflife
15:46 josephwb if you are around mtholder i sent you an email
16:05 blackrim have to go to a meeting and some lunch but will be around later
16:49 jar286 joined #opentreeoflife
16:58 codiferous joined #opentreeoflife
17:04 mtholder joined #opentreeoflife
17:30 jar286 joined #opentreeoflife
17:33 josephwb jar286 you want me to submit a treemachine PR?
17:34 kcranstn joined #opentreeoflife
17:36 mtholder joined #opentreeoflife
17:36 josephwb hi mtholder
17:36 mtholder hi
17:37 josephwb results look good, except i am unclear about:
17:37 josephwb XX named internal nodes supported by an input tree expanded tip node
17:37 josephwb unclear, because these are subproblem newicks
17:37 josephwb can the full original newick (say, Metazoa) be pruned to Mammalia?
17:38 mtholder it is expanding a tip to a clade.
17:38 mtholder so it notes that the support is questionable.
17:38 josephwb but is it *really* a tip in the original tree, or just in a particular subproblems?
17:38 mtholder it looks to the taxonomy to see if the OTT ID is terminal or not
17:39 josephwb but most subproblems will not have terminal taxa, right?
17:39 mtholder correct
17:40 josephwb a "good" (tree-supported) taxon will have its own subproblem, but also appear as a tip in another
17:41 josephwb say, Primates
17:41 mtholder yes. that tool was not really designed to work on a fragment of the tree. How would it know that there is more of the tree below a tip?
17:41 josephwb exactly
17:42 josephwb so, can i use the original newicks?
17:42 josephwb i.e. how would i prune them to the correct level easily?
17:42 josephwb will it just ignore "outgroup" taxa?
17:42 mtholder you'd need to prune them down to the leaf set of the synth.
17:43 josephwb how do i do that?
17:43 josephwb oct-?
17:43 josephwb otc-?
17:44 mtholder otc-induced-subtree might work.
17:44 josephwb ok, that looks promising
17:45 mtholder or you could use the taxonomy part of the subproblem as the taxonomy.
17:45 mtholder then otc would think that the leaves were leaves.
17:48 codiferous we have a code review in 10 minutes right?
17:49 codiferous are we wanting to look at the treemachine pr? i think joseph just submitted that pr now
17:52 jar286 joined #opentreeoflife
17:52 jimallman codiferous: yes, i believe we’re trying to review PRs again at 2pm
17:54 blackrim joined #opentreeoflife
17:54 jimallman assuming that all components now use the same version of jade, ot-base, etc
17:55 josephwb seems like we should just re-head (and possibly re-name) the treemachine branch. comparison is oranges and orioles
17:58 pmidford2 joined #opentreeoflife
18:01 codiferous what is the procedure for the pr review? are we doing a call?
18:01 kcranstn we usually do these over irc
18:02 kcranstn but to be honest, we’ve never had one this big
18:02 codiferous ok
18:02 codiferous irc does not seem like high enough bandwidth to get make much progress with understanding the code
18:02 kcranstn I think the PR broke the github UI
18:03 codiferous haha! seriously? i didn't think it was *that* bad
18:04 jar286 hello
18:04 kcranstn I honestly don’t know how to effectively review this many changes at once
18:04 mtholder I think clicking on "merge" is the efficient way.
18:04 kcranstn ha
18:05 jar286 https://github.com/pulls?user=OpenTreeOfLife
18:05 mtholder It's kinda like looking at the synth tree in a browser - there is not much that will be caught by looking when it is this big.
18:05 codiferous i don't think an actual pr review is going to be very effective. we could however spend some time looking at tricky bits of code if that would be helpful for anyone
18:06 jar286 usually for each issue there are few questions that are useful to cover, even if the review is not thorough
18:06 kcranstn this affects both synthesis and the web services
18:06 kcranstn right?
18:07 jar286 there are four connected PRs here
18:07 jar286 I get the impression they have to be done in lockstep
18:07 kcranstn treemachine, jade, ot-base and ??
18:07 jar286 opentree
18:08 josephwb oti?
18:08 josephwb yeah
18:08 jar286 oh, yes, oti, not jade
18:08 codiferous jade and oti
18:08 jar286 we don’t control jade.  did the jade maintainers make a change that we should know about?
18:08 codiferous except i believe jade was already done
18:08 kcranstn is jade even in opentreeoflife?
18:08 codiferous no, that is a historical mistake
18:08 josephwb jade is good
18:08 codiferous it should be
18:08 jar286 is it backward compatible?
18:09 jar286 I hate lockstep changes.
18:09 jar286 I’m going to ask that repeatedly
18:09 jar286 let’s start by talking about jade. are the jade changes backward compatible?
18:09 codiferous i don't know
18:10 jar286 i’ll dredge up the repo account
18:10 jar286 https://github.com/FePhyFoFum/jade
18:11 codiferous https://github.com/FePhyFoFum/jade/pull/1/files
18:11 jar286 hmm, the maintainers look sort of familiar.  someone named chinchliff and someone named josephwb. must be a coincidence
18:12 jar286 why the @%$# is the java version number in the pom file
18:12 jar286 so, it’s not backward compatible.
18:12 blackrim codiferous, anything from the jadetree/tree right?
18:12 jar286 looks like we have a flag day.  rollback will be almost impossible
18:13 jar286 what is hppc?
18:13 blackrim there isn't anything parallel in this is there?
18:14 codiferous http://labs.carrotsearch.com/hppc.html
18:14 codiferous not that i know of
18:14 jar286 for 0.0000000000000000000001  I am pretty sure there’s a static value in the Double class that is the least positive value
18:14 kcranstn ?
18:14 codiferous jar286, if you are talking about '+                <target>1.8</target>'
18:15 codiferous i do not remember adding that. it could have been changed by eclipse behind the scenes
18:15 codiferous and i do not know if it is necessary to specify
18:15 josephwb i updated it
18:15 josephwb the 1.8 thing
18:15 codiferous is it necessary?
18:15 josephwb just making it consistent with treemachine
18:15 josephwb no?
18:16 jar286 I won’t pursue the least positive double thing, we don’t have time
18:16 josephwb change it back
18:16 codiferous that is fairly irrelevant anyway, that change is inconsequential
18:16 codiferous least positive double, i mean
18:17 codiferous the bigger issue are the changes to use the TreeNode interface, which were intended to set the stage for a lot of simplifying code changes elsewhere
18:17 jar286 I don’t see how to avoid this being a flag day, so I’d say leave the 1.8 (it’s sort of odd anyhow, since I don’t think we have 1.6 on the servers, so I don’t know how 1.6 worked)
18:17 codiferous those require updates to other tools, such as oti
18:17 blackrim yeah, i was wondering if TreeNode/JadeNode would be an issue for that
18:18 codiferous yeah, anything using JadeNode is likely to need casts now to preserve the old behavior
18:18 codiferous oti has been updated to do so
18:18 codiferous those changes are in the oti pr
18:19 jar286 the problem is if we get 3/4 of the way through updating production, and then find a problem around 1/4, we can’t just roll back the broken module, we have to roll back everything
18:19 jar286 that’s why i hate flag days (lockstep changes)
18:19 codiferous sorry, not understanding the implication of your 3/4 1/4 statement
18:20 jar286 we have 5 repos that have to be updated together.  that’s what I mean by lockstep
18:20 jar286 the changes are not independent
18:20 jar286 that makes rollback very difficult
18:20 codiferous ok, clarifying there were not some other elements also involved
18:20 jar286 you can’t just rollback the failing module, you have to roll back them all, since they are interdependent
18:20 codiferous yes, that part was clear
18:21 codiferous do we have all five of these repos currently running in their updated states on dev?
18:21 codiferous my impression was that we did
18:21 josephwb yes, we do
18:21 josephwb all using java 8
18:22 jar286 we will do another round of checking before going to production.  commit to master, then deploy master on dev, then deploy to production
18:22 jar286 I will try to do that this afternoon, after we do our merges
18:23 jar286 so is jade’s master branch the correct branch for us to use?
18:23 josephwb for the new code, yes
18:23 codiferous yes
18:23 jar286 ok… so jade is out of the way.  how about ot-base
18:24 jar286 working bottom up
18:24 jar286 https://github.com/OpenTreeOfLife/ot-base
18:24 jar286 https://github.com/OpenTreeOfLife/ot-base/pull/14
18:25 jar286 is this a backward compatible change?
18:25 codiferous depends on changes to jade
18:25 codiferous so no
18:25 jar286 that’s not what I mean.
18:25 jar286 if you install the new ot-base, will the old neo4j apps still work?
18:26 jar286 I already asked that, I see...
18:26 codiferous not unless you also install the new jade, which will not work with the current master branches of the neo4j apps
18:26 jar286 cody said “I think it is likely not backward compatible”
18:26 jar286 let me try again. suppose you have new jade and new ot-base. will old treemachine still work?
18:27 codiferous no
18:27 codiferous new jade is not backwards compatible with old treemachine
18:30 codiferous was it something i said?
18:30 kcranstn :)
18:30 blackrim yes
18:31 kcranstn he seems to have disappeared offline entirely
18:32 josephwb aneurysm, most likely
18:32 blackrim because of codiferous ?
18:32 codiferous i hope not
18:33 josephwb b/c of backwards compatibility
18:33 codiferous maybe a large angry gopher just discovered his internet connection
18:33 blackrim bill murray
18:34 codiferous oh, yes groundhog is much better. misplaced revenge for the death punxsutawney phil
18:36 jar286 joined #opentreeoflife
18:36 kcranstn jar286!
18:36 jar286 sorry about that
18:37 mtholder he's back, but I think he's been drinking.
18:37 jar286 i’ve been trying to get a key for 5 weeks now and the key person just accosted me
18:37 kcranstn he’s was gone long enough for a few shots
18:37 blackrim bill murray
18:37 jar286 hm. ok, so y’all merged the ot-base pr, right?
18:37 codiferous shooting at the ground hog?
18:38 kcranstn y’all? do they say that in boston?
18:38 jar286 it’s a perfectly good pronoun, fills an empty niche
18:38 codiferous we reserved the honor for you
18:38 codiferous i like y'all. it's better than y'uns in any case
18:38 jar286 ordinarily i would like better notes on what this pr is about
18:39 jar286 I can read the commit messages, but they’re sort of elliptical
18:39 jar286 it’s almost as bad as the PRs I do for the reference-taxonomy repo
18:39 jar286 you should do as i say, not as i do
18:40 codiferous added a bit of description in the comments thread
18:41 jar286 that was an excellent message
18:42 kcranstn merging?
18:42 josephwb you won't like this one: https://github.com/OpenTreeOfLife/treemachine/commit/cc1e865e80d7aa2ff6d48337f8b0902e8afc2db9
18:42 jar286 I moved your comment up to the PR issue description (at the top)
18:43 codiferous i see, ok
18:43 jar286 whitespace improvements are always welcome… looks like there’s some other changes too, must not be important
18:44 jar286 merged
18:45 jar286 https://github.com/OpenTreeOfLife/oti/pull/37
18:46 jar286 this one is pretty clear.
18:46 jar286 and it’s deployed and works, right?
18:47 jar286 anyone there?
18:47 codiferous yes
18:47 jar286 ok, i’ll merge
18:48 jar286 looks like there’s no opentree PR to do - we must have already done it
18:49 jar286 oh, i was supposed to merge ot-base pr, thought i did but didn’t. doing so now
18:49 jar286 done
18:50 jar286 ok, one left. https://github.com/OpenTreeOfLife/treemachine/pull/176
18:50 josephwb opentree repo has 92 branches!
18:50 kcranstn yeah, needs cleanup
18:50 jar286 I tried doing a diff for this pr a while back and it was… interesting
18:51 jar286 ok, here’s a question.  we have readmes and other documentation that says how to use treemachine. do these need to be (or have they been) updated?
18:52 jar286 i.e. if we can’t review the code, can we at least review the interface to it?
18:52 josephwb woefully out of date
18:52 blackrim do you mean server calls or cli calls?
18:52 jar286 by the way i ran the germinator repo tests against the dev treemachine and they passed…
18:53 jar286 i mean both.
18:53 jar286 my guess is that the api is pretty stable, except for the one removed method
18:53 josephwb it isn't moved
18:53 josephwb removed
18:53 josephwb yet
18:53 josephwb don't know where to redirect
18:53 josephwb https://github.com/OpenTreeOfLife/treemachine/issues/170
18:54 jar286 that’s fine, we can let that go today
18:54 josephwb it is just broke at the moment
18:54 josephwb yes
18:54 codiferous writing some description info about the treemachine pr
18:55 jar286 i’d like the README to be at least tell no lies. does it say anything that’s incorrect?
18:56 jar286 s/be at/at/
18:57 jar286 it talks about ottol_dumpv1
18:58 jar286 the choice here is between (a) hold this pr hostage to documentation fixes, or (b) merge the pr and extract a promise from someone to update the readme
18:58 jar286 kcranstn, what think you?
18:58 kcranstn how many changes to the readme needed?
19:00 jar286 it’s not that long, but someone who knows treemachine will have to check it and fix it as needed
19:00 jar286 maybe 1/2 hour ? I don’t know
19:01 * jimallman was distracted (first chick hatched in our incubator!), reviewing now….
19:01 kcranstn anyone else more familiar with treemachine have a thought on how long to update the readme?
19:02 kcranstn I assume all of this is also out of date: https://github.com/OpenTreeOfLife/treemachine/wiki
19:02 jar286 I’m willing to let it be a debt
19:02 kcranstn can we get an issue?
19:02 jar286 we can create an issue for it and nudge it along
19:03 jimallman regarding https://github.com/OpenTreeOfLife/opentree/pull/637 (PR for opentree), this was merged yesterday, before we decided to postpone
19:03 jar286 doing so now
19:03 blackrim the README itself doesn't require much editing
19:04 jar286 https://github.com/OpenTreeOfLife/treemachine/issues/177
19:04 blackrim those tutorial pages seem to have been put there as placeholders (the first part of the wiki , before "Information...")
19:04 jar286 scanning through the PR conversation…
19:05 jar286 doesn’t the readme have to talk about subproblems?
19:06 * jar286 looking at travis log
19:06 blackrim I was referring to the comment about it no lying more than it being complete.
19:08 jar286 well, misleading by omission is tantamount to lying…  but if the information is correct and useful maybe that’s ok for now
19:09 jar286 so, yes, I think a bit more than not lying would be nice. e.g. someone reading about treemachine might go to the repo and ask ‘how do i replicate this?’ - the answer should be somewhere
19:10 jar286 Failed to execute goal on project ot-base: Could not resolve dependencies for project org.opentree:ot-base:jar:0.0.1-SNAPSHOT: Could not find artifact org.opentree:jade:jar:0.0.1-SNAPSHOT in sonatype-snapshots (https://oss.sonatype.org/content/repositories/snapshots/) -> [Help 1]
19:10 blackrim agreed. and right now we rely on ot-cetera for the subproblems so the README will be updated to reflect that info without a doubt
19:10 jar286 Looks like someone has to install jade in a maven repository, if travis is to be happy?
19:10 jar286 it’s fine for a README to point the reader in the right direction
19:12 jar286 the four previous PRs all passed travis.  what happened?
19:12 codiferous new dependencies on jade to reduce the amount of replicated classes
19:12 codiferous treemachine previously had its own copies of the jade* classes
19:13 codiferous travis... that should just be in travis yaml scripts right?
19:13 jar286 I guess so
19:14 codiferous hm, looking for them, but i don't remember where the travis config is. anybody know off the top of their head?
19:14 jar286 it’s a file in the top level of the git repo
19:14 jar286 might be a dot-file
19:14 jar286 .travis.yml
19:14 jimallman travis-config folder: https://github.com/OpenTreeOfLife/treemachine/tree/master/travis-config
19:14 codiferous ah, yes it was
19:15 codiferous hm it looks as though it should be installing jade as a dependency
19:15 jar286 Failure executing javac, but could not parse the error:
19:15 jar286 javac: invalid target release: 1.8
19:16 jar286 looks like travis is set up to use 1.7 or 1.6
19:16 jar286 (see, this is what i was talking about before)
19:16 jar286 (well not exactly. 1.8 is necessary in this case)
19:17 codiferous did you change the line in .travis.yml to tell it to use java 8?
19:17 jar286 not me
19:17 jar286 I was just looking at the travis log
19:17 codiferous i can do that. it is currently set to use java 7
19:17 jar286 ok. i don’t think that should hold us up
19:18 jar286 kcranstn, what do you want to do with this pr? if we don’t merge then i won’t be able to push the new tree to production
19:18 jar286 but everyone’s getting tired of this i think
19:18 kcranstn merge
19:19 jar286 ok…
19:19 jar286 here goes…
19:20 jar286 hmm, travis build is in progress
19:20 codiferous i don't think its going to work
19:21 jar286 just merged & closed the PR.
19:22 jar286 would it help to have an issue for travis build or will you just take care of it?
19:22 codiferous i told travis to use java 8 but it still seems to be using java 7
19:22 codiferous and i don't think i am going to get to this, lots of stuff to be done on the doc
19:22 jar286 i’ll do an issue then.
19:24 jar286 done with pr review
19:24 jar286 thanks all
19:27 codiferous indeed
19:28 codiferous i noticed looking through the irc log from yesterday that there has been some confusion about the new procedures. i am happy to try and clarify that if anybody has questions
19:29 codiferous jar286, i am working on commenting the hell out of the testsums procedure, i think it could be helpful to do an orientation on that with you and mtholder and blackrim and myself
19:31 jar286 it says “the testsum procedure, explained below” but i don’t see any occurrences of the string ‘testsum’ below
19:31 kcranstn can we schedule a general Q&A for Tuesday?
19:31 kcranstn (I am out of town through Monday night)
19:32 codiferous sure
19:32 mtholder it is the convoluted test for "supported by"
19:32 jar286 the root merger business is a part of the procedure i’ve skipped over so far.  i felt it would make more sense if i understood other things first
19:32 jar286 ah.
19:32 jar286 i’d be happy to go over it, whenever
19:32 codiferous is the root merger confusing?
19:32 kcranstn yes
19:33 jar286 it relies on testsum (or supported by) and testsum is confusing
19:33 codiferous actually i don't think the root merger uses the testsum
19:33 mtholder josephwb: new args to find-unsupported-nodes have been implemented that look for taxonomic problems.
19:33 mtholder see https://github.com/OpenTreeOfLife/otcetera#checking-for-unnamed-nodes-in-a-full-tree-that-have-no-tree-supporting-them
19:33 codiferous the testsum procedure is used only for the non-root merger nodes
19:34 jar286 is there a second kind of node support then? or is that it?
19:34 codiferous nope, that is it
19:34 mtholder (still talking to josephwb) I did a much more efficient impl, which means that we can now compare the synth to the taxonomy and look for nodes that should have taxonomic names, but are unlabelled.
19:35 codiferous well, that is it as far as the loading procedure goes. otcetera uses a different node support test
19:35 jar286 Mark had reverse engineered it from the code and put the result in the doc. did that get deleted?
19:35 josephwb thanks mtholder
19:35 codiferous jar286, what did mark reverse engineer?
19:36 jar286 the supported by check.  maybe i misremember
19:36 codiferous hm, i don't remember that
19:36 mtholder codiferous wrote that.
19:36 jar286 oh.
19:36 codiferous oh, the mess of notation, yes. that is not very helpful i realize. it was about as good as i could do since i didnt really understand it either
19:36 codiferous making progress now
19:37 jar286 i can’t find it now
19:37 codiferous search for 2.2.3
19:38 jar286 oh there it is. is this the same as testsum?
19:38 codiferous yes
19:38 codiferous and i wouldn't recommend trying to make sense of that description in the doc
19:39 josephwb mtholder have you tested the big tree with the new code?
19:39 mtholder yes.
19:39 josephwb any new problems [please say no]
19:40 mtholder just 12 examples of things we weren't checking for before (and which are less serious).
19:40 mtholder here they come:
19:40 mtholder 12 unamed internal nodes which should have been named.
19:40 mtholder MRCA of "Idiospermum australiense ott545857" and "Pseudowintera axillaris ott59243" should have been named ott31010.
19:40 mtholder MRCA of "Phaeophila dendroides ott231897" and "Schizomeris leibleinii ott401401" should have been named ott275989.
19:40 mtholder MRCA of "Anthericum sp Weigend Weigend 2000 154 ott848231" and "Agave scaposa ott3996135" should have been named ott361629.
19:40 mtholder ancestor 1 node(s) before MRCA of "Eriozamia latifolia ott5456051" and "Dioon pygmaea ott4001703"
19:40 mtholder should have been named ott614459.
19:40 mtholder ancestor 1 node(s) before MRCA of "Triphysaria chinensis ott3882423" and "Seymeria cassioides ott3882475"
19:40 mtholder should have been named ott5144558.
19:40 mtholder MRCA of "Johnsonina eriomma ott3631376" and "Triacanthus nieuhofii ott3631366" should have been named ott803539.
19:40 mtholder MRCA of "Callithrix sp DB-2003 ott274346" and "Callithrix humilis ott298261" should have been named ott1015231.
19:40 mtholder MRCA of "Nematistius pectoralis ott776560" and "Trachurus japonicus ott43502" should have been named ott785725.
19:40 mtholder MRCA of "Alectis indica ott900270" and "Trachurus japonicus ott43502" should have been named ott710014.
19:40 mtholder MRCA of "Aponogeton junceus ott680123" and "Isidrogalvia duidae ott4001293" should have been named ott399481.
19:40 mtholder MRCA of "Ianthella basta ott314613" and "Aiolochroia crassa ott595809" should have been named ott619374.
19:40 mtholder MRCA of "Helicophyllum torquatum ott643657" and "Rhacocarpus apiculatus ott3856301" should have been named ott47736.
19:41 mtholder oops and a line-wrapping bug in my cod.
19:41 mtholder code.
19:41 mtholder that was a taxonomic bug in my comment.
19:41 jar286 I really wish there would be more first-principles discussion. It would help to explain what looks like a strew of arbitrary choices. Why would a node be in the TAG or not, why would an edge be in the TAG or not. For example…
19:42 jar286 you could make a graph in which there was a node for every clade hypothesis over N tips (3^n of them), and you could have an edge from u to v if and only if u is NCO v (order (3^n)^2 of them).  Why would that not be a good TAG?
19:43 jar286 it seems obvious there are too many nodes and edges, but why?
19:43 jar286 how do you tell a node or edge that should be there from one that shouldn’t?
19:43 codiferous that is an interesting question. it might be better than what we are doing depending on what you want
19:44 jar286 this is why ‘supported by’ seems central.
19:44 codiferous we have indeed made a bunch of arbitrary choices for the sake of efficient runtimes
19:44 mtholder thanks for the comments in the code, Cody. Looks like a different def of "supported by"
19:44 codiferous i think its pretty likely we are not getting the best answer. and how good our answer is, seems pretty arbitrary
19:44 jar286 if it can be shown that it’s an approximation to some ideal, that would be a step in the right direction
19:45 jar286 I think you’re misunderstanding my intent. I’m not looking for ‘best’
19:45 jar286 I’m looking for what constitutes admissible, with this particular TAG generator, one hopes, generating admissible graphs
19:45 codiferous i think i understand what you mean, but i don't think there's a straightforward "ideal". it is good enough to pass the tests
19:46 jar286 it’s ok if it’s an approximation, but it would be very helpful to know what it’s approximating
19:46 jar286 the graph i described above is obviously silly because it doesn’t depend on the input trees
19:46 codiferous yeah. i'm not sure how to describe the approximate except as a string of arbitrarily cut corners
19:47 jar286 but corners cut off of what?
19:47 codiferous but if you filtered that graph to only those edges supported by the input trees, then it would not be silly
19:47 codiferous off of the entire set of all possible nodes and all possible supported nco edges
19:48 jar286 but you don’t want the entire set.  that’s obvious, right?
19:48 codiferous mtholder, no problem. i think there are some ways it can be made simpler as well, just going slow to avoid screwing things up
19:48 jar286 you are looking for something that depends on the input trees
19:48 mtholder FWIW: it is not well tested (and the -r option is not working) but https://github.com/OpenTreeOfLife/otcetera#checking-for-additional-splits-that-could-be-added describes how to resolve the synth tree some more.
19:48 mtholder by going back through the input trees and resolving polytomies.
19:49 codiferous jar286, i think ideally we would want the entire set of phylogenetic statements supported by the input trees
19:49 codiferous the synthesis method should return the correct answer if provided with a tag containing that
19:50 jar286 that’s what i said above - ‘supported by’ is central. (and I don’t understand it.) that may explain the nodes, but it doesn’t explain the edges - you’re also applying a support criterion to them (and I don’t understand why it’s the right one)
19:51 jar286 the overall problem has got to be fairly simple to state. but I have seen no such statement.
19:51 jar286 overall TAG problem that is
19:52 codiferous ah, ok. there are two different defs of supported by. we should probably try to choose a different name for one of them for the sake of clarity
19:52 jar286 yes, I think I said that in a doc comment
19:52 codiferous i agree
19:52 codiferous node 'supported by' means that there is information in the input trees to divide everything in the ingroup from everything in the outgroup
19:53 codiferous that is what the testsums procedure is trying to identify
19:53 jar286 see, that’s the kind of sentence that needs to be in the document.  (codiferous 3:52pm)
19:54 kcranstn I’ll admit that this is the type of up-front discussion taht would have been helpful to my understanding
19:54 mtholder In my docs, I use the phrase to mean something different. "minor-minimal" is the jargon. see sect. 1.3 of http://phylo.bio.ku.edu/ot/summarizing-taxonomy-plus-trees.pdf
19:54 codiferous haha, indeed, me too
19:55 codiferous it has taken a bit of time for me to distill that from discussions with stephen and analysis of the code
19:55 codiferous edge supported by is slightly less straightforward to explain...
19:55 blackrim where is edge supported by?
19:56 codiferous it is the condition used in the map trees step
19:57 codiferous whether or not to make an streechildof edge between two nodes
19:57 jar286 I looked at map trees, and it’s a recursive definition. my version is better i think, it’s not recursive (the ‘filtering…’ section)
19:57 codiferous fine with me
19:57 codiferous do you understand the mapping of the tree root nodes?
19:57 jar286 who are you asking?
19:58 codiferous asking jar286. it looked like there was some question about that in the doc
19:58 jar286 I really haven’t looked at the mapping of the tree root nodes.
19:58 jar286 yet.
19:58 kcranstn I am still confused about whether root here refers to the subproblem root or the input tree root
19:58 codiferous input tree root
19:58 kcranstn stephen’s diagram seems to indicate otherwise
19:58 mtholder I don't think that jar286's edge creation is an accurate depiction of how the STREE nodes are created.
19:59 codiferous stree edges?
19:59 mtholder yes. sry.
19:59 jar286 probably not. my writing is meant to be provocative in places
20:00 mtholder from looking at the code, I think the recursive def is needed because the root mappings get the ball rolling.
20:00 codiferous yes, i agree, the root mappings are critical
20:00 mtholder and then it goes preorder down the source trees.
20:00 jar286 are you talking about m() ?
20:00 codiferous yes
20:00 mtholder I don't understand why the root nodes must go to taxonomy nodes, though.
20:01 jar286 I tried to figure out how m() was made, drew a picture, and read off the condition from the picture.
20:01 jar286 I don’t think the map trees description gives the base case
20:01 jar286 it’s a recursion with no base case
20:01 codiferous i think the last sentence does
20:01 codiferous looking back now to check
20:01 mtholder the base is the root nodes. but I agree that is not clear in the doc.
20:01 mtholder have to look at the code.
20:02 codiferous " starting at root(t), where root(t) is first imaged to all taxon nodes u such that ingroup(root(t)) ⊆ ingroup(u)."
20:02 codiferous the root is mapped to all taxon nodes in the subproblem whose ingroups contain all the tips in the tree
20:02 mtholder yeah. sry must have missed that.
20:02 mtholder by why taxon nodes?
20:02 codiferous it is easier to understand in plain text i agree
20:02 mtholder but not by
20:02 jar286 what is m(parent(x)) if x is a root?
20:03 jar286 empty?  it doesn’t say
20:03 codiferous nothing. there is no parent of the root node
20:03 codiferous that is like saying m(null)
20:04 codiferous mtholder: but not by?
20:04 jar286 but that’s not said.  it’s just ill formed the way it’s written.
20:04 codiferous i agree, sloppily formed
20:04 mtholder "but why taxon nodes?" is what I meant
20:04 codiferous because mapping root nodes promiscuously leads to cycles
20:04 codiferous for reasons we don't really understand that well
20:06 codiferous mapping to taxon nodes alone definitely limits the extent to which trees are mapped, but it isn't clear what other nodes should or should not be used as starting positions
20:08 codiferous jar286, there is no need to keep the map trees section i wrote
20:08 mtholder slight change of topic: I found this the other day: http://www.mdpi.com/1999-4893/6/3/396 which is an approximation to a supertree problem - which is not the problem that we are solving, but close
20:08 jar286 i need it as a check against what i wrote.
20:08 codiferous i just wanted to make sure what i was trying to say was clear so you can formulate it in a way that makes more sense to you
20:08 mtholder I think just cleaning up that sentence that you quoted could be enough.
20:09 jar286 mtholder says what i wrote was wrong, so I need to repair it, or merge with what you wrote
20:10 jar286 if there is no way to express it without reference to some recursive procedure, that’s very unfortunate, but if it’s all we have, so it goes
20:10 mtholder I agree w/ jar286 that it does not make sense because the imaging function requires a parent. Just saying "As the base of the recursion, the set m is initially populated with the root nodes..." could do it
20:11 jar286 the use of ‘iff’ with a subsequent correction doesn’t fly.  ‘iff’ is a very strong connective and needs to be categorically true
20:11 codiferous all of this is fine
20:11 codiferous jar286 can i leave it to you to compose something acceptable from my original text and the notes we've made here?
20:12 jar286 sure, i will have to do some work to see if there’s a nonrecursive way to characterize this condition
20:13 jar286 it’s clearly a very important condition, so saying it’s the product of some code is pretty unsatisfactory. code is fine as implementation, just not as description
20:13 codiferous mtholder, that looks like an interesting method. probably better than the mls method we were using before
20:14 mtholder I'm not sure if there is an impl.
20:14 mtholder and it is only approximate.
20:14 codiferous jar286 if there's anything i can do to help figure out a better way to describe it just let me know
20:15 jar286 i’m looking to complete the sentence ‘after m has been determined, x is in m(y) if and only if <condition> is true’
20:15 jar286 if you can find <condition> that would be helpful
20:15 jar286 I made an attempt, but it seems to have a bug
20:16 mtholder I'm working on an impl of modified BPMF from http://www.sciencedirect.com/science/article/pii/S0166218X10000983 FWIW.
20:16 codiferous is that sentence currently in the doc?
20:18 jar286 you mean the buggy version? Yes, in the ‘filtering’ section
20:24 kcranstn I don’t see it there
20:25 jar286 it’s not there word for word, but it’s equivalent. 2nd sentence.
20:26 jar286 instead of talking about y in m(x), it talks about an edge from x to y (or maybe the other way around, i don’t remember)
20:26 codiferous well, i can't see a way around the recursive nature of the mapping procedure, because it all depends on that first node
20:27 kcranstn codiferous - in the section “Merger nodes from the root of one tree to nodes in other trees”
20:27 jar286 but there *has* to be some reason, or intuition, for what it has chosen to do
20:27 kcranstn how do we know which TAG nodes correspond to roots of input trees?
20:27 codiferous but if there's some way to add a condition to that sentence like "and the parent of x is mapped to some parent that is mapped ... to the root of the tree" (obviously not that sloppily, but to that effect)
20:27 kcranstn as opposed to roots of subproblem fragments?
20:28 kcranstn is that info in the subproblems?
20:28 codiferous kcranstn, is there a specific sentence you are looking at?
20:29 kcranstn Let the TAG node representing the root of input tree j be denoted rj.
20:31 codiferous do we talk about the roots of subproblems somewhere?
20:31 codiferous (other than the earlier parts that describe the subproblem procedures themselves, that is)
20:31 kcranstn in Loading the subproblems we mention the roots
20:31 kcranstn the synthesis subtrees selected within each subproblem connect to one another at the subproblem root nodes to form one large tree that spans the entire taxonomy.
20:32 kcranstn and “Each subproblem contains a set of trees (including a subtree induced from the taxonomy) that span the taxonomic region between the uncontested taxon at the root of the given subproblem and those uncontested taxa at the roots of other subproblems.”
20:32 codiferous right. so there are lists of nodes that are known to be subproblem roots
20:32 codiferous they are known before the loading procedure (bc of subproblem division by otcetera) and do not change during loading/synth
20:33 codiferous so there is nothing we really need to do to identify them except check if they're in the list
20:33 codiferous but they are fundamentally different from the roots of input trees
20:33 codiferous i guess i'm not sure what the basis is for the confusion
20:33 kcranstn so are the “roots” in the merger nodes sections roots of subproblems or input trees?
20:33 codiferous they are the roots of input trees
20:34 kcranstn how do we know which nodes are roots of input trees?
20:34 kcranstn if the inputs to the graph are subproblems?
20:34 codiferous ah, i think i see
20:34 kcranstn in stephen’s figure, the root nodes
20:34 codiferous when we say "roots of input trees" we mean the root nodes of the input tree fragments that come through from the subproblem
20:34 kcranstn are subproblem roots
20:35 codiferous subproblem roots are taxon nodes in the graph
20:35 kcranstn ok, that makes sense to me now
20:35 codiferous ok
20:36 codiferous is that sufficient? do you think we need to update the text somewhere?
20:36 kcranstn I am simply going to add “fragment” after “input tree"
20:36 codiferous ok
20:37 codiferous mtholder, did you write the new merger nodes section?
20:37 mtholder mostly me yes.
20:38 mtholder I think (I'm haven't looked to see if it morphed).
20:38 codiferous ok. what do you think about adding the condition i mentioned that there can only be one merger node made on any given root-tip path in the second tree?
20:39 mtholder let me pull it up again...
20:39 kcranstn why is that condition necessary?
20:41 codiferous talking to stephen about it now
20:43 mtholder is it identifyNestedChildBiparts ? (in the code)
20:44 mtholder this is where the commented out break (line 609) made we think that it was possible to merge multiple.
20:45 mtholder perhaps something prevents that. (one of the other tests).
20:45 mtholder I'm definitely fine with your edit if that is what the code does.
20:46 codiferous the one-merger-per-root-node-path condition is in processBipartsForTree
20:47 codiferous according to stephen, that commented break on line 609 prevents the same condition from being enforced for the root sums
20:48 codiferous root merger nodes, that is
20:48 codiferous we merge each root node with *all* possible nodes from the other tree
20:49 codiferous but for non-root tree nodes, we only make a merger node using p and the *first* merge-compatible node we find on a given root-tip path in the second tree
20:50 kcranstn ok, folks. I am outta here. Going away dinner for Graeme Lloyd (who has been at nescent for the past month), then Kids in the Hall show then backpacking from Sat - Mon
20:50 codiferous have fun!
20:50 codiferous oh, before you go
20:50 kcranstn there are supposed to be baby feral ponies on this trail
20:51 kcranstn yes…
20:51 kcranstn ?
20:51 codiferous any updates on the changes to the plan for submission?
20:51 codiferous are we still going for pnas asap?
20:51 mtholder pnasap is almost a palindrome
20:52 kcranstn We can discuss Tuesday about resubmission (the PNAS response was always a “please resubmit” rather than “accept with revisions”)
20:52 codiferous ok
20:52 kcranstn Depends on state of the supplemental and how confident people are feeling about their understanding of the methods
20:52 kcranstn this discussion now helped
20:53 codiferous good. i am available for more of these. didn't realize what was happening on irc yesterday until i looked at the log today
20:53 codiferous feel free to rope me in whenever
20:55 mtholder I'm afraid that i'll have to look at the code when I'm fresh. but I definitely didn't work through all of processBipartsForTree before. So, I certainly would not trust my text.
20:57 codiferous ok
20:57 blackrim joined #opentreeoflife
20:57 codiferous i think my changes should make it correct
20:58 codiferous but my description of the one-node-per-root-tip-path condition is not good
20:58 codiferous if you have a way to describe that better that would be helpful
20:59 codiferous but i'm not sure if it makes sense
20:59 codiferous i mean, not sure if it makes sense to you

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary