Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2014-02-27

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:48 ilbot3 joined #opentreeoflife
00:48 Topic for #opentreeoflife is now Open Tree Of Life | opentreeoflife.org | github.com/opentreeoflife | http://irclog.perlgeek.de/opentreeoflife/today
02:22 mtholder joined #opentreeoflife
02:41 jimallman joined #opentreeoflife
03:13 travis-ci joined #opentreeoflife
03:13 travis-ci [travis-ci] OpenTreeOfLife/api.opentreeoflife.org#316 (translatingnexson - efe30c4 : Mark T. Holder): The build was broken.
03:13 travis-ci [travis-ci] Change view : https://github.com/OpenTreeOfLife/api.opentreeoflife.org/compare/5e2f7455a605...efe30c449b8e
03:13 travis-ci [travis-ci] Build details : http://travis-ci.org/OpenTreeOfLife/api.opentreeoflife.org/builds/19701761
03:13 travis-ci left #opentreeoflife
03:18 mtholder jimallman: while I refactor, nexson validation has been disabled on the translatingnexson branch (of API). The API should now accept PUTs and POSTs without validating. I'm working on getting the validation done on v1.2 nexson. then I'll reenable it.
03:19 mtholder ot7 is running the api with translatingnexson branch
03:19 jimallman awesome, thanks. i'm wrapping up some changes in the tree editing (rooting, etc) UI, then i'll get back on this.
03:19 jimallman so i should point to ot7 for testing now..?
03:20 mtholder Yes, i think that it should work.
03:20 mtholder it is passing the ws-tests
03:20 mtholder (in the api repo)
03:21 mtholder note that it is using the phylesystem_test repo as the doc store.
03:21 jimallman sounds good, my "favorite" studies are in this subset.
03:21 jimallman mtholder: do you have time for a few quick questions about tree-rooting logic?
03:21 mtholder yes its like "open tree: greatest hits volume 1. the early years"
03:21 mtholder yup
03:22 jimallman mellow gold
03:22 jimallman i'm trying to reconcile the curator's tree editor with recent conversations.. but we have lots of (overlapping?) properties for rooting
03:23 mtholder yes we do.
03:23 jimallman there's legacy stuff like @root flags on individual nodes, then of course the "natural" root based on Nexson's tree structure.
03:23 mtholder i think @root is almost useless, but should be kept equal to the node with no parent
03:24 jimallman karen suggests ignoring (dropping) the study-level property ot:notUsingRootedTrees..
03:24 mtholder almost useless because it will be set to true for arbitarily rooted trees from treebase
03:24 jimallman ah, ok. should we respect / preserve the old @root flags?
03:25 mtholder I think we should move @root around with the root (for nexml conformance), but not rely on it to mean anything
03:25 mtholder looking up notUsingRootedTrees now...
03:25 jimallman do we expect incoming trees to possibly have conflicting "natural" root (based on edges) and @root markers?
03:26 mtholder we should not. my validator will treat that as an error.
03:26 jimallman re: notUsingRootedTrees, this was actually a request of mine, just a hint to suppress "improvement" hints to the user, and possibly relax validation.
03:26 mtholder that sounds fine to me.
03:26 mtholder we are in a netherworld when we first see a tree.
03:26 jimallman sorry, i'll try to ask one question at a time. what sounds fine?
03:27 mtholder sorry. keeping notUsingRootedTrees sounds fine.
03:27 jimallman ok, will do.
03:27 mtholder it is not too useful, but could flag intent.
03:28 mtholder we are in a netherworld when we first see a tree, and until we see notUsingRootedTrees or specifiedRoot we don't know what to do
03:28 jimallman got it.
03:28 mtholder ot:notUsingRootedTrees could be better placed on each tree, I suppose.
03:28 mtholder a few studies might have a mix of rooted and unrooted.
03:29 mtholder which is to say intentionally unrooted and intentionally rooted.
03:29 jimallman i think this was Karen's thought as well. we have a property on each tree.. let's see...
03:29 jimallman i thought it was based on the presence (or absence) of ot:specifiedRoot
03:30 jimallman ... but i think i found another that seemed more definitive...
03:30 mtholder a new tree will have neither specifiedRoot nor any flag indicating that it is intentionally unrooted.
03:30 mtholder so we need a flag for each intension, I guess.
03:31 jimallman here it is... tree['^ot:unrootedTree']
03:31 mtholder that looks good.
03:32 jimallman i thought we'd resolve it something like this:
03:32 jimallman 1) if a tree has to:unrootedTree == true, that's the last word.
03:32 mtholder If '^ot:unrootedTree' nor specifiedRoot is set, then that means "hassle the curator to affirmatively pick one or the other.
03:32 jimallman 2) if not, but it has a specifiedRoot, we treat that as authoritative
03:32 mtholder those rules look good to me.
03:32 jimallman 3) if not, we treat it as uncertain and draw based on the "natural" root (nexson structure)
03:33 jimallman but i'm not sure what to do with @root if it exists and disagrees with nexson structure. ok to clobber these?
03:33 mtholder yes to 3. with a "as far as we know this rooting is arbitrary. please confirm." UI indication.
03:34 jimallman ! that's interesting
03:34 mtholder I think it is fine to clobber misplaced @root. Once I have the validation working, if you see one, it should be a bug report to me (peyotl repo, I suppose).
03:34 jimallman i've added this as a checkbox
03:34 mtholder sounds good.
03:34 jimallman [x]  Tree root is arbitrary (not biologically correct)
03:35 jimallman so we'd either need to check that when in doubt... or reverse the text, something like
03:35 jimallman [x]  Tree root is biologically correct (not arbitrary)
03:36 mtholder That should work.
03:36 jimallman it's tough. if they agree with the "natural" rooting we initially present, there's no explicit action they can take.
03:36 jimallman initial (uncertain) state should have been
03:37 jimallman []  Tree root is biologically correct (not arbitrary)
03:37 mtholder It could be unchecked with the latter text.
03:37 mtholder right []
03:37 jimallman (unchecked) right
03:37 jimallman thoughts on old/"wrong" @root flags? ok to clobber these?
03:37 jimallman or should we retain and mark as "multiply rooted" (the current behavior)
03:38 mtholder Fine to clobber. If you see one from a GET, it should be a bug report to me (peyotl)
03:38 jimallman ah, ok. so i shouldn't expect to see there.
03:39 jimallman these
03:39 jimallman but yeah, i thought they might sneak in via import
03:39 mtholder correct (once validation is working again).
03:39 jimallman neat!
03:39 mtholder They might, but that would be a bug.
03:39 mtholder on my part.
03:40 mtholder I can make the validator fix them on your POSTs/PUTs, or you could unset @root for the old root and set it on the new root node.
03:40 mtholder whenever the curator reroots
03:40 jimallman yes, no problem, i'll handle it on the client.
03:40 jimallman one other thing that made me more cautious about "prior" rooting...
03:41 jimallman there's a property 'ot:originalRoot' in the NexSON wiki page
03:41 mtholder we can ignore that for display, but preserve it as the tree is rerooted.
03:41 jimallman this is intended to flag the original root node the first time we re-root. i suppose this is about provenance?
03:41 mtholder yes.
03:42 mtholder we can get the original root from version control.
03:42 mtholder but curators might unintentionally reroot and want an "undo"
03:42 jimallman ok, i'll set this on the "natural" root (topmost in nexson tree) and ignore any conflicting @root for this purpose.
03:42 jimallman (sorry, that shouldn't happen. it's late)
03:43 mtholder I should add that flag on import.
03:43 mtholder for phylografter trees, I don't think we'll know it.
03:43 jimallman you'll add ot:originalRoot flag? or you mean @root?
03:44 mtholder originalRoot
03:44 jimallman sure, why not?
03:44 mtholder I just don't think the mysql db stores the history
03:44 mtholder we could ask them to start maintaining that bit of provenance.
03:44 jimallman (why not? was rhetorical)
03:45 mtholder ah. got it.
03:45 jimallman :)
03:45 jimallman about re-rooting.. karen pointed out that the current interaction is probably not right. i have the user select a root and make it the tree root, while she says the proper gesture would be to select an EDGE and "split" it with a new root node.
03:45 jimallman upon re-rooting, we would basically delete the old root node and create a new one that splits the chosen edge.
03:46 mtholder unfortunately, she is correct. Unfortunate, because that implies creating new node and edge IDs
03:46 jimallman this makes me itch a little, since it changes the graph structure in new ways, but i think i understand her point.
03:46 jimallman what if i retain the original root node and sort of "re-attach" it?
03:47 jimallman maybe it drags along an extra edge as well... this is completely weird, but would avoid some churn and new objects/ids.
03:47 mtholder well. if the tree is originally rooted at a polytomy we might need 1 more node and edge than we originally have.
03:48 jimallman whoah
03:48 mtholder polytomy = more than 2 children.
03:48 jimallman right, so the root has 2 or more children
03:48 jimallman hmm
03:48 mtholder We could have the rule be:
03:49 mtholder 1) the curator js can create 1 node edge if the curator roots on an existing edge.
03:49 mtholder 2) if the curator move the root, keep moving that newly minted node and edge.
03:49 mtholder so we wouldn't be creating a large # per tree.
03:50 mtholder It does seem like the newly minted ones should get some flag for provenance purposes.
03:50 * jimallman is sketching this for clarity
03:51 jimallman by "create 1 node edge" , do you mean "create 1 new edge"?
03:51 jimallman (rule 1 above)
03:52 mtholder sorry. I meant 1 new node and 1 new edge
03:52 jimallman how would we need a new node? don't we always have an "extra" node to move around?
03:52 mtholder If the original tree is (A, B, C)
03:52 mtholder and the curator clicks on the branch between A and the root...
03:53 mtholder then we should get (A, (B, C))
03:53 jimallman !
03:53 mtholder which has a root and an internal.
03:53 mtholder I think that is what she means by clicking on an edge and splitting it.
03:53 jimallman ah, so the edge between A and root is treated as splitting A from (B,C)...
03:54 mtholder yes.
03:54 jimallman i get it when clicking between two non-root nodes. just hadn't thought of clicking on an edge that's already adjacent to the root
03:54 jimallman it seemed like a meaningless act, but i can see how it would be different in a polytomy
03:55 mtholder if the curator clicks on a node and reroots, then it should result in a polytomy.
03:55 mtholder (A, (B,C)) <click on the parent of B+C   ===>  (A, B, C)
03:56 mtholder when that happens we gain an edge and a node that are no longer needed.
03:57 mtholder we could hold onto those "extraNodeId" and "extraEdgeId" so we are not continually regenerating Ids as the curator moves the root around.
03:57 jimallman agreed, i'll come up with a convention for this. i need to play with some example trees to make sure i understand what (A, (B,C)) looks like
03:58 jimallman is there an unnamed sibling of A that is the parent of B and C?
03:58 mtholder It is nice to know which edge is "extra" (created by the curator app for the purpose of rerooting) because it has no provenance for its @length.
03:58 jimallman maybe we could use 'node0' and 'edge0' for this?
03:59 jimallman these?
03:59 mtholder in (A, (B,C)) the leaf A is sister it the group B+C and B+C has an internal node.
03:59 mtholder do you mean 'node0' as the ID or as a property name?
03:59 jimallman ah, of course. the names refer to leaves. i keep drawing these trees with all labeled nodes. :-/
04:00 jimallman i thought perhaps as the IDs for the root node and its "extra" edge
04:00 jimallman if we're not using these. but then we're going to get trees with an existing root, and it'll already have an ID
04:00 mtholder the gotcha there is that on nexml import someone could use those ids for other nodes
04:00 jimallman true.
04:00 jimallman it's the wild west out there!
04:01 mtholder yes.
04:01 mtholder with less gold.
04:01 mtholder relatively low chance of rattlesnake bites, too.
04:01 jimallman well, if we dance the rest of the tree around a persistent root node, we'll certainly know where to find it (topmost in the nexson tree)
04:02 mtholder Yes. it is just the fact that it comes and goes (depending on root on edge or root on node choices) that is really tricky.
04:02 jimallman so the new edge is the one to watch (and possible new root node, i guess)
04:02 jimallman ah, so we *might* choose to root on a node after all? internal or otherwise?
04:03 mtholder unfortunately, yes.
04:03 mtholder most times it will be on an edge.
04:03 jimallman easy enough, we have that functionality now.
04:03 jimallman (rooting on node, i mean)
04:03 jimallman i'll just need to add edge highlighting to the UI, and figure out how we mutate the tree (previous conversation)
04:04 mtholder a pair of tree property: "manufacturedNodeId" and "manufacturedEdgeId" then we can tell if the root node (and one of its descendant edges) were generated after import.
04:04 mtholder tree *properties*
04:05 mtholder by which I meant. We could use a pair of tree properties...
04:05 jimallman hm, that works. that will also work as a quick check of whether they exist already... i assume we'd want to destroy them (and clear these tree properties) if the user re-roots on an existing node.
04:06 mtholder Or we could hold onto them in case we need that pair of node+edge again. No matter to me as long as we know the convention.
04:06 jimallman i like that, if it's no harm done to have a "loose" node and/or edge floating around in the nexson
04:07 jimallman i assume that this edge (if not being used) can have blank source and target, right?
04:08 mtholder we could make a special case (relaxed validation) for the edge that is the specially manufactured edge (so that it is OK if it lacks a source/target)
04:09 jimallman it seems cleaner to me to remove these elements if they're not being used, honestly.
04:09 mtholder that is fine with me, too.
04:10 jimallman i'll start that way and see if there's a problem. less special-case logic to chase for both of us.
04:10 jimallman (one more question, i promise)
04:10 mtholder we still want to flag this node edge pair (when they are present)
04:10 jimallman right. and i suppose they're always attached, with the root node as the source of the edge
04:11 mtholder yes.
04:11 jimallman cool, that helps.
04:11 mtholder and if the root moves to an internal node, then they go away
04:11 mtholder and the other edge that had attached to the root stays.
04:11 jimallman got it.
04:12 jimallman so in the typical case that a tree enters the curator with a designated root node, and we re-root to another edge, should we move the original root node and one of its edges in the graph?
04:12 jimallman (to "split" the chosen edge, i mean)
04:12 jimallman or is this mangling the graph?
04:13 jimallman karen suggested we'd clobber the original node and an edge...
04:14 mtholder we can get rid of them, if they are not the targets of annotations... ugh to checking that.
04:14 mtholder we should also sum branch lengths for the two edges that spanned the orginal root.
04:14 jimallman the plot thickens
04:15 jimallman sum the lengths, got it
04:15 mtholder I would not have a problem with leaving the node in (for provenance), but we should see if that make life tough on the Smith lab.
04:15 mtholder it would be weird: a node with a parent edge and only one daughter edge.
04:15 mtholder but interpretable.
04:16 jimallman i can try that, so the result would be something like (A, (B(C)), D)..?
04:16 mtholder It would also mean that the ot:originalRoot property would not go stale.
04:16 mtholder Yes to your question.
04:16 jimallman like you say, an "only child" with an only child
04:16 jimallman good point about the originalRoot. that's what makes me itchy about moving it around in the graph
04:17 mtholder So the rule could be...
04:17 jimallman we could have a radically altered tree, and end up with it looking like we're still on the original root
04:17 mtholder if the root is moving and the current root is "manufactured" move the current root node (and its manufactured edge).
04:17 mtholder otherwise manufacture a new root node and edge.
04:18 jimallman yes, that works.
04:18 mtholder if the root is moving and the current root is not manufactured, don't delete it.
04:20 jimallman i could cosmetically hide the "ghost" root, if it's the only node in a chain from its parent to its only child.. or is that bogus?
04:21 mtholder I think that is fine for our purposes.
04:21 mtholder if we later support display of generic annotations, we might need to make it visible.
04:21 mtholder or at least visible when annotated.
04:21 mtholder off topic a bit: note that honey badger fish v1.2 has a tree property ot:rootNodeId that should always point to the node with no parent. that property says nothing about whether the root is arbitary. It is like @root, but it needs to be on the tree
04:22 jimallman ok, i'll try and make that happen. if someone re-roots on the visible edge between its parent and child, i'll just need to restore it as the root
04:22 mtholder sounds great.
04:22 jimallman hm, so this is not the same as 'ot:specifiedRoot'?
04:22 jimallman it just reflects the "natural" root based on nexson tree structure?
04:23 mtholder no. specifiedRoot implies intent. rootNodeId is all about knowing what node to start at when you build a tree.
04:23 jimallman (i see why you'd add that, for faster tree building if nothing else)
04:23 jimallman right
04:24 jimallman so if the original root is a polytomy (has 3+ children), and we re-root the tree, it survives as an normal internal node, but one of its children becomes its parent (as a result of changing edge directions). correct?
04:24 mtholder yes,
04:24 mtholder if it had 3 children, then it will go from polytomy to bifurcating node.
04:24 jimallman yep
04:25 jimallman awesome, thanks
04:25 mtholder sure. I've got to sign off soon...
04:25 jimallman that's all my questions. this is going to take some wrangling...
04:25 jimallman but i think i've got it clear in my mind
04:26 mtholder may the force be with you.
04:26 mtholder or the force-directed edges be with you.
04:26 mtholder whatever.
04:26 jimallman i.. have no snappy response. good night!
04:26 mtholder good night.
04:26 jimallman (what the hell is an aluminum falcon?)
04:27 mtholder that is not in OTT
04:27 mtholder we'll need fuzzy matching.
04:27 jimallman oh, can't we add it?
04:27 mtholder and with that. I'm out of here....
04:27 jimallman adios
07:10 jimallman joined #opentreeoflife
14:49 towodo joined #opentreeoflife
15:35 jimallman joined #opentreeoflife
15:45 PEM joined #opentreeoflife
15:47 pmidford joined #opentreeoflife
16:42 josephwb joined #opentreeoflife
16:46 jimallman joined #opentreeoflife
16:49 towodo joined #opentreeoflife
18:06 pmidford joined #opentreeoflife
18:41 josephwb Are you there Jonathan?
19:00 towodo josephwb, yes
19:10 josephwb we were just having problems with the amazon server, but seem to have it under control now.
19:51 mtholder joined #opentreeoflife
20:07 jimallman joined #opentreeoflife
22:36 mtholder joined #opentreeoflife
23:22 pmidford left #opentreeoflife

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary