Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2015-05-01

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:34 kcranstn joined #opentreeoflife
00:42 kcranstn joined #opentreeoflife
00:56 kcranstn joined #opentreeoflife
01:51 kcranstn joined #opentreeoflife
02:13 kcranstn joined #opentreeoflife
02:27 kcranstn joined #opentreeoflife
03:37 josephwb joined #opentreeoflife
03:37 josephwb hey ilbot3
10:45 josephwb joined #opentreeoflife
11:59 kcranstn joined #opentreeoflife
12:13 jar286 joined #opentreeoflife
12:38 mtholder joined #opentreeoflife
12:44 josephwb joined #opentreeoflife
13:42 kcranstn joined #opentreeoflife
14:02 kcranstn joined #opentreeoflife
14:09 mtholder It looks like 7 subproblems led to unsupported nodes:
14:09 mtholder 23373
14:09 mtholder 39538 unsupported nodes=2
14:09 mtholder 46248
14:09 mtholder 601168 unsupported nodes=3
14:09 mtholder 712383 unsupported nodes=3
14:09 mtholder 79861
14:09 mtholder 921871
14:37 josephwb thanks mtholder
14:37 jar286 new version of apache isn’t compatible… slogging through allow/require quagmire
14:38 kcranstn not compatible with java 8?
14:38 mtholder would it be possible to create a target in maven for the server side stuff that does not need java 8?
14:38 mtholder (in the treemachine maven)
14:38 jar286 not a java problem.  just basic site configuration
14:39 jar286 something about ‘require all granted’ in the vhost config… tons of search hits for the problem, jus ttrying to understand what’s going on
14:41 jar286 this is a result of upgrading from apache 2.2 to 2.4
14:42 jar286 http://httpd.apache.org/docs/2.4/upgrading.html#access
15:36 kcranstn joined #opentreeoflife
15:40 jar286 http://asterales.opentreeoflife.org/opentree/argus/otol.draft.22@1  = jessie + apache2.4 + openjdk-8
15:49 kcranstn w00t!
16:01 josephwb joined #opentreeoflife
16:09 kcranstn josephwb?
16:10 josephwb yerp
16:11 kcranstn what’s the relationship between blackrim/avatol_nexsons and josephwb/synthesis_trees?
16:11 josephwb not sure what blackrim/avatol_nexsons is, but it is certain to be out of date
16:11 josephwb is that bitbucket?
16:12 kcranstn ah, ok
16:12 kcranstn and the synthesis trees are in what state of pre-processing/
16:12 kcranstn ?
16:12 josephwb not sure i understand
16:13 kcranstn how have the trees in that repo been changed from what is in phylesystem?
16:13 josephwb the subproblems have not been updated there
16:13 josephwb oh
16:13 josephwb pruning of duplicate taxa
16:13 josephwb labels are just ottids
16:15 josephwb is that what you want?
16:15 kcranstn yup, thanks
16:15 kcranstn crap, that repo is huge
16:15 josephwb yeah
16:15 josephwb used to store way more stuff there
16:16 jar286 IMO derived files should never be put under source control
16:17 josephwb this is not supposed to be an official archive. just convenient.
16:17 jar286 IMNSHO
16:18 kcranstn probably should publish this whole repo along with the paper (because not easy to go from ott + phylesystem to these files)
16:18 josephwb new smaller repo, you mean
16:18 kcranstn what’s the difference between Newicks_OTT and Source_nexsons?
16:18 josephwb format?
16:19 josephwb newicks have been processed
16:19 kcranstn are the Source_nexons identical to what is in phylesystem?
16:19 josephwb dups pruned, labels -> ottids
16:19 josephwb versions are involved, but yeah
16:19 kcranstn ok
16:22 kcranstn the trees in Newicks_OTT do appear to be pruned to the ingroup only
16:23 josephwb yes
16:23 josephwb whatever is defined as the ingroup in the curator
16:24 kcranstn when does the pruning happen?
16:24 josephwb it is a separate step in treemachine, before everything else
16:25 kcranstn confused about the comment thread in the suppl doc
16:25 kcranstn “This is sort of irrelevant; definition of ingroup occurs at the curator-app stage. Treemachine doesn't do anything with pruning of anything but duplicate taxa.”
16:25 josephwb to be clear: outgroups are not pruned, they are not read in to begin with. only dups are pruned (or tips without any mapping)
16:26 kcranstn so, the outgroup is not “pruned” it is “not imported”?
16:26 josephwb ok
16:27 kcranstn the inputs to treemachine are trees with ingroup + outgroup, but outgroup discarded
16:27 josephwb i guess
16:27 kcranstn or is the removal of outgroup done in another sofware packate?
16:27 kcranstn software package?
16:27 josephwb i would say the curator app is where the "pruning" occurs
16:28 kcranstn no, because the curator writes the tree with outgroup + ingroup to phylesystem
16:28 josephwb but fine, treemachine
16:28 josephwb reads in nexson, stops at ingroup node
16:28 kcranstn ok
16:29 josephwb so treemachine does not know anything about "outgroups"
16:30 kcranstn only that it only reads in descendents of the node labelled as ingroup / focal clade
16:30 josephwb yeah
16:30 kcranstn so description of ingroup important
16:30 josephwb but that is a curation step
16:30 josephwb not an analysis step
16:31 kcranstn it is both
16:31 kcranstn it is a piece of metadata entered by the curator, which is then used by treemachine when importing trees
16:31 josephwb i guess...
16:32 kcranstn the TAG does not include outgroups, but the curator app shows trees with outgroups, so we need to explain how those get removed
16:33 josephwb sounds a little semantic. it is immaterial what software reads in the nexson; the curator is what determines the ingroup (and so, what, if anything, doesn't make it into analysis)
16:33 josephwb treemachine does do the dup pruning
16:34 josephwb that is important, because alternative pruning is possible
16:34 kcranstn but not done for this version
16:34 josephwb what isn't done?
16:34 kcranstn sorry, forget that last comment
16:34 kcranstn you are talking about duplicate pruning
16:35 josephwb yes
16:35 josephwb if there are a lot, alternate pruning is possib;e
16:35 kcranstn how do we pick which node is pruned?
16:35 josephwb arbitrary
16:36 kcranstn repeatable? or random?
16:36 josephwb that is why it is important to have those newicks
16:36 kcranstn agreed
16:36 josephwb not random, but arbitrary
16:36 kcranstn we just keep the first one in the file? or some other rule?
16:36 josephwb erg. i can look. probably either first or last.
16:37 kcranstn there is a question in the doc about that
16:37 josephwb ok, i will look
16:38 josephwb is "arbitrary" not sufficient
16:39 josephwb rotating clades will lead to alternate prunings. curator has some ordering, nexson reader has its own, pruner traverses in some way.
16:40 kcranstn arbitrary is not repeatable unless you know what rule you are using
16:40 josephwb someone processing the same nexson may get a unique newick string
16:40 josephwb does it have to be repeatable? we provide the processed trees
16:40 kcranstn you mean running the same treemachine code a second time might result in different pruning?
16:41 josephwb no
16:41 kcranstn why can’t we just say what rule we use?
16:42 josephwb for a full description, we would need to provide: 1) how the curator writes the nexson (and maybe which nexson version), 2) how thenexson reader constructs the tree, 3) how the pruner traverses the tree
16:42 josephwb postorder, preorder
16:43 josephwb [i'm not trying to be difficult ;) ]
16:43 josephwb there are just so many steps involved where different decision could give different results
16:44 kcranstn I understand that, but I want to be clear about what decisions we made for this version of the tree
16:46 josephwb i thin khow the curator writes the nexson is just as relevant as any other step, right?
16:46 josephwb but that level of detail is silly
16:46 kcranstn I don’t think it is
16:47 kcranstn we say in the text “we prune off all but one exemplar”
16:47 kcranstn explaining how we do that seems important
16:47 kcranstn how we choose the exemplar
16:47 josephwb if a clade is rotated in the curator, we can get a different tree
16:48 josephwb nevertheless, i will get you the answer
16:48 kcranstn thanks
16:48 josephwb for the last decision
16:59 josephwb i'm digging, but stephen agrees with my take on this
16:59 kcranstn he said it was random
17:00 josephwb it is arbitrary, not random
17:00 josephwb given all of the steps involved, it is essentially random
17:00 blackrim joined #opentreeoflife
17:00 josephwb but treemachine is deterministic
17:27 josephwb don't know if you hashed this out with stephen, but we keep the first encountered tip amongst dups
17:28 kcranstn yup
17:28 kcranstn thanks
17:29 kcranstn output from curator is always consistent (no option to rotate nodes), unless someone deletes a tree and uploads a new newick with different rotation
17:49 blackrim joined #opentreeoflife
18:06 jar286 joined #opentreeoflife
18:28 jimallman joined #opentreeoflife
18:50 josephwb hey jimallman
18:51 josephwb do you do anything with the date string returned from treemachine?
18:51 jimallman hi josephwb. i’ve been on the reoad, but will dig into your recent changes tonight
18:51 jimallman date string… hm, not sure what you mean
18:54 josephwb i guess not, then ;)
18:55 josephwb it is the date that the synthetic tree was constructed
18:55 josephwb you might not touch that
18:57 josephwb it is not in the getsynthetictree call that you use
19:00 jimallman no, i don’t think so
19:00 josephwb ok
20:02 josephwb hey jimallman
21:49 jimallman josephwb: still here?
21:50 jar286 joined #opentreeoflife
22:37 josephwb hi jimallman
22:37 jimallman hi!
22:38 josephwb was just windering when we should change (and break, temporarily) the name of the synth tree
22:38 josephwb were you windering that too?
22:38 jimallman are you talking on devtree/devapi, or production?
22:38 josephwb dev
22:38 jimallman i think the web apps will handle it gracefully, not sure where we’ll see problems except maybe in tests
22:39 jimallman and examples
22:39 josephwb i can flip it over at anytime
22:40 jimallman see my comments here for details on web apps:  https://github.com/OpenTreeOfLife/opentree/issues/629#issuecomment-98072045
22:40 josephwb since we are in the midst of yet another re-running of things, i can just push a db to dev for you to mess with
22:40 josephwb i saw that
22:40 jimallman i need to review what’s been deployed to dev, since i had a setup with lots of funky branches and config tweaks
22:40 jimallman i assume that’s been replaced..?
22:41 josephwb yes
22:41 josephwb should all be master at this point
22:41 josephwb (i think)
22:41 josephwb i can push the same db with just the tree name changed
22:42 josephwb so you can test
22:42 jimallman sure, give it a shot. i’ll be working tonight, so i can try to handle any fallout and create PRs for the needed cahnges.
22:42 jar286 josephwb, for future reference we tend to deploy the ‘development’ branch on devtree and devapi
22:42 josephwb ok, i will get it going right now
22:43 josephwb jar286 yes.
22:43 jar286 I sent email about this a while back, but the policy is probably not ‘discoverable’
22:43 josephwb did i do something different than that?
22:43 jar286 you said master
22:43 josephwb no
22:43 josephwb the code is a master version
22:43 josephwb the db is new
22:43 jar286 oh. just going by what you said above ‘should all be master at this point’
22:43 josephwb ah
22:43 josephwb sorry
22:44 josephwb just swapping out the db
22:44 jar286 oh, sorry, ‘development’ is just for the opentree repo...
22:44 jar286 sorry about that, doesn’t apply to treemachine
22:44 jar286 I’m meddling clumsily
22:44 josephwb welcome to my world
23:05 jar286 joined #opentreeoflife
23:09 josephwb gleesh these neo4j dbs are unweildy
23:10 josephwb jimallman: the db is finally starting to be pushed
23:11 josephwb synth tree name is "opentree3.0"
23:11 josephwb ack. gotta run for my bus now. will be back when i get home.
23:17 josephwb gah, won't make it
23:18 jar286 where do you live? i could give you a ride
23:35 jimallman_ joined #opentreeoflife
23:40 jar286 joined #opentreeoflife
23:43 josephwb jimallman: at the moment, you will need to provide the correct synth tree name
23:44 josephwb the new code has a good default
23:44 josephwb not deployed at the moment, and i think there is a bit of tweaking yet
23:53 josephwb plus, java 8 is needed
23:53 josephwb [not really, but maven thinks so]
23:58 josephwb why does ot10 have a timestamp of 2May?
23:58 josephwb am i pushing to the future!
23:59 jar286 GMT
23:59 jar286 josephwb ^

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary