Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2014-06-23

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:27 kcranstn joined #opentreeoflife
01:54 towodo joined #opentreeoflife
02:08 towodo joined #opentreeoflife
08:50 scrollback joined #opentreeoflife
09:46 kcranstn joined #opentreeoflife
11:06 towodo joined #opentreeoflife
11:56 kcranstn joined #opentreeoflife
13:19 towodo joined #opentreeoflife
13:34 josephwb joined #opentreeoflife
14:10 towodo joined #opentreeoflife
14:14 kcranstn joined #opentreeoflife
14:14 jimallman kcranstn: how’d it go?
14:14 kcranstn good!
14:14 jimallman did you crowd surf?
14:15 kcranstn generally not done at these things, although it was sufficiently well-attended such that surfing would have at least been possible
14:15 jimallman :)
14:20 blackrim joined #opentreeoflife
14:21 7F1AAIML5 joined #opentreeoflife
14:22 blackrim josephwb: ok, i can make some procedures in treemachine to dump a taxonomy tree with ottids as the labels
14:22 blackrim then i can make a procedure as part of the mrp dump which is the newick with the ott ids
14:22 josephwb sounds good
14:22 josephwb canid example takes 150.253 secs
14:22 blackrim ok, that is a little scary cause that is very very small and it goes by polynomial time
14:23 blackrim So we can start with dipsacales which is our smallest example
14:44 lcoghill joined #opentreeoflife
14:47 josephwb blackrim: where can i find the supinfo from our science submission? googledoc?
14:47 kcranstn dropbox
14:47 kcranstn you have access?
14:47 kcranstn dir = synth_for_Science
14:47 josephwb nope
14:48 kcranstn just a sec
14:49 kcranstn share
14:49 kcranstn d
14:50 josephwb thanks
14:54 blackrim josephwb: I have the trees output with the right uids. Just give me a sec to get the taxonomy in the right format and you should be good for a test
15:00 josephwb blackrim: so, that bird tree is unrooted, right?
15:00 blackrim correct
15:01 josephwb doesn't look great
15:04 blackrim not surprising
15:04 josephwb sent you a recent one that i made
15:05 lcoghill jimallman: I think it was mentioned before, and may still be a wip, but the bib refs page on devtree is blank for me. (xubuntu / chrome).
15:06 jimallman yes, i can chase this down.. but i’m pretty sure it’s just data in oti or treemachine out of date (or empty). i’ll confirm, though.
15:06 blackrim josephwb: thanks. yeah, we can demonstrate this as perhaps a case showing the different methods
15:06 jimallman lcoghill: ^
15:15 blackrim josephwb: ok, i have it
15:15 blackrim what is the method called?
15:15 blackrim mls right
15:16 josephwb blackrim yup
15:20 blackrim ok, there is a load_mls.py in gcmdr. it has instructions at the top for getting a taxonomy tree as well
15:20 blackrim pull gcmdr and treemachine
15:21 josephwb *pulling*
15:22 blackrim little error in load_mls.py , one sec
15:23 blackrim got it. pull again
16:02 josephwb blackrim: added printing out of taxonlist to treemachine
16:03 blackrim sounds good. any results from a test yet?
16:06 josephwb not yet
16:15 josephwb gah. crashes on simple examples i try.
16:16 blackrim even the one that comes with the method? (their examples)
16:16 josephwb no, they work
16:16 josephwb e.g. canids
16:16 josephwb i am getting:
16:16 josephwb java.lang.ArrayIndexOutOfBoundsException: 1
16:16 blackrim ah, missing something somewhere
16:16 josephwb yup
16:16 blackrim maybe it is that taxlabe lstuff?
16:17 josephwb it reads those fine
16:17 blackrim hmph
16:19 josephwb figured it out
16:23 josephwb now stuck again. it thinks my tree(s) is unrooted, but it is.
16:23 josephwb 8o|
16:25 josephwb erg. need to explicitly include "[&R]" in tree definition
16:31 josephwb blackrim: ok, gives exact same result for the 2 examples in our science supinfo
16:32 josephwb that would argue against reviewer #3
16:35 kcranstn joined #opentreeoflife
16:36 blackrim you mean, we get the same as they do
16:36 blackrim yeah, i don't think reviewer 3 knows what they are talking about in regard to the supp figure (TLDR: they are wrong)
16:56 josephwb yes
17:18 josephwb blackrim: trying Aves example
17:18 josephwb ran out of memory with 4 GB
17:19 josephwb re-running with 16 GB
17:24 blackrim wow
17:32 josephwb for smaller datasets, used ~1600% cpu
17:32 josephwb for this big one, just 100$
17:32 josephwb 100%
17:33 blackrim hm, also interesting. maybe it isn't to the stage where it can go parallel yet
17:33 blackrim is it opensource?
17:33 josephwb i only have the jar
17:35 josephwb http://www.atgc-montpellier.fr/supertree/mls
17:35 josephwb do you have a smaller example you want me to try out?
17:37 jimallman lcoghill: fyi, i’ve updated the Bib. References page to understand the new source descriptions.. but we’re a little shy of sources (looks like treemachine on devapi just has a little test data): http://devtree.opentreeoflife.org/about/references
17:38 blackrim josephwb: no source code it seems. lovely. well that is something we can talk about in the paper
17:38 blackrim i will send along an example, one sec
17:45 lcoghill jimallman: looks good. tested and working for me now.
17:45 towodo joined #opentreeoflife
17:46 blackrim josephwb: i fixed something in treemachine for reading nexsons. so for an example do this...
17:46 blackrim taxonomy uid: 659705
17:47 blackrim studies :
17:47 blackrim "pg_424_532", #Lonicera
17:47 blackrim "pg_1962_6580",#Viburnum Clement and Donoghue 2011
17:47 blackrim "pg_915_1802",#Viburnum
17:47 blackrim "pg_915_1803",#Valerianaceae
17:47 blackrim "pg_09_1",#Campanulidae
17:47 blackrim "pg_2828_6578"#caprifolieae Smith 2009 NEW
17:47 blackrim lets see what it does with those (only 600 taxa or so as well)
17:47 josephwb will do
17:50 josephwb made minor cheng to TM
17:50 josephwb change
17:54 kcranstn joined #opentreeoflife
18:08 josephwb birds finished after 2300 seconds
18:08 josephwb 2 trees + taxonomy
18:10 josephwb blackrim: did you break the nexson reader?
18:10 josephwb not correctly reading study 915
18:11 josephwb maybe because it has two trees?
18:12 josephwb when I do:
18:13 josephwb python load_mls.py
18:13 blackrim hm, i ran it right before i sent it. let me try again
18:13 josephwb the trees for pg_915_* are empty
18:18 blackrim got it ; git pull
18:21 josephwb looks good. thanks.
18:31 josephwb erg. one tree has a taxon that is not in the taxonomy (862535)
18:31 blackrim ok, let me see real quick
18:32 josephwb do i need to go one node back in the taxonomy?
18:32 blackrim ah, outgroup
18:32 blackrim hm, just for experimentation, can you prune it from the tree just for the MLS run
18:32 blackrim if we step back it will be too big
18:32 kcranstn joined #opentreeoflife
18:33 josephwb just started it without the tree
18:33 josephwb but can prune too
18:33 blackrim that works too. which tree is it? 09
18:33 josephwb 23 seconds (?)
18:33 josephwb Campanulidae tree
18:33 blackrim yeah
18:33 josephwb yes, 09_1
18:33 blackrim did the run finish though?
18:33 josephwb yes
18:34 blackrim ok, you want to email the resulting tree (you can change the names to readable names first if you like otherwise i will)
18:34 josephwb yup
18:34 josephwb do you want me to put 09 back in before?
18:35 josephwb seems to be the biggest tree
18:35 blackrim if you can prune it and do it , that would be great. if not, send along without it
18:35 josephwb *trying*
18:35 blackrim this should be an interesting test because there is a decent amount of conflict with a couple trees
18:36 blackrim just opened the tree and you could probably just take the larger of the first deep split in the tree. i can send if you like
18:36 josephwb sure
18:36 josephwb there is more than one problematic taxa
18:37 josephwb how the hell did we get stuff done before irc?!?
18:38 blackrim emailed
18:39 blackrim hopefully that solves the problems
18:40 kcranstn joined #opentreeoflife
18:40 josephwb still no-go
18:41 blackrim what is wrong now
18:41 josephwb 849194 is outside the taxonomy
18:41 blackrim checking on it
18:41 josephwb i'll just send the results without the tree
18:41 blackrim ok, i will find where to cut this thing
18:45 blackrim ok, here you go
18:45 blackrim ((((((((((759046,393345),(459096,945854)),459087),1035979),(((69531,916996),965921),1035960)),(((459103,344166),68247),702435)),(((275391,275395),1014066),819508)),((((275370,717595),225985),649885),502668)),(243305,30739)),((((330645,757497),724393),1042137),757503));
18:46 blackrim i will email
18:46 kcranstn do we have a metadata field for fossil taxa?
18:46 blackrim towodo has it in taxonomy
18:46 josephwb working…
18:47 blackrim ok, good. this will have more resolution
18:47 josephwb went from largest tree to smallest
18:47 blackrim yeah :-p
18:47 towodo 90224|90215|Ornithischia|order|ncbi:436488,irmng:10455||edited,extinct_direct|
18:47 josephwb ok, fixing names, will send soon
18:47 blackrim i forgot that it is significantly deeper and didnt' realize that it would be as much of a problem
18:52 blackrim yep. looks good.
18:52 blackrim so the question is what is the limit and how can we scale this in an automated way
18:52 blackrim with the bird one it seems like maybe the memory is the problem? can we try it somewhere else and see if it will run?
18:53 * blackrim brb
19:10 kcranstn @towodo: will the OTT identifers be stable from version to version?
19:10 josephwb blackrim this is running
19:10 kcranstn (talking to james rosindell from OneZoom)
19:11 josephwb will likely finish after i leave early today
19:11 towodo I do my darnedest to keep the ids stable.
19:11 josephwb i've screened it, so will be able to check in on it later
19:11 josephwb i'll sned you the inputs too in case you want to play around with it
19:12 josephwb crap, 16 GB is not enough
19:12 josephwb ^o)
19:12 towodo kcranstn, id stability is implemented using the generic alignment algorithm...
19:12 kcranstn I don’t know what that menas
19:12 kcranstn means
19:13 towodo when a new version is generated, it gets aligned with the previous version.  so the ids in the new version get 'imported' from the old version
19:13 josephwb kcranstn: you don't know "gaa"?
19:13 kcranstn @josephwb: um, no
19:13 towodo so ids should be robust against, say, id reassignment within ncbi or gbif
19:16 mtholder joined #opentreeoflife
19:24 josephwb blackrim: ran out of memory with 32 GB
19:24 josephwb yeah, no way of doing a big tree here
19:25 towodo kcranstn, does that make sense?  it says: here is a node in new version, without an id.  Is there an node in the previous version of the tree that matches it? If so give the new node the id from the old node.
19:30 josephwb blackrim: dies with 62 GB; 20650 names (including internal), 17 trees + taxonomy.
19:31 towodo maybe it's the SHAs?
19:31 josephwb gotta run
19:31 kcranstn @towodo: yes, thanks
19:37 towodo I can't believe how long it takes to prepare a 5 minute talk
19:42 mtholder 4 minutes?
19:42 kcranstn interpretive dance?
19:43 towodo I finally found a use for garageband - for timing the damn thing
20:24 josephwb joined #opentreeoflife
20:35 josephwb joined #opentreeoflife
21:02 josephwb joined #opentreeoflife
21:18 towodo joined #opentreeoflife
22:33 josephwb joined #opentreeoflife

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary