Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2014-06-16

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:18 kcranstn joined #opentreeoflife
01:46 kcranstn joined #opentreeoflife
02:07 mtholder joined #opentreeoflife
02:38 mtholder joined #opentreeoflife
05:19 7F1AADEMT joined #opentreeoflife
05:25 scrollback joined #opentreeoflife
06:09 17SAAC7EF joined #opentreeoflife
09:59 kcranstn joined #opentreeoflife
11:15 kcranstn joined #opentreeoflife
11:33 kcranstn joined #opentreeoflife
11:48 kcranstn joined #opentreeoflife
11:51 kcranstn joined #opentreeoflife
12:03 mtholder joined #opentreeoflife
13:26 mtholder joined #opentreeoflife
13:54 mtholder joined #opentreeoflife
13:57 josephwb joined #opentreeoflife
13:59 blackrim joined #opentreeoflife
14:25 josephwb blackrim: looking at your changes to gcmdr conf
14:25 josephwb you use "phylesystem"
14:25 josephwb shouldn't it be "phylesystem-1"?
14:26 blackrim where
14:26 josephwb in stephen_laptop.conf
14:26 blackrim oh, I don't clone the repo, instead I use the api. so that dir can be whatever.
14:26 josephwb oh, okay
14:27 josephwb it just conflicts with an existing repo name
14:27 blackrim the api call is in def get_study_opentreeapi(studyid, studyloc):
14:27 blackrim in general_tm_utils
14:28 blackrim yeah, i dont' clone any of those repos. just use the apis
14:28 josephwb gotcha
14:33 mtholder joined #opentreeoflife
14:35 mtholder note that if you were to clone things, https://github.com/OpenTreeOfLife/phylesystem *is* a repo. It is the top level (umbrella) repo. https://github.com/OpenTreeOfLife/phylesystem-1 is the one (and only) shard in that repo. If we add more shards later, I'll update the scripts in https://github.com/OpenTreeOfLife/phylesystem
15:01 blackrim josephwb: I am adding the pg_ to the names in the files_forsub2.0
15:10 josephwb ok
15:18 josephwb blackrim: got things configured correctly, and running some tests
15:18 josephwb lots of 'missing property' warnings
15:18 josephwb crap. died.
15:19 blackrim yeah, we can turn that off. you might need to pull again real quick because I just made a change to fix one thing that was dying
15:19 josephwb in tree_reader.py, read_tree_string
15:19 josephwb ok, will do
15:19 blackrim that will likely be the java error. might check the TEST file
15:21 josephwb ok, same error in read_tree
15:21 josephwb i'll track it down
15:21 blackrim is it the java thing spitting into that file?
15:21 josephwb ooph, TEST is empty
15:22 blackrim hmph, that might be something else
15:24 josephwb can you try 2876_6670?
15:25 josephwb nevermind
15:25 josephwb i don't think it is a problem with the study
15:25 josephwb rather, a conf problem
15:25 josephwb it is supposed to write the newick to TEST, yes?
15:25 blackrim ok, i am doing the fungi right
15:26 blackrim yes it is
15:26 josephwb that is where it is broke for me
15:26 blackrim it just writes the TEST in the working directory.
15:26 josephwb the trashlog shows that it is processed correctly in treemachine
15:26 blackrim you can check whereever you have generallogfileloc in conf
15:27 * blackrim is going to get some bread
15:31 * blackrim has returned from the bread adventure
15:41 mtholder joined #opentreeoflife
15:43 josephwb what version of ott is the curator currently using?
15:44 mtholder whatever taxomachine serves is used by the curator
15:44 josephwb ok
15:44 mtholder (the curation app does not have any other taxonomic resources other than taxomachine)
15:55 blackrim seems like we switched from unclassified_direct and inherited to unclassified
15:55 blackrim 4107956|352914|fungal sp. 3 CCG-2012|species|ncbi:1171858||unclassified,sibling_higher|
15:55 blackrim adding this to treemachine
16:00 mtholder joined #opentreeoflife
16:19 mtholder jimallman. Do you happen to know of a commit to phylesystem-1 or phylesystem-0 that shows the nexson generated by rerooting in the current version of the curator app?
16:20 mtholder I can generate one, if needed.
16:20 jimallman mm, i’d have to look. i know i did some re-rooting recently…
16:21 mtholder I can dig through the commits. no worries..
16:21 jimallman this one, maybe? https://github.com/OpenTreeOfLife/phylesystem-1/commit/dfe565ca6cd03f4bb58e25a6f1a1640140a6de5c
16:22 mtholder I'm looking for one that changes "ot:rootNodeId"
16:22 mtholder I'll just generate one locally. no prob.
16:23 jimallman ok. let me know if something’s not changing, that should,
16:23 mtholder I think that commit that you posted is a curator asserting that the root is not arbitrary. It looks fine to me, just less of a change than I need for testing the "smart diff" tool.
16:24 josephwb i can confirm that mtholder
16:24 josephwb i.e. asserting not arbitrary
16:25 mtholder thanks
16:25 josephwb no re-rooting there (my trees are always correctly rooted 8-|)
16:29 mtholder ugh. just noticed (despite glanced at it many times) that we are using "ot:substitutionCount" for what phylografter displays as "substitutions per site"
16:29 mtholder (in branch length mode)
16:32 towodo joined #opentreeoflife
16:37 blackrim towodo: running synth on just fungi with just the hibbett tree to make sure things are good. something definitely funky with some of the other trees in fungi that we had been using. we will see. more to come
16:38 jimallman mtholder: i believe we have an open issue on expected values for inference methods, not sure if that’s related… https://github.com/OpenTreeOfLife/opentree/issues/226
16:39 towodo blackrim, think about what needs to be done about taxonomy and when. I'm swamped this week and will be traveling the next week.
16:40 blackrim ok, worst case scenario, can we back off back to 2.6 or are there really important things in 2.7->2.8 draft 5?
16:40 towodo umm… I don't know, I'd have to check the commit log (which you could do)
16:40 blackrim ok, don't worry about it just yet
16:41 towodo well, I refreshed NCBI taxonomy for 2.8draft5, but I had refreshed it once before that
16:41 towodo and then there are IRMNG species, that's very new and might help with the microbe diversity issue (don't know, haven't measured)
16:42 jimallman towodo: have you recently made any changes to DNS for devapi.opentreeoflife.org? in the last few minutes i lost access to it (ping “cannot resolve devapi.opentreeoflife.org: Unknown host”)
16:43 towodo nope.  try 'dig'
16:43 jimallman done. no ANSWER SECTION for ‘dig devapi.opentreeoflife.org’
16:44 jimallman devtree reports just fine
16:46 mtholder devapi pings for me (and http works for me, too)
16:46 * jimallman must be having a local or subnet problem… odd.
16:46 mtholder would /etc/hosts mess with it?
16:47 towodo oh yeah, there is no evapi.
16:47 towodo there is no devapi.
16:47 josephwb blackrim: got things working (kinda).
16:47 jimallman just checked.. there’s nothing relevant in my /etc/hosts
16:47 josephwb looks like you implemented the prefix "pg_" in treemachine
16:47 josephwb but not yet in gcmdr?
16:47 jimallman towodo: ? i was just testing API method URLs with curl,a dn it was working ~30 min ago
16:48 towodo sorry, I was confused.  devapi should be ot10
16:48 mtholder it is the same machine, but the DNS for devapi.opentreeoflife.org is working for me.
16:48 towodo works for me.
16:48 josephwb it is passing non-prefixed values, treemachine adds the prefix, but in subsequent gcmdr calls the values are un-prefixed
16:49 jimallman hmm.. i’ll try clearing my local DNS cache.
16:49 jimallman thanks for checking!
16:50 jimallman …. and it’s back with no further action on my part. odd.
16:56 josephwb thanks blackrim. testing birds now
16:57 towodo testing - how a propos -that's what we're about to talk about
16:57 towodo so hold that thought
17:00 towodo 'the video call ended because of an error'
17:00 mtholder peter, cody and I are on the call.
17:01 mtholder or at least we are on a call
17:01 mtholder sas, too
17:35 kcranstn joined #opentreeoflife
17:50 kcranstn_ joined #opentreeoflife
17:53 kcranstn joined #opentreeoflife
17:56 kcranstn joined #opentreeoflife
18:02 kcranstn joined #opentreeoflife
18:05 josephwb joined #opentreeoflife
18:07 kcranstn joined #opentreeoflife
18:10 kcranstn joined #opentreeoflife
18:11 towodo still trying to fix google hangouts
18:13 kcranstn joined #opentreeoflife
18:14 towodo reinstalled google talk plugin, now finally hangouts work
18:16 kcranstn joined #opentreeoflife
18:18 kcranstn I’ve unleashed the hounds (um, I mean curators) onto the new interface
18:24 towodo So I've got to make sure those 2.8draft5 ids are stable.
18:24 kcranstn joined #opentreeoflife
18:27 kcranstn joined #opentreeoflife
18:27 kcranstn on the Duke guest network, and wireless is flaky
18:27 blackrim haha, that explains a lot
18:30 kcranstn joined #opentreeoflife
18:30 josephwb mtholder: here i was assuming QWERTY. lame.
18:31 mtholder I think that at one point karen used dvorak. not sure if she can still use that as an excuse...
18:32 kcranstn joined #opentreeoflife
18:32 kcranstn it's been a long time since I used dvorak
18:32 mtholder my dabbling with dvorak is certainly part of the reason that I'm a terrible typis
18:32 mtholder the other is my total lack of dexterity
18:32 josephwb pretty funny how a single character transition changes the whole meaning of the message
18:32 kcranstn made it rather snarky, didn't it?
18:33 kcranstn scrollback seems less bothered by flaky wireless, FYI
18:36 kcranstn jimallman - you around?
18:37 jimallman kcranstn: hi! here now
18:37 towodo aoeuidhtns
18:39 kcranstn I am going to hide the items in the About menu that have no content (synthetic tree manager, materials and methods, etc). It seems that this requires editing the menu in each of the *.html files under opentree/webapp/views/about. That seem right?
18:40 towodo "we are not ready" should be "we are now ready" ? but I think people will figure it out
18:41 towodo have been reading about 'partial truth' over the weekend, this is a good example
18:41 kcranstn @towodo - I sent a correction
18:41 towodo oh. in a separate threads. I see
18:53 jimallman kcranstn: yes, the menu is repeated in all About pages. i’m happy to do this, if you like.
18:55 kcranstn already done
18:55 kcranstn now I am just fighting with local deployment...
18:56 kcranstn I might just pull request the branch for you to check
18:57 jimallman absolutely, glad to do it
18:57 jimallman (there have been some tweaks in my local test environment, from dev => devtree for instance)
19:00 kcranstn ok, done
19:03 * jimallman is cleaning up the About drop-down nav menu …
19:05 blackrim josephwb: ok, fixed that one bit we were talking about
19:05 blackrim towodo: have the taxonomy over to romina to see what they think
19:06 towodo good.
19:06 blackrim i noted that you went from unclassified_higher or lower to just unclassified. that is correct right (fixed the import to reflect that)
19:07 towodo that's right, and necessary.  I think I sent email about this earlier, and it's on the ott download page
19:07 towodo unclassified_indirect is still there
19:07 towodo but means something different from unclassified.
19:08 josephwb blackrim: saw that. trying metazoa now...
19:11 blackrim towodo: on the note about stability -- is it too hard (expensive , etc.) to have a server running a new synth db that I could have another set of my d3 files pointing to? just as a way of having a stable synth and pages (what it is now) and a development synth with just another set of d3 pages pointing to it?
19:12 towodo Yes, but why not use the development system (ot10)?  that's sort of what it's for.
19:13 jimallman kcranstn: OK, new About menu is live on devtree.
19:14 jimallman i’d like to spiff up the Contact page a bit, maybe add the Kiwi IRC button we use in the main opentree repo: https://github.com/OpenTreeOfLife/opentree/blob/master/README.md
19:14 guest|46644 joined #opentreeoflife
19:14 guest|46644 this is jima, doing a quick test of Kiwi link from GitHub... yep, she's a working
19:14 guest|46644 left #opentreeoflife
19:15 kcranstn thanks, @jimallman!
19:16 blackrim is this ot10?
19:16 towodo blackrim, FYI, the files.opentreeoflife site is now under version control… so if you change files there you might also want to change the original (repo originals are copied to files dir from time to time)
19:16 towodo 'this' = ?
19:16 blackrim sorry is ashby ot10
19:17 towodo ot10 = devapi = ashby
19:17 towodo it runs phylesystem-api, oti, taxomachine, treemachine
19:19 blackrim ok, so the d3 files are already there, I can just make other copies I suppose.
19:19 blackrim how can I change the links here http://dev.opentreeoflife.org/about/developer-resources
19:19 jimallman blackrim: these are in web2py now.. just a sec..
19:20 jimallman https://github.com/OpenTreeOfLife/opentree/blob/master/webapp/views/about/developer_resources.html
19:21 jimallman if you have cloned opentree repo, it’s there. if you prefer, just branch and edit through-the-web, and make a pull request.
19:21 blackrim OK, I will branch, edit and pull request
19:21 blackrim oh, wait, I already have it cloned. will do that.
19:24 mtholder joined #opentreeoflife
19:26 blackrim ok, i pushed, but I suppose it will need to be pulled on that main server (I don't know how to do that but whenever someone does it, it should be better for the d3 bit)
19:27 blackrim towodo: ok, is there a treemachine instance already up on that machine?
19:27 towodo blackrim, if you make new files in the files directory on ot10, that's fine, they won't be deleted, but they won't persist on 'release' unless they go into git.
19:27 towodo ot10 runs phylesystem-api, oti, taxomachine, treemachine
19:28 towodo its treemachine provides the tree you see when you go to devtree.opentreeoflife.org
19:28 blackrim ok, is that the one that we see in dev.opentreeoflife
19:28 mtholder no
19:28 towodo no
19:28 mtholder that is on ot12 (aka api.opentree...)
19:28 blackrim ok
19:28 towodo hmm. not that either.
19:29 towodo dev.opentreeoflife.org -> ot9 to get the tree
19:29 towodo api.opentreeoflife.org -> ot12 to get the tree
19:29 towodo it's not consistent, it's that way to avoid interference between tree release and api release
19:30 blackrim so then I can replace the treemachine on ot10 without major issues (and then make some copies of the d3 bits so they point there)
19:30 towodo eventually the overall production system will have only one treemachine.  right now it has two
19:30 towodo yes.
19:31 towodo ot10 is serving files.opentreeoflife.org … that's also temporary I think, it probably should be on ot9 or ot12 ...
19:32 blackrim ok, I will make a synth just of taxonomy and try and put it over there
19:32 towodo the only thing to watch out for is if taxomachine on ot10 were broken then the curator app would also be open, but supposedly that's among friends
19:33 towodo that's fine with me.  at worst we can revert to the old database (you might want to move it aside in the downloads dir)
19:34 blackrim the treemachine.db.tgz?
19:34 towodo here, I'll do it.
19:35 blackrim just doing it copying to treemachine.db.lastgoodone.tgz
19:35 towodo ouch. no need to copy, it's big. mv would be fine.
19:35 blackrim ok, will del the old one
19:36 towodo doesn't matter now, i was thinking about time not size. machine is hung up now while copy happens
19:36 towodo mv is instantaneous
19:37 blackrim oky doky
19:37 jimallman blackrim: should i deploy your changes? or have you already?
19:37 blackrim go for it (don't know how so surely faster for you)
19:38 towodo blackrim, you remember how to do push.sh install-db , yes?
19:39 towodo oops, that's push-db, not install-db
19:39 blackrim we will see. it has been a while but I was going to look back at https://github.com/OpenTreeOfLife/deployed-systems. will probably need an updated ot10.config because those seem to change a bit
19:39 blackrim i thought it was push.sh
19:40 * jimallman is already pushing…
19:40 mtholder the script is opentree/deploy/push.sh
19:40 mtholder you have to give it a command
19:40 mtholder push-bd is a command
19:40 mtholder db not bd
19:41 towodo if everyone sticks to their own components when pushing there should be no interference.  maybe an unlikely race condition on the log file.
19:41 blackrim i see, is there an updated readme somewhere? i am not doing it right now so if not, i will just ask when i am there
19:41 mtholder https://github.com/OpenTreeOfLife/opentree/tree/master/deploy
19:41 jimallman as a rule, always pull the master branch of deployed-systems before you deploy
19:42 towodo I think the readme is still valid as far as it goes.
19:42 towodo but now you don't need to prepare your own config file.
19:42 blackrim readme says $ cd opentree/deploy
19:42 blackrim $ ./push.sh -c ../../deployed-systems/development/ot3.config opentree
19:42 towodo create the .tgz as directed, then say ./push.sh -c deployed-systems/development/ot10.config push-db
19:42 jimallman yes, that’s typical now, assuming side-by-side repos for opentree and deployed-systems
19:43 blackrim oh, i see, the one that mark sent is more up to date. got it
19:43 towodo looks good to me.  did you create the tar file with . = graph.db ?
19:44 blackrim don't have one yet so not created yet
19:44 jimallman fyi - https://github.com/OpenTreeOfLife/deployed-systems/blob/master/README.md#running-the-push-scripts is actually an accurate description with side-by-side repos. but you can skip the final argument ‘opentree’ if you just want to install all default components for (in this example) ot3
19:45 towodo during development nobody should be installing all default components… just the ones they're working on ...
19:46 towodo or if they are using an unshared server, then it's ok
19:50 jimallman ah, good point.
20:04 jimallman mtholder: i’m retracing Romina’s steps, importing some TreeBASE studies. # 12586 works, but it’s suprisingly slow to import and load (~2 min) into the curation app. Looks like we’re returning from import with lots of work left to be done? because subsequent loading into the app is MUCH quicker. maybe we’re taxing the server today?
20:05 jimallman treebase study #13306 is slower still, may be broken in fact. i haven’t seen it finish importing yet.
20:05 * jimallman should put all this in a ticket, sorry, going there now
20:08 jimallman i see that #13306 has generated a web2py ticket, chasing this now...
20:08 mtholder on ot11?
20:11 jimallman yes, apparently a key error in peyotl:
20:11 jimallman https://github.com/OpenTreeOfLife/peyotl/blob/cf0a03937081d35b677f1c188b0b7bd6059738be/peyotl/external.py#L181
20:11 jimallman i’m creating a ticket for this, with all details.
20:12 mtholder I see the prob. stand by...
20:13 jimallman https://github.com/OpenTreeOfLife/peyotl/issues/16, just fyi
20:21 towodo oh my goodness… libre office understands svg ...
20:23 mtholder ok jim, I think it is fixed and deployed. I'll see if I can import 13306...
20:26 mtholder It went in quickly (ot_14)
20:33 blackrim ok, what do I need to change to this to send over treemachine database (with new taxonomy) to ot10?
20:33 blackrim ./push.sh -c {configfile} pushdb newlocaldb.db.tgz {app}
20:45 blackrim think i got it. new taxonomy going over as new treemachine now
20:45 blackrim i am out
20:46 jimallman mtholder: sweet! thanks, i’ll close the issue.
20:47 jimallman well, unless we can do something about the slooow imports like 12586
20:55 mtholder joined #opentreeoflife
20:56 mtholder sorry, jimallman. I had to take a call. I'm back. Do you know whether 12586 is slow on the opentree/curator operation or phylesystem-api?
20:57 jimallman i believe it’s in the API, because subsequent loading of the study (after the import) is very fast.
21:20 josephwb joined #opentreeoflife
21:47 mtholder jimallman. the slowness has disappeared. I don't think that it was anything that I did. I suspect that the machine was under heavy load (or some similar issue that does not involve the code).
21:48 mtholder http://devtree.opentreeoflife.org/curator/study/edit/ot_16 is an import of S12586 from treebase. took about 4 seconds.
22:19 mtholder joined #opentreeoflife
22:30 jimallman mtholder: awesome, thanks!
22:47 travis-ci joined #opentreeoflife
22:47 travis-ci [travis-ci] OpenTreeOfLife/phylesystem-api#565 (master - a1ee2e2 : Mark T. Holder): The build passed.
22:47 travis-ci [travis-ci] Change view : https://github.com/OpenTreeOfLife/phylesystem-api/compare/351efc29093f...a1ee2e2bfe25
22:47 travis-ci [travis-ci] Build details : http://travis-ci.org/OpenTreeOfLife/phylesystem-api/builds/27726732
22:47 travis-ci left #opentreeoflife
22:49 josephwb joined #opentreeoflife
23:34 mtholder joined #opentreeoflife

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary