Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2014-07-28

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:02 towodo Comment login: 500 Sorry, there was a server error.
00:02 jimallman OK, i’ll chase that.
00:02 towodo was just going to check the apache error log
00:02 towodo but you can do it if you prefer…
00:03 jimallman sure, i’ll do it.
00:06 jimallman hm, the main webapp (tree browser) is getting an error from GitHub API:
00:06 jimallman No access_token found in data: {'error_uri': 'https://developer.github.com/v3/oauth/#incorrect-client-credentials', 'expires': None, 'error_description': 'The client_id and/or client_secret passed are incorrect.', 'error': 'incorrect_client_credentials'} 'access_token'"
00:06 jimallman checking the CLIENT_ID and CLIENT_SECRET now… probably the wrong domain name
00:09 towodo probably
00:10 jimallman hm, TREEVIEW_GITHUB_CLIENT_ID matches what we have in Bos/tree.config, which is tied to domain tree.opentreeoflife.org… so that should be right
00:10 jimallman would you mind pushing your latest deployed-systems to origin/master?
00:10 towodo ok
00:10 towodo looking up DOI at http://tree.opentreeoflife.org/curator/study/edit/pg_581  - hangs
00:10 towodo ‘an error occured’
00:12 towodo pushed.
00:12 jimallman hm, the call is hanging up on our web2py proxy for some reason… http://tree.opentreeoflife.org/curator/search_crossref_proxy/dois?q=...
00:12 jimallman (the call that looks up DOI, that is)
00:15 jimallman regarding the failed login, there’s a bad GITHUB_CLIENT_SECRET in /home/opentree/repo/opentree/webapp/private
00:17 jimallman i’m guessing your local file ~/.ssh/opentree/treeview-GITHUB_CLIENT_SECRET-tree.opentreeoflife.org is out of date? or missing, so we just kept an old/outdated file from before?
00:18 jimallman (probably a stale client secret from a previously registered tree.opentreeoflife.org app on GitHub)
00:20 jimallman towodo: the DOI lookup is working again now, must have been a brief outage on crossref.org
00:20 jimallman see notes above re: ~/.ssh/opentree/treeview-GITHUB_CLIENT_SECRET-tree.opentreeoflife.org
00:21 jimallman i can push opentree with my current CLIENT_SECRET, then you can refresh yours from ot14… OK to go ahead with this?
00:21 towodo looking. I didn’t do anything to make those files right so they’re probably wrong
00:21 towodo ok
00:22 towodo tree secret ends 3b33, no curation secret.
00:22 towodo I don’t understand where they’re kept on the deployed machine
00:23 towodo somewhere in the web2py tree I suppose
00:25 jimallman right, the latest secret ends 61
00:25 towodo can’t figure out how I could have obtained it, if I had tried.
00:25 jimallman /home/opentree/repo/opentree/webapp/private/GITHUB_CLIENT_SECRET
00:26 jimallman /home/opentree/repo/opentree/curator/private/GITHUB_CLIENT_SECRET
00:26 jimallman these have special names in your local ~/.ssh/opentree stash...
00:26 jimallman i believe i’ve sent email with scp commands that sets the right local filenames… checking now…
00:28 towodo sorry if I lose track of these things. copying info to my journal
00:29 jimallman no problem, re-sent email on the way.
00:29 jimallman agreed this should probably be documented somewhere..?
00:30 jimallman as a general procedure for propagating updated credentials, or recruiting a new push-er…
00:30 jimallman anyway, if i have your latest deployed-systems, i’ll push opentree again now...
00:30 towodo Sure.  Set a URL for a study… seems to be winning… going through the integration tests now
00:31 jimallman $  ./push.sh -c ../../deployed-systems/Cavia/tree.config opentree
00:31 jimallman that should update CLIENT_SECRET files for both tree viewer and curator
00:32 jimallman meanwhile, i’ll add a more sensible error msg for crossref.org outages (vs web2py’s default “Reload this page”.. ugh)
00:33 jimallman OK, push is complete and tree-browser login is working. you should be able to pull the latest SECRET files as shown in the email.
00:33 jimallman those scp commands should work as-is
00:33 towodo tree looks good - maybe the legend should say what red and grey mean on tip labels
00:33 jimallman assuming you have ot14 defined in your .ssh/config, that is
00:34 towodo I don’t… never bothered to learn about .ssh/config. now is as good a time as any
00:34 jimallman yes, probably so. in case it’s not obvious, red are conflicting taxon mapping (w/ no exemplar)
00:35 jimallman re: .ssh/config, it’s a huge convenience: https://gist.github.com/jimallman/d77ae7682e0273ceec95
00:36 jimallman here’s a typical entry for ot14, i have several more of course for OpenTree servers
00:37 jimallman for (many) more options, see    $ man ssh_config
00:51 kcranstn joined #opentreeoflife
00:51 towodo ok, all integration tests seem to pass. I’m declaring victory.
00:52 towodo and with 8 minutes to space
00:52 towodo s/space/spare/
00:52 jimallman go team!
00:59 towodo joined #opentreeoflife
01:12 towodo joined #opentreeoflife
01:23 towodo joined #opentreeoflife
12:38 towodo joined #opentreeoflife
12:55 kcranstn joined #opentreeoflife
13:14 josephwb joined #opentreeoflife
13:24 josephwb jimallman towodo the "Update MRCA" doesn't seem to be working. I get an error at the bottom of the page.
13:24 josephwb Tried on two studies
13:24 josephwb NullPointerException
13:25 josephwb Want the stack trace?
13:26 towodo I can’t do much with the stack trace… maybe email it to Jim?
13:26 josephwb {
13:26 josephwb "exception" : "NullPointerException",
13:26 josephwb "fullname" : "java.lang.NullPointerException",
13:26 josephwb "stacktrace" : [ "opentree.plugins.GoLS.getDraftTreeMRCAForNodes(GoLS.java:105)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGra
13:26 josephwb }
13:26 josephwb ok. is he not around?
13:42 towodo haven’t seen him this morning.
13:51 josephwb ok, emailed him
14:21 kcranstn joined #opentreeoflife
14:39 josephwb towodo: there is a way to search GH issues, right?
14:40 josephwb oh, just "search repository"?
14:45 towodo don’t know.
14:46 towodo yes, the search bar at the top seems to do the trick. I find it confusing too
14:52 josephwb thanks
14:52 josephwb jimallman still not around?
14:56 josephwb guess not
15:26 jimallman josephwb: sorry, was at a dentist appointment
15:28 jimallman re: searching issues on GitHub, yes, the default search is current repository, and results have two “tabs” (code and issues)
15:29 jimallman useful tips here: https://help.github.com/articles/searching-issues
15:56 josephwb thanks jimallman
15:57 josephwb jimallman: I asked you some time ago to map Genus_sp. taxa to Genus. This worked well for "normal" studies, but not for microbe studies. Can we maybe trun that off (i.e. make the curator do it explicitly)?
15:57 josephwb trun == turn (Canadian spelling)
15:58 kcranstn :)
15:58 jimallman tourn
15:59 josephwb Basically: we don't want taxa mapped to taxa AND their ancestor (e.g. Genus_species and Genus)
15:59 jimallman i don’t recall how that mapping was handled, but it’s almost certainly on the server side. (i’m not munging names in the curation app itself.)
15:59 josephwb if "Genus_sp." is the only representative of "Genus", we want to map it
15:59 josephwb If there are species from that Genus, we do NOT want to map "Genus_sp." to "Genus"
15:59 josephwb make sense?
16:00 jimallman ah, that suggests an awareness of the larger context (other mappings), whereas each OTU mapping operation is currently done in isolation.
16:00 kcranstn makes sense to me
16:00 josephwb good
16:00 josephwb treemachine prunes things, but sometimes gets confused if 100s of taxa are pruned
16:00 jimallman josephwb: ^ see my last msg above
16:01 josephwb see that, thanks.
16:02 jimallman i suppose that the initial mapping (during import) could handle this more circumspectly, but OTU mapping within the web app would need to become more circumspect as well (pass along the entire current tree, i suppose). sounds like a slowdown.
16:04 josephwb I suppose it could be a potential validator check: are taxa sampled to ancestors and descendants?
16:16 jimallman hm, yes. so the desired behavior would be to map to either (a) the genus, if none of its species / descendants are found, or (b) just one species, if *other* species are found from the same genus. in any case, map all instances if “Genus_sp.” to the same taxon. does that sound right?
16:21 kcranstn josephwb requested “If there are species from that Genus, we do NOT want to map "Genus_sp." to "Genus””
16:22 jimallman i think that’s what i said, in case (a) above
16:22 kcranstn at any rate, we probably want to provide explicit documentation on what we are doing so that users can tell
16:22 jimallman agreed.
16:23 jimallman i’ll add a peyotl ticket for this, and link here for review/confirmation
16:26 josephwb Desired behaviour for "Genus_sp." taxa: if no other descendants of "Genus" are present, map to "Genus". Otherwise, do not map to "Genus".
16:26 josephwb where "Genus" can represent any taxonomic level
16:26 josephwb e.g. if a tip is Family_sp.
16:38 jimallman ok, so if “Genus” is actually a family, and there are other descendants found inside it, we’d have a child node “Family_sp.” directly under the family (since we don’t know its proper parent). Are there other likely suffixes besides “sp.” for “genus within”, “family within”, etc?
16:41 josephwb Hmm…
16:41 josephwb probably
16:41 josephwb maybe?
16:41 jimallman sounds good!
16:42 josephwb I think "_sp./_sp" covers most
16:43 josephwb I think I garbled things with my horrible example
16:44 josephwb if X is a family, Y is a genus in X, and Y is sampled in the tree, we do not want to map "X_sp." to "X"
16:44 josephwb But if not descendant of X exists in the tree, map "X_sp." to "X"
16:54 jimallman right, we’re on the same page. i’ll write up a proper ticket after today’s hangout.
16:54 josephwb good
16:54 josephwb this is not urgent
16:54 josephwb but curators should be aware
16:58 jimallman towodo: OK for me to push a (needed, low-risk) fix to production? i broke OTU mapping with a typo  :-/
16:58 kcranstn yes
16:59 kcranstn should talk about setting up tests for curator / webapp
16:59 kcranstn that should have been caught
17:00 towodo_ joined #opentreeoflife
17:00 towodo_ isn’t there a hangout now?
17:00 jimallman agreed. i’ll add this to our integration test (we should also save a study after some tests, since the validator is catching some things that don’t cause problems in the app itself)
17:01 josephwb supposed to be towodo
17:01 towodo only 2 people so far, where are y’all?
17:02 josephwb don't see invite
17:02 kcranstn https://plus.google.com/u/0/events/cdkage4uo87uf1p2n5iejjhv8kk?authkey=CJTFgJqc9ubuSg
17:02 josephwb think i am arriving
17:07 pmidford joined #opentreeoflife
18:10 josephwb jimallman I haven't been able to get Update MRCA with any tree
18:14 towodo joined #opentreeoflife
18:45 jimallman josephwb: odd, this one works for me:  http://tree.opentreeoflife.org/curator/study/edit/ot_98?tab=trees&tree=tree1
18:45 josephwb hmm
18:45 jimallman ah, this one gives a different error (TaxonNotFoundException): http://tree.opentreeoflife.org/curator/study/edit/pg_2760/?tab=trees&tree=tree6375
18:46 jimallman it seems this is a brittle operation, will investigate further
18:48 jimallman i’ve added the cURL call (second failure) to https://gist.github.com/jimallman/9f4a4b63f4c8d8a840d3
18:48 jimallman all the ottids (from tips) look sensible… Is it possible that we’re using the wrong version of taxonomy to test for MRCA?
18:50 jimallman fyi, i’ve added another (working) cURL call to the gist above
19:00 josephwb the taxonomy version shouldn't be an issue (I think)
19:01 towodo treemachine taxonomy = 2.6  TNRS taxonomy = 2.8
19:01 towodo but they’re 99% or more identical
19:20 jimallman i thought perhaps the TaxonNotFoundError meant a new taxon ID (from 2.8) that wasn’t found in the current TNRS (2.6). but that shouldn’t happen if the TNRS is newer, unless we’ve filtered out some existing taxa in the latest TNRS.
19:28 josephwb i've only tried on PG trees, so old OTT and probably no new names
19:31 jimallman i tinkered with the cURL call above (the one that generates the TaxonNotFound error), and it seems the problematic taxon ID is 83477
19:31 jimallman when i remove this one, i get the MRCA
19:46 josephwb taxon seems stable across taxonomies
19:50 jimallman here’s what i’m seeing when i download ott2.6, 2.7, 2.8 and test for this taxon ID:   https://gist.github.com/jimallman/13dbd25a3d8f64df3a97
19:51 jimallman it looks like this ID is found in the taxonomy.tsv of all tree versions, but it comes and goes in the other files (hidden.tsv, otu_hidden.tsv, differences.tsv, otu_differences.tsv). any clues here?
19:57 josephwb not for me
21:27 josephwb jimallman: looks like the Update MRCA uses all taxa in a tree, not just the declared ingroup, yes?
21:28 jimallman no, i’m pretty sure i filter to just the ingroup before submitting the request
21:28 josephwb hmm, not working that way for me.
21:28 jimallman (and if there’s no ingroup chosen, we stop and prompt the user to choose one)
21:28 jimallman example? maybe i dropped a stitch there..
21:29 josephwb for the tree i'm looking at, the ingroup are all in a genus, but inferred MRCA is much deeper
21:29 josephwb ot_75
21:32 josephwb i noticed this with some other trees too
21:36 josephwb buts seems to work on some
22:11 jimallman josephwb: i’ve added the actual (cURL) request from ot_75 to the gist:  just now
22:11 jimallman https://gist.github.com/jimallman/9f4a4b63f4c8d8a840d3#comment-1271371
22:12 jimallman It shows the four taxa submitted (the ingroup, as expected) and the rsulting MRCA, which does seem rather distant.
22:45 * jimallman is finally moving all this to a proper GH issue in treemachine...
22:53 jimallman https://github.com/OpenTreeOfLife/treemachine/issues/111
23:40 kcranstn joined #opentreeoflife
23:53 towodo joined #opentreeoflife

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary