Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2015-05-19

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:43 jar286 joined #opentreeoflife
01:48 jar286 joined #opentreeoflife
02:47 josephwb joined #opentreeoflife
02:48 kcranstn joined #opentreeoflife
07:14 mtholder joined #opentreeoflife
10:48 josephwb joined #opentreeoflife
11:04 mtholder joined #opentreeoflife
11:13 kcranstn joined #opentreeoflife
11:37 jar286 joined #opentreeoflife
11:50 josephwb joined #opentreeoflife
11:52 josephwb joined #opentreeoflife
12:31 kcranstn joined #opentreeoflife
13:09 josephwb joined #opentreeoflife
13:44 josephwb joined #opentreeoflife
14:46 mtholder joined #opentreeoflife
15:04 jar286 joined #opentreeoflife
16:02 jar286 josephwb, hangout starting now, is cody around?
16:04 josephwb on our way
17:12 mtholder left #opentreeoflife
19:12 josephwb jimallman i (believe i) fixed the arguson issue. can you please take a look/test?
19:12 jimallman will do. thanks! (it’s on dev now, right?)
19:12 josephwb yes
19:12 josephwb https://github.com/OpenTreeOfLife/treemachine/issues/180
19:13 josephwb i tried the curl calls in that issue
19:13 josephwb will the tree browser use that new code, or does something need to be restarted?
19:16 josephwb oh, does "Loading supporting studies..." still not work?
19:16 josephwb jimallman ^
19:18 jimallman “Loading supporting studies” has actually been doing the AJAX fetch as intended. It only looks broken because we’ve  been getting an empty list for matching studies. I’ll add an error message so we can tell the difference.
19:19 jimallman the curl calls in #180 should match what the app is sending.
19:19 jimallman checking now…
19:19 josephwb curl calls work
19:20 jimallman odd. i’m still seeing empty study lists in the webapp…
19:21 josephwb are they using different phylesystems?
19:22 jimallman perhaps. the webapp is calling api.opentreeoflife.org, which (afaik) is the “correct” oti instance to get this information.
19:22 josephwb is this the issue that jar286 uncovered?
19:23 jar286 dev webapp should be calling dev phylesystem/api
19:23 jar286 if it’s not then the config needs to be fixed
19:23 jar286 devtree -> api  is not correct any more
19:24 jar286 exactly because of the current situation (testing one against the other)
19:24 jimallman OK, will tweak the configuration now.
19:25 josephwb what is the curl call using?
19:26 jimallman curl calls in #180 are generally using devapi
19:33 jimallman ok, i’ve updated the devtree config in deployed-systems/master. would you mind pulling this change and pushing with the new config? or should i?
19:33 jimallman josephwb: ^
19:34 josephwb pushing what?
19:35 josephwb oh, the new treemachine?
19:35 jar286 jimallman I suggest you push devtree and joseph push devapi
19:35 josephwb do i need to repush devapi?
19:35 jimallman devtree is the sticking point in this case. i’ll push from here.
19:35 jimallman (no need to update devapi, i think. the webapp is just calling the wrong host)
19:35 josephwb ok
19:38 josephwb let me know when it is up and running jimallman
19:38 jimallman i’m testing it now…
19:40 jimallman i’m getting consistent results (just 3 studies) now for node [Phaseolus + Glycine + ...]
19:42 josephwb me too
19:42 josephwb but some still do not work, it seems
19:42 jimallman checking the second test in the issue...
19:42 jimallman ?
19:42 josephwb https://devtree.opentreeoflife.org/opentree/argus/ottol@81461/Aves
19:43 josephwb try edge leading to Palaeognathae
19:43 josephwb curl seems to work
19:44 josephwb curl 'https://devapi.opentreeoflife.org/treemachine/v1/getSyntheticTree' -H 'Content-Type: application/json' --data-binary '{"treeID":"opentree3.0","format":"arguson","maxDepth":"3","subtreeNodeID":"788774"}'
19:46 jimallman hm, the “good” news is that Palaeognathae has consistent results (empty list) as both the target node and in its parent’s view.
19:46 jimallman the bad news of course is that’s apparently a bogus result.
19:48 jimallman hm, and it looks like there should be matching studies:
19:48 jimallman supportedBy: ["pg_420_522_a2c48df995ddc9fd208986c3d4225112550c8452",…]
19:48 jimallman 0: "pg_420_522_a2c48df995ddc9fd208986c3d4225112550c8452"
19:48 jimallman 1: "pg_2926_6757_a2c48df995ddc9fd208986c3d4225112550c8452"
19:48 jimallman 2: "taxonomy"
19:48 jimallman (that’s the arguson ‘supportedBy’ value for Palaeognathae, in the Aves view)
19:48 jimallman two matching studies, but neither is found
19:50 jimallman …and the captured curl call fails, as expected:
19:50 jimallman curl 'https://devapi.opentreeoflife.org/oti/v1/singlePropertySearchForStudies' -H 'Origin: https://devtree.opentreeoflife.org' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.152 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'Accept: */*' -H
19:50 jimallman 'Referer: https://devtree.opentreeoflife.org/opentree/opentree3.0@788774/Aves' -H 'Connection: keep-alive' --data 'property=ot%3AstudyId&value=pg_2926&verbose=true' --compressed
19:52 josephwb what's going on?
19:52 jimallman the study is there (via phylesytem api): http://devapi.opentreeoflife.org/v2/study/pg_2926
19:52 jimallman … so i’m guessing the oti is out of date, or still failing to index some studies
19:52 jimallman see link above, the study IS in phylesystem-0
19:52 josephwb pg_420 works for other nodes:
19:52 josephwb https://devtree.opentreeoflife.org/opentree/opentree3.0@788776/Neognathae
19:53 jimallman interesting!
19:55 jimallman hm, it looks like we try to load each study in turn. if one fails, we stop trying.
19:55 josephwb yeah
19:56 jimallman so i’ll add a sensible error message instead of “Loading…” and we should focus on the failed-indexing bug from yesterday.
19:59 jimallman https://github.com/OpenTreeOfLife/oti/issues/40
19:59 jimallman so we’re back to the challenge of getting a fresh version of ot-base?
19:59 jimallman or oti, i guess
19:59 jimallman we can do this!
20:00 jimallman http://i.imgur.com/q7BBjPb.jpg
20:01 josephwb i'll ask cody if he can do it; i'm not sure i'm capable of anything at this point
20:15 jimallman josephwb: can we try a trivial change to oti master? just something tiny to prompt for git-refresh?
20:15 * jimallman is pushing a simple error message when supporting study fails to load
20:16 jimallman ok, that’s working (should cut down on some confusion)
20:20 josephwb about to push an update to ot-base
20:28 jimallman i’m also guessing you might need to commit a tweak to oti, to get it to recompile (no?)
20:28 jimallman josephwb: ^
20:33 josephwb yes
20:36 jar286 yes, 2926 is one of the 8 that didn’t get indexed (see other bug)
20:37 jar286 the losers are pg_2926.json pg_2739.json ot_29.json ot_19.json ot_38.json ot_39.json
20:37 jar286 ot_32.json ot_33.json
20:37 jar286 on dev
20:38 jar286 josephwb, you can just use the -f flag with a push of oti
20:38 jar286 didn’t you implement that?
20:38 josephwb yes
20:38 josephwb had to make a compensatory change to oti for the ot-base update
20:38 josephwb crap, didn't compile...
20:38 josephwb did locally
20:39 jar286 ok, gotta run…
20:50 josephwb ok jimallman i finally got things compiled on dev
20:51 jimallman sweet. i’ll test it shortly
20:52 josephwb thanks. i don't know how to test it if the curl calls are working fine
20:53 josephwb yaya!: ERROR loading supporting study: pg_2926
20:53 josephwb yay
20:53 jimallman why is that good again?
20:53 josephwb we get an error now
20:53 jimallman i think that’s my message, actually (in tree-viewer, right?)
20:54 josephwb there was no error before i deployed my change though
20:54 josephwb recall the exception refused to include the error
20:54 josephwb error message
20:55 josephwb or is that all your changes?
20:55 josephwb does oti take a while to index?
20:55 jimallman are we getting an errorMessage back from AJAX? because that ttext (above) is mind, i’m afraid
20:55 josephwb i don't know how to do the AJAX thing
20:55 jimallman yes, oti does take awhile and chatters at you while it’s indexing.
20:55 jimallman maybe we need to trigger that explicitly.. looking that up…
20:56 josephwb i just compiled. do i have to do something to instaigate reindexing?
20:56 jimallman yes, like push.sh -c CONFIG_FILE indexoti
20:56 jimallman (or index, or index-db, if you prefer)
20:57 josephwb like that or exactly that?
20:57 josephwb *checking*
20:57 josephwb ok, see it
20:58 josephwb want me to do it?
20:58 jimallman sorry if my last point was unclear. what i mean is that push.sh recognizes a few “synonyms” that all do this:
20:58 jimallman https://github.com/OpenTreeOfLife/opentree/blob/master/deploy/push.sh#L144-L145
20:58 jimallman sure, go for it!
20:58 josephwb will do
20:58 josephwb should i screen this?
20:59 jimallman not a bad idea. it will be a long-running console with the status of each study
20:59 josephwb thanks
20:59 jimallman this is probably already being logged in deployment logs, though
20:59 jimallman so not a big deal. the only concern is whether you disconnecting might abort the indexing
21:00 josephwb requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR
21:01 jimallman hrmm
21:02 josephwb emailed you the error
21:02 jimallman if that’s all you’re seeing, we’ll need to check the oti (neo4j?) logs i guess
21:04 josephwb do i need to do this: https://github.com/OpenTreeOfLife/opentree/tree/master/deploy#initializing-the-oti-database
21:04 josephwb no, probably not
21:05 jimallman this is almost certainly the source of that 500 error: https://github.com/OpenTreeOfLife/oti/blob/master/index_current_repo.py#L47-L48
21:05 jimallman my bad! i meant the individual indexing request: https://github.com/OpenTreeOfLife/oti/blob/master/index_current_repo.py#L57-L58
21:06 jimallman josephwb: ^
21:06 josephwb i see
21:07 josephwb something wrong with the url?
21:07 jimallman though it might just be the first link above (call to study_list method). this is a simple GET, let’s try it in a browser.
21:08 josephwb brb
21:08 jimallman josephwb (when you’re back): do you see a line like this in the log? “Using the studies from: …”
21:09 jimallman this should give us the value of api_url from index_current_repo.py
21:10 jimallman once we have that value, we should be able to just paste (api_url + "study_list”) into a browser’s address box.
21:10 josephwb Using the studies from: http://devapi.opentreeoflife.org/phylesystem/v1/../
21:12 jimallman ok, i’m trying http://devapi.opentreeoflife.org/phylesystem/v1/../study_list
21:12 josephwb jimallman ^
21:12 jimallman the browser might nudge this address to http://devapi.opentreeoflife.org/phylesystem/study_list
21:13 jimallman in any case, there IS an error there, from web2py. i can chase this from here…
21:13 josephwb great
21:13 jimallman but weird, since we haven’t touched this code in a long time
21:13 jimallman i’ll give a shout here if i find something interesting
21:14 josephwb ok
21:16 josephwb fwiw your working link from before no longer does:
21:16 josephwb http://devapi.opentreeoflife.org/v2/study/pg_2926
21:17 jimallman yeah, i think this is badly borked, maybe by some of my recent peytol changes?
21:18 josephwb oh, i'm sure it is me
21:18 jimallman “<type 'exceptions.ValueError'> No git repos in ['REPO_PAR']"
21:18 jimallman is coming from the default web2py controller in phylesystem-api
21:19 jimallman REPO_PAR looks suspiciously like a placeholder
21:22 jimallman and yet, we should have deployed branch ‘master’ of peyotl, which doesn’t have any of these changes
21:22 jimallman maybe it’s you :)
21:23 josephwb *sigh*
21:23 jimallman i’m checking to see if that error message predates these changes.
21:29 josephwb this is weird:
21:29 josephwb phylesystem-api -> ../../repo/phylesystem-api
21:30 josephwb oh, but it is old. still, weird.
21:33 jimallman it looks like the phylesystem-api (web2py) config file isn’t set up properly.
21:33 josephwb not this: https://github.com/OpenTreeOfLife/oti/commit/b3c961b7c35d2d2fb4290e999ea358bcb87c35ab
21:34 jimallman it should have two variables set (repo_path and repo_par), but they’re still showing placeholders
21:34 jimallman good point about jar’s changes!
21:35 jimallman i forgot that was new
21:35 josephwb index_current_repo.py changed on 12 May
21:36 jimallman i’m still walking the backtrace from your email. it points to this config problem in phylesystem-api.
21:36 josephwb ok. i can't help with this. good luck.
21:36 josephwb if the ot-base/oti changes are bad, i can revert to master and try indexing again
21:37 jimallman oh, you can actually. do you know if oti was recently rebuilt on devapi?
21:37 josephwb yes, i did it
21:37 jimallman from master (with jar’s recent changes)?
21:37 jimallman or with that stuff merged from master, i guess..?
21:37 josephwb with my changes
21:38 josephwb built off of master
21:38 josephwb branch test-fix
21:38 josephwb (same branch name for oti and ot-base)
21:38 josephwb i had to update oti to work with changes to ot-base
21:38 jimallman ok. have you merged recent stuff (incl. jar’s commit) to that branch of oti?
21:38 josephwb aren't multiple interdependent repos phun?
21:39 jimallman awesome sauce.
21:39 josephwb jar's commit?
21:39 josephwb let me check
21:39 jimallman https://github.com/OpenTreeOfLife/oti/commit/b3c961b7c35d2d2fb4290e999ea358bcb87c35ab
21:39 josephwb it is in there
21:40 josephwb https://github.com/OpenTreeOfLife/oti/commits/test-fix
21:40 jimallman gotcha. just tried a comparison view, and it confirms:  https://github.com/OpenTreeOfLife/oti/compare/test-fix...master
21:41 jimallman ok, that’s helpful. i’ll chase the api-config error and see where it leads.
21:41 jimallman thanks again.
21:44 josephwb it doesn't matter that i am on the development branch of opentree, does it?
21:45 jimallman no, that’s appropriate
21:46 * jimallman must attend to the animals. back in a bit..
22:06 josephwb good luck jimallman

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary