Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2015-07-16

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:14 jar286 joined #opentreeoflife
01:48 ilbot3 joined #opentreeoflife
01:48 Topic for #opentreeoflife is now Open Tree Of Life | opentreeoflife.org | github.com/opentreeoflife | http://irclog.perlgeek.de/opentreeoflife/today
02:48 scrollback1 joined #opentreeoflife
07:29 jar286 joined #opentreeoflife
07:50 jar286 joined #opentreeoflife
11:01 kcranstn joined #opentreeoflife
12:18 jar286 joined #opentreeoflife
12:59 jar286 joined #opentreeoflife
13:40 kcranstn joined #opentreeoflife
14:15 kcranstn joined #opentreeoflife
14:25 jar286 joined #opentreeoflife
14:35 kcranstn just to check… when testing locally (and without adding support for https) login to github does not work?
15:05 jimallman kcranstn: that sounds right, because of the OAuth redirect URL (which wants to bounce your browser back to https://)
15:06 kcranstn ok
15:06 kcranstn just wanted to make sure that was expected behaviour
15:06 jimallman i’m testing this now. it’s likely that you could register a new app with GitHub, using the http:// URL for redirect instead.
15:08 jimallman hm, i was mistaken. the current github_redirect_uri uses http://…, so I’m not sure why this would block login.
15:08 kcranstn I get a 404
15:09 jimallman is your webapp config set to ignore HTTPS?  [security] secure_sessions_with_HTTPS = false
15:09 kcranstn yes
15:10 jimallman i’ll need to remove my multi-port config to reproduce this. just a sec.
15:10 jimallman what’s the URL where you get a 404?
15:10 kcranstn http://127.0.0.1/curator/user/login?_next=/curator
15:11 kcranstn which goes to https://github.com/login/oauth/authorize?state=5c5b6f22a05c3b140db9833af9dda8c7&redirect_uri=REDIRECT_URI_NOT_FOUND&response_type=code&client_id=CLIENT_ID_NOT_FOUND&scope=public_repo
15:11 jimallman ah, REDIRECT_URI_NOT_FOUND and CLIENT_ID_NOT_FOUND
15:11 jimallman a clue!
15:11 kcranstn amazing how helpful error messages are...
15:12 jimallman it looks like your webapp config files are missing values in the [apis] section
15:12 kcranstn is this because I am loading the page as 127.0.0.1 rather than devtree
15:13 jimallman that’s odd, but i don’t think so (the same link works for me)
15:14 jimallman compare your webapp config files to the [apis] values here:  https://github.com/OpenTreeOfLife/opentree/wiki/Installing-a-local-curator-and-tree-browser-test-server#create-local-webapp-config-files
15:14 jimallman (NOTE the different values for synth-tree viewer, versus curation app)
15:15 kcranstn yup. both webapp/private/config and curator/private/config match
15:16 jimallman hrmmm
15:18 jimallman i wonder if some kind of glitch in the config files could be breaking the parser?
15:18 jimallman (extra square brackets? not sure what would do this)
15:20 * jimallman is looking for a simple config-file validator, or a way to test in python…
15:23 kcranstn this is happening both with the webapp and curator
15:23 jimallman hm, do you have a ‘localconfig’ file that’s hogging the spotlight? (i never use this, so maybe we should remove the check for this file)
15:24 kcranstn in the same directories?
15:24 kcranstn webapp/private and curator/private?
15:24 kcranstn no, no localconfig
15:25 jimallman gotcha. i’m trying to put together a quick validation script in python, should just be a few minutes...
15:35 jimallman kcranstn: please give this a spin inside the private/ folder of each web app:  https://gist.github.com/jimallman/fa79cddd3a0a0fa4733f
15:35 jimallman $ python config-test.py
15:36 jimallman it should parrot back the sections and values in the local ‘config’ file, or complain if something’s terribly wrong
15:37 kcranstn [apis]
15:37 kcranstn github_client_id = 8cdc1fa7f5a3a416f958
15:37 kcranstn github_redirect_uri = http://devtree.opentreeoflife.org/opentree/user/login
15:37 kcranstn [apis]
15:37 kcranstn github_client_id = d731965529a15ef9d529
15:37 kcranstn github_redirect_uri = http://devtree.opentreeoflife.org/curator/user/login
15:38 kcranstn yours?
15:40 jimallman [apis]
15:40 jimallman github_client_id = 8cdc1fa7f5a3a416f958
15:40 jimallman github_redirect_uri = http://devtree.opentreeoflife.org/opentree/user/login
15:40 jimallman [apis]
15:40 jimallman github_client_id = d731965529a15ef9d529
15:40 jimallman github_redirect_uri = http://devtree.opentreeoflife.org/curator/user/login
15:41 jimallman looks like we match exactly. so why isn’t the internal config-parser (using the same built-in SafeConfigParser) finding these values? maybe it’s misreading the path to the file…. reading more closely now..
15:41 jar286 joined #opentreeoflife
16:03 jimallman kcranston: i’m working on error reporting for config problems.. should have a tiny PR for you to try shortly.
16:03 kcranstn cool
16:14 jimallman kcranston: here’s the new code in a pull request: https://github.com/OpenTreeOfLife/opentree/pull/678
16:15 jimallman you should be able to merge this branch, since it was created from master
16:20 jimallman kcranstn: i see a potential cause for this..  see https://github.com/OpenTreeOfLife/opentree/blob/master/webapp/models/db.py#L117-L125
16:20 jimallman here we try to set all three OAuth-related values. note that one (the client secret) was pulled in from a separate, secure file here: https://github.com/OpenTreeOfLife/opentree/blob/master/webapp/models/db.py#L16-L19
16:21 jimallman i’m guessing this file is not in the expected place, or with the expected name?
16:22 jimallman should be private/GITHUB_CLIENT_SECRET, alongside the config file
16:24 * jimallman is adding this step (copying these files to {webapp}/private/ folders) to the wiki page...
16:24 kcranstn it already is
16:24 kcranstn https://github.com/OpenTreeOfLife/opentree/wiki/Installing-a-local-curator-and-tree-browser-test-server#copy-secure-files-from-a-live-opentree-server
16:26 kcranstn ah, the instructions put those files in ~/.ssh/opentree
16:26 jimallman that step copies the secure files to your local machine and stashes them in ~/.ssh, but it doesn’t copy them into the {webapp}/private/ directories
16:30 kcranstn and it is expecting those files to be named GITHUB_CLIENT_SECRET?
16:31 jimallman yes, see https://github.com/OpenTreeOfLife/opentree/wiki/Installing-a-local-curator-and-tree-browser-test-server#add-client-secret-files-alongside-config-files
16:32 jimallman it’s also likely you’ll need the DEVtree versions of these (instead of ‘*-tree.opentreeoflife.org’)
16:33 kcranstn so *-devtree.opentreeoflife.org
16:33 kcranstn should we update the instructions?
16:34 jimallman ah, i think my mistake is in renaming them (in second step of wiki page) with the production domain name. since we’re scp’ing them from devtree, we should keep its domain name in the local files.
16:34 jimallman so it’s the right files, but with the wrong name (confirming this now)
16:36 jimallman confirmed, my local ~/.ssh/opentree has both curation-GITHUB_CLIENT_SECRET-tree.opentreeoflife.org and curation-GITHUB_CLIENT_SECRET-devtree.opentreeoflife.org, etc.
16:36 * jimallman is correcting the wiki page...
16:36 kcranstn does ot16 have both files on it?
16:37 kcranstn i.e. becuase I didn’t want to change back by /etc/hosts file, I copied from ot16
16:37 kcranstn rather than from devtree
16:37 kcranstn scp -i ~/.ssh/opentree/opentree.pem opentree@ot16.opentreeoflife.org:repo/opentree/webapp/private/GITHUB_CLIENT_SECRET ./treeview-GITHUB_CLIENT_SECRET-tree.opentreeoflife.org
16:38 kcranstn is that the production file or the dev file?
16:39 jimallman see latest revision to wiki page: https://github.com/OpenTreeOfLife/opentree/wiki/Installing-a-local-curator-and-tree-browser-test-server/_compare/c9db112998528ffb52b11c13fbe4821efef39745%5E...c9db112998528ffb52b11c13fbe4821efef39745
16:39 kcranstn the only diff is what you are naming the file, right?
16:40 jimallman right. the domain name should match the site (tree.opentreeoflife.org == production), but my error in the wiki page gives production names to the dev files. D’oh!
16:41 kcranstn which shouldn’t matter in the end, because they get renamed to GITHUB_CLIENT_SECRET
16:41 kcranstn the thing that matters is that you copy it from the right server
16:41 jimallman right. but in the meantime, we’ve stashed them in your local ~/.ssh/opentree with the wrong names, and potentially clobbered legit keys with those names
16:41 jimallman so there could be problems if you do later deployments
16:42 jimallman (the deployment tools expect to find domain-specific secret files in your local ~/.ssh/opentree, so it will assume the names match the domains)
16:43 jimallman sorry, security always makes things more confusing
16:45 jimallman brb, grabbing a snack...
16:58 jimallman Added a little explanation for GitHub client-secret files, revised link is https://github.com/OpenTreeOfLife/opentree/wiki/Installing-a-local-curator-and-tree-browser-test-server#add-github-client-secret-files-alongside-config-files
17:44 kcranstn ok, getting farther along this time
17:45 kcranstn now getting web3py internal error
17:45 kcranstn <type 'exceptions.Exception'> No access_token found in data: {'error_uri': 'https://developer.github.com/v3/oauth/#redirect-uri-mismatch2', 'expires': None, 'error_description': 'The redirect_uri MUST match the registered callback URL for this application.', 'error': 'redirect_uri_mismatch'} 'access_token'
17:47 kcranstn redirect_uri:"http://devtree.opentreeoflife.org/curator/user/login"
17:49 jimallman ok, let’s compare the redirect_uri sent and received
17:51 jimallman i assumed so. checking now…
17:54 jimallman yes, i can download both files as described in the wiki (second step), and each has 41 bytes (look right to me)
17:58 jimallman can we repeat the test before, checking passed values in the curation app’s OAuth request?
17:59 kcranstn sure, just not sure where to look
17:59 jimallman if you’re passing the expected values (instead of the error messages as previous), we can check these values against the registered app on GitHub (in the opentreeoflife org account)
17:59 jimallman we got the passed arguments from the browser’s Network tab
17:59 jimallman during login
18:01 kcranstn client ID and callback URL match
18:02 jimallman at this point, we seem to have the same values in config (above), so i’m not sure what could be going wrong. i can login to the curation app just fine from here
18:02 jimallman let’s try to get the login request (from browser) and copy as cURL
18:03 jar286 PRs?
18:04 kcranstn yes
18:04 kcranstn I am going to go get a beverage first
18:04 kcranstn 1 minute
18:04 jar286 me too
18:05 kcranstn back
18:06 jar286 ok…. just finishing one email… re oti
18:07 jar286 ok.
18:08 jar286 my ff is screwed up
18:09 jar286 https://github.com/pulls?user=OpenTreeOfLife bottom to top
18:09 pmidford2 joined #opentreeoflife
18:10 jar286 skipping bottom 3
18:10 jar286 what about tree properties. https://github.com/pulls?user=OpenTreeOfLife
18:10 kcranstn on hold
18:11 jimallman was pending review, i think..?
18:11 jar286 yes.  next…
18:11 jimallman still cooking on tree collections
18:11 kcranstn why are there 3 PRs?
18:11 jimallman different repos, all work in tandem
18:11 kcranstn ah, 3 repos
18:11 jar286 https://github.com/OpenTreeOfLife/peyotl/pull/120
18:11 kcranstn is there any of that work that can get merged?
18:12 jimallman in #120, or tree-repo branches?
18:12 kcranstn tree collections in general. Hoping to prevent the PR too large to review
18:12 kcranstn 120 failing travis
18:13 jimallman gotcha. #120 includes heavy refactoring of peyotl. it works in my testing, and it’s been running on devtree (off and on) for awhile
18:13 jimallman looking at travis failures now
18:15 jimallman assertion test fails:   self.assertTrue(r['newick'].startswith('('))     …  maybe a quirk in python 3.4?
18:15 kcranstn ok, so not ready for review?
18:16 jimallman no, but i’ll chase this later
18:17 kcranstn does 120 need to be done before https://github.com/OpenTreeOfLife/phylesystem-api/pull/148
18:17 jar286 so… https://github.com/OpenTreeOfLife/germinator/pull/20
18:17 kcranstn ?
18:17 jar286 oops, sorry
18:18 jimallman yes, i think 120 is required for 148
18:18 kcranstn ok, so 20 then
18:18 jimallman (all three collections pulls are entangled, mainly because of deployment changes)
18:20 jar286 I think 20 is OK, but should look at it more carefully. I think it only affects treemachine and the ott build
18:21 jar286 I will need to update the ott build to process the new taxon list format
18:21 jimallman yes, it seems pretty isolated
18:21 jar286 let’s move on, I’ll look at it offline
18:21 kcranstn https://github.com/OpenTreeOfLife/taxomachine/pull/95
18:22 jar286 I was going to ask the list.  Was shy, have gotten over it
18:22 jimallman tiny changes, big travis failures
18:22 jimallman [ERROR] Failed to execute goal on project ot-base: Could not resolve dependencies for project org.opentree:ot-base:jar:0.0.1-SNAPSHOT: Could not find artifact org.opentree:jade:jar:0.0.1-SNAPSHOT in sonatype-snapshots (https://oss.sonatype.org/content/repositories/snapshots/) -> [Help 1]
18:23 jar286 not related to what I did.
18:23 jar286 should have failed in a previous pull
18:23 jimallman possibly a transient maven problem, or due to change from 1.7 to 1.8?
18:23 jar286 cody made a bunch of changes to jade, ot-base, and taxomachine
18:23 jar286 this will need study.  i say table this one too
18:24 kcranstn https://github.com/OpenTreeOfLife/treemachine/pull/185
18:26 jar286 hmm.
18:26 jar286 should be ok, the examples he gives are fine
18:27 kcranstn ok
18:27 jimallman gah, whitespace changes are cluttering the diff (can’t recall how to suppress these)
18:27 jar286 I should try my smoke tests, although I don’t know if this is tested there
18:27 jar286 should add a test if not
18:28 jimallman fyi, this is the diff ignoring whitespace:  https://github.com/OpenTreeOfLife/treemachine/pull/185/files?w=1
18:28 jimallman ?w=1 is the trick
18:28 jar286 Funny, I get  "Mandatory argument \"git_sha\" not supplied."
18:29 jar286 table it.
18:29 kcranstn why?
18:29 kcranstn oh, I see
18:30 kcranstn https://github.com/OpenTreeOfLife/opentree/pull/671
18:31 jimallman trivial text change (so merge-able!)
18:31 kcranstn yay
18:31 kcranstn done
18:32 kcranstn https://github.com/OpenTreeOfLife/opentree/pull/672
18:33 jimallman i’ve tested this PR (and its companion in phylesystem-api) pretty thoroughly, i think it’s ready
18:33 jar286 I’ll trust you, but maybe kc should try it
18:33 kcranstn don’t think this branch is deployed right now
18:33 kcranstn but the code change is minor and clear
18:34 jimallman no, i pivoted development back to testing tree collections
18:34 jimallman (i believe this was forced by the phylesystem-api changes, since that repo has no ‘development’ branch for pending stuff)
18:34 jar286 but that’s why we had a development branch - so we could combine experiments?…
18:35 jimallman see above
18:35 jar286 ok
18:35 jar286 we could make one
18:35 jimallman it adds a little song-and-dance to dev deployment, but i’m ok with that.
18:36 jar286 up to you.  just too bad we couldn’t review this.  I stilll trust you
18:36 jimallman oh, i can switch dev* in a minute
18:36 kcranstn yes, let’s do that
18:36 jimallman (didn’t think to do it beforehand)
18:36 jimallman ok, one sec
18:37 kcranstn (we may have a bunch of people adding studies when the paper comes out, and this shouldn’t be broken)
18:37 josephwb joined #opentreeoflife
18:37 josephwb jar286 which service is giving you the mandatory git_sha error?
18:38 kcranstn the example from the PR
18:38 jar286 see irc log.
18:38 kcranstn the ‘Good call'
18:38 josephwb ok, thanks
18:39 josephwb hmm
18:39 jar286 josephwb, source_tree  https://github.com/OpenTreeOfLife/treemachine/pull/185
18:39 josephwb doesn't look the fix is deployed
18:40 jar286 but it was at one time.  wonder why it got reverted
18:40 jar286 May 14
18:40 josephwb dunno
18:41 josephwb looking now
18:42 jimallman OK, new checks for CC0 should be working on dev: https://devtree.opentreeoflife.org/curator/study/create
18:42 kcranstn *goes off to change /etc/hosts*
18:42 jar286 the big PR merge was june 11 for treemachine, so that probably clobbered it
18:42 jimallman kcranstn: ;-)
18:43 jar286 jimallman, what did you think about using proxy config to do the name switcheroo?
18:43 jar286 in browser
18:43 jar286 then you could just switch browsers to get the ‘truth’
18:43 jimallman oops, missed this comment. i haven’t done much with that.
18:43 jar286 just an idea.  might not work
18:43 jimallman i’ll check it out. thanks!
18:44 kcranstn Does license option #2 ever apply for TreeBASE?
18:44 kcranstn “These data already have an applied waiver or license"
18:45 jar286 could.
18:45 jimallman i believe i saw something like that in a prior chat (links in related issue)
18:45 josephwb it seems like my PR is not deployed?
18:45 jar286 that’s right, not on dev because the big PR overwrote it
18:46 kcranstn ok
18:46 josephwb i deployed it again last week sometime, i think
18:46 kcranstn I wondered if curators importing from TreeBASE might wonder if TreeBASE applies a waiver / license and get confused
18:46 jar286 oh.
18:46 josephwb i can do it again
18:47 jar286 then I have no idea.  yes, I think that would be nice, although it would be just for the one test you put in the issue comments
18:49 josephwb ok, works now
18:49 josephwb although this service should be deprecated
18:51 josephwb i thought i had put in a link to files.opentree.org...
18:51 josephwb looking...
18:51 jar286 well, then why are we worrying about it?
18:52 josephwb who is worrying about what?
18:52 jar286 the cleanup PR
18:52 josephwb just responding to comments
18:52 jar286 oh.  there are other things in there, though, right?  or not?
18:52 jar286 I’ll look
18:53 josephwb mostly documentation stuff
18:53 jar286 oops need to find it
18:53 jar286 #185
18:53 kcranstn https://github.com/OpenTreeOfLife/treemachine
18:53 kcranstn oops
18:54 kcranstn https://github.com/OpenTreeOfLife/treemachine/pull/185
18:54 josephwb wait, it might be what working
18:54 jar286 why would this service be deprecated?  you went to the trouble to make it work, so we should review it to see that it does (in combination with the June PR)
18:55 jar286 I have every reason to think it does
18:55 jar286 but I agree it’s not a good long term design
18:55 josephwb ok, i was confused
18:55 josephwb everything is cool
18:56 josephwb i thought it was pulling from the graph DB, but it does get them from files.opentree...
18:56 jar286 right, you did good
18:56 josephwb it was so long ago
18:56 jar286 62 days
18:57 josephwb i think you were right that it got lost with the big PR
18:57 jar286 I say merge it
18:58 jar286 doing so now, ok?
18:58 jimallman works for me
18:58 josephwb phine by me
18:59 jar286 done
18:59 josephwb next one will deal with all the references to "node_id"
18:59 kcranstn w00t!
18:59 jar286 waiting for kc review of cc0 business?
18:59 kcranstn I wondered if curators importing from TreeBASE might wonder if TreeBASE applies a waiver / license and get confused
18:59 kcranstn i.e. if option #3 might add  “(Note that TreeBASE does not apply a waiver or license)”
18:59 josephwb i bet they do
19:00 jar286 could add a note like they do in turbotax “this would be unusual”
19:00 josephwb get confused
19:00 jimallman kcranston: like it’s automatically applied, on export?
19:00 kcranstn just that people don’t know much about waivers or licenses, so it would be good if we can be helpful
19:00 jimallman i see what you mean. if CC-BY is the expected default on TreeBASE, we might say this too.
19:01 kcranstn we know that treebase does not automatically apply a waiver / license
19:01 kcranstn there is NO license / waiver on treebase data
19:01 jimallman though these kinds of messages might encourage lazy click-throughs without checking…? esp. for someone else’s trees
19:01 jimallman ah yes
19:02 jimallman right, the confusion was whether TreeBASE data (by its nature) was fact and in the public domain
19:02 kcranstn but we cannot make that claim
19:02 kcranstn although it is probably true
19:02 jar286 “ordinarily no license or waiver is applied to treebase submissions. check with original author before claiming that one is”
19:03 kcranstn awkwardly worded
19:03 jar286 i wrote it for intent, not style
19:03 jimallman “Ordinarily no license or waiver is applied to treebase submissions. When in doubt, check with the original author.”
19:03 kcranstn option #2: “These data already have an applied waiver or license (for TreeBASE data, this would be highly unusual)”
19:04 kcranstn option #3: “These data have no applied waiver or license, and I have no authority to apply a waiver (TreeBASE does not apply any waiver or license)"
19:04 jimallman re #2, yes. i can even show the parenthetical phrase only if TreeBASE is chosen above
19:04 kcranstn fancy!
19:04 jimallman true for both, actually
19:04 jimallman yes, all the bells and whistles, and just a few show-stopping bugs!
19:05 jar286 only people can apply waivers.
19:06 jar286 the question is about the speech act, not about what is merely said.
19:06 jimallman only the author can apply them, you mean?
19:06 kcranstn I so often have no idea what jar286 is talkin about
19:06 kcranstn talking
19:06 jar286 only the rightsholder, yes.  otherwise it’s not felicitous
19:06 jar286 http://plato.stanford.edu/entries/speech-acts/
19:07 kcranstn how about “(TreeBASE does not provide the ability for authors to assign waivers or licenses to data)"
19:07 jar286 that’s certainly true
19:07 jimallman this has a bearing on our UI. what about when someone submits a study by other authors (a grad student, an OpenTree curator)?
19:07 jar286 think of it as like marriage.
19:08 jar286 If I say you are married, that does not make you so, in general.
19:08 jar286 But it could be true, even if I did not do the ceremony.
19:08 * jimallman nods
19:08 jar286 And if I did the ceremony felicitously, then it does become true by virtue of my saying so.
19:08 jar286 licenses are just the same.
19:08 kcranstn we trust that curators will not apply a waiver unless they are the author or have permission of the author
19:09 jimallman yes, permission is the bit i’m curious about.
19:09 jimallman i suppose that is equivalent to the speaker/author making the assertion directly?
19:09 jar286 you can tell others that I am married, if I am.  otherwise you cannot.
19:09 kcranstn how about option #3: “These data have no applied waiver or license, and I have no authority to apply a waiver (This would usually be the case for data from TreeBASE)"
19:10 jar286 yep.   that’s sort of like turbotax
19:10 kcranstn awesome. Our UI aspires to be as pleasant as turbotax
19:10 jimallman re: “speech acts” interesting stuff!
19:10 jimallman :-/
19:11 kcranstn you want me to record those on the PR or the issue?
19:11 kcranstn the new phrasing
19:11 jimallman let’s make it in the PR (which is a sort of issue)
19:13 kcranstn https://github.com/OpenTreeOfLife/opentree/pull/672
19:13 kcranstn I shortened the phrasing to be very clear
19:13 kcranstn less change of confusion
19:13 kcranstn chance
19:13 jimallman cool, thanks!
19:13 kcranstn thank you
19:14 kcranstn my numbering linked to issues. oops
19:14 jar286 so will the change be done in time for this afternoon’s redeploy?
19:14 kcranstn yes!
19:14 kcranstn ( I can do it if need be)
19:14 jar286 ok, we start in 15 minutes, but I have until about 6
19:15 jimallman true! either way is fine. of course you’ll want to checkout the PR’s branch and push the changes there
19:15 kcranstn let’s get through the rest of hte PRs
19:15 jimallman oh, the fancy JS hide/show logic would take a few minutes to test
19:15 kcranstn let’s make it static to get it out this afternoon
19:15 kcranstn make an issue for the fancy
19:15 kcranstn this is a bad bug
19:16 jar286 +1
19:16 jimallman agreed
19:16 jar286 https://github.com/OpenTreeOfLife/opentree/pull/677
19:16 kcranstn what about 152?
19:16 kcranstn https://github.com/OpenTreeOfLife/phylesystem-api/pull/152
19:17 kcranstn companion to previous
19:17 jar286 oh right, sorry, skipped over it
19:17 jimallman yes, and tested fairly thoroughly by me on devapi
19:17 kcranstn looks good to me
19:17 jar286 ‘in tandem’ = both merges should happen at pretty much the same time.
19:18 kcranstn the other one will happen in a matter of minutes
19:18 kcranstn right, jimallman?
19:18 jar286 ok
19:18 jimallman should be quick, yes
19:20 kcranstn https://github.com/OpenTreeOfLife/opentree/pull/677
19:20 jar286 merge 677? pics look good
19:21 jimallman looks good to me (nice simplification in the nav)
19:22 kcranstn I don’t think this does address issue 259 (i.e. screen width not dependent on width of menu items)
19:22 jimallman one minor nit: will someone who’s looking for EXISTING studies+trees think to click ‘Add trees’?
19:22 kcranstn but does address 661
19:22 jimallman yes, screen width is triggered by Bootstrap (or our CSS) at a predetermined width in pixels.
19:23 jimallman i can modify that, but it’s not sensitive to actual text crowding in the page.
19:23 kcranstn I though about the Add trees vs Add / Edit / View but couldn’t come up with a good replacement that was actionable
19:23 jimallman ok, i will ponder. this is where i’d opt for adding both links, personally.
19:23 jar286 yes
19:24 kcranstn having two links that go to the same place seems silly
19:24 jar286 you’re not going to add a tree from a phone
19:24 jar286 add tree could go directly to the add tree page
19:24 kcranstn would go directly to the login page
19:24 kcranstn which might be confusing
19:25 jar286 need more ‘stop, read this’ pages generally
19:25 jimallman two links *is* silly from an architectural standpoint, but they’re friendly for actual humans (distracted, tired, timid)
19:25 kcranstn ok, I can add both links
19:25 jar286 table this?
19:25 kcranstn no, it’s fast
19:26 kcranstn will get it done before deployment. I think it is important
19:26 kcranstn “Browse trees"?
19:27 jimallman earlier i thought we needed “source trees” to be clear
19:27 jimallman trees is trees
19:28 jimallman but maybe a synthetic tree is such a special case that “trees” would be assumed mean contributing phylogenetic trees.
19:28 jar286 but the tree is not trees.  moving right along…
19:29 jimallman “Browse trees” is a decent starting point, i think. users will let us know if it’s confusing.
19:30 jar286 ‘browse/add trees’ ‘browse or add trees’ ‘view or add trees’ …
19:30 kcranstn and now menu wraps on my laptop screen
19:31 kcranstn and looks terrible
19:31 jar286 yeah, well… I took my turn
19:31 kcranstn what does that mena?
19:31 kcranstn mean?
19:32 jar286 never mind.  just throwing out more ideas
19:32 jimallman i like ‘browse or add trees’, actually
19:32 kcranstn I can do that
19:32 jar286 And we can’t say just ‘trees’ because of what Emily said…
19:33 jimallman ?
19:33 jar286 isn’t that how this got started? ‘source trees’ was not suggestive?
19:33 jar286 and ‘trees’ is not significantly different from ‘sources trees’ (better because shorter, but otherwise not)
19:34 jar286 so ‘browse or add trees’ would be informative and accurate, but is too wide?
19:34 * jimallman is doing the text changes for CC0 + TreeBASE…
19:35 kcranstn https://github.com/OpenTreeOfLife/opentree/pull/677
19:35 jar286 there’s lots of space between menubar items
19:35 kcranstn doesn’t matter
19:35 kcranstn see new
19:35 kcranstn like?
19:36 jar286 looks fine to me
19:36 jar286 no, not just fine, good
19:36 kcranstn :)
19:36 jimallman i like it!
19:36 jimallman and the menu-item spacing is by design (to clarify multi-word items)
19:37 jar286 merge?
19:38 kcranstn I am not going to tell you to merge my own PR ;)
19:38 jimallman yes, it’s good
19:38 jar286 I’ll do it
19:38 jimallman do it!
19:38 jar286 done
19:39 jimallman kcranstn: thanks for cleaning up some dead markup in there. hope springs eternal...
19:39 kcranstn np
19:39 jar286 https://github.com/OpenTreeOfLife/opentree/pull/678
19:39 kcranstn thanks for this one
19:39 jar286 has been put to use, yes?
19:40 jimallman yes
19:40 jar286 kc, do the honors?
19:40 jimallman and rest assured, this still handles a tester who doesn’t want or need GitHub registration details.
19:40 kcranstn done
19:41 jar286 https://github.com/OpenTreeOfLife/ot-base/pull/16
19:41 kcranstn hack!
19:41 jar286 yes, the correct fix is very big
19:41 jar286 and requires mark I think
19:42 kcranstn so explain to me what this fixes?
19:42 jar286 studies failing to index
19:42 jar286 studies not being listed in tree browser
19:43 kcranstn all of the errors?
19:43 jar286 failure to provide the promised answer to “why” questions
19:43 jar286 no.
19:43 jar286 most of them.  working on the others.
19:43 kcranstn ok
19:43 jimallman this problem (missing OTU on tips) is in stuff imported from TreeBASE, yes?
19:43 jar286 yes
19:43 jimallman does assigning a null OTU make cleanup (later) a breeze? should we assign something more grep-friendly?
19:43 jar286 there’s a peyotl issue
19:44 jar286 I have no idea
19:44 jar286 these things are very easy to handle in short python scripts
19:44 jar286 at least to analyze
19:44 jimallman just a thought. this makes sense to me.
19:44 kcranstn ok to me
19:45 kcranstn merging?
19:45 pmidford2 joined #opentreeoflife
19:45 jar286 I see your point. but I think we should move ahead
19:45 kcranstn merged
19:45 jar286 everything gets re-indexed (like today) so we can change it later
19:45 kcranstn (I delete branches when I merge)
19:46 jar286 that’s fine
19:46 kcranstn ruthlessly
19:46 jar286 so I’m waiting for 672 before deploying, yes?
19:46 jimallman a good habit. please share your ruthless pruning incantation.
19:47 jar286 doesn’t gh offer it?
19:47 jimallman re: #672, i’m doing this now
19:47 jar286 easy to just go to the branches list, it tells you which have been merged.
19:47 kcranstn after you merge a PR, there is an option on the same page to delete the branch
19:48 kcranstn it’s exciting to be writing code again :)
19:49 kcranstn expect a flood of UI changes
19:50 jar286_ joined #opentreeoflife
19:56 jimallman jar286: #672 is (imo) ready to merge. has new hints regarding TreeBASE norms.
19:56 jimallman i went ahead and added the JS toggle, so they’ll only appear if TreeBASE is the chosen source.
19:57 kcranstn yay, thanks
19:58 kcranstn I can try and tackle https://github.com/OpenTreeOfLife/phylesystem-1/issues/6 tomorrow
19:59 jar286 joined #opentreeoflife
20:00 jar286 ok, I’ll merge, then deploy master to dev, and we can take a look
20:01 jar286 672 has been merged
20:01 jar286 proceeding with deploy… in 3 minutes
20:03 jimallman coffee break!
20:06 jar286 joined #opentreeoflife
20:09 jar286 pushing master to devapi …
20:11 jar286 pushing master to devtree…
20:12 jar286 done… checking
20:12 jimallman same here, the usual smoke test
20:13 jar286 I want to be able to move that middle bar to the right
20:13 jar286 ...
20:13 jimallman menus look unchanged… was that not merged?
20:14 jar286 I thought it was
20:14 jar286 let me check
20:14 jar286 browser cache?
20:15 jimallman jar286: i think i had logic to slide those dividers at one time. it’s not the Responsive Bootstrap Way, but i could potentially restore those handles on a wide screen.
20:15 jar286 it should be automagic, in an ideal world.  like tex
20:15 jimallman :)
20:16 jimallman not sure what’s up. my /etc/hosts is not overriding. you’re looking at the latest on https://devtree.opentreeoflife.org/  ?
20:16 jar286 trying to
20:17 jimallman hm, i still see the old menus (cleared cache)
20:17 jimallman i will wake up my long-slumbering Safari browser to try a fresh look
20:17 jar286 same here
20:17 jimallman same result there (old menus)
20:18 jar286 agreed
20:18 jimallman ah, let’s see where kcranston set this PR to merge…
20:18 jimallman well that looks OK too … merging to master   https://github.com/OpenTreeOfLife/opentree/pull/677
20:19 jar286 did safari purge ‘view source’?  if so I’ll remove it from dock
20:19 jimallman no, safari has all the usual goodies
20:19 jimallman so did something clobber these changes on master? checking now...
20:20 jimallman nope, here’s the changes (“Add / browse trees”) in master branch:  https://github.com/OpenTreeOfLife/opentree/blob/master/webapp/views/layout.html#L161
20:20 jimallman a deployment error (unable to pull ‘opentree’ repo), perhaps?
20:24 jimallman jar286: it looks like there are unsaved changes(!) in the ‘opentree’ repo on devtree.
20:24 jar286 ahh…. that would explain it
20:24 jimallman two languages/ files in curator (probably noise generated by web2py)
20:24 jar286 I’ll look over my shell transcript
20:24 jimallman and one that we care about, the requirements.txt file.
20:24 jar286 thanks for catching that
20:24 jimallman someone messed with this, perhaps a test?
20:25 jimallman (they removed distribute==0.6.34 from ‘requirements’ text)
20:25 jar286 oh, it’s on the wrong branch, too
20:25 jar286 i got confused
20:26 jimallman i just checked: we did implement the handy “which branch am i on?” indicator in the webapp footer! see https://devtree.opentreeoflife.org/contact
20:26 jimallman it still won’t detect a failed pull, but a quick sanity check for moments like these
20:27 jimallman shall i clobber the unsaved changes?
20:27 jimallman (i was waiting in case you wanted a look)
20:27 jar286 hang on, let me try to get it right
20:27 jar286 i changed the branch for devapi but not for devtree
20:28 jimallman oh, in the server-config files?
20:28 jar286 yes
20:28 jar286 redelpoying now
20:28 jar286 thanks for adding that feature
20:29 jar286 much better now.
20:29 jar286 ok, more smoke.
20:29 jimallman hm, i should add latest SHA to the branch indicator
20:30 jimallman i am smoke-free here. what do you see?
20:30 jar286 as in smoke test.
20:31 jimallman right, i assumed smoke == problems.
20:31 jar286 i need to look up that metaphor
20:31 jar286 is it about pipes?
20:32 jimallman i believe it comes from “magic smoke”, when a resistor or other component fails in a circuit board test
20:32 jar286 looking for smoke when powering electrical items for the first time
20:32 jar286 - wp
20:32 jar286 ok, so not what i thought.
20:32 jar286 no smoke here.
20:33 jar286 moving on to production.  will do the same, then re-index
20:33 jimallman same. i imported a TreeBASE study and chose not to assign a license or waiver. looks good.
20:33 jar286 doing api
20:35 jar286 we should deal with the requirements.txt glitch
20:35 jar286 From https://github.com/OpenTreeOfLife/opentree
20:35 jar286 * branch            development -> FETCH_HEAD
20:35 jar286 Mcurator/languages/cs.py
20:35 jar286 Mcurator/languages/es.py
20:35 jar286 Mrequirements.txt
20:36 jimallman yes, i’m surprised we were able to deploy. i guess since we weren’t changing branches, git didn’t make a fuss.
20:36 jimallman like i said, someone must have been testing an idea by removing ‘distribute’ from the requirements.
20:37 jimallman there’s a requirements.txt.save, but it’s identical, so no clue there
20:39 jar286 indexing ot_31 works now.
20:40 jar286 I’m going to do apt-get update/upgrade just for fun
20:40 jimallman hm, the changes in language files (curator/languages/es.py) look quite deliberate. not sure how these are appearing, but i’m guessing we’ve had multiple versions of web2py over time. maybe we’ve been on ‘development’ branch for so long that they’re from an older web2py
20:41 jimallman i’m inclined to checkout fresh copies (from master) over all three of these files, unless you want to ask around for the requirements.txt changer..
20:42 jar286 no, go ahead, maybe post email after the fact
20:44 jar286 update/upgrade ok on api.
20:46 jar286 oh man, lots of action, doing tail -f on the apache log, and it looks real.  browsing that is (looking on api).
20:46 jar286 we should put up the maintenance in progress banner
20:47 jar286 and I always forget how… it
20:47 jar286 s in a readme or something?
20:48 jar286 jimallman ^
20:48 jimallman yes, and in the curation app’s private/config file
20:49 jimallman in the [maintenance] section. i’m there now...
20:50 jimallman i just changed the date and activated the message. we need to restart apache… (doing this now)
20:51 jimallman that suspends study editing. or did you want the big redirect?
20:51 jimallman jar286: ^
20:52 jar286 no it’s ok, looking at activity on tree. and it’s nothing serious, not nearly as busy
20:52 jimallman OK, study curation is now suspended with a message.
20:52 jimallman should i restore curation?
20:52 jar286 no
20:53 jar286 it will be clobbered when I deploy.  going to update/upgrade, then deploy tree
20:55 jimallman gotcha.
20:55 jimallman fyi, here are basic instructions for the big standby page: https://github.com/OpenTreeOfLife/opentreeoflife.github.io#scheduled-maintenance-page
20:55 jimallman “scheduled maintenance page”, that is
20:56 jimallman and here are the full instructions, in the deployment README:  https://github.com/OpenTreeOfLife/opentree/blob/51a96f3885c5eecc3a14547d6c664a0e714d49b8/deploy/README.md#notifying-users-of-scheduled-downtime
20:57 jar286 deployment readme… ok
20:57 jar286 our docs need to be better organized.  it’s on my to do list
20:57 jar286 pushing the tree now
20:58 jimallman agreed, i had to follow the breadcrumb trail to find these notes again
20:58 jar286 and looking for M’s
20:58 jar286 long list.
20:58 jar286 in opentree repo.  including requirements and a gajillion languages
20:59 jar286 more importantly, in webapp/controllers/default.py
20:59 jimallman that is most unexpected. heading there now…
20:59 jar286 i would do: git diff, followed by git reset (with the appropriate flag, I’ll look it up)
21:01 jar286 I find this to be incredibly useful: http://marklodato.github.io/visual-git-guide/index-en.html
21:01 jimallman ok, default controller has harmless (but embarassing) test code, looks like mine.
21:01 jimallman oh, that’s nice!
21:02 jimallman this visual cheatsheet is also good for clarifying the relationship between workspace, index, etc:  http://ndpsoftware.com/git-cheatsheet.html
21:02 jar286 mayb just do ‘git stash’
21:02 jar286 that’s the easiest way to blow away local changes
21:02 jimallman or ‘git reset --hard’ should dump all changes
21:03 jar286 right, just found that.
21:04 jimallman permission denied, as admin user. would probably work for opentree (logging out now)
21:04 jimallman done
21:04 jimallman (as opentree)
21:04 jar286 good.  thanks for checking
21:04 jar286 right, the idea is to make absolutely minimal use of the privileged account
21:05 jimallman i see that requirements.txt had the very same change here (‘distribute’ was removed). definitely worth an email to the software list...
21:05 jar286 so as much as possible is done as opentree
21:05 jar286 I see the new menu.  I’m going to start reindexing now
21:06 jimallman ok. should i suspend curation, to avoid funky TNRS/etc results in the meantime?
21:07 jimallman jar286: ^
21:08 jar286 no, taxomachine is unaffected.
21:08 jar286 but the study list is affected...
21:08 jar286 and the browser…
21:08 jar286 so sure, if you’re going to be around in 40 minutes when it finishes.
21:08 jimallman yes, i’ll be here. is there a sign when it’s done?
21:09 jar286 no
21:09 jar286 I see it on the console
21:11 jar286 ok here goes.
21:11 jar286 lots of   "message" : "The specified root node node435772 is different from the observed root of the tree in the NexSON object hierarchy. This is nonsensical.",
21:12 jar286 lots and lots
21:12 jar286 I need to make it forge ahead with the wrong rooting.  Does the curator app fix this?… is it robust to it?…
21:12 jimallman ok. we’re bouncing to the maintenance page for now: http://opentreeoflife.github.io/maintenance.html
21:12 jar286 jimallman ^
21:12 jimallman where are these messsages appearing?
21:13 jimallman in indexing?
21:13 jar286 in my shell buffer
21:14 jimallman the curator (on saving) will clean up some ambiguous rooting, i think. and it should tolerate whatever it can load.
21:14 jar286 the curator must be robust to this. since it’s been dealing with it all along.
21:14 jimallman yes, it should be very tolerant
21:14 jar286 Cody just made ot-base intolerant without checking or telling anyone
21:14 jar286 I am not happy
21:15 jimallman eek
21:15 jar286 I think I can do the same thing I did with the other problem - turn the throw into a print (to the neo4j console)
21:16 jimallman sounds reasonable. i see you encountered this earlier today, but just on one study: https://github.com/OpenTreeOfLife/oti/issues/34#issuecomment-122037540
21:16 jimallman oh, and i chased this for awhile back in May:  https://github.com/OpenTreeOfLife/oti/issues/40#issuecomment-103948239
21:17 jimallman when it was already affecting several studies
21:19 jimallman note Cody’s comment, which points to a fix (branch ‘test-fix’) that has NOT been merged:  https://github.com/OpenTreeOfLife/oti/issues/40#issuecomment-103974875
21:21 jimallman here are the uncommitted changes, not sure what’s the story here:  https://github.com/OpenTreeOfLife/ot-base/compare/test-fix?expand=1
21:21 jimallman brb, time for coffee
21:21 jar286 that’s the code he added, which made these unindexable
21:26 jimallman it looks like an unmerged fix that might fix the problem...
21:27 jimallman OLD (current master):  if (specifiedRoot != null && specifiedRoot != observedRoot) {
21:27 jimallman NEW (unmerged text-fix):  if (specifiedRoot != null && !specifiedRoot.equals(observedRoot)) {
21:27 jar286 ahhh…
21:27 jar286 i didn’t look closely
21:28 jimallman not sure why it was abandoned
21:28 jimallman maybe josephwb would know?
21:29 jar286 ok, dev now how that check completely suppressed…
21:29 jar286 but I will try the fix now too
21:42 jar286 jimallman  - cascade https://github.com/OpenTreeOfLife/ot-base/commit/7da7b4ad53d89b48fb78c223df7baf50811016e8
21:43 jar286 choice now, reindex without the ‘validation’, or spend time now adding exception declarations to oti
21:43 jimallman hmmm
21:44 jimallman it would be interesting to see the results of a “reckless” (non-validating) reindex
21:46 jar286 that’s what i think i’ll do.  starting with dev
21:47 jar286 deployed…
21:47 jar286 tested…
21:47 jar286 can you see if anything is broken with study ot_232 ? I’m not sure how to test it (on dev)
21:48 jar286 it is probably perfectly fine but it would trigger the erroneous validation check, were it not suppressed
21:48 jar286 jimallman ^
21:48 jimallman it won’t load in the dev curator:  https://devtree.opentreeoflife.org/curator/study/view/ot_232, checking production now…
21:49 jimallman d’oh! maintenance page..
21:49 jar286 I need to leave in about 20 minutes (but can stretch that a bit)
21:50 jar286 indexing finished on production
21:50 jar286 571 occurrences of “specified”
21:50 jimallman it can’t be retrieved from phylesystem-api: {"description": "Study #ot_232 GET failure", "error": 1}
21:50 jimallman checking phylesystem-0 now...
21:51 jar286 ok, so implementing this ‘fix’ will not make things any worse.  I think I’ll go ahead
21:51 jimallman ah, there IS no ot_232 in phylesystem-0
21:52 jar286 but…
21:52 jar286 oh yeah… tricky
21:52 jimallman https://github.com/OpenTreeOfLife/phylesystem-0/tree/master/study/ot_32
21:52 jimallman so why/how is it in the index?
21:52 jar286 I indexed it in the dev oti index, even though it’s not in the dev phylesystem
21:52 jar286 manually
21:52 jimallman ah, ok
21:53 jar286 so… no way to test.  but what I say above stands
21:53 jimallman agreed, no worse
21:53 jar286 ok.  I’ll do a proper PR, then merge to master, etc
21:54 jimallman ok. that should clear the redirect to maintenance page, and i guess the curation blocker
21:56 jar286 jimallman, https://github.com/OpenTreeOfLife/ot-base/pull/17  - this is deployed on dev fwiw, but not really tested there, but etc.
21:57 jar286 do you want to look or should I just deploy to production.
21:57 jimallman taking a quick look...
21:58 jimallman looks good to me
21:58 jar286 oh wait, there are lots of pg_ examples
21:59 jar286 let me see…
21:59 jar286 pg_719
21:59 jar286 pg_713
21:59 jar286 and 500 more
22:00 jar286 but I don’t think it matters.
22:00 jar286 actually maybe you can do the indexing, it’s easy
22:03 jar286 Mcurator/languages/zh.py
22:03 jar286 Mwebapp/languages/zh.py
22:04 jar286 patch now on production.
22:04 jar286 jimallman, you there?
22:05 jimallman hi, catching up now…
22:05 jar286 This takes 40 minutes: ./push.sh -c ../../deployed-systems/production/api.config index
22:05 jimallman sure, walk me through the indexing (i’ve done it before, but don’t have the incantation handy)
22:05 jar286 that’s it
22:05 jimallman ok, will do
22:06 jar286 I need to go to dinner in 10 minutes, meaning my laptop won’t be awake to run the process if I start it now
22:06 jar286 or rather monitor
22:06 jimallman ok, it’s running now…
22:06 jar286 the process
22:06 jar286 great.
22:07 jar286 any errors yet?
22:07 jimallman yep. i’m seeing the printed exception messages in console.. oh, do i need to pull ‘opentree’ to get changes to deployment tools?
22:07 jar286 hmm, shouldn’t need that
22:07 jar286 which error are you getting?
22:07 jar286 root/root?
22:08 jimallman "The specified root node tree6129_ROOT is different from the observed root of the tree in the NexSON object hierarchy. This is nonsensical."
22:08 jar286 production?
22:09 jimallman it does say “Indexing failed” for this study. does that mean i’m not running the latest?
22:09 jimallman yes, production. i ran the ./push command exactly as you pasted it above
22:09 jar286 it’s all on the server.
22:09 jar286 hang on
22:10 jimallman thought so. but i expected to report (then ignore) the error, so indexing would work for the “nonsensical” trees.
22:10 jar286 I didn’t push it out
22:10 jar286 hang on
22:11 jimallman ok to cancel the reindexing in progress?
22:11 jar286 sure
22:12 jar286 need to kill it on the server probably, i’ll look
22:12 jimallman i was wondering about that..
22:14 jar286 no, it went away properly
22:14 jar286 so restart now
22:15 * jimallman restarted, is watching output…
22:15 jar286 you should know by now.
22:16 jimallman no error messages now
22:16 jimallman i guess your change is printing them elsewhere
22:16 jar286 excellent.
22:16 jimallman yeah, nice!
22:16 jar286 yes, it’s in neo4j-oti/data/log/console.log
22:16 jar286 there are no study ids, so the output is useless
22:16 jimallman yep.
22:16 jar286 but I have all the study ids in my shell buffer.
22:17 jar286 gotta run. thanks for al your help. i’ll check in probably around 9:30.
22:17 jimallman i’ll watch this and remove the blockers (maint. page, etc) when it’s done. yes?
22:17 jar286 yes
22:17 jimallman cool. talk to you later!
22:17 jar286 bye
23:17 jimallman just for the record, reindexing completed with (i believe) no reported errors, in 33 minutes. i’ve removed the maintenance page and curation blocker. after a quick smoke test, everything looks good to me.

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary