Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2015-02-12

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:36 pmidford2 joined #opentreeoflife
02:46 jimallman_ joined #opentreeoflife
02:48 ilbot3 joined #opentreeoflife
02:48 Topic for #opentreeoflife is now Open Tree Of Life | opentreeoflife.org | github.com/opentreeoflife | http://irclog.perlgeek.de/opentreeoflife/today
02:49 josephwb1 hah. there s/he is.
02:49 jimallman_ *whew* hey, little buddy!
02:49 josephwb1 that was disconcerting
02:49 jimallman_ i figured he was @6A4ABFQ2L (bogus handle after an IRC netsplit)
02:49 jimallman_ i see a few different versions of you (josephwb) and me…
02:50 josephwb1 some things you just rely on being there. gravity. oxygen. ilbot3.
02:50 jimallman_ yeah, netsplit is an interesting phenomenon.. multiple universes!
02:50 josephwb1 woah.
02:51 jimallman_ there are three of you, two of me, and just one pmidford2 (so 6A4ABFQ2L is probably the original pmidford)
02:51 josephwb1 i don't trust that josephwb_ guy. seems evil...
02:51 jimallman_ yeah, the trailing underscore is like a digital goatee
02:51 josephwb1 hah.
02:53 jimallman i’m all better now, but you’ve still got a clone
02:53 jimallman or two
02:57 josephwb1 i'll try not to make any decisions, lest i trigger another dimensional split.
02:58 jimallman do you watch Community (TV series)?  i’ve been watching early seasons, and that’s the premise of a great episode.
02:58 jimallman Evil Troy and Abed!
02:58 josephwb1 i missed that show.
02:59 jimallman it all starts with a roll of the die. good stuff.
02:59 josephwb1 but the evil twin from a parallel dimension trope is well trodden
02:59 jimallman indeed
03:00 jimallman http://vignette3.wikia.nocookie.net/community-sitcom/images/e/e7/Evil_Troy_and_Evil_Abed.png/revision/latest?cb=20120325174706
03:01 jimallman (that mess on the table is a pile of construction-paper goatees)
03:01 josephwb1 heh.
03:01 josephwb1 wait, don't you sport a goatee?
03:02 josephwb1 e-vil.
03:14 jimallman it comes and goes. currently sporting something like a full board…
03:15 * jimallman shaves the neck area, risking his geek cred
03:23 josephwb1 jimallman, i don't think you ever need to worry about anyone questioning yer geek cred
03:25 jimallman aww, thanks!
03:26 josephwb1 ;-)
04:18 pmidford2 joined #opentreeoflife
13:24 jar286 joined #opentreeoflife
14:24 guest|58334 joined #opentreeoflife
16:53 kcranstn joined #opentreeoflife
16:55 kcranstn guten abend!
17:32 jimallman howdy (it’s quiet in here today)
17:32 kcranstn I am in Germany
17:32 kcranstn and technically on vacation
17:32 jar286 in middle of intense hangout
17:32 jimallman wunderbar!
17:49 kcranstn joined #opentreeoflife
18:14 kcranstn joined #opentreeoflife
18:50 pmidford2 joined #opentreeoflife
19:00 jimallman PRs for today:  https://github.com/pulls?user=OpenTreeOfLife
19:00 jar286 hello
19:00 jimallman hi!
19:01 jar286 https://github.com/OpenTreeOfLife/treemachine/pulls
19:01 jar286 https://github.com/OpenTreeOfLife/treemachine/pull/165
19:02 jar286 josephwb, you’re good with this, right? it will give you enough logging for debugging purposes?
19:02 jimallman looks pretty cut-and-dried
19:02 jar286 (if you’re there…)
19:03 jar286 I’ll merge
19:04 jar286 https://github.com/OpenTreeOfLife/opentree/pull/536
19:04 jar286 still cooking I presume
19:04 jimallman no recent progrss on this
19:04 jimallman yeah, please skip for now
19:05 pmidford2 Agree
19:05 jar286 marking it ‘needs discussion’
19:05 jar286 https://github.com/OpenTreeOfLife/opentree/pull/591
19:06 jar286 two dependencies noted
19:06 jar286 there’s no PR for phylesystem-api.
19:06 jar286 ?
19:06 jimallman support in phyleslystem-api was pushed (by me) to master, which was a break in protocol
19:07 jar286 oh, ok
19:07 jimallman apologies, it seemed small and free of side effects (but i did in fact need to tweak it)
19:07 jar286 fine
19:07 jar286 the second ‘note that’ is the same as the first I presume
19:08 jar286 ok to merge?
19:08 jar286 I observed it working on devtree
19:08 jimallman one caveat: i have (on one occasion) had the tree-view JSON fail, and needed to restart devapi to clear cache and get it working again.
19:08 jimallman not sure of the cause, or if it’s a one-time aberration.
19:09 jar286 what do you think are the chances of this happening on production?
19:09 jimallman hard to say without pinpointing the cause.. i tried flooding the cache with lots of major clades, and it held up fine.
19:09 jimallman so i don’t think it was a memory shortage
19:10 jar286 ‘let’s risk it’
19:10 jimallman i’ve deployed a few more times since then, and can’t reproduce.
19:10 jar286 merging
19:10 jimallman easy to fix by restarting apache on api
19:10 jimallman i can document that one-liner… but where?
19:10 jar286 the problem is we don’t have monitoring
19:11 jimallman true.
19:12 jar286 https://github.com/OpenTreeOfLife/opentree/pull/593
19:12 jar286 the thing I didn’t quite get about this one… caching is important because this is an operation that happens on every browser ‘page’ load, right?
19:12 jar286 most of the time it’s null
19:12 jimallman right. it’s pretty cautious at this point, just caches for 5 minutes) but it should
19:13 jimallman but it should help under load (lots of visitors moving over major clades)
19:13 jar286 ok, I say merge
19:13 jimallman can’t hurt, might help
19:13 jar286 https://github.com/OpenTreeOfLife/opentree/pull/596
19:14 jimallman mtholder realized that it’s possible to “import trees” using a NEXSON (or other) file that actually has no trees defined in it
19:14 jimallman we now handle this gracefully, with an error message in the curation UI.
19:14 jar286 this reminds me of the idea of importing all mendeley papers into phylesystem, without trees
19:15 jar286 (the opentree mendeley group)
19:15 jimallman right..
19:15 jimallman this particular code path is “import trees”, and merge trees + OTUs into an existing study.
19:15 jar286 merge, ok?
19:16 jar286 done.
19:16 jimallman i feel good about this one (tested on devtree)
19:16 jar286 yep
19:16 jimallman i’m working on the latest tree properties now, should have them up on devtree by end of day
19:17 jar286 ok… too bad karen didn’t review, she’s a big proponent of reviews
19:18 jimallman but… vacation!
19:18 jar286 according to the nescent informatics calendar she’s back in the fray next week
19:19 jimallman sounds good. i hope to have some prototype pages for curator profile and collections next week
19:19 jar286 jimallman, I need to do MORE shoveling… if you could set up the work-in-progress notice aroun 3:15 that would be helpful. i hope i’ll be done by then
19:19 jar286 I want to upgrade the servers again and reboot them this time
19:19 jimallman sure!
19:20 jar286 see you in a bit
19:21 jimallman pmidford2: i’ll keep you posted regarding Nexson changes to tree properties, or watch issue #555 in opentree, esp. https://github.com/OpenTreeOfLife/opentree/issues/555#issuecomment-74102857
19:21 jimallman this is all tentative, based on discussions to date
19:22 pmidford2 jimallaman - ok thanks
19:44 pmidford_ joined #opentreeoflife
20:19 jar286 rebooting ot20
20:19 jar286 well maybe I’ll wait until 3:30
20:25 jimallman maintenance notice is up (a little late)
20:25 blackrim joined #opentreeoflife
20:31 blackrim josephwb: are you using pgloadind
20:32 blackrim or which gcmdr command? just looking to get trees with the ott ids. we did them for mls (I can use those trees for now)
20:33 josephwb load_studies_inf_mono.py, i think
20:36 blackrim do you have the individual trees from the mls (with ott ids as tips)? can't seem to find it
20:37 josephwb just a sec
20:38 josephwb in files_for_submission_v2.0
20:38 blackrim yeah, i see the python files but not the tree files
20:38 josephwb MLS_MPR_files_v2.0
20:39 jar286 doing duke conflict of interest thing…
20:39 josephwb oh, the original tree files?
20:39 blackrim right
20:39 josephwb let me look
20:42 jar286 rebooting ot20
20:43 jar286 upgrading ot14
20:43 josephwb blackrim: files didn't make it through OS transition. but can be remade with load_mls.py
20:45 jar286 ot20 isn’t coming back… will check console
20:47 jar286 oh it’s fine, it just doesn’t respond to ping
20:47 jimallman looks like the call to api (findAllStudies) is failing
20:47 jimallman i’m trying to load the study list… retrying now...
20:47 jar286 humph.
20:47 jar286 i’ll run the germinator tests
20:47 jimallman same result in curl (503 error)
20:48 jar286 503 service temporarily unavailable… I have to start the servers manually
20:48 jar286 might as well do a deploy. hang on
20:49 jimallman FYI, oti is borked but other APIs seem to be working
20:50 jar286 doing a deploy now, let’s try again after that
20:50 josephwb blackrim: in general_tm_utils.py there is "load_nexson" specify test=True and it won't commit.
20:51 jar286 Warning: you are leaving 1 commit behind, not connected to
20:51 jar286 any of your branches:
20:51 jar286 666b898 Merge pull request #12 from OpenTreeOfLife/newick-illegal
20:51 jimallman hm, this does not ring a bell
20:51 jar286 Using branch master of repo phylesystem-1
20:51 jar286 From https://github.com/OpenTreeOfLife/phylesystem-1
20:52 jar286 6731e29..de6f6f8  master     -> origin/master
20:52 jar286 Branch master set up to track remote branch master from origin.
20:52 jar286 Warning: you are leaving 1591 commits behind, not connected to
20:52 jar286 any of your branches:
20:52 jar286 6731e29 adding more tag (with 9 "delete me" tags so far)
20:52 jar286 bf0d0e3 adding bomb tag (7 "delete me" tags)
20:52 jar286 57641cd adding bazz tag (test of removing redundant tags)
20:52 jar286 d93ad80 Added DOI and updated publication reference
20:52 jar286 ... and 1587 more.
20:53 jimallman detached head, probably (i think this happens routinely when refreshing and saving studies)
20:53 jimallman let’s check the last few commits and make sure they made it to GitHub..
20:55 josephwb blackrim: sent Aves MLS input
20:55 blackrim thanks. i am trying a derivative of that script to load_mls to get the trees
20:55 blackrim signing off to go to the talk
20:56 jar286 Successes: 10 Failures: 0
20:56 jar286 ok, moving on to ot14…
20:57 jar286 will reboot it now
20:57 jimallman i see that both phylesystem-1 repos are on master and clean
20:57 jimallman jar286: that fixed the study list on tree.opentreeoflife.org
20:57 jar286 figured
20:58 jar286 rebooting...
20:58 jar286 (having upgraded)
21:01 jar286 ot14 is back up. time to deploy
21:01 jar286 done. time to smoke-test
21:02 jar286 Whoops!
21:02 jimallman ?
21:02 jar286 Whoops! The call to get the tree around a node did not work out the way we were hoping it would. Show details
21:03 jar286 show details is singularly uninformative
21:03 jimallman weird. i was just looking at the major clades
21:03 jimallman the fix is to restart api.opentreeoflife.org, but i’d like to take a closer look...
21:03 jar286 completion is failing
21:04 jimallman completion?
21:05 jar286 sorry. taxon name lookup. search box.
21:06 jar286 curator app seems to work ok
21:06 jimallman this is odd…https://api.opentreeoflife.org/cached/treemachine/v1/getSyntheticTree returns “invalid function”
21:07 jar286 is this like the failure we saw before?
21:07 jimallman could be what i saw on dev recently.. but there *is* a ‘cached’ function in the default controller…
21:09 jimallman i wonder if it’s a cached 404 response, if someone tried to fetch a tree view while api server was rebooting..?
21:09 jar286 i might have done that
21:09 jimallman since it’s a RAM cache, i don’t have an easy way of checking from here
21:10 jar286 restart apache to test?
21:10 jar286 but why would my taxon lookup fail? that was a minute or two later
21:10 jimallman yes, that should fix it, but i won’t know the cause
21:10 jimallman yeah, and i get the same 404 for other nodes in the tree:  https://tree.opentreeoflife.org/opentree/argus/otol.draft.22@132234
21:11 jar286 should we turn off caching?
21:12 jimallman yes, we can modify webapp/private/config to do this
21:12 jimallman change CACHED_* method urls to match the normal versions
21:12 jimallman one sec, please…
21:13 jar286 cache is a classic case of leaky abstraction
21:14 jimallman confirmed that the phylesystem-api/controllers/default.py is the right version, and that it does have a ‘cached’ function.. not sure why we get 404 for this.
21:14 jimallman unless routing is screwing up somehow?
21:15 jar286 but we’re running the same setup on dev
21:16 jar286 let’s try restarting one or the other of the apaches, and if that doesn’t work, retract caching
21:16 jimallman ok, restarting api now...
21:17 jar286 by the way you can restart apace with ‘push.sh -c … apache’
21:17 jar286 although that’s not much easier than the ssh sudo incantation you gave
21:17 jimallman ok, i’ll try that
21:17 jar286 s/apace/apache/
21:17 jar286 it’s the same thing
21:18 jimallman hm, no dice.
21:19 jimallman looks like a successful restart, i mean, but no joy in the tree viewer
21:19 jar286 did you restart both apaches?
21:20 jimallman no, just api
21:20 jimallman doing tree now...
21:22 jimallman OK, i see a routing(?) problem…
21:22 jimallman i’d like to try a hotfix in the webapp/private/config, if you don’t mind
21:24 kcranstn joined #opentreeoflife
21:26 jar286 ok
21:27 jimallman this is working, but it seems CORS checking is more stringent now (needs to match Access-Control-Allow-Headers)…
21:27 jar286 I will upgrade ot10 and ot16 now
21:28 jimallman i can modify the cached URLs for now and stop the pain. these are useful clues, at least.
21:29 jimallman OK, we’re skipping the cache for now
21:29 jar286 ot16 was already updated, but maybe it needs a reboot.. doing one now
21:30 jimallman skipping cache for arguson, that is. comments are still cached and looking good so far.
21:31 jar286 actually, i’ll do ot20 first…
21:31 jar286 oops I meant ot10
21:34 jar286 ot10 is rebooting
21:37 jar286 deploying to ot10
21:39 jimallman i think i found the routing problem, it’s a difference in configuration between devapi and production api
21:39 jar286 Successes: 9 Failures: 1   - which is as good as it was before.  do devapi is back up, fresh and alert
21:39 jimallman looks like the default web2py app on production is the boilerplate ‘welcome’ app, as you can see here:  http://api.opentreeoflife.org/
21:39 jar286 yes, I set that up
21:39 jimallman on devapi, it’s the phylesystem-api app: http://devapi.opentreeoflife.org/
21:40 jar286 it’s configured in the deploy config file
21:41 jimallman my cached URLs assumed that phylesystem is the default app in web2py.. if we don’t need the welcome app, i’d like to modify production config to match dev (or i can force the cached URLs to use it in apache config, if you prefer)
21:41 jar286 ouch. that seems like incorrect modularity.  what if phylesystem and opentree are running on the same server? (e.g. asterales)
21:42 jimallman good point.
21:42 jimallman in that case, i’m inclined to add this to apache shared config (/cached URLs go to this app)
21:43 jar286 that would work I guess.  or how about a new web2py app devoted to this purpose?
21:43 jimallman i could do that, yes. it’ll be tiny :)
21:43 jar286 tiny is ok
21:43 jimallman and might include lots of boilerplate junk
21:44 jar286 - (any objections to my rebooting / redeploying devtree? development branch?)
21:44 jimallman note that the caching for comments is different, just the usual @cache decorator around a normal web2py method (no CORS or other complications)
21:44 jar286 up to you.  I just consider apache hacking to be less robust.
21:45 jimallman ok, i’ll take a look at both options and touch base in a bit. meanwhile, we’re skipping the cache for tree view. (and now i’m wondering how in the heck it worked for me on production, before it broke!?)
21:46 jar286 browser cache?
21:47 jar286 I’m going to reboot devtree now
21:47 jimallman but browser cache would’ve been keyed to a cached URL on production.. unless the main page was cached and i didn’t notice.
21:47 jimallman ok, that makes sense.
21:48 jimallman i’d like to change the devapi config to also use welcome as the default web2py app. just to flush out any similar default-routing snafus
21:49 jimallman that should help to reproduce the problem on devapi and test solutions
21:49 jar286 right, go ahead
21:50 jar286 redeploying devtree now
21:50 jimallman for now, keep in mind that the master branch of api will deploy with broken caching. i can hot-fix the master branch, or we can just tread lightly til a proper fix is in.
21:51 jar286 I hope to not deploy it any time soon, but will try to remember
21:51 jar286 devtree should be back in business, i’ll try it
21:52 jar286 cellular orgnisms loads …
21:52 jar286 search for taxon works…
21:52 jimallman yes, looking good from here.
21:52 jar286 smoke tests pass
21:54 jimallman i’m not sure the cached views are much faster, to be honest. the lion’s share of the delay is probably rendering SVG. which we could try to optimize, or push for conversion to d3 viz.
21:54 jar286 agreed
21:54 jimallman but that’s a headache for another day
21:55 jimallman in any case, it might be invaluable once more people are browsing the tree at once.
21:55 * jimallman is going for coffee, back in a few
21:56 jar286 i’m going out to buy groceries in advance of the coming storm
21:56 jimallman man, you guys cannot get a break
21:56 jimallman is it supposed to let up soon?
21:56 jimallman ah, i see the forecast is sunny for Monday. but cooold.
21:57 jar286 it’s brutal… I already have to throw shovelfuls of snow over 6’ piles.
21:57 jar286 for the driveway, I pick up a shovelful, walk 20 feet to find a place to put it
21:57 jimallman sometimes i miss New England, but not right now.
21:57 jimallman gah
21:57 jar286 I’m not sure I’ve ever seen it this bad
21:58 jar286 we had 4’ of snow in ithaca once, but the storm ended so it wasn’t so bad
21:59 jimallman you’re … what… 3 or 4 weeks into this weather now?
22:00 jimallman at some point there’s just no place to put it all. the melt is going to be epic.

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary