Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2014-03-06

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:09 mtholder joined #opentreeoflife
00:44 josephwb joined #opentreeoflife
02:13 mtholder joined #opentreeoflife
03:11 mtholder joined #opentreeoflife
04:30 jimallman joined #opentreeoflife
10:15 scrollback joined #opentreeoflife
13:34 mtholder joined #opentreeoflife
14:02 towodo joined #opentreeoflife
14:05 mtholder joined #opentreeoflife
14:56 travis-ci joined #opentreeoflife
14:56 travis-ci [travis-ci] OpenTreeOfLife/api.opentreeoflife.org#344 (master - b038b33 : Mark T. Holder): The build passed.
14:56 travis-ci [travis-ci] Change view : https://github.com/OpenTreeOfLife/api.opentreeoflife.org/compare/8c70171a4207...b038b33df48f
14:56 travis-ci [travis-ci] Build details : http://travis-ci.org/OpenTreeOfLife/api.opentreeoflife.org/builds/20213133
14:56 travis-ci left #opentreeoflife
15:04 josephwb joined #opentreeoflife
15:05 travis-ci joined #opentreeoflife
15:05 travis-ci [travis-ci] OpenTreeOfLife/api.opentreeoflife.org#346 (pga - fe6b391 : Emily Jane McTavish): The build passed.
15:05 travis-ci [travis-ci] Change view : https://github.com/OpenTreeOfLife/api.opentreeoflife.org/compare/c6b709508ee8...fe6b391a2ef5
15:05 travis-ci [travis-ci] Build details : http://travis-ci.org/OpenTreeOfLife/api.opentreeoflife.org/builds/20213760
15:05 travis-ci left #opentreeoflife
15:15 travis-ci joined #opentreeoflife
15:15 travis-ci [travis-ci] OpenTreeOfLife/api.opentreeoflife.org#347 (pga - 82a9d80 : Emily Jane McTavish): The build passed.
15:15 travis-ci [travis-ci] Change view : https://github.com/OpenTreeOfLife/api.opentreeoflife.org/compare/fe6b391a2ef5...82a9d8050c2c
15:15 travis-ci [travis-ci] Build details : http://travis-ci.org/OpenTreeOfLife/api.opentreeoflife.org/builds/20214391
15:15 travis-ci left #opentreeoflife
15:31 jimallman joined #opentreeoflife
17:08 jimallman joined #opentreeoflife
17:24 pmidford joined #opentreeoflife
19:44 jimallman_ joined #opentreeoflife
20:23 towodo jimallman, you there?
20:24 jimallman towodo: yes, hi!
20:24 towodo question about the webapp...
20:24 towodo it uses approximately 8 services from treemachine and taxomachine, according to the sample config file...
20:25 towodo are these services called from javascript on the client, or from python on the server?  or a mix?
20:25 towodo that's the question.
20:25 jimallman a mix, i'm afraid (mostly JS)
20:26 jimallman typical server-side calls are to populate UI elements or if data dictates the URL (e.g., for a taxon-search result page)
20:26 towodo ok, and it's depending on the proxypass setup to deal with SOP?
20:27 towodo or is it using CORS?...
20:32 towodo looks like it's using CORS.
20:32 towodo doing GET methods inside our neo4j plugins is looking intractable.
20:33 towodo I conclude it's using CORS by looking at the deployed apache config on ot3.
20:34 towodo the /treemachine URLs on ot3 couldn't possibly work. ergo, javascript is contacting ot9 directly, which works due to CORS.
20:35 * jimallman is catching up now (got distracted in my code)
20:35 towodo hmm, looks like web2py has a caching feature...
20:36 jimallman yes, we're generally using CORS, but web2py is pretty bad at it (any error on the server causes the CORS response to fail, so no response at all)
20:37 jimallman re: caching neo4j responses, can't we introduce a proxy cache to store these?
20:37 jimallman (assuming some kind of SHA or Etag to ensure we have the latest)
20:37 towodo no, that's the whole problem.  neo4j uses POST only, and those aren't cachable
20:38 jimallman ah, so it blocks GET requests by design. wow.
20:38 jimallman i remember now, calling these w/ GET in curl gives a service description
20:38 towodo well, they're not blocked, they just have poor support, and work completely differently than the POST setup that Cody and Stephen use
20:39 towodo so I'm thinking maybe do the POST using python under web2py, and cache the results using http://web2py.com/books/default/chapter/29/4#cache
20:41 towodo maybe this is getting too hairy.  I don't have proof that neo4j will be a bottleneck, just a hunch
20:41 jimallman if i'm reading the web2py cache docs correctly, it looks like it would be pretty easy to cache the results of GET requests to the API.
20:43 jimallman looks like we should investigate cache.action...
20:44 mtholder joined #opentreeoflife
20:44 towodo it would just require rewriting all the spots where web2py is currently doing POST, and introducing shims that javascript can use. not too too bad
20:45 mtholder hi, jimallman: can you pull and try again (responding to an email from jim via irc)
20:45 jimallman mtholder: will do
20:46 jimallman towodo: maybe i'm not understanding you. can't we just directly cache the response to API calls that use GET? these pass through web2py already .
20:47 towodo ahh… hmm...
20:47 guest|54705 joined #opentreeoflife
20:47 jimallman mtholder: just out of curiosity, where did you fix the config problem?
20:48 mtholder i commented out the call in peyotl that used peytol's config.
20:48 mtholder we did not intend that to be used.
20:49 towodo jimallman, I'm not sure what you mean, but note that the cache wants to be shared among all users
20:49 mtholder the api is now using peytol for (local) git operations.
20:49 jimallman mtholder: i'm re-deploying the api to ot10.. let me know if there's a pull request somewhere, i didn't see any
20:50 towodo jimallman: but you're right, we could just put even an Apache cache in front of some of the web2py pages, such as tree views
20:50 mtholder i just pushed a new commit to peyotl master.
20:50 jimallman hm, ok
20:57 jimallman mtholder: now we get a different save-study error, an opaque 400 with no apparent cause. progress!
20:57 jimallman towodo: i'm guessing the auth values we send with API GET requests will be enough to cache separate responses for each curator
20:58 towodo I don't care about caching the curator app… I would have exempted it entirely
20:58 towodo I'm thinking mainly about the browser
20:59 jimallman ah, of course.
20:59 towodo where cached pages would be shared among all users
20:59 mtholder progress indeed. ;-( we are seeing some tests failing. So guest|54705 and I are trying to diagnose...
21:02 16WAA0LTF joined #opentreeoflife
21:03 towodo jimallman, https://httpd.apache.org/docs/2.2/caching.html
21:06 towodo but… the tree views aren't separate pages, they are managed by javascript… so this doesn't work
21:06 towodo so I think the POSTs need to be turned into GETs and cached either in web2py or in apache
21:08 jimallman yes, my apologies in steering you wrong before. calls to getSyntheticTree are of course outside of the Open Tree API, and never went through web2py
21:08 jimallman but i suppose we could call web2py using GET, or apache for that matter, and mutate the requests to a POST if we need to pass it on to treemachine
21:09 towodo maybe the thing to do is to create a new web2py app that shims all of the tree and taxomachine calls, converting GET to POST.  then use mod_cache to cache the responses
21:11 jimallman yes, or possibly a controller in the main webapp
21:12 jimallman that is, a 'treemachine' controller and a 'taxomachine' controller, so the URLs would be the same a now
21:12 jimallman (never mind, URLs would change..)
21:15 jimallman i suppose we'd end up with GET requests like this:
21:15 jimallman http://dev.opentreeoflife.org/treemachine/getSyntheticTree?treeID=otol.draft.22&format=arguson&maxDepth=3&subtreeNodeID=215
21:15 towodo yes
21:16 jimallman leaving a pretty straightforward transformation to the real POST request. yep, that should work.
21:17 jimallman ideally with some kind of forced expiration of the entire cache when we update the synthetic tree
21:17 towodo yes
21:19 towodo I might be able to handle this...
21:26 towodo ok, heading out, thanks for your help
21:30 jimallman sure thing. i'll let you know as soon as i have my head above water...
21:59 guest|54705 ok! I think the api is behaving on ot7 now
22:01 jimallman guest|54705: hi, is this Emily?
22:01 guest|54705 Yes!
22:01 guest|54705 I am in disguise...
22:01 jimallman cool, i was just sending you stuff.. pls disregard if it's working now...
22:02 * jimallman goes off to test again
22:03 guest|54705 Hmmm. I not actually convinced that it will fix the bug you just sent.
22:04 guest|54705 I will think on it, but unfortunately (also fortunately) I'm going to be off line till Monday AM.
22:04 jimallman yes, this is a good test case (sudden introduction of focal-clade taxon id from "upstream"). maybe we need to merge those changes to all WIP branches..?
22:05 jimallman ok. i'll try some other studies and see if we can't save *anything* (kind of a bummer, but oh well)
22:05 jimallman plenty to do here, regardless. have a great weekend!
22:07 jimallman guest|54705: if you don't mind, i'll push your latest to the API server for dev.opentreeoflife.org. it's an improvement in any case, since i'm getting the very latest changes when i save and re-load a study.
22:07 guest|54705 Sounds good to me!
22:08 guest|54705 glad it helped at least a bit.
22:08 guest|54705 Have a good weekend!
22:58 mtholder joined #opentreeoflife
23:07 josephwb joined #opentreeoflife

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary