Perl 6 - the future is here, just unevenly distributed

IRC log for #opentreeoflife, 2014-10-20

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:02 kcranstn joined #opentreeoflife
00:57 towodo joined #opentreeoflife
10:50 jimallman joined #opentreeoflife
11:54 kcranstn joined #opentreeoflife
11:59 towodo joined #opentreeoflife
13:51 mtholder joined #opentreeoflife
13:59 kcranstn joined #opentreeoflife
14:19 jimallman joined #opentreeoflife
15:08 towodo jimallman, I have a web2py question…
15:08 jimallman hi! let’s see if i can help
15:09 towodo it’s about state.  the web2py model is that state lives in a database…
15:10 towodo I’m wondering about doing expensive operations during initialization of a model or controller .py file
15:10 towodo the question is, how often do these .py files (and python as a whole) get reloaded?
15:10 towodo is it only when apache is restarted?
15:10 jimallman believe it or not, these files are run on every request
15:10 towodo omg.
15:11 jimallman some of the code (like models.py) is obviously sensitive to existing state and is usually a no-op
15:11 towodo I’m looking at applications/welcome/models/db.py
15:12 jimallman so that’s kind of the model. models.py or modules.py is probably where this kind of thing belongs.
15:12 towodo so, not in models/, but in modules/ - or where?
15:13 towodo I’’m thinking about an expensive, but re-doable operation, that creates a data structure that I want my controllers to access
15:14 jimallman in principle, models is for db setup and other persistence. modules is for (you guessed it) organized python code called from within the app.
15:14 jimallman sounds like models is what you want
15:15 kcranstn except for that “expensive operation” part
15:15 jimallman i take it “re-doable operation” means the user can trigger this on demand..? or automatically, based on circumstances?
15:15 towodo I really want it done when web2py starts up
15:16 jimallman that sounds like models/ then. it’s generally used to set up conditions required for the app to work.
15:16 towodo can be redone each time python starts
15:16 towodo so, all the .py files in models get loaded each time web2py starts?
15:17 towodo but not on every page load?
15:17 jimallman i believe it’s on every page load, with enough smarts to know whether the conditions are already satisfied.
15:17 jimallman so we might need a flag of some kind to trigger it running again.
15:17 towodo oh, I see, that’s what the ‘if’ is about in db.py ?
15:17 jimallman crude (but effective) would be to clobber the old data.
15:18 * jimallman is looking for “if” there now...
15:18 kcranstn waht are you trying to set up, towodo?
15:19 towodo hmm, no, that ‘if’ is about database type
15:19 towodo so you can store state inside of… what? the session? but I want it stored across all sessions
15:19 jimallman yes, the tests in welcome/models/db.py are mostly for the runtime environment (Google App Engine needs special handling), etc.
15:20 jimallman general system state could be in the database, perhaps a dedicated table with some flags? or just a JSON file, i suppose.
15:20 jimallman (we have a state-tracking table in the Fossil Calibrations site that has worked pretty well)
15:21 towodo I’m trying to avoid creating a database.  it’s just some csv files.  (taxonomy and related)
15:21 towodo even if it were a database i’d want to cache it in memory. so an external database would be pointless
15:22 jimallman got it.
15:23 towodo kcranstn, evaluating alternatives for displaying information from taxonomy (at least four technology alternatives her)
15:23 towodo s/her/here/
15:23 towodo maybe a module in modules/ would do the trick
15:24 jimallman so this expensive operation is to update the .csv files? or build the display?
15:25 towodo no, the ooperation is to load the .csv files into data structures in memory
15:25 towodo maybe 30 seconds
15:27 jimallman i see.. lemme see what i can find about persistent in-memory structures in web2py...
15:27 towodo not your job really, I can do the research. just thought you might know
15:29 jimallman not off-hand. as far as i know, the data structures would need to be loaded in each request (though you could of course only do this in a relevant controller or action)
15:31 jimallman some possibly relevant discussions here: https://groups.google.com/forum/#!topicsearchin/web2py/persistent$20AND$20data
15:37 jimallman this looks like web2py’s cache is the best option (shared across users and sessions), if you want to avoid creating a db:
15:37 jimallman http://web2py.com/books/default/chapter/29/4#cache
15:40 towodo did I ever tell you how might I hate frameworks
15:40 towodo s/might/much/
15:41 jimallman i hear you. i’d like to explore Pyramid, it strikes a nice balance with much more control.
15:42 jimallman fwiw, i think web2py’s modules/ is a decent method for escaping the limitations of web2py
15:44 towodo cache.ram() doesn’t look so bad… in fact we could use it to speed up the study list display, maybe some other things
15:44 towodo like the root tree view
15:45 towodo can do cache.clear() when a study is added or deleted
15:46 jimallman agreed
15:46 jimallman i’ll look into this for devtree
15:47 towodo although we don’t *really* know where the time is being spent (server side or client side)
15:48 towodo web2py as an alternative to http caching
15:49 towodo must… resists… looking at cache.action
15:49 kcranstn :)
16:12 towodo kcranstn, I’ve just made a new draft of the taxonomy… with worms
16:13 towodo the problem is that there are 42 deprecated taxa, and they need to be examined because half the time they reflect errors that could sabotage synthesis
16:13 towodo … so I’m thinking about how to speed up the ‘examination’ process.  that led to the web2py question
16:19 towodo you’re probably eating lunch. I think I will go do so now
16:44 pmidford2 joined #opentreeoflife
16:55 mtholder joined #opentreeoflife
17:41 towodo joined #opentreeoflife
18:54 mtholder joined #opentreeoflife
19:10 mtholder left #opentreeoflife
19:27 kcranstn did you get any feedback from Keith & Chris?
19:28 kcranstn about the taxonomy?
19:38 towodo not yet

| Channels | #opentreeoflife index | Today | | Search | Google Search | Plain-Text | summary