Perl 6 - the future is here, just unevenly distributed

IRC log for #webwork, 2013-04-02

| Channels | #webwork index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:37 mgage joined #webwork
01:10 rbeezer joined #webwork
01:29 goehle_ joined #webwork
01:37 goehle_ hey
01:37 goehle_ just got your email
01:37 mgage hi
01:37 mgage the comment is also to help set rules for the future :-)
01:38 mgage I think we'd be glad of help and advice from the CS guys.  (That doesn't mean we won't try to modify the advice once we get it -- but info from professionals will be helpful.)
01:39 goehle_ great
01:39 goehle_ I talked to a bunch of them
01:39 mgage Was the final verdict that a global table for past_answers was a good idea?  -- I was a little confused by the final comments.
01:39 goehle_ yeah, there was a lot of back and forth
01:39 goehle_ I didn't want to write it all out until there was consensus
01:40 goehle_ they said that mysql can certainly handle big tables, but there is less margin for error
01:40 goehle_ suboptimal coding (like putting database calls inside for loops and not using the correct index setup) will have a much bigger impact if you have 10 million rows
01:40 goehle_ than if you have 100,000
01:41 mgage I think there is a lot to think about -- so I'm not surprised there were differences of opinion.  I'd like to eventually mine the past_answers using BigData techniques to try to create information about either problem effectiveness or types of students working on the problems or both -- and one big table helps that a bit.
01:41 Paul_Pearson joined #webwork
01:41 goehle_ for now their suggestoin was to go to per course tables
01:41 goehle_ since we already have all the tools to do that
01:41 goehle_ it also solves our upgrade problem
01:41 mgage true enough.  there are also security issues that have to be watched more carefully if you mix rows from different courses.
01:42 goehle_ since moving to per course tables will automatically create the course tables with the correct key structure
01:42 goehle_ something more interesting, though.  Both of the cs guys in the department (one of whom specializes in databases) are interested in looking more at webwork
01:43 goehle_ so sometime at the end of the month I was going to spend some time going over the current code (focusing on the database)
01:43 goehle_ then maybe we could all have a conference about what to do with the future database setup
01:43 mgage I see -- so we want a short term solution that allows us to put release/2.7 out in the next month -- so moving to per course tables does make that easier. Also means that searches are automatically reduced in size.
01:43 goehle_ right
01:43 goehle_ so suboptimal coding and whatnot doesn't matter so much
01:44 goehle_ they had lots of ideas about how to make a big past_answers database work
01:44 goehle_ but they all involved some work
01:44 mgage that sounds like a plan -- and I'm glad to consider including them in the vancouver camp if they are interested.
01:44 goehle_ for example, if you are using many many rows then its better to index over numerical fields
01:44 goehle_ they very well might be
01:44 mgage it would bring another perspective to the group.
01:45 goehle_ it cant help with planning the next iteration of the database
01:45 mgage if we are using tables then it helps a lot to keep the tables narrow and use joins on numeric indices -- I've gotten that much rule of thumb
01:45 goehle_ hmm cant not help mabye
01:45 goehle_ right
01:45 goehle_ they were saying that
01:46 goehle_ (which we don't do much of)
01:46 goehle_ so a new database might have a lot more crosslisting of table rows and the like
01:46 mgage so did you see the comments that Danny sent in from his CS database guy?  some comments on noSQL database types -- as long as we are designing it's worthwhile to think big for a while at least.
01:47 goehle_ yeah, I did see those
01:47 goehle_ once I run the cs guys through the code
01:47 goehle_ there could be some big picture conversations about what exactly we will use for the next generation backend
01:47 mgage moodle is the database that I studied a bit a few years ago -- it has many narrow global tables and that was my original model.  However things have evolved in even the two years since I studied that closely.
01:48 goehle_ one of my issues is I"ve never really studied it, so I'm a bit out of my depth
01:48 mgage it's worth taking a look at it's design for comparison.
01:48 mgage that is noodle's design
01:48 goehle_ also a good idea
01:48 mgage moodle's design
01:49 mgage in general I think that WW has been successful partially because it was developed by math instructors for their own needs
01:49 mgage that inadvertently is good HCI design since the designers and the target users are one and the same
01:50 goehle_ I agree.  Its easy for us to priorities things
01:50 goehle_ I did convince the cs guys that we needed to work in the existing framework
01:50 goehle_ no trying to rewrite the whole thing in python/django
01:50 mgage but we've evolved to the point where we could use some help from the pros on a number of fronts.  -- both backend databases,  organization of open source communities and more.
01:51 goehle_ in general I think their interest is how to bring exisitng systems into modern frameworks without rewriting everything
01:51 mgage that's good -- that is not a trivial task -- and it may be eventually that new problems will be written in a python dialect instead of a perl dialect -- that hook is kind of there already but hasn't really been used.
01:52 mgage I also suspect that people will find that perl is better than the expected as a language for constructing new DSL's (domain specific languages) of which WW is an example.
01:52 mgage they expected
01:52 goehle_ yeah, its flexibility can be a curse, but there is no denying its helpful
01:53 goehle_ that was just an example though
01:53 goehle_ I think the big thing is they agree that changes to the database backend have to preserve the existing accessor methods
01:53 goehle_ how close is 2.7 supposed to be to release?
01:53 mgage I think that's wise -- and for the most part not that difficult to manage.
01:54 mgage I'd like to release it with in the next month or month and a half.  I'd like it to have a solid version of your essayQuestions -- but perhaps not the final version.
01:55 mgage It won't have the new front end that Peter and David have been designing and which is now (working as far as I can tell) in the develop branch of openwebwork.
01:56 goehle_ will it have math4?
01:56 mgage I'd like to have that ready by mid summer to the end of summer -- and possibly add database stuff if that is coming along well enough.  If it's incremental and we can add it a bit at a time then we can also incrementally test it which is a huge advantage.
01:57 goehle_ sounds like a plan.  For what its worth my recommendation is to play it safe with past answers and make it a native table
01:57 mgage I think math4 can go in to release/2.7 -- it's not that big a change -- and there is still a month to shake problems out on hosted2 during the next month.  After that real testing slows down considerably because all of the classes are out.
01:58 goehle_ I've been looking through my code and I do a lot of inefficient things like database calls in for loops
01:58 goehle_ (when its probably better to call the database with an array and then loop through the results)
01:58 mgage ok -- I'm willing. Do you think you might be able to pull a version of release/2.7 as it stands now and make the changes in the way the tables are created?
01:58 goehle_ those kinds of things will matter less if the tables are smaller
01:58 goehle_ sure
01:59 goehle_ so I shoul dpull release/2.7
01:59 mgage ok -- that would be very helpful.
01:59 goehle_ change just the database stuff and repull
01:59 mgage yes then send a pull request back to release/2.7
01:59 goehle_ ok
01:59 goehle_ so how do I get a fresh copy of release/2.7 to work with?
02:00 mgage you may need to make changes in pg   release/2.6  as well (that already had essayQuestion stuff in it and I haven't renumbered it.)
02:00 mgage brb
02:01 mgage probably the best way would be
02:02 mgage git branch -t fresh2.7 origin/release/2.7
02:02 mgage assuming origin is the remote name for github.com/openwebwork/webwork2
02:02 mgage (I think I have the syntax right )-- this creates a branch that tracks release/2.7.
02:03 goehle_ yeah that worked I think
02:03 mgage when you are done   git branch fresh2.7
02:04 mgage then git push   mygithub    fresh2.7
02:04 mgage will push this branch up to your github.  Finally issue a public pull request from your github.   (SourceTree will do some of this stuff for you graphically which can be helpful.)
02:04 goehle_ right
02:05 mgage I'm nudging people to get more documentation out on this.
02:05 mgage but some approximation of what I just said should work.
02:06 goehle_ its looking like it is
02:06 mgage the image is a triangle -- pull from openwebwork ---- work on your changes --- pull from openwebwork again to make sure that openwebwork hasn't changed in the meantime   ---push the result to your github and issue a public pull request to openwebwork
02:07 goehle_ if only it were ever that simple
02:07 goehle_ looks like math4 isnt on openwebwork/release/2.7
02:07 mgage the last step means potential changes are always out in the open and those who are only casually following along can view what is happening if they want to.
02:07 goehle_ my server is set to math 4
02:07 goehle_ how do i change that manually?
02:08 goehle_ simple.conf?
02:08 mgage your sever is set to math4 theme? or math4 branch? -- you called one of your branches math4
02:09 goehle_ right
02:09 goehle_ my server is set to the math4 theme
02:09 mgage git branch -vv will tell you what branches you have and what they are connected to upstream
02:09 mgage you can change that in Course Configuration -- it's set course by course
02:09 mgage (which changes simple.conf )
02:10 goehle_ I cant see course config
02:10 goehle_ because it wont load any pages, since system.template is missing
02:10 goehle_ I manually changed simple.conf and fixed it though
02:10 mgage yeah -- we need to fix that bug someday -- you can lock yourself out from the web if you are not careful
02:11 goehle_ :D
02:11 goehle_ ok
02:12 goehle_ so I changed the database stuff
02:12 goehle_ it seems to work
02:12 goehle_ I had to upgrade my courses using the usual method
02:12 goehle_ and then past answers seemed to be fine
02:13 mgage anything need changing in the admin module? or in pg modules?
02:13 mgage you had some update code in the admin module if I recall
02:13 mgage but you probably changed all of that in converting to native tables
02:13 goehle_ pg should just be for the essay macro stuff
02:13 goehle_ I've left the code in admin
02:14 goehle_ but its not used now because its native
02:14 goehle_ so the upgrade/creation happens via the usual course upgrade process
02:14 goehle_ one nice thing is that I dont think any of the essay code is affected
02:14 goehle_ since it was using the accessor methods
02:14 goehle_ and they should work however the database is structured
02:14 mgage ok -- that's good.
02:15 mgage sounds like you are still creating a global past_answer table as well -- should we stop doing that for now?
02:15 mgage also remember to check that past_answers continues to work -- since you were using the answer table for that as well
02:17 goehle_ I check past_answers
02:17 goehle_ and that works
02:17 goehle_ I haven't check this
02:17 goehle_ but since the non_native flag
02:17 goehle_ is no longer in the database.conf file
02:18 goehle_ it won't create a global past_answer database table anymore
02:18 mgage ok -- sounds good.  remember to make the change in database.conf.dist  before you submit the pull request -- and don't add the database.conf file itself to the commits
02:19 mgage I've got to get some advising awards submitted tonight so I'm going to work on that -- but I'll be able to look at a pull request tomorrow
02:20 goehle_ right
02:20 goehle_ hmm
02:20 goehle_ I'm getting a http request failed 403 error
02:21 mgage for the push to your github?
02:21 goehle_ yeah, but I think I figured it out
02:24 goehle_ ok
02:24 goehle_ pull request sent
02:25 goehle_ if the new look is already in develop
02:25 goehle_ how much effort should I put into math4 at this point?\
02:27 goehle_ actually, well catch up on that some other time
02:27 mgage I don't know the answer to that for sure -- I think some of the incremental changes in math4 theme could be added early -- one can back out of them easily and having them around and played with helps with decisions involving the current develop -- switching to the current develop is something of an all or nothing step that people will be more reluctant to try out.
02:27 mgage that's fine
02:27 goehle_ :D
02:27 goehle_ have a good one, let me know if that stuff pulls cleanly
02:28 mgage thanks much geoff.  I'll be in touch
02:28 goehle_ old past_answer entries won't be transported over (of course) but they wont be deleted either
02:28 goehle_ by
02:28 goehle_ e
02:28 goehle_ left #webwork
10:28 Tim3 joined #webwork
11:38 mgage joined #webwork
12:52 goehle_ joined #webwork
12:52 goehle_ left #webwork
12:52 goehle_ joined #webwork
12:52 goehle_ left #webwork
12:52 goehle joined #webwork
12:53 goehle_ nick goehle
13:10 mgage joined #webwork
14:39 rbeezer joined #webwork
14:44 rbeezer_ joined #webwork
15:17 mgage joined #webwork
15:28 mgage rbeezer_: hi
15:42 mgage joined #webwork
15:52 Paul_Pearson joined #webwork
16:32 mgage joined #webwork
16:41 rbeezer_ mgage: back from a meeting, sorry to hear you can't make Sage Days
16:42 mgage_ joined #webwork
16:43 rbeezer_ mgage_: just saw your email.  Go ahead and fiddle with essay questions
16:43 mgage_ me too, but the summer was getting thoroughly crowded
16:43 rbeezer_ a good thing - but you don't want it too crowded
16:43 rbeezer_ I've passed on a few trips myself
16:43 mgage_ kk -- I'll do that and put up a note when I do -- make sure you and your students let me know if anything weird is going on -- I'd like to get this tested before school lets out
16:44 rbeezer_ I'm not authoring essay questions - getting experience with NPL/OPL so I can add my essay questions in the fall for everybody to use
16:45 mgage_ great -- if I get time I'm going to add an essay question or two to my multivariable calculus class just to get some experience
16:45 mgage_ any chance you would be able to drop in at the Vancouver code camp?
16:46 rbeezer_ Great!  It'll be good to have another experimenter.  Easy written feedback to students (via email) is the only big feature I see that it needs.  Anything else would be tweaks (IMHO).
16:46 mgage_ I'm hoping to have more info posted this weekend.
16:46 mgage_ you'll get feedback in this next version I believe.
16:47 rbeezer_ Perfect.  ;-)
16:48 mgage_ speaking of feedback - have you used Piazza? -- I've been using it this semester instead of "email your professor" -- it works fairly smoothly.
16:49 mgage_ it will take some time to see how effective it will be -- the student culture has to get used to it.
16:49 mgage_ https://piazza.com/
16:49 rbeezer_ enabling cookies for Piazza....
16:50 rbeezer_ mathjax-enabled, nice
16:50 mgage_ you can view my class here: https://piazza.com/demo_login?nid=hbxxlhlswds5p6&auth=94892a8
16:50 mgage_ Professors appreciate Piazza best when they see how it is being used.
16:50 mgage_ Allow colleagues to view your class through a demo link - a restricted, read only version of your class where all students’ names are anonymized and all student information hidden.
16:51 mgage_ the guest view is an example of nice little touches
16:52 rbeezer_ yes, that looks nice - like an instructor "endorsement" of an answer
16:56 mgage_ I suspect it would take a semester or so of use before it reaches its full potential -- but I've been pleased with how its gone even with the multivariable calc course being the only math course that uses it (and this is the first semester)
17:00 mgage_ off to school -- have office hours this afternoon -- ttyl
17:14 Paul_Pearson joined #webwork
18:03 mgage joined #webwork
18:59 Paul_Pearson joined #webwork
20:15 rbeezer joined #webwork
20:27 Paul_Pearson joined #webwork
22:09 mgage joined #webwork
23:08 mgage joined #webwork
23:12 Paul_Pearson joined #webwork

| Channels | #webwork index | Today | | Search | Google Search | Plain-Text | summary