Perl 6 - the future is here, just unevenly distributed

IRC log for #webwork, 2012-08-23

| Channels | #webwork index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:28 djun joined #webwork
02:44 goehle joined #webwork
02:44 goehle hey mgage
02:44 goehle quick question while you are here
02:45 mgage hi
02:45 mgage sure go ahead
02:45 goehle I want to switch back to the old version of pgEditor
02:45 goehle because the new one has some kinks that are confusing the other profs
02:45 mgage oh and by the ay I'll write you an email -- probably tomorow
02:45 goehle no worries
02:45 goehle its not due until october
02:46 goehle and it only has to be a paragraph
02:46 goehle the main thing is that we are under the boyer model
02:46 goehle so it has to be "reviewed" in some fashion to count
02:46 mgage boyer?
02:46 mgage ah right -- I'll review it.  I'd like to set up a more formal process as well such as the one sage has but that might take some time
02:46 goehle http://en.wikipedia.org/wiki/Boyer's_model_of_scholarship
02:47 goehle well
02:47 goehle review is a stretchy word
02:47 goehle I dont want to cause a lot of extra work
02:47 goehle take a look at it by any means, but in some sense you are reviewing it by integrating it into the main code
02:48 mgage np -- as to the pgEditor -- if you just enable the first version does that work as you expect (there are three versions of the PGproblemeditor
02:48 mgage the first version should be pretty similar to what they were used to -- I don't think there were any changes to it -- at least not on purpose.
02:49 goehle my question was how to enable it
02:49 goehle I set a variable
02:49 goehle pgproblemeditor1 => 1 in
02:49 goehle postlocal.conf
02:49 goehle or something like that
02:49 goehle but it didnt seem to do much
02:50 mgage in localOverrides.conf  there is a similar thing -- you want to set the other editors to zero.  You might like to upgrade to the version 2.5.1 which is in webwork2-dev -- I think it is stabler right at the moment
02:51 mgage I'm trying to get these versions stabilized and then placed in webwork2
02:52 goehle yeah
02:52 goehle this is on my production machine
02:52 goehle so I am hesitant to upgrade
02:52 goehle at least until th enext big break
02:52 mgage the main reason for going to 2.5.1 is that it has Danny Glin's version of the configuration files using sites.conf default.config and localOverrides.conf instead of prelocal, global, postlocal.  Not only are his names much better but he has cleaned up the configuration
02:53 mgage yeah -- I realize that is a problem You should be able to set pgproblemeditor =>1
02:53 mgage and the others to set to 0 and you will only see the old problem editor.
02:53 mgage you might have to reboot the server to see the effect
02:54 goehle hmm still looks the same
02:54 goehle pgproblemeditor1   => 1,                                                \
02:54 goehle pgproblemeditor2   => 0,                                                \
02:54 goehle pgproblemeditor3   => 0,
02:55 goehle # );
02:55 goehle is what I have in my config file
02:55 mgage what looks different from what you are expecting??
02:57 goehle it seems like the same problem editor I was using before
02:57 goehle there are two "save" tabs for example
02:57 goehle (one should be save as)
02:57 mgage ok -- I know what it is
02:58 mgage there was a period where the pgproblemeditor2 got copied over to the pgproblem editor 1  -- (and the cs on problemeditor 2 is a bit wonky -- the under bar didn't play well with css
02:59 goehle ok
02:59 mgage is your production server using git?
02:59 goehle it is
03:01 mgage here is what I would suggest -- and I grant that I'm less risk adverse than most people.  Create a new branch in git and download the master copy of openwebwork/webwork2-dev and pg-dev
03:01 mgage you can configure them -- but don't upgrade any of the courses.
03:02 mgage I'm pretty sure that the verson of PGProblemEditor1.pm in 2.5.1 is the original pgProblemEditor version
03:03 mgage so you can make a copy of it, switch back to your main branch and replace the copy you currently have (there is probably a fancy way to do this with git but I don't know it reliably)
03:04 mgage I think you will be able to switch back and forth between the two branches easily just by switching and booting the server - that way you'll get the new stuff available but you can revert back to the old stuff if you need to.
03:05 goehle nice thanks for the advice
03:05 goehle I did it a bit more old school
03:05 goehle I just used wget to get the file from github
03:06 mgage the only thing I'm not 99% sure about is upgrading the courses -- it could be if that you upgrade one you can't go back to the previous ww branches -- so I would only upgrade a test course at first
03:06 goehle and replaced the .pm file in my install
03:06 mgage did it do what you expected?
03:06 goehle yup
03:07 mgage kk -- for the sites I'm running I'm working on setting them up so that I can switch between versions quickly -- still don't have it completely in place where I can explain the process to others clearly.
03:07 goehle when I set up th elatest dev server I used your link trick
03:08 goehle I've got webwork-old
03:08 goehle and webwork-dev
03:08 goehle and I can change which one i'm using by changing which one webwork2 points to
03:08 goehle since this was a single fix of one page I figured just replacing the .pm file would be easiest
03:08 mgage that's good, but once you get used to it git makes it even easier to switch between branches -- it makes testing much more feasible and less scary
03:09 mgage it was the easiest for this one -- particularly since I was pretty sure what was wrong
03:09 goehle yeah
03:09 goehle I like what you guys are doing with the branches and the version nubmers
03:09 goehle that seems like a good way to go
03:10 goehle in any case, I have to get to bed
03:10 goehle thanks for the help with the pgeditor
03:10 mgage 'nite   np
03:10 goehle left #webwork
03:45 djun joined #webwork
12:31 goehle joined #webwork
15:05 aubreyja joined #webwork
15:06 aubreyja mgage: hi
15:07 mgage hi
15:08 mgage aubreyja: hi
15:09 aubreyja So, the start-of-the-semester storm seems to have passed, so I can work on webwork stuff
15:10 mgage the webwork phone call is coming up soon, but after that I could use someone looking over my shoulder as I try to get a stable 2.5.1 (call it 2.5.1.1) up into webwork2.  I've started more or less by editing the release notes for weBWorK 2.5.1 about what will be included
15:11 mgage and I'm now trying to match the description and the release
15:11 aubreyja Our college algebra and intermediate algebra courses went to webwork this semester from mymathlab - we've now got around 3500 students, and next week it will go up about 4200 when the stat 1200 course starts using it again
15:11 aubreyja sure thing
15:11 mgage wow
15:11 aubreyja yeah, I was nervous.
15:12 aubreyja The only problem that came up was that we forgot to up the mysql max_connections from the default of 150
15:12 aubreyja so, we got a few minutes of 'too many connections' before we increased it to 5000
15:12 aubreyja which is hopefully way more than we'll need at any one moment
15:13 mgage gavin is understandably reluctant to use 2.5.1 -- I want to get the authen business ironed out and get just the features we have advertised into 2.5.1
15:13 aubreyja I've had trouble with the authen stuff too.  We can talk about this afternoon
15:14 aubreyja I ended up commenting out some of what Bill added
15:14 mgage the master version of openwebwork/webwork2-dev is the current model for 2.5.1
15:14 aubreyja cool - I'll get it up and running on my laptop during our call
15:14 mgage yes -- I think we should just make it work and then commit it.  If Bill wants additional changes we'll do it later.
15:15 aubreyja sure - have you started a wiki page with release notes? I think I saw one
15:15 mgage yes -- there is a whole category
15:17 mgage actually it is a little unclear still what we should put in to 2.5.1 and what we should put into 2.5.2  -- that's one of the things I'd like to talk about.
15:17 aubreyja to clarify 2.5.1 will be the official 'release' and 2.5.2 will be the next release?
15:18 mgage yes -- but for example I want sites.conf and localOverrides.conf in 2.5.1 so that not many people have to switch to both prelocal.conf and then sites.conf  etc.
15:19 mgage on the other hand -- do we want MathAchievements in 2.5.1 or in 2.5.2? -- it requires upgrading courses.  On the other hand I'd like to get Geoff's stuff in quickly so that he gets credit for a "publication"
15:19 mgage so I'm inclined at the moment to include MathAchievements in 2.5.1
15:20 mgage 2.5.3 I know will be reserved for adding Aaron's white board.  That will require a lot of changes and will be pretty fragile for a while.  I have a version in mgage/webwork2-dev
15:23 djun joined #webwork
15:26 aubreyja sorry - visitor - Teri (college algebra coordinator) just came by to say how happy she is with webwork :)
15:27 mgage cool
15:27 aubreyja Yes, that's right - I agree on both of those. Also, my hs users really want achievements
15:28 mgage ok -- the other thing we'll try to get in is themes -- so that we can incrementally work on cleaning up the css
15:28 mgage ww conference call
15:29 aubreyja Good - I like djun's modifications maybe for the future we can build in some capability for modifications by course professors...
15:29 aubreyja yep, calling in now
15:30 djun what are the call details for the call?
15:31 aubreyja just a sec
15:44 djun joined #webwork
16:41 dbrianwalton joined #webwork
16:42 dbrianwalton I have a question/issue with user authentication (or maybe general database issues).
16:43 mgage hi
16:43 dbrianwalton Several months ago I made a remark about our WW server slowing down.  On the forums, it was suggested that perhaps it could be resolved with Apache2::SizeLimit.
16:43 dbrianwalton We implemented that change (it looks like with too small a threshold, but anyway...), but we are still getting some stalling on the server.
16:44 aubreyja Bill Wheeler recently found and fixed a memory leak in WeBWorK::Authen
16:44 mgage yes -- I haven't tested yet whether that is making a big difference
16:44 dbrianwalton I enabled the debug output and yesterday afternoon caught a stall and identified it in the error.log file.  The stalling was always occurring in the Authen module where the database was being queried about an open key.
16:45 mgage you main to check whether the key was current?
16:45 aubreyja That's interesting...
16:46 dbrianwalton I think that is correct.  Give me a second to track down exactly which routine I was stalled in.
16:48 dbrianwalton Multiple sessions successfully entered "verify" but get bogged down in the middle of "verify_normal_user"
16:49 dbrianwalton The stall lasted multiple minutes, and then suddenly all of the requests were simultaneously freed, successfully exiting the verify routine.
16:49 dbrianwalton I'm suspecting that the database was somehow locked up and then reset and finished the queries.
16:49 aubreyja strange - did you have a count of how many sessions? It could be a mysql max_connections issue
16:50 dbrianwalton I didn't have any access to the mysql information directly at the time.
16:51 dbrianwalton Based on the logs, there were three users involved.  One was an admin user (not me) who was running a script using an automated web-scripting utility.
16:51 aubreyja about how many users do you have on the machine? max_connections by default in mysql 5.5 is set at 150 (I just had this error yesterday)
16:52 aubreyja ah, ok
16:52 dbrianwalton Classes don't start until next week.  That is why I had the debug turned on while the load was light, but we were still having some issues.
16:53 dbrianwalton I have a feeling the script had something to do with something, as it was logging into essentially every course and making a modification to one of the files through File Manager.
16:53 aubreyja ok, so that's probably not the issue
16:54 djun_afk joined #webwork
16:54 dbrianwalton Since each class has its own database tables, could it have something to do with trying to buffer too many tables?
16:57 aubreyja maybe, that's getting beyond what I know about mysql though
16:57 aubreyja also though, it sounds like your users weren't actually accessing very many tables
16:58 aubreyja just three users would have queried at most 3 key tables for the key, right?
16:59 dbrianwalton I am unsure exactly what the script was accessing, but I think it may have opened the Classlist Editor prior to modifying a file.
17:00 dbrianwalton I just noticed another discrepancy.
17:00 dbrianwalton One minute before the stalling began, it appears the dispatch routine returned an "UNDEF" value instead of 0.
17:01 dbrianwalton never mind, it looks like that isn't uncommon
17:01 aubreyja I'm interested to know how you identified these function calls as the bottleneck? Did you put in extra debug stuff?
17:03 dbrianwalton I actually just turned on the Debug option in WeBWorK::Constants.
17:03 dbrianwalton The dispatch routine then recorded step-by-step updates of requests.
17:04 dbrianwalton When I learned of the stall and the time it occurred, I looked through the log file to see if there were any surprises.
17:04 aubreyja ah, ok. Just a sec - phone call
17:06 dbrianwalton Each request made it up to and including:  "[Wed Aug 22 16:04:03.820891 2012] WeBWorK::Authen::verify: BEGIN VERIFY " and suddenly stopped.
17:07 dbrianwalton New requests still made it into the queue, usually ending at the same point.
17:07 dbrianwalton Requests for a new login screen completed successfully.
17:11 dbrianwalton But I caught one log-in attempt by a user that showed a few extra lines:
17:11 dbrianwalton [Wed Aug 22 16:06:33.433169 2012] WeBWorK::Authen::verify_normal_user: sessionExists='1' keyMatches='' timestampValid=''
17:11 dbrianwalton [Wed Aug 22 16:06:33.433302 2012] WeBWorK::Authen::LDAP::checkPassword: LDAP module is doing the password checking.
17:15 aubreyja ok, back
17:15 dbrianwalton I added a few more lines of explanation.  (see above)
17:17 dbrianwalton So I think in "verify_normal_user", existing sessions calling Authen::check_session hit a stall, but new log-ins made it through check-session and got stuck a few steps later.
17:20 aubreyja ok, so let's see verify passes it to do_verify which passes it to get_credentials and then check_user…where is verify_normal_user called?
17:20 aubreyja ah, nevermind do_verify
17:22 aubreyja so, those extra lines you pasted sessionExists='1' keyMatches='' was empty? Does that seem right?
17:23 dbrianwalton Well, the script would have been acting like it was logged into a different course and then suddenly tries logging into a new course.
17:23 dbrianwalton I assumed that is why keyMatches would be ''
17:23 aubreyja ok
17:25 dbrianwalton I could have had more information, but I disabled the debug from get_credentials since it would have logged raw password information.
17:26 dbrianwalton So I don't know what happened in that routine.
17:26 dbrianwalton ...
17:27 dbrianwalton Anyway, I'm thinking I need to figure out how to get information about the database itself.
17:27 dbrianwalton I'm a little confused with the code for the database interface, so I'm not sure where I would add debugging information.
17:28 dbrianwalton Or is there an external way of tracking what's happening to the sql server?
17:28 aubreyja you can turn on the mysql log and log all of the raw sql queries
17:28 aubreyja could it be your ldap server isn't getting back to you quickly?
17:29 dbrianwalton If there is a file_connections or other limit, would the logging allow me to track that as we go as well?
17:29 aubreyja the number of connections you can watch in real time - there is a mysql monitoring tool - I forget what it's called though - comes with mysql
17:30 dbrianwalton ok.  Will look into that
17:30 aubreyja oh, just mysqladmin -u -p status, for example
17:30 dbrianwalton great!
17:31 aubreyja let us know what you find!
17:31 dbrianwalton will do
17:31 dbrianwalton Thanks for hearing me out.
17:32 aubreyja I've been trying to figure out authen stuff lately too actually, so going through it is good for me
17:44 dbrianwalton left #webwork
20:07 aubreyja mgage: HOw's 2.5.1 coming along?
20:20 Paul_Pearson joined #webwork
20:52 Paul_Pearson Hi Mike.  I have a low priority feature request: the ability to enable or disable DragMath via the Course Configuration -> PG problem display / answer checking page.  If there is currently another way to enable or disable DragMath, I'm unaware of it and would like to know how to do it.  Let me stress that this is very low priority.
20:53 aubreyja Hi Paul
20:55 Paul_Pearson Hi Jason.  Has school started for you yet?  We start on Tuesday.
21:04 aubreyja Yep, we started this week…I think the storm has passed already though
21:05 Paul_Pearson Do you have time to help me with git right now?  Or, is there another time?
21:05 aubreyja Sure - what's up?
21:05 Paul_Pearson Here's the problem:
21:05 Paul_Pearson Paul@PAVEL /c/webwork-git (master)
21:05 Paul_Pearson $ ls
21:05 Paul_Pearson pg  webwork2
21:05 Paul_Pearson Paul@PAVEL /c/webwork-git (master)
21:05 Paul_Pearson $ git clone http://github.com/openwebwork
21:05 Paul_Pearson Cloning into 'openwebwork'...
21:05 Paul_Pearson fatal: http://github.com/openwebwork/info/refs not found: did you run git update
21:05 Paul_Pearson -server-info on the server?
21:05 Paul_Pearson Paul@PAVEL /c/webwork-git (master)
21:05 Paul_Pearson $ git update-server-info
21:05 Paul_Pearson Paul@PAVEL /c/webwork-git (master)
21:05 Paul_Pearson $ git clone http://github.com/openwebwork
21:05 Paul_Pearson Cloning into 'openwebwork'...
21:05 mgage hi -- I'm back -- I've been talking to shivram.venkat
21:06 aubreyja ah, you need to do .../openwebwork/webwork2-dev
21:06 aubreyja .git
21:06 Paul_Pearson I'm trying to checkout / download the OPL.
21:06 mgage a contact from michael pearson who is interested in putting together a collection of prblems
21:06 Paul_Pearson OK, I'll try that right now.
21:07 aubreyja for the OPL it would be git clone https://github.com/openwebwork/webwork-open-problem-library.git
21:07 aubreyja mgage: great - problem collection for what?
21:08 aubreyja Also, Paul - you'll get OPL-update with that, and to configure it set the problem library version to 2.5 in the conf files (I don't know which conf file has the original setting)
21:08 mgage math, science -- includes digitizing problems in public domain from old textbooks.  -- he's been focusing on making such a collection searchable.  -- and he's got some good ideas about
21:09 aubreyja ah, neat
21:09 mgage I think he'll want to talk to you as well Jason
21:09 aubreyja oh, I remember him - I set up a course for him on test
21:10 mgage there might be some areas we can help each other.  -- yes -- that is the one -- he's played around with it some
21:10 Paul_Pearson @ jason: should the page http://webwork.maa.org/wiki/Category:Developers be changed to state
21:10 Paul_Pearson git clone https://github.com/openwebwork/webwork-open-problem-library.git
21:10 Paul_Pearson Also, should svn be removed from that wiki page altogether?
21:11 mgage particularly in discussions about how to organize the library he might have relevant ideas.  -- his project is slightly grander -- not just math and not just questions that are appropriate for electronic grading.
21:11 mgage 2.5.1 is coming along -- I think I have finally squashed at least some of the CSS bugs.
21:11 aubreyja yes, well, the main problem is that it's ambiguous, people need to get webwork2-dev, pg-dev and webwork-open-problem-library
21:12 aubreyja the git clone http…./ on that page is the beginning of each of the three required clones
21:12 aubreyja mgage: That sounds interesting - we should get djun involved too
21:12 mgage My local copy won't do Achievements -- but I think it is missing some files -- once I have it so that I can't break it immediately I'll push it up to a branch on openwebwork/webwork2-dev and let you guys help.  Should be in an hour or two
21:16 aubreyja ok - I'll be leaving to get my kids soon, then back on around 8
21:17 mgage ok -- I'll have something ready by 8 CDT --
21:18 aubreyja sounds good -
21:19 Paul_Pearson So, I've made changes to the pg file in
21:19 Paul_Pearson http://svn.webwork.maa.org/npl/trunk/NationalProblemLibrary/FortLewis/DiffEq/1-First-order/04-Linear-integrating-factor/KJ-2-3-12/
21:19 Paul_Pearson and I want to commit them.  What are the git commands for this?
21:20 aubreyja You made the change in a copy of the open problem library or a copy of the npl?
21:21 Paul_Pearson I made a git clone and then changed my local copy of the pg file.
21:21 Paul_Pearson the opl
21:21 aubreyja ah, I see - just a sec - phone call
21:22 Paul_Pearson (The NPL address above was just being used to view the problem source before I had the git clone.)
21:23 aubreyja ok, I guess there are two routes
21:24 aubreyja the standard git workflow would have you fork the OPL, push changes to your fork, and then issue pull requests to the openwebwork copy of the OPL
21:24 Paul_Pearson OK, so I've screwed up already.
21:24 aubreyja well, not necessarily - we actually haven't made any decisions about this I think
21:25 Paul_Pearson I can start over from scratch.  (The pg file edit was super easy.)
21:25 mgage there is a wiki page on github that has some of this -- it needs more work
21:25 mgage (a webwork wiki page -- search for github
21:26 Paul_Pearson If you have time, I just sent a google hangout request.  If you've got to run, we can do this later.
21:26 aubreyja The 'svn like' workflow would be just for us to give all former npl committers and future OPL commuiters  push rights on the openwebwork opl
21:27 mgage you can send it to me as well michaelegage@gmail.com
21:31 aubreyja ok - on my way out -
21:31 Paul_Pearson Mike: for future reference, do you prefer gmail or your rochester mail?
21:31 mgage rochester mail for mailing to me -- I use michealegage@gmail.com for google docs, google+ etc.
21:33 Paul_Pearson @ mike: I'll try to figure out git with Jason's help later (you're generally busy with other, perhaps more urgent, things).
21:33 mgage I also have another mail alias on google but that just helps my rochester address distribute the mail
21:34 mgage ok -- mostly it's just selecting the rihgt pieces ot get together for a release -- but it takes concentration.
22:27 aubreyja joined #webwork
22:27 aubreyja joined #webwork
23:23 djun joined #webwork

| Channels | #webwork index | Today | | Search | Google Search | Plain-Text | summary