Perl 6 - the future is here, just unevenly distributed

IRC log for #webwork, 2013-03-18

| Channels | #webwork index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:06 Paul_Pearson joined #webwork
00:39 mgage_ joined #webwork
01:20 Paul_Pearson joined #webwork
02:38 Paul_Pearson joined #webwork
03:10 mgage_ joined #webwork
03:20 Budd joined #webwork
04:59 Budd joined #webwork
05:32 Budd joined #webwork
10:39 Tim2 joined #webwork
11:58 mgage_ joined #webwork
13:01 goehle_ joined #webwork
13:14 Tim2 joined #webwork
13:16 goehle_ hey mgage
13:27 Tim2 joined #webwork
13:54 aubreyja joined #webwork
14:58 Budd joined #webwork
15:04 Paul_Pearson joined #webwork
15:18 yannick1 joined #webwork
16:07 yannick1 left #webwork
16:42 Budd joined #webwork
16:48 Paul_Pearson joined #webwork
17:25 darwyn joined #webwork
17:58 darwyn Hi Jason or Mike
18:08 mgage hi -- back
18:38 Paul_Pearson joined #webwork
19:06 darwyn left #webwork
19:25 Tim2 joined #webwork
20:09 goehle_ hey mgage
20:55 mgage goehle_: hi   I'm back again -- haven't been around for most of the day.
20:55 goehle_ hey
20:55 goehle_ I just wanted to ask you about the index stuff
20:55 mgage did you see Sam's note?
20:55 mgage (I included him because he wrote the original.)
20:56 goehle_ I did
20:56 goehle_ it looks like there is already support for indexing in the db code
20:57 goehle_ I seem to remember that there was a reason why I removed the key flag from the course/user/set ids in PastAnswer
20:57 goehle_ but I dont remember what it was
20:57 mgage It seems to me that you might be able to hack his first suggestion -- just add a specific line that adds an index for the four fields described -- test it in a branch version of yours and see if it works and does speed up search -- we can have compass test it on their classes and Robin can check on her's
20:57 goehle_ and it could have just been that I didn't know what it did
20:57 goehle_ it wont speed up mine
20:57 goehle_ since I don't have many rows in pastAnswer
20:58 goehle_ (my test machine is still on an older version)
20:58 mgage there is support for indexing but according to Sam they just added indexing for adjacent pairs of keys
20:58 mgage kind of arbitrary
20:58 goehle_ what do you mean adjacent pairs of keys?
20:58 mgage that was what Sam said in his email -- I haven't looked at the code closely yet.
20:59 mgage I have time now -- I had a bunch of things that had to get done the end of last week but they are done now
21:00 mgage from Sam:  There is code in in NewSQL/STD.pm (in
21:00 mgage _create_table_stmt) that creates indexes on all possible adjacent
21:00 mgage combinations of keyfields. It's a bit naive and will create way too many
21:00 mgage indexes.
21:01 goehle_ yeah
21:01 goehle_ I was just looking at that
21:02 goehle_ I think what he is saying is that it creates an index with all of the specified key columns
21:02 goehle_ when I look at _create_table_stmt
21:02 mgage I think that it's enough that you can test that a hack doesn't cause side effects -- then Robin and compass can pull your branch down temporarily and see if it speeds things up.
21:02 goehle_ I see it going through the keyfields
21:03 goehle_ and pushing them into the index components array
21:03 mgage I'm not too worried about temporary hacks because I think in the long term this whole section of code will be written for a new database.
21:03 goehle_ which it builds into the string to create the table
21:05 goehle_ so an easy fix
21:05 goehle_ is to add key=>1 to course_id, user_id, set_id, problem_id in PastAnswer.pm
21:05 goehle_ it will make a big index
21:05 goehle_ but thats ok, since we actually do query on all of those things
21:06 goehle_ unfortunatelly, this wont update existing tables
21:06 goehle_ it will just ensure that new tables have an index
21:06 mgage that seems to be what Sam suggests as his first proposal -- I need to look at _create_table_stmt() to see what it does exactly
21:07 goehle_ sure
21:10 goehle_ if you think that all of that code is going to be upgraded
21:10 mgage ok -- I think I understand what it does.  I suggest that we do exactly what you propose.  test it on your machine and see if it causes trouble -- such as slowing things down on small tables.
21:10 goehle_ sure
21:10 goehle_ I'll do that
21:10 mgage if it works we can use it for now.
21:10 goehle_ ok, what about existing tables?
21:12 mgage I have to think about how to upgrade those tables.  -- I've got a student now so I have to go.  One thing that might be less drastic would be to grap the $field_string at line 163 of Std.pm/_create_table_stmt  and figure out how to add one more index for the 4 keys we define above.  --that might be easier to upgrade
21:12 mgage be back after I talk to the student.
21:20 goehle_ kk
21:26 goehle_ I have to head out
21:27 goehle_ I"ll be back on in a few hours
21:53 Tim2 joined #webwork
22:55 goehle_ hey mgage
23:28 Tim2 joined #webwork
23:29 Paul_Pearson joined #webwork
23:49 mgage_ joined #webwork

| Channels | #webwork index | Today | | Search | Google Search | Plain-Text | summary