Camelia, the Perl 6 bug

IRC log for #parrot, 2012-04-15

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:28 dalek ParrotStore: 84feff6 | Whiteknight++ | mysql/pmc/mysqldata (2 files):
00:28 dalek ParrotStore: [MySQL] Add in a few more vtables so we can iterate over a table, and loop over row elements
00:28 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/84feff6b0e
00:28 dalek ParrotStore: 0605f32 | Whiteknight++ | mysql/pmc/mysqldata (2 files):
00:28 dalek ParrotStore: [MySQL] Add in a few interfaces for iteration. MySqlDataTable is its own iterator, so implement that interface. MySqlDataRow uses a HashIterator over the column names hash. In the future we might use something more custom, since rows should be able to be iterated by column name and by raw indexed value
00:28 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/0605f32569
00:28 dalek ParrotStore: 77caf91 | Whiteknight++ | .gitignore:
00:28 dalek ParrotStore: Hide the dynext directory in .gitignore
00:28 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/77caf911c3
00:28 whiteknight more or less, we have MySQL bindings on Parrot now
00:37 contingencyplan joined #parrot
00:47 contingencyplan joined #parrot
00:49 jsut joined #parrot
00:54 jsut_ joined #parrot
00:59 jsut joined #parrot
02:04 dalek ParrotStore: f6489a7 | Whiteknight++ | mysql/pmc/mysqld (3 files):
02:04 dalek ParrotStore: [MySql] Condense .query and .query_no_results into a single method that returns a table for a select, and returns an integer count for insert, update and delete for number of rows affected. Several other code cleanups.
02:04 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/f6489a7ee4
02:04 whiteknight left #parrot
02:51 plobsing joined #parrot
04:26 l3l1p joined #parrot
06:49 fperrad joined #parrot
07:10 fperrad joined #parrot
07:18 aloha joined #parrot
07:19 perlite joined #parrot
07:55 jevin joined #parrot
08:22 fperrad joined #parrot
08:25 dngor_ joined #parrot
08:36 _mayank joined #parrot
08:41 elmex joined #parrot
09:13 rich joined #parrot
09:51 dalek rakudo/nom: 64f70e8 | jnthn++ | src/Perl6/Grammar.pm:
09:51 dalek rakudo/nom: Fix a thinko.
09:51 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/64f70e8862
09:59 dalek rakudo/nom: 0227a99 | jnthn++ | src/Perl6/World.pm:
09:59 dalek rakudo/nom: Remove a workaround I couldn't figure out why was needed; turns out it was just the buggy call from token subset.
09:59 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/0227a99e45
10:14 dalek rakudo/nom: 1639834 | jnthn++ | src/Perl6/Grammar.pm:
10:14 dalek rakudo/nom: Eliminate the last remaining use of parse_name in the Grammar.
10:14 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/1639834cc2
10:14 dalek rakudo/nom: e07d6e4 | jnthn++ | src/Perl6/Actions.pm:
10:14 dalek rakudo/nom: Toss an unused action method that mostly dates back to the pre-nom days.
10:14 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/e07d6e4411
10:14 dalek rakudo/nom: a803152 | jnthn++ | .gitignore:
10:14 dalek rakudo/nom: Add .exp and .lib to .gitignore.
10:14 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/a80315280a
10:19 dalek Heuristic branch merge: pushed 630 commits to parrot/ayardley/documentation_revisions by ayardley
10:30 dalek rakudo/nom: a648357 | jnthn++ | src/Perl6/Actions.pm:
10:30 dalek rakudo/nom: Eliminate the majority of the remaining uses of parse_name; just one left in Actions now.
10:30 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/a64835723c
10:35 dalek parrot: c245d17 | alvis++ | README:
10:35 dalek parrot: There is no 'NEWS' ([t]here's the truth of the signal, and then there's the puppet theatre .... :).
10:35 dalek parrot: review: https://github.com/parrot/parrot/commit/c245d17151
10:38 lucian joined #parrot
10:42 whiteknight joined #parrot
10:42 whiteknight good morning, #parrot
10:42 nine Good morning, whiteknight
10:51 tadzik hello #parrot
10:51 nine Hi tadzik
10:52 JimmyZ joined #parrot
11:05 dalek rakudo/nom: 8f94276 | jnthn++ | src/Perl6/ (2 files):
11:05 dalek rakudo/nom: Eliminate last remaining parse_name call, whihc was in compilation of variables. Another tiny speedup from avoiding the reparsing.
11:05 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/8f9427678c
11:05 dalek rakudo/nom: 1421472 | jnthn++ | src/Perl6/Grammar.pm:
11:05 dalek rakudo/nom: Toss parse_name.
11:05 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/1421472184
11:07 dalek parrot: 446acba | alvis++ | docs/project/release_manager_guide.pod:
11:07 dalek parrot: Added myself as a Release Manager for the June release.
11:07 dalek parrot: review: https://github.com/parrot/parrot/commit/446acbaed9
11:10 nine whiteknight: another data point. Sharing the interp lock between all threads, thereby serializing access to all GCs does not help at all, indicating that the problem is not concurrency related.
11:10 whiteknight nine: You said you found the place where the PMC is initially allocated?
11:12 nine whiteknight: no. It's pretty much random PMCs. Minor variations in test data or parrot's code change which PMC is found. I've seen the "sayer" sub, callcontexts, an Integer and even the other thread's interp.
11:13 * JimmyZ last heard 6model will be in parrot core before parrot 4.0, but ....
11:14 nine whiteknight: from all I've seen I'm pretty sure that somehow threads share some parts of the pointer arrays but all attempts to produce evidence for this failed so far.
11:15 whiteknight in src/gc/gc_gms:gc_gms_allocate_pmc_header, why do we lock after gc_gms_maybe_mark_and_sweep?
11:15 whiteknight It seems like we should lock before that, right?
11:16 nine whiteknight: gc_gms_maybe_mark_and_sweep does the locking itself
11:16 nine whiteknight: IIRC I did it this way because mark_and_sweep can be called from outside the GC itself.
11:17 whiteknight ok
11:19 nine In the long run I want to get rid of these locks anyway...
11:31 schmooster joined #parrot
11:40 nine whiteknight: disabling the reuse of removed cells in pointer arrays makes the problem disappear!
11:40 whiteknight that....seems weird
11:40 whiteknight so it's a problem with pointer arrays?
11:42 nine whiteknight: well it would fit with everything I've seen so far. Although I'd like to be cautious here. I've already wrongly suspected pointer arrays once.
11:44 nine whiteknight: maybe it's not the pointer arrays themselves but some call to Parrot_pa_remove which is wrong. So far we've concentrated only on Parrot_pa_insert
11:44 whiteknight ah, that's a good point
12:08 nine whiteknight: this is it!
12:08 nine whiteknight: write barriers!
12:11 nine whiteknight: doing a PARROT_GC_WRITE_BARRIER on a shared PMC (which is owned by the main thread, not the current one) causes it to be removed from another interp's objects list and put on the wrong interp's dirty list
12:13 nine whiteknight: and PARROT_ASSERT_INTERP didn't catch this because it explicitely ignores shared PMCs since Parrot_gc_mark_PMC_alive knows how to hande those.
12:15 whiteknight nine: I thought write barrier just set a flag
12:16 nine whiteknight: no, in addition to setting the flag it does a Parrot_pa_remove(interp, self->objects[gen], item->ptr); followed by item->ptr = Parrot_pa_insert(interp, self->dirty_list, item);
12:17 whiteknight oh man
12:17 nine whiteknight: I think the flag is just to not repeat this for already dirtied objects
12:18 whiteknight I might have never thought to look there
12:18 nine I added a PARROT_ASSERT(Parrot_gc_pool_is_owned(interp, self->pmc_allocator, item)); to every place where Parrot_pa_remove is called. The stacktrace told the rest of the story
12:18 whiteknight very clever
12:19 whiteknight I bet performance tanked with that assertion active
12:19 nine Everything makes so much sense now. It takes some time for the bug to appear because it only happens with objects in older generations
12:19 nine oh yes it did :)
12:19 bacek joined #parrot
12:20 nine bacek: I just found the source of my GC/threading problem :)
12:21 whiteknight nine: commit a fix and I'll test it here too
12:21 nine The question remains: how can I actually fix it?
12:21 whiteknight oh yes, I guess that's the first thing to do
12:22 whiteknight Parrot_pa_remove(pmc->orig_interp, ...)
12:22 nine Using orig_interp for anything but assertions can only be the very very last resort
12:22 whiteknight if (!PObj_is_shared_TEST(pmc)) { ... mark dirty ... }
12:24 nine But then the object wouldn't be marked dirty even though it should be. May even be that it fixes the test program, but I'd bet that extremely hard to debug problems would still remain...
12:27 nine Well, we don't really need orig_interp since shared PMCs can only belong to the main interp. Storing a pointer to the main interp in a global variable or the parent_interpreter or something like that would suffice.
12:27 whiteknight We do store an "emergency interp" somewhere for generating backtraces
12:28 whiteknight If we populate that with the main interp reference, we can probably use that
12:30 whiteknight src/interp/api.c static Interp* emergency_interp = NULL;
12:30 whiteknight We can add an accessor for that, and make sure it gets set to the main interp only
12:31 nine I don't really like the name though. It sounds very different from what I'd be using it for.
12:31 whiteknight we can rename it
12:31 whiteknight it was only put in for backtraces, in cases where things blow up and we don't have an interp reference handy
12:32 whiteknight but we can rename it to "main interp", and use it for other things
12:32 whiteknight or you can add your own variable too.
12:38 nine Finding a way to remove the lock from gc_gms_write_barrier will be a very interesting exercise...
12:43 PacoAir joined #parrot
12:43 whiteknight "interesting" is a fun word for it
12:46 nine well I already got an idea :) I could add a second dirty_list to the GC where access is protected by a lock same as I did for the block_gc_mark counter. All accesses by other threads have to go to the protected counter/list while the owning thread may use the lock free original counter/list. The Parrot_pa_remove(interp, self->objects[gen], item->ptr); would have to go though. Maybe I can move this to mark_and_sweep where I hold the locks
12:46 nine This way a single threaded program only has to care about locking during the GC run which is done sparsely anyway
12:48 nine alloc_test.pir is running stable for five minutes now with constant memory usage :) No sign of stopping
12:48 dalek rakudo/nom: 99e7750 | moritz++ | src/ (3 files):
12:48 dalek rakudo/nom: try to silence some compiler warnings (RT #112310, #112308)
12:48 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/99e77502ad
12:48 whiteknight nine++
12:56 benabik nine++
13:23 dalek parrot/threads: ab1d2d4 | nine++ | / (3 files):
13:23 dalek parrot/threads: Perform GC write barriers for shared PMCs only on the main interp
13:23 dalek parrot/threads:
13:23 dalek parrot/threads: Performing write barriers on a different interp than the one owning a
13:23 dalek parrot/threads: PMC leads to corruption of the GC's objects and dirty lists since the
13:23 dalek parrot/threads: write barrier removes the PMC from the objects list of the generation
13:23 dalek parrot/threads: the PMC belongs to. This causes the next_free pointer of the main
13:23 kid51 joined #parrot
13:23 dalek parrot/threads: thread's GC's objects list to point to a place in the child interp's
13:23 dalek parrot/threads: objects list.
13:23 dalek parrot/threads:
13:23 dalek parrot/threads: Fixed by checking the shared flag and delegating the call to the
13:23 dalek parrot/threads: main_interp if the current interp is a child.
13:23 dalek parrot/threads: review: https://github.com/parrot/parrot/commit/ab1d2d4451
13:23 dalek parrot/threads: 145ba94 | nine++ | src/thread.c:
13:23 dalek parrot/threads: MUTEX_INIT the interp_lock
13:23 dalek parrot/threads: review: https://github.com/parrot/parrot/commit/145ba94e80
13:26 mdupont joined #parrot
13:26 mdupont_ joined #parrot
13:38 dalek nqp/cstruct-work: 7ff1038 | (Arne Skjærholt)++ | src/6model/reprs/CStruct.c:
13:38 dalek nqp/cstruct-work: General cleanup.
13:38 dalek nqp/cstruct-work: review: https://github.com/perl6/nqp/commit/7ff1038057
13:38 dalek nqp/cstruct-work: f6a394e | (Arne Skjærholt)++ | src/6model/reprs/CStruct. (2 files):
13:38 dalek nqp/cstruct-work: Implement getting reference type members in CStruct.
13:38 dalek nqp/cstruct-work: review: https://github.com/perl6/nqp/commit/f6a394e1ff
13:57 nine threads_playground seems to be very, very stable now :) It just runs and runs...
13:58 whiteknight awesome
13:58 benabik joined #parrot
13:59 whiteknight if the single-threaded case continues to work, passes all tests and doesn't have a big performance degredation, and if we can add windows support, I think we can start talking merger
14:00 dalek rakudo/nom: f524138 | jnthn++ | src/core/Str.pm:
14:00 dalek rakudo/nom: A few little optimizations to some Str builtins. Found by looking at profile of code from masak++; seems to shave a few percent off it.
14:00 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/f524138d1d
14:07 JimmyZ_ joined #parrot
14:15 dalek rakudo/nom: de1eaf5 | moritz++ | src/core/Str.pm:
14:15 dalek rakudo/nom: remove redundant conditional in Str.match-list
14:15 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/de1eaf515e
14:16 dalek ParrotStore: e0a7dc1 | Whiteknight++ | Makefile:
14:16 dalek ParrotStore: Add mysql to the make clean target
14:16 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/e0a7dc1b1e
14:16 dalek ParrotStore: b1d907e | Whiteknight++ | Makefile:
14:16 dalek ParrotStore: Re-add mongodb to the build. Fix build for mongodb and mysql so the shared libs always get built.
14:16 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/b1d907e7df
14:16 dalek ParrotStore: a09b843 | Whiteknight++ | / (2 files):
14:16 dalek ParrotStore: Make sure we include mysql in the install. Update readme with information about the three current projects, how to get prereqs, and how to build/install them
14:16 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/a09b84308f
14:16 dalek ParrotStore: 32146e0 | Whiteknight++ | / (4 files):
14:16 dalek ParrotStore: [MongoDB] Several fixes so we can create a simple BsonDocument and save it successfully to the DB
14:16 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/32146e0395
14:16 dalek ParrotStore: 387bbb5 | Whiteknight++ | mongodb/pmc/mongodbcontext.pmc:
14:16 dalek ParrotStore: [MongoDB] Add a get_bool to determine if the db is connected. Add a disconnect method to disconnect
14:16 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/387bbb5384
14:16 benabik What is this ParrotStore?
14:16 nbrown joined #parrot
14:16 benabik I mean, can I buy an African Blue?
14:18 benabik Hm.  Three disconnected database drivers?
14:20 * JimmyZ wonders what difference is between ParrotStore and libffi which can conect mysql
14:21 benabik JimmyZ: It seems to present MySQL as actual objects instead of opaque pointers you have to pass to FFI functions.
14:22 JimmyZ benabik: as a user, I'm not aware of it,
14:23 tadzik as a user of what?
14:23 JimmyZ who connect mysql through parrot
14:24 nine Well except for some codingstd fails and expected task fails everything looks fine :)
14:24 nine off to a Rhapsody concert now :)
14:25 tadzik oh, concerts
14:26 benabik JimmyZ: I think ParrotStore is new.  First commit appears to be Apr 12, 2012
14:28 JimmyZ benabik: yeah
14:35 dalek rakudo/nom: d6cd1e2 | jnthn++ | src/core/MapIter.pm:
14:35 dalek rakudo/nom: When I added FIRST/NEXT/LAST, it was idiomatic but not quite so fast. This makes it faster. Another little bit of masak++'s program.
14:35 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/d6cd1e2bd1
14:47 preflex_ joined #parrot
15:14 jsut_ joined #parrot
15:19 jsut joined #parrot
15:28 jsut_ joined #parrot
15:33 jsut joined #parrot
15:38 jsut_ joined #parrot
15:43 dalek nqp: 3a23406 | jnthn++ | src/6model/reprs/CStruct.c:
15:43 dalek nqp: Don't try to allocate 0 bytes in CStruct REPR.
15:43 dalek nqp: review: https://github.com/perl6/nqp/commit/3a23406005
15:43 jsut joined #parrot
15:55 jashwanth joined #parrot
16:00 whiteknight benabik: Yeah, it's just a bunch of disconnected DB bindings
16:01 whiteknight I might build up from there, but I wanted to add bindings and I didn't want to create three separate repos
16:11 plobsing joined #parrot
16:29 l3l1p joined #parrot
16:32 alvis joined #parrot
16:52 jsut_ joined #parrot
17:04 l3l1p joined #parrot
17:08 jsut joined #parrot
17:35 benabik joined #parrot
17:44 dmalcolm joined #parrot
17:55 dalek rakudo/nom: 9df76e9 | jnthn++ | src/ (3 files):
17:55 dalek rakudo/nom: Implement various cases of binding into pseudo-packages.
17:55 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/9df76e9363
17:55 dalek rakudo/nom: 442db9f | jnthn++ | t/spectest.data:
17:55 dalek rakudo/nom: Turn on S02-names/pseudo.t.
17:55 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/442db9f058
18:16 lucian joined #parrot
18:33 dalek ParrotStore: 1169764 | Whiteknight++ | / (3 files):
18:33 dalek ParrotStore: [MySQL] Read out the rows from the result set eagerly, MySQL doesn't seem to intend for the result objects to be long-lived
18:33 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/1169764916
18:33 dalek ParrotStore: d3c4d95 | Whiteknight++ | mysql/pmc/mysqldbcontext.pmc:
18:33 dalek ParrotStore: [MySQL] Support multiple results from multi-statement queries or from stored procs. A few other small code cleanups
18:33 dalek ParrotStore: review: https://github.com/Whiteknight​/ParrotStore/commit/d3c4d95473
18:36 brrt joined #parrot
19:43 Coke ugh, I hate the mentor GSOC mailing lists.
19:44 moritz I mostly just ignore it these days
19:44 moritz except root-level mails from Carol
19:44 benabik The student list isn't much better.
19:46 Coke IWBNI people speaking on behalf of google were marked as "moderators". I think that would help slightly.
20:04 lucian joined #parrot
21:04 mdupont joined #parrot
21:20 perlite_ joined #parrot
21:40 aloha (parrot/parrot) Issues opened : 756 (Parrot Build Fails at Build Test) by gtoast : https://github.com/parrot/parrot/issues/756
21:43 kid51 joined #parrot
22:06 gtoast joined #parrot
22:07 gtoast Seems like the tutorial here(http://www.parrotblog.org/2008​/03/targeting-parrot-vm.html) is a bit out of date. Is there an updated version?
22:07 gtoast As is the auto-generated tutorial.
22:09 kid51 joined #parrot
22:17 nbrown joined #parrot
22:42 contingencyplan joined #parrot
23:03 kid51 joined #parrot
23:30 davidfetter joined #parrot
23:54 gtoast joined #parrot

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced