Camelia, the Perl 6 bug

IRC log for #parrot, 2012-07-02

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 pmichaud ooc, would there be a way to use the mark algorithm to find all of the PMCs referenced from a given PMC?
00:01 whiteknight pmichaud: I don't *think* so right now. But I suspect such a thing could be added
00:01 whiteknight Actually.... Maybe not
00:02 whiteknight I'll have to look at it
00:04 dukeleto pmichaud: i would also suggest sending an email to parrot-dev and asking for input from bacek++ and nwellnhof++, who know the gc internals best
00:05 dukeleto pmichaud: they don't seem to be on IRC very often, but they usually chime in on parrot-dev when they are needed
00:06 pmichaud dukeleto: will do.
00:06 pmichaud 'sweep 1' helps a bit, though; it helps to distinguish what PMCs are truly "alive" and which are just hanging around waiting to be collected
00:12 whiteknight I wonder if the profiling core could be augmented to show allocations and deallocations through valgrind
00:17 dukeleto whiteknight: sounds useful
00:48 dalek Rosella: 7a79016 | Whiteknight++ | src/date/Date (2 files):
00:48 dalek Rosella: [Date] Add some new ways to get shorter weekday and month names, for formatting
00:48 dalek Rosella: review: https://github.com/Whiteknig​ht/Rosella/commit/7a7901643b
00:48 dalek Rosella: 3de1543 | Whiteknight++ | src/unstable/feed/ (2 files):
00:48 dalek Rosella: [Feed] Several fixes so we can write a basic RSS feed
00:48 dalek Rosella: review: https://github.com/Whiteknig​ht/Rosella/commit/3de1543683
00:48 dalek Rosella: cbc78ca | Whiteknight++ | src/unstable/feed/Writer.winxed:
00:48 dalek Rosella: [Feed] Add version='2.0' to the <rss> tag.
00:48 dalek Rosella: review: https://github.com/Whiteknig​ht/Rosella/commit/cbc78ca5ef
00:49 Coke I literally meant 'ulimit', not an option to parrot to limit memory.
00:51 Coke the problem with calling sweep is that, if it's anything like java, you end up forcing an expensive option more often than you need to.
00:51 kid51 joined #parrot
00:51 sorear pmichaud: you want ulimit -v KILOBYTES
00:55 whiteknight Coke: Yes, you don't want to force GC too early in terms of performance, but it's necessary in terms of analytics
00:55 whiteknight and if it's an option you can turn on later, that's fine
02:06 aloha joined #parrot
02:39 bacek_at_work joined #parrot
02:39 brambles joined #parrot
02:40 brambles_ joined #parrot
04:07 davidfetter joined #parrot
04:42 japhb joined #parrot
06:02 dalek rakudo/nom: 16db643 | pmichaud++ | src/core/Str.pm:
06:02 dalek rakudo/nom: Add fix for Str.perl to be more robust when ICU isn't present (RT #113886).
06:02 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/16db64382f
07:02 fperrad joined #parrot
07:11 brrt joined #parrot
07:47 Khisanth joined #parrot
08:26 lucian joined #parrot
08:27 Liz joined #parrot
08:28 preflex joined #parrot
10:53 Patterner joined #parrot
11:33 schm00ster joined #parrot
11:53 kid51 joined #parrot
13:01 PacoAir joined #parrot
13:08 whiteknight joined #parrot
13:09 whiteknight good morning, #parrot
13:11 brrt good afternoon
13:30 brrt oh, whiteknight, i have some evidence that the embedding api doesn't properly catch all errors
13:30 brrt i.e.
13:30 brrt it sometimes causes my web server to exit()
13:30 brrt which is a /huge/ pain
13:30 brrt and wrong, and evil, etc
13:39 bluescreen joined #parrot
13:49 dalek nqp: f16f2c5 | jnthn++ | src/how/NQPModuleHOW.pm:
13:49 dalek nqp: Stop nqp::can(EXPORTHOW, 'foo') from exploding by adding some missing bits to NQPModuleHOW.
13:49 dalek nqp: review: https://github.com/perl6/nqp/commit/f16f2c591c
13:59 whiteknight brrt: Blah, sounds wonderful.
13:59 whiteknight brrt: Do you have an example?
14:00 whiteknight brrt: There are some errors in Parrot's core, like panics and core dumps and those things that do force an exit() call.
14:00 whiteknight I've tried to minimize those kinds of things
14:00 brrt no, a regular exception
14:00 brrt i'm going to try and recreate it outside of apache
14:00 whiteknight okay, that's extremely worrisome
14:00 brrt but it has been hard
14:01 brrt it is
14:01 whiteknight Okay, pass any information to me as soon as you get it. We don't want that kind of creepy behavior happening
14:02 whiteknight https://github.com/parrot/parrot/blob​/master/src/embed/embed_private.h#L17
14:03 whiteknight For your records, the embed API sets up a jmpbuf in the interp. When we throw an exception, if there are no handlers, the interp jumps to that jmpbuf
14:03 brrt oh.. wait
14:04 brrt thats interesting
14:04 brrt what happens if the jmpbuf cannot be set?
14:04 whiteknight It may be possible that the interp->api_jmp_buf field gets corrupted somewhere, or maybe if we throw an exception from a child interp that field might not have been initialized
14:04 whiteknight brrt: gnashing of teeth, weeping of women, the end of days
14:04 brrt hahaha
14:05 whiteknight If the jmpbuf is null, I think we give up and exit
14:05 brrt lets see if i can reproduce this with the minimal amount of effort
14:05 brrt exit with.. which response code?
14:05 whiteknight whichever we've passed to the routine
14:05 brrt well, the program exits with one
14:05 whiteknight https://github.com/parrot/parr​ot/blob/master/src/exit.c#L70
14:06 whiteknight Parrot_x_jump_out is what we call to exit the interp and fly back to that jmpbuf
14:06 brrt i trust that function
14:06 brrt .. .i assume that jmpbuf becomes null somewhere
14:06 whiteknight So whatever status value is passed to that is what's used. That value is extracted from the exception object, and I think it defaults to 1
14:07 whiteknight do a search for "Parrot_x_jump_out(interp, 1)"
14:07 whiteknight actually, that probably wont find anything
14:08 whiteknight Can you attach a debugger to apache and set breakpoints?
14:09 brrt i can very easily attach a debugger to apache,
14:10 brrt that is called 'make debug' ;-)
14:10 brrt but breakpoints, i do not know how
14:11 brrt oh, break
14:13 whiteknight "break Parrot_x_jump_out"
14:13 whiteknight Just a warning, gdb doesn't seem to handle longjmp very well. If you try to step through that, you'll get very confused about what's happening next
14:13 whiteknight but breaking until that point and checking the backtrace will yeild lots of good information
14:14 brrt ok, maybe i can see if you can get the same results
14:14 brrt do you have a recent checkout?
14:14 brrt or at least, my last?
14:15 brrt hasn't updated since friday iirc
14:15 whiteknight I'm probably a few days behind that
14:15 whiteknight I'll update and test tonight when I get home, if you want
14:15 brrt yes, i'd like that
14:15 whiteknight sure thing
14:16 brrt ... hmm, it doesn't really seem to work
14:16 brrt (breakpointing)
14:16 brrt thats annoying
14:16 brrt it seems to have not defined the functions on start
14:17 whiteknight yeah, mod_parrot is probably loaded dynamically
14:17 whiteknight double-check your spelling and just continue
14:18 brrt apache is quite resilient against debugging as far as i can see
14:18 brrt but, i can probably recreate the enviroment
14:20 brrt .. i'm going to try it out at home
14:20 brrt i can get the 'crash' to work easily, but i can't get any breakpoints
14:25 whiteknight okay, that's wonderful
14:25 brrt maybe exit
14:25 brrt maybe exit() can be breakpointed
14:25 brrt who knows
14:26 whiteknight maybe
14:27 brrt nope, can't be set, not here at least
14:27 whiteknight blah
14:56 whiteknight_ joined #parrot
14:59 alester joined #parrot
14:59 pmichaud https://github.com/parrot/parrot/issues/795
15:03 atrodo pmichaud: that seems useful...
15:07 whiteknight pmichaud: I
15:07 whiteknight 'll look at it tonight
15:09 dmalcolm joined #parrot
15:13 whiteknight I *suspect* the threshold calculation to determine the amount of memory to use before running a GC sweep doesn't take ulimit into account
15:14 whiteknight so it thinks it has more room to grow, attempts an allocation, the allocation fails and it panics
15:14 pmichaud the corollary to the above report is that any Parrot program of any size will ultimately use at least 2% of the available memory, whether it actually needs that much memory or not.  On my system, 450000 sub calls is sufficient to cause the process to allocate 180MB of memory
15:14 whiteknight right, that's the dynamic threshold at work
15:14 pmichaud whiteknight: I would think that if an allocation fails, though, that it would at least try a GC sweep before panicking.
15:15 whiteknight pmichaud: yes. It should
15:15 whiteknight and, after I'm done with it, it will
15:16 pmichaud ooc, is it really a "dynamic" threshold if it's not responding to actual memory constraints?
15:17 pmichaud seems to me that the threshold is more of an "initial" threshold than a dynamic one.
15:17 pmichaud I guess it just depends on what one considers "dynamic".  :-)
15:18 pmichaud afk for a while
15:31 contingencyplan joined #parrot
15:33 Coke pmichaud++, nice ticket.
15:37 jashwanth joined #parrot
16:02 whiteknight the "dynamic" part of the threshold has to do with the total amount of memory. I don't know if parrot can query the current ulimit
16:02 whiteknight I have to look at it
16:15 dalek nqp/toqast: e5cccaa | jnthn++ | src/QAST/Compiler.nqp:
16:15 dalek nqp/toqast: Provide a way to handle loadlibs.
16:15 dalek nqp/toqast: review: https://github.com/perl6/nqp/commit/e5cccaacdb
16:19 dalek rakudo/toqast: a9770e0 | jnthn++ | src/QPerl6/Actions.pm:
16:19 dalek rakudo/toqast: Translate GLOBAL installation (still something vm-specific for now).
16:19 dalek rakudo/toqast: review: https://github.com/rakudo/rakudo/commit/a9770e05ab
16:19 dalek rakudo/toqast: 6e0b080 | jnthn++ | src/QPerl6/World.pm:
16:19 dalek rakudo/toqast: Some QAST translation of setting loading; still will need some generalizing by VM.
16:19 dalek rakudo/toqast: review: https://github.com/rakudo/rakudo/commit/6e0b080c2a
16:19 dalek rakudo/toqast: f62e3e3 | jnthn++ | src/QPerl6/World.pm:
16:19 dalek rakudo/toqast: Update loadlibs handling to QAST::VM approach.
16:19 dalek rakudo/toqast: review: https://github.com/rakudo/rakudo/commit/f62e3e3e5b
16:26 dalek nqp/toqast: 2e6ad4a | jnthn++ | src/HLL/Compiler.pm:
16:26 dalek nqp/toqast: Make HLL::Compiler more aware of QAST.
16:26 dalek nqp/toqast: review: https://github.com/perl6/nqp/commit/2e6ad4aa16
16:43 Liz joined #parrot
16:45 dalek nqp/toqast: a39b82e | jnthn++ | src/QAST/Compiler.nqp:
16:45 dalek nqp/toqast: Improve error reporting when we find a PAST node inside a QAST tree.
16:45 dalek nqp/toqast: review: https://github.com/perl6/nqp/commit/a39b82e61d
16:46 Liz joined #parrot
16:54 tuxit joined #parrot
16:55 Liz joined #parrot
16:58 LizM joined #parrot
17:08 Coke how to watch a ticket in github?
17:22 whiteknight no idea.
17:24 lucian joined #parrot
17:49 rich joined #parrot
17:57 dukeleto joined #parrot
17:58 dukeleto jashwanth: i am around if you need some help, but I will be on a phone call in a few minutes
17:58 woolfy joined #parrot
19:01 brrt joined #parrot
19:01 brrt ping whiteknight
19:01 whiteknight pong
19:02 plobsing joined #parrot
19:02 brrt so, at home, gdb works
19:03 brrt and whats more
19:03 brrt api_jmp_buf is very much zero
19:03 whiteknight okay, do you have the backtrace?
19:04 brrt ehm, i guess
19:04 whiteknight if you can gist that, I'll take a look at it
19:05 brrt will do
19:10 brrt .... it doesn't crash now
19:11 brrt oh, this is very interesting
19:12 brrt i have a separate case which does not crash
19:13 brrt very interesting indeed
19:15 brrt https://gist.github.com/3035045
19:15 brrt is the gist
19:18 brrt ... i just can't get it done outside of apache
19:18 rich left #parrot
19:18 brrt extermely similar code in all respects, doesn't crash
19:18 brrt wait
19:19 brrt nope, doesn't happen
19:21 brrt1 joined #parrot
19:24 whiteknight hmm
19:24 whiteknight That's extremely weird
19:25 brrt1 joined #parrot
19:26 whiteknight it definitely should not be null there
19:27 brrt2 joined #parrot
19:27 brrt2 my connection is a bit flaky today
19:28 brrt joined #parrot
19:28 whiteknight I see that
19:28 brrt :-)
19:28 brrt on my mobile phone now
19:28 brrt much more reliable
19:30 whiteknight if it only happens in apache, I wonder if that means apache does something weird with setjmp
19:30 whiteknight actually, that doesn't make any sense either
19:30 brrt ioh, but hilariously, it doesn't in all cases
19:31 whiteknight awesome
19:31 whiteknight I'll have to step through it at home, to try and figure out when that field becomes null
19:32 brrt ok, its easy to reach
19:32 birdwindupbird joined #parrot
19:32 brrt basically,  edit echo.winxed to throw an exception at the last line
19:32 whiteknight ok
19:33 whiteknight Can you write me up all the details as an email?
19:33 brrt will do
19:33 whiteknight that way I wont need to ask a bunch of questions while you're sleeping
19:35 brrt ... could it have something to do with dlfunc?
19:35 brrt no, not likely
19:36 brrt .. or maybe
19:36 whiteknight you're very decisive
19:39 brrt joined #parrot
19:39 brrt i am, am i not? :-)
19:39 dukeleto joined #parrot
19:49 dukeleto joined #parrot
19:56 brrt it does not depend on nci
19:56 brrt full report per e-mail
20:02 brrt well, i have an answer, i think
20:03 brrt it depends on the case where an NCI-called function (so, from within the interpreter) calls Parrot_api_* functions
20:03 brrt in my case
20:03 brrt to make a hash
20:03 brrt which causes the interp->api_jmp_buf to be zeroed
20:05 brrt which does not meet my personal definition of either awesome or correct
20:05 dukeleto brrt: indeed. we call that LTA around here: "Less Than Awesome"
20:06 brrt :-)
20:07 brrt thats a good term for this
20:09 whiteknight oh yeah, Parrot_api_* are not reentrant
20:09 whiteknight you can't call Parrot_api_* from NCI
20:12 brrt .. o
20:12 brrt well
20:12 brrt that is LTA
20:12 brrt or, challenging
20:12 brrt however you want to put it
20:13 brrt or, a bit of a setback
20:14 brrt hmm... i'll have to think about this
20:16 whiteknight you can use NCI to call the functions that the api routines call
20:17 whiteknight What are you trying to do with that?
20:17 brrt i create a hash
20:18 brrt basically, i create a hash, then iterate over the apache-specific table structure, and fill the hash with that data
20:18 whiteknight okay
20:19 brrt that hash is then returned to the 'loader'
20:19 whiteknight look in src/hash.c for routines to work with hashes
20:19 brrt i.e., the interpreter
20:19 whiteknight or the hash PMC
20:20 brrt i guess i should try that again
20:20 brrt wasn't the greatest of successes tbh
20:20 whiteknight ok
20:20 brrt lots of header loading issues
20:20 brrt but yeah, i see where the api jump buffer is zeroed
20:21 brrt its in EMBED_API_CALLIN
20:21 whiteknight yeah
20:22 * brrt could conceivably use a variable to hold the old jump buffer?
20:22 brrt or is that a bad idea for reasons i do not know?
20:22 whiteknight We can probably set it up to use a linked list to make it reentrant
20:22 whiteknight I'll look at it
20:22 brrt the other strategy is (still) extening parrot
20:23 brrt creating a pmc to 'wrap' the apache data types
20:23 brrt or use StructView
20:23 brrt well, thanks anyway :-) this cleared up a lot
20:23 whiteknight no problem
20:24 whiteknight I'm wrapping up now. I'll look at some stuff later tonight
20:25 brrt bye
20:25 * brrt is leaving as well &
20:25 brrt left #parrot
20:44 brambles joined #parrot
21:03 woolfy joined #parrot
21:24 LizM joined #parrot
21:25 LizM joined #parrot
21:25 woolfy joined #parrot
21:31 woolfy left #parrot
21:46 LizM joined #parrot
21:47 woolfy joined #parrot
22:04 dalek parrot/chromatic/early_pmc_gc: 2cd0e7e | chromatic++ | / (2 files):
22:04 dalek parrot/chromatic/early_pmc_gc: Added early GC PMC reclamation to GC GMS.
22:04 dalek parrot/chromatic/early_pmc_gc:
22:04 dalek parrot/chromatic/early_pmc_gc: This makes the "sweep 0" opcode reclaim all PMCs in the youngest generation
22:04 dalek parrot/chromatic/early_pmc_gc: which have their early GC flag set. Set this with the "needs_destroy" opcode.
22:04 dalek parrot/chromatic/early_pmc_gc:
22:04 dalek parrot/chromatic/early_pmc_gc: If you don't know what you're doing, you can cause great harm. If you do know
22:04 dalek parrot/chromatic/early_pmc_gc: what you're doing, you can force a cheap GC run every few thousand PMC
22:05 dalek parrot/chromatic/early_pmc_gc: allocations to reclaim PMCs you know won't be used ever again without running a
22:05 dalek parrot/chromatic/early_pmc_gc: full mark and sweep or waiting to bump up against a memory limit.
22:05 dalek parrot/chromatic/early_pmc_gc:
22:05 dalek parrot/chromatic/early_pmc_gc: This comes with the appropriate warning: YOU are performing manual escape
22:05 dalek parrot/chromatic/early_pmc_gc: analysis. If you get that wrong, you'd better have a spectacular test suite to
22:05 dalek parrot/chromatic/early_pmc_gc: demonstrate that you have it wrong.
22:05 dalek parrot/chromatic/early_pmc_gc: review: https://github.com/parrot/parrot/commit/2cd0e7e931
22:05 dalek parrot/chromatic/early_pmc_gc: 7ec2caf | chromatic++ | src/gc/gc_gms.c:
22:05 dalek parrot/chromatic/early_pmc_gc: Ran headerizer on new lazy GC code.
22:05 dalek parrot/chromatic/early_pmc_gc: review: https://github.com/parrot/parrot/commit/7ec2cafe38
22:05 dalek parrot/chromatic/early_pmc_gc: 8047502 | chromatic++ | src/gc/gc_gms.c:
22:05 dalek parrot/chromatic/early_pmc_gc: Added documentation to new GC GMS functions.
22:05 dalek parrot/chromatic/early_pmc_gc: review: https://github.com/parrot/parrot/commit/8047502390
22:05 dalek parrot/chromatic/early_pmc_gc: 1742672 | chromatic++ | src/gc/gc_gms.c:
22:05 dalek parrot/chromatic/early_pmc_gc: Added headerization.
22:05 dalek parrot/chromatic/early_pmc_gc: review: https://github.com/parrot/parrot/commit/1742672e05
22:17 whiteknight joined #parrot
22:28 whiteknight good afternoon, #parrot
22:37 plobsing joined #parrot
22:39 pmichaud 17:08 <Coke> how to watch a ticket in github?
22:39 pmichaud At the bottom of the ticket page, there's an obscure link that says "Notifications for new comments on this issue are {on|off}".  Make sure it's set to "on".  :-)
22:40 pmichaud I'm guessing that will enable email notifications for that issue for you.
23:02 kid51 joined #parrot
23:16 woolfy left #parrot
23:36 lucian joined #parrot
23:37 whiteknight msg moritz: I've tested the parrot-iocleanup1 branch against my most recent changes. Looks like the encoding-related issues are gone, but we've still got the numchars issues. How do you want that fixed?
23:37 aloha OK. I'll deliver the message.

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced