Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2014-01-26

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
23:12 jeffreykegler joined #marpa
00:44 sivoais joined #marpa
01:36 jeffreykegler left #marpa
04:27 jeffreykegler joined #marpa
04:43 jeffreykegler left #marpa
04:52 jdurand callgrind done - anybody interested in the .out binary file - this of linux 32bits ? otherwise I will just post some of my conclusions
06:06 ronsavage joined #marpa
08:35 jeffreykegler joined #marpa
08:35 jeffreykegler I'm interested in both the .out and the conclusions.
08:35 jeffreykegler left #marpa
12:09 jdurand Jeffrey, are there cases where read()'ing a input could be a memory hog ?
14:10 jdurand I used dl.free.fr to send the callgrind and a png - png is self-explanatory and is really where all consumption in terms of CPU cycles happen - have to go for a while, sending process is still ongoing - a notification to me and Jeffrey by mail will happen if it succeeds...
14:10 jdurand otherwise I'll use another free file hosting service
14:16 jdurand Re memory hog - ok I was trying a grammar on an input file that ha a size of  375Mb - that probably explain the memory hog /if/ the following is true:
14:17 jdurand support that the grammar is configured to have no action, neither on G1, neither on G0, then there is no reason for the lexeme values to be kept inside Marpa isn't it ?
14:17 jdurand So the question is: does Marpa keep track of all lexeme values when it is reading the input ?
14:19 jdurand If the answer is yes, then this might be a problem when parsing a big file, going orthogonal with the eventual concept of a-la-SAX streaming with Marpa, that can be easily simulated using events
14:19 * jdurand is noww absent for a while, children need to eat to something!
15:10 jdurand Epilogue: I guess, and might file an issue if that is not the case, that Marpa should not keep track of lexeme values if lexeme default action would be set to ::undef - unless this is an issue for Marpa internals, but I doubt
15:10 * jdurand is now eating
15:54 jdurand Jeffrey, I sent you in PM callgrind data and one PNG - sorry but it seems dl.free.fr hosting service is failing
16:55 AndChat|306516 joined #marpa
17:45 jeffreykegler joined #marpa
17:50 jeffreykegler judrand: Thanks.  I looked at the callgrind.  Re CPU, it seems as if the routine doing all the work is taking all the time.
17:51 jeffreykegler Libmarpa uses a lot of in-lining, but the way, which makes its working very obscure from a subroutine-focused point of view.  If you time, you might want to turn off in-lining, and see what the callgrind looks like ...
17:51 jeffreykegler although the overhead of calling subtroutines would distort things a bit.
17:54 jeffreykegler Re memory, yes, Marpa does track lexeme values at the Perl level.  Libmarpa also tracks them, but as int's, which it passes to the Perl layer so it can look them up.
17:55 jeffreykegler jdurand: * "If you time" should be "If you have time"
18:05 jeffreykegler jdurand: Marpa, since it is table-driven, is more like DOM than SAX.  Marpa understands the structure of the whole document, even if it is extremely complex.
18:07 jeffreykegler SAX discards information as it proceeds, which is OK if you no longer need it.  Marpa expects you'll need it.
18:08 jeffreykegler There are ways to use Marpa within a SAX-ish approach, which could in certain cases give you the best of both worlds.
18:09 jeffreykegler The SLIF, for example, uses Marpa for lexing, but once each lexeme is recognized, it throws all data about the internal structure of the lexeme, possible alternatives, etc., etc., away ...
18:10 jeffreykegler and just returns the start/end locations of the lexemes to the upper layer (G1).
18:34 jeffreykegler I'm going to start Phase 3 in a day or two.  The phases should not be as big a deal as they were ...
18:34 jeffreykegler Phase 1 involved new math, and the possibility that, if I'd gotten the math wrong, I'd have to revert the whole thing.  That and major changes to the internals of the parse engine.
18:35 jeffreykegler Phase 2 was the same, but to a lesser degree.  After the success of phase 1, the math seemed a pretty sure bet, but the changes to the parse engine internals were still very far-reaching.
18:36 jeffreykegler Phase 3 will add value: "cheap forgiveness"
18:36 jeffreykegler The math is now confirmed -- the last developer's release could not be performing if the math wasn't OK.
18:38 jeffreykegler And, I'll now be simply adding features to the parse engine, rather than taking it apart and putting it back together.  Much easier.
18:39 jeffreykegler Re issues: I've marked two "in progress" and two "pending".  The one's in progress I plan to finish and get into a developer's release before I start Phase 3.
18:39 jeffreykegler The ones "in progress" were both raised by jddurand ... a small problem with an error message ...
18:41 jeffreykegler and more flexibility in dealing with lexemes when they are inaccessible from the G1 start symbol -- the ability to go ahead and parse anyway.
18:41 jeffreykegler By number, they're Github issues #112 and #104
19:22 AndChat|306516 joined #marpa
19:25 AndChat|306516 Re about memory then I'll have to understand why my valgrind grammar eats twice the size offre the input file
19:27 jeffreykegler AndChat|306516: If you take a string, add structure to it, and record that structure as Perl data structures, it's gonna grow.
19:27 AndChat|306516 Otherwise  marpa performs great and yes will redo profiling with no inlining
19:28 jeffreykegler Somebody might want to look at who exactly is using the space -- perhaps there are some savings that can be found.
19:29 jeffreykegler But basically we are turning a string into a fully structured tree (while usually keeping the original string as well)
19:30 jeffreykegler So if it's just twice the size of the original, that's surprisingly good :-)
19:31 jeffreykegler AndChat|306516: are you jdurand ?
19:33 AndChat|306516 Yes using my tiny mobile do not know How to become jdurand ...
19:34 jeffreykegler By the way, moritz is *not* the right guy to contact to be added to the "no paste" menu
19:34 jeffreykegler He tells me it's run by Shadowcat Systems, so we probably need to find someone there
19:37 AndChat|306516 Ok thx  for having contacting him :-)
19:43 AndChat|306516 If i pause lexeme before and do myself lexeme_read without à value may i succeed to reduce memory. ..
19:44 jeffreykegler Not sure
19:44 jeffreykegler Don't you need the values?
19:45 AndChat|306516 Once just for collecting stats nothing else
19:46 AndChat|306516 Ok will return home AFK
19:46 jeffreykegler bye
19:53 lucs I just asked mst (shadowcat) to add #marpa to the nopaste dropdown; I guess he'll be responding soonish.
19:53 jeffreykegler lucs: much thanks!
19:53 lucs No problem.
20:14 jdurand Jeffrey, I see there is support of argument Marpa-debug gfor Build, that will do -fno-inline -Wno-inline, is that the way to compile without inlining or does it involve much other things
20:14 jdurand in the later case I will do remove inline directly from the Buid and/or Makefile
20:16 jeffreykegler judrand: I should have been clearer.  I meant remove inlining from Libmarpa, *not* the XS compile.
20:16 jdurand Ah ok - thx - will do it like that
20:18 jeffreykegler XS should always be compiled the same way as the Perl it's being combined with.  That's because the XS generation process wants to make assumptions about the Perl it is a part of.
20:18 jdurand yes no pb with that
20:18 jdurand inline appears in the following files: ./libmarpa_dist/marpa_ami.c ./libmarpa_dist/marpa_slif.c ./libmarpa_dist/marpa_avl.c ./libmarpa_dist/marpa.c ./libmarpa/avl/marpa_avl.c
20:18 jdurand I remove it in all of them, ok?
20:19 jeffreykegler autoconf should honor CFLAGS
20:20 jeffreykegler I forget the exact arguments, but you can call "configure" and pass it a new set of flags
20:23 jeffreykegler An example of a configure command for Libmarpa being assembled is the code in cpan/inc/Marpa/R2/Build_Me.pm
20:26 jeffreykegler I put the args together in @configure_command_args -- and as it happens, there's an example in the code where I do turn off inlining for gcc when I set certain debugging flags.
20:28 jdurand ok - I will just hack a little bit Build_Me.pm this will be enough a priori
20:28 jeffreykegler Sounds great
20:29 jeffreykegler Btw, speaking from the experience of having wasted time by omitting this step ...
20:29 jeffreykegler You want to double-check the flags carefully when they are echoed, not just to see if your new flags got set ...
20:30 jeffreykegler but also if you didn't inadvertently turn off any others
20:30 jdurand sure -; I have the experience of having falled into such traps as well
20:32 * jeffreykegler is AFK for a bit
20:59 jeffreykegler left #marpa
22:48 jdurand Uninlined profiling send to Jeffrey in PM
22:49 jdurand Jeffrey, there will be one thing you'll might explain to me... I do not see where is hiden to call to R->Value() -;

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary