Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2015-06-21

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:05 Cheery joined #marpa
00:20 ronsavage joined #marpa
00:25 ceridwen_ joined #marpa
00:26 ronsavage Yes, the order-preservation is deliberate. I will switch to push, pop, reverse. In my defense I will say I copied the code :-). I like more-and-more the idea of outputting to a tree managed by a tree-oriented module.
00:29 ronsavage1 joined #marpa
00:36 ronsavage I'm thinking of using Tree::DAG_Node, despite that fact I sometimes use Tree::Simple. My thinking is that it's better to go with the module providing the most features, so end users can have it without the problem of wanting it (or something similar) while also stuck with my choice of Tree::Simple because it's simple.
00:37 aredridel joined #marpa
00:48 ronsavage1 Hmmm. Looks like the 1st posting was lost at more-or-less exactly when the backlog was at it's local midnight. Another mystery!
03:08 CQ_ joined #marpa
07:39 lwa joined #marpa
07:46 ronsavage joined #marpa
13:07 koo5 joined #marpa
15:20 koo5 joined #marpa
16:12 Cheery joined #marpa
16:37 koo5 joined #marpa
17:39 idiosyncrat joined #marpa
19:03 Cheery does marpa have some benchmarks I could look into?
19:04 Cheery turns out, if I run my parser on: print("hello world" -1 * (2 + 3))
19:04 Cheery that line repeated 1000 times, it takes 2 seconds to parse
19:20 idiosyncrat Cheery: most of the overhead in a typical SLIF parse occurs at the Perl level
19:20 idiosyncrat Typically, that is
19:20 idiosyncrat A goal of the Kollos project is to convert the Perl layers of the SLIF to Lua, where they will be much faster ...
19:21 idiosyncrat and can be made faster yet via Luajit or a selective rewrite into C
19:22 idiosyncrat All this is hand-wavy, and without looking at your code (I'm immersed in Kollos), but I hope it helps
19:35 idiosyncrat More directly responsive to the request for benchmarks, all the synopses in the POD docs are in the test suite.
19:36 idiosyncrat You might try running against these for comparison,
19:36 idiosyncrat Note that repeatedly creating new grammars and recognizers does have a cost, so short inputs
19:37 idiosyncrat where you create a recognizer over and over again, or a new grammar each time, ...
19:37 idiosyncrat are a pessimal case for Marpa.
19:37 idiosyncrat This is one advantage regexes have -- typically they are very low overhead for each startup
19:42 Cheery at this point it's not very pressing issue.
19:42 Cheery should also fix the traversing code a bit. It crashes when the input exceeds 1000 lines. :)
19:52 RichardMichaels joined #marpa
21:13 ernimril Cheery, hmm, that is not very much... for my java parser I actually found some real code where _one_ file was 1.2M lines (very silly auto generated code, but still valid). It did cause some memory problems for me at first, but now I can handle it
21:14 ernimril Cheery, that comes out to about 16.5M earley states
21:25 Cheery how did it handle that?
21:27 ernimril Cheery, I mark some rules as "partial completions" and clean out states that were not used in that part. So when I reach state 100, I realize "some part is done" filter down state 80-100 so that they only contain the one or two states that are really used (instead of the average ~35 I have in my java grammar)
21:28 Cheery ok
21:30 Cheery but how did it handle that? how long did it took?
21:31 ernimril Cheery, about 80 sec for that file
21:33 ernimril Cheery, but my normal test is another set of 11k files and that takes about 37 seconds on my computer (very old one)
21:34 Cheery apparently I can likely then improve the performance a bit
21:37 ernimril Cheery, most probably and probably me to. I have a tokenizer and it takes about 2 seconds, then I have about 31 seconds of parse and then a few for semantic analysis. Parse is the big thing for me. Also, I work in java and parse on all cpu cores at once
21:58 koo5 joined #marpa
22:31 Cheery funny thing
22:31 Cheery I can probably make my parser about 50 times faster if I switch away from ordered dicts
22:43 ronsavage joined #marpa
23:22 koo5 joined #marpa
23:28 koo6 joined #marpa

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary