Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2015-01-11

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:50 jeffreykegler joined #marpa
05:20 mcedit joined #marpa
05:21 sirdancealot joined #marpa
05:31 ronsavage joined #marpa
09:51 lwa joined #marpa
09:53 basiliscos joined #marpa
11:33 basiliscos joined #marpa
11:38 basiliscos joined #marpa
12:38 pczarn joined #marpa
13:22 sirdancealot joined #marpa
14:06 pczarn hello
14:36 basiliscos joined #marpa
15:06 sirdancealot joined #marpa
15:14 basiliscos joined #marpa
15:15 sirdancealot joined #marpa
15:16 pczarn I'm starting to write more grammars with the "racing version" of bindings
15:17 pczarn I'm going to benchmark JSON
15:18 basiliscos joined #marpa
15:21 basiliscos joined #marpa
15:45 basiliscos joined #marpa
16:21 basiliscos joined #marpa
16:29 pczarn JSON to AST in 4.15s for 1.7M file
16:35 pczarn I'd like to parse a lot more complex language with similar speed
16:45 basiliscos joined #marpa
16:47 basiliscos joined #marpa
16:49 pczarn libmarpa's json.c does that in 0.259s, whoa
16:50 Aria Wow.
16:51 pczarn the lexer in json.c is hand-written though
16:52 pczarn I'm using an NFA-only regex...
16:52 pczarn as my "lexer"
16:53 pczarn so excluding my regex and everything until tree creation, it's 0.062s, but I'm not sure when marpa's work happens
17:15 Aria Lots of work up front to make sure that further parsing is fast.
17:18 pczarn right, I just realized I'm skipping the point at which NoParse might be returned
17:27 basiliscos joined #marpa
17:29 pczarn got rid of that binary search, AST construction went down to 0.0526s by 0.009s
17:51 pczarn Finally, the main part takes 0.218s + 0.052s.
18:40 Aria Woot!
21:26 jeffreykegler joined #marpa
21:28 jeffreykegler pczarn: re http://irclog.perlgeek.de/marpa/2015-01-11#i_9918210 -- as is often the case, I am not sure that I understand the question, but ...
21:29 jeffreykegler almost all of libmarpa's processing will happen during earleme completion ...
21:29 jeffreykegler marpa_r_earleme_complete()
21:30 jeffreykegler This is excluding evaluation, and assuming a long parse, which allows me also to exclude grammar precomputation.
21:32 jeffreykegler rns: re http://irclog.perlgeek.de/marpa/2015-01-08#i_9906276 and marpa_r_terminals_expected() ...
21:33 jeffreykegler in my owns plans, I suspected  marpa_r_terminal_is_expected() might work better -- that is, just before trying each lexeme, check to see if it is expected.
21:34 jeffreykegler This saves the overhead of creating a data structure for that information outside of Libmarpa, and allows IMHO cleaner code.  I've never measured, but I suspect it's also faster.
21:36 jeffreykegler pczarn, Aria: nice to see timings, by the way.  Thanks!
21:38 jeffreykegler An advantage of Marpa's more general, algorithmic, mathematical approach is that it allows for most of the work of parsing to be done by logic which couild, reasonably, be encoded in silicon, the way floating point and graphics are.
21:40 jeffreykegler So just imagine these number 10x faster -- all of a sudden, for lot of things you had never imagined formulating as a general BNF parsing problem, that now becomes the optimized approach.
21:43 ronsavage joined #marpa
22:05 pczarn joined #marpa
22:13 pczarn jeffreykegler: thanks for clarification, processing during earleme completion is necessary for Ruby Slippers, I suppose.
22:15 pczarn why did you call that approach a "racing version"?
22:18 jeffreykegler "earleme completion" is when the Earley sets are actually built -- which is 99% of the parse engine's work.
22:18 jeffreykegler "racing version" may have been originally my phrase -- I used when guessing at someone's intent -- I now forget the context.
22:23 pczarn ok, all passes are working on each Earley item in the current set, not all sets at once. my intuition was wrong
22:24 pczarn Loup's tutorials cover Leo's algorithm etc. so I'd like to start an implementation
22:25 jeffreykegler Right.  For efficiency, I may go back and recompute certain indexes to the Earley set, etc., but basically the Earley sets are build one at a time and each one is completely built, and will not change, before I move on to the next one.
22:26 jeffreykegler Note that Marpa rearranges the parse engine, so that the order of operation is not the same as in Leo, Loup's tutorials, or Earley's original.
22:27 pczarn jeffreykegler: in case I finish an efficient Earley parser implementation for a specific grammar, how many thousands of lines it might have?
22:27 jeffreykegler Huh?
22:27 jeffreykegler Libmarpa is 15,000 lines of code, plus
22:28 jeffreykegler so that could be taken as a maximum.
22:28 jeffreykegler Loup's aesthetic is that he wants the thing to fit on one page, ...
22:28 jeffreykegler which is nice if you can do it and still do what you need to do.
22:29 pczarn yes, some of Libmarpa is cycle detection and other stuff I probably don't need
22:29 jeffreykegler If you want to reimplement, look at the pseudo-code in my theory paper.
22:30 jeffreykegler If you want to remimplement Marpa, that is.
22:32 jeffreykegler By the way, in later phases of Kollos, I'll rewrite Libmarpa to be smaller and simpler, ...
22:32 jeffreykegler but it still won't fit on one page. :-)
22:34 pczarn hmm, it might be too hard for me to get a reimplementation I need :)
22:35 jeffreykegler If I may suggest, not just to you, but generally ...
22:36 jeffreykegler while I do *NOT* discourage people looking at the algorithm itself, on the contrary it develops your intuition, and I think it's a very constructive thing to do ...
22:36 jeffreykegler if you are out to make a mark for yourself ...
22:37 jeffreykegler I think they are lots of techniques at the higher level that remain undiscovered, techniques nobody has explored because the underlying parser did not exist before, ...
22:37 jeffreykegler I've picked off a lot of these -- Ruby Slippers, general BNF parsing, and hopefully strand parsing will be another ...
22:39 jeffreykegler but there are others out there I'm sure.
22:39 pczarn that's a good observation about discoveries in general
22:40 jeffreykegler One approach would be find some tool/language, etc. which is currently hobbled by traditional parsing and its limits ...
22:40 jeffreykegler and think out of the box about it.
22:41 jeffreykegler Because from a parsing point of view, a lot of our tools these days are absolutely pathetic.
22:41 pczarn I'm trying to do that
22:41 jeffreykegler And unfortunately new ones are being designed pathetic.
22:42 jeffreykegler But of course basically if you're going to be creative you've got to follow your own instincts,
22:43 jeffreykegler which also means ignoring what I've just said if it is not what excites you.
22:44 jeffreykegler But, while I am striving for inspirational aphorisms :-)
22:45 jeffreykegler I think one intuition to cultivate is the one that allows you to distinguish between the excitement of being creative, and the excitement of following trends.
22:45 jeffreykegler And to indulge the first, and learn to suppress the 2nd.
22:55 pczarn jeffreykegler: Of course, macros are deficient tools that can be designed better
23:02 pczarn marpa is great for defining DSLs. By putting both together, we get useful syntactic macros
23:14 jeffreykegler pczarn: That's a good example -- now that you have Marpa to parse macros, there is probably a lot more that can be done with them ...
23:15 jeffreykegler and I've got so much else going, I personally just won't have the time to explore.
23:16 jeffreykegler But in particular the idea of true higher level languages -- languages which specify languages, may have applications there.
23:17 jeffreykegler Previously, if your parser generator was yacc or PEG, and you automatically generated a grammar, the changes your parser generator could generate a parser for it without hand-twiddling were almost zero.
23:18 jeffreykegler With Marpa it's a sure thing, and with a bit of care, you can also be sure the parser will be linear.
23:20 pczarn_ joined #marpa
23:21 pczarn_ I know what G1 stands for
23:21 pczarn_ the United States
23:23 ronsavage pczarn: Re: http://irclog.perlgeek.de/marpa/2015-01-11#i_9919192. That's why I wrote Text :: Balanced :: Marpa. I said to myself: Using Marpa, I can do better that the original.
23:25 pczarn_ good night.
23:25 jeffreykegler Yes, and it helps guide my efforts -- I think in BNF, so that event-driven is an afterthought for me ...
23:25 jeffreykegler pczarn: good night
23:26 jeffreykegler that is, I think of Marpa as a general BNF parser which also do procedural parsing ...
23:27 jeffreykegler so the opposite perspective -- Marpa as a procedural parsing framework that includes the ability to general BNF parsing -- is "new" to me.
23:29 ronsavage pczarn: Perhaps you can re-write Text::MacroScript or Text::Template!
23:48 jeffreykegler Above, in the list of new techniques that I have created with Marpa, there's a typo -- the list should be
23:49 jeffreykegler " Ruby Slippers, general *precedence* parsing, and strand parsing"
23:49 jeffreykegler Marpa does a lot to make general *BNF* parsing more accessible, but it was around long before I came along.
23:51 jeffreykegler "general precedence parsing" is the generalization of precedence parsing beyond "operator precedence parsing" so that it no longer depends on identifying and classifying operators.
23:51 jeffreykegler It's what makes the SLIF's precedence statements a lot more flexible and easier to write than their equivalent, for example, in yacc.

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary