Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2016-06-18

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:12 ronsavage joined #marpa
00:32 ronsavage MarpaX::Grammar::Parser no longer passes its tests, so I'm working on that.
01:48 ilbot3 joined #marpa
01:48 Topic for #marpa is now Start here: http://savage.net.au/Marpa.html - Pastebin: http://scsys.co.uk:8002/marpa - Jeffrey's Marpa site: http://jeffreykegler.github.io/Marpa-web-site/ - IRC log: http://irclog.perlgeek.de/marpa/today
03:40 dvxd joined #marpa
03:40 dvxd evening all!
03:42 dvxd I'm doing a little project in the languages arena, and MARPA looks great.  My question: is BNF or EBNF overkill for parsing sexp (LISP expressions)?
03:42 dvxd Another question: is it possible to specify LISP using BNF or EBNF.
03:44 dvxd Ok, I did some more googling, looks like LISP can be specified in BNF.  Is there a way to crunch the BNF down into a C program or C function I can call from other C code?  So I don't have to use Perl every single time I want to do the parsing?
03:44 dvxd https://gist.github.com/pstuifzand/5930450
03:49 dvxd also, does the MARPA EBNF parse support UTF-8?
04:01 kaare_ joined #marpa
04:06 idiosyncrat_ About to sign off for the day, but re the questions ...
04:07 idiosyncrat_ LISP is an extremely simple grammar, and Marpa is in a sense overkill.
04:07 idiosyncrat_ But if the alternative is hand-coding, Marpa may be the easiest way to do it.
04:08 dvxd thanks.
04:08 dvxd you available for a few more questions?
04:08 idiosyncrat_ Marpa::R2 supports UTF-8 in its input and output
04:08 dvxd I'm reading through the libmarpa documentation.
04:08 idiosyncrat_ But there's a problemwith using it in the DSL specs themselves, unfortunately.
04:09 dvxd Ok.  My goal is to put LISP in an EBNF grammar, have it crunched down into some sort of C code that I can compile, which uses libmarpa calls (if necessary) to do the parsing
04:09 dvxd for the LISP I'm doing, shouldn't need UTF8 in the spec itself, except to specify that all characters are UTF.
04:10 idiosyncrat_ So you should be OK
04:10 dvxd Ok, am I misunderstanding Marpa?  Is there a clear path through Perl to transform EBNF into a C function that will do the parsing for me?
04:11 idiosyncrat_ Marpa is not like yacc which generates C -- although you cold set it up to do so.
04:11 idiosyncrat_ s/cold/could/
04:12 dvxd what time investment would I be looking at?
04:12 idiosyncrat_ I won't say there.
04:12 dvxd I guess I was thinking of it as a yacc replacement.  yacc... *shudder*
04:13 dvxd Ok, another thing; the EBFN is translated into a series of calls to libmarpa?  and then libmarpa returns a parse tree in some format?
04:13 idiosyncrat_ You might want to look at other Marpa-powered LISP parsers -- IIRC there is one out there, though I can't remember where. :-)
04:13 idiosyncrat_ Yes, with Marpa it's easy to get an AST and proceed from there.
04:13 dvxd I think I just pasted the link to it; it uses Perl, but it seems the dependancy on perl never goes away
04:13 dvxd I was just hoping for a small blob of C I can incorporate in another program.
04:14 idiosyncrat_ That's right, it's dependent on Perl
04:14 idiosyncrat_ Maybe you want to write the parser in Lua?
04:14 dvxd I don't know Lua well enough.
04:15 dvxd The EBNF for LISP was so small and beautiful, I wanted to ship it as the language "source", and I don't mind going through Perl for an initial stage, but the end binary I want to be independant of Perl.
04:15 idiosyncrat_ If you don't want anything fancy, there's probably C code you can simply lift.
04:16 dvxd For parsing LISP, it is true, I can grab lots of simple C parsers.
04:16 dvxd but... Marpa is the coolest parser out there. I love your promise of speed.
04:16 idiosyncrat_ It sounds like they are in fact what you're looking for.
04:16 idiosyncrat_ Thanks!
04:16 dvxd and I love even more the idea of using EBNF as the language that the program is written in.
04:17 dvxd yacc is almost unusable; but if you could use EBNF directly to generate a C parser the way yacc does, I could see Marpa usage really taking off.  People don't want to be tied to Perl.  At least, I don't.  I'm people ;)
04:17 idiosyncrat_ Yes, it was something I dreamed about for decades.  Folks told me it couldn't be done.
04:18 idiosyncrat_ I believe someone is at work on a Marpa-to-C parser, with no tie to Perl, but this person likes to work in quiet.
04:18 dvxd Am I correct, EBNF isn't part of Marpa; it is the Perl layer that has the EBNF to Marpa converter?
04:19 idiosyncrat_ There's the SLIF (which you can call an EBNF if you like) and yes these days it's only implementation is in Perl.
04:20 idiosyncrat_ If you've a big question, ask it now, because I really am about to drift off.
04:21 dvxd Sleep well... hope to talk to you again tomorrow.
04:21 dvxd I still haven't given up.
04:21 dvxd oh, parse trees
04:21 dvxd does marpa return a parse tree, or is that generated by the Perl layer also.
04:22 idiosyncrat_ Libmarpa returns a parse forest, but there's no syntactic sugar.
04:22 idiosyncrat_ So the practical answer to your question is that it requires the Perl layer.
04:22 idiosyncrat_ Good night!
05:02 jdurand joined #marpa
05:03 jdurand Jeffrye, Jeffrey, I was looking to the "untested" methods in libmarpa: the doc does not say if marpa_r_clean() will generate events corresponding to the previous recognizer state - does it? Thanks.
05:08 jdurand Also, looking to zwa, since the doc does not say if there are restrictions about the default value when using marpa_g_zwa_new(Marpa_Grammar g, int default_value), how one does know when marpa_r_zwa_default(Marpa_Recognizer r, Marpa_Assertion_ID zwaid) fails ? I suppose(d) that the default value must be a positive integer(?).
05:37 dvxd jdurand: are you using marpa directly in C?
05:49 jdurand dvxd: yes
06:29 dvxd jdurand: do you mind talking about it?
06:43 dvxd jdurand: if you are close to a C program that reads BNF and creates a parse tree, without using Perl, will that be available at some point?
06:43 dvxd otherwise I have to write it. :)
06:56 jdurand dvxd: c.f. https://github.com/jddurand/c-marpaWrapper
06:57 jdurand Value is already available - Right now I am implementing ASF, because it appears to me that this is only safe way to deal with EBNF exception in a BNF parser
06:59 jdurand The content of the value itself is a per libmarpa semantics: user choice. libmarpa think in terms of generic stack, this is why I also wrote https://github.com/jddurand/c-genericStack. The test suite in c-marpaWrapper is an example of its usage.
07:02 dvxd thank you, looking at it now.
07:02 dvxd just compiled libmarpa; wow, 800K!
07:08 jdurand Jeffrey, yet another question: in Marpa::R2, there is the undocumented factoring_max option for ASF. Can you elaborate on this?
07:10 jdurand dvxd: c-marpaWrapper have two required dependencies: c-genericStack (for the tests, I could have prevented CMakeLists.txt to fail if it does not find it, though), and c-genericLogger (c.f. https://github.com/jddurand/c-genericLogger). You will have noted that everything is CMAKE based and should compile/install everywhere, including on Windows.
07:14 jdurand dvxd: the tests are run with "make check" and not "make test" - side effect of (IMHO) cmake bad semantics with "make test".
07:24 dvxd thank you, still reading
07:25 dvxd also wondering why the difference between lexing and parsing.
07:25 jdurand where do you see this difference?
07:26 dvxd in what Jeffrey wrote; in his BNF also, there is the ::= and ~ difference
07:27 dvxd nice documentation, looks very thorough and clear (still reading it)
07:27 jdurand This is a SLIF subtility: "~" is nothing else but refering to a subgrammar.
07:28 dvxd I was wondering about that; since the definition of a symbol might change depending where it is in relation to other symbols.
07:28 jdurand In SLIF there are two grammars implicitely: the "::=" is the top-level one. "~" forces the right hand side to be a terminal in the top level grammar.
07:29 dvxd ah.
07:29 jdurand This mean that, in the SLIF, the RHS of a "~" rule cannot be an LHS of a "::=". All RHS's of "~" rules denote nothing else but a sub-grammar.
07:31 jdurand And this is preciely why all adverbs like "action", "event" etc have no effect in the "~" subgrammar: it is hardcoded to take the longest parse tree value, the latest to be injected in the top-level "::=" grammar as an explicit terminal.
07:31 dvxd ok much better, compiling libmarpa with -Os instead of -g -O2 in the CFLAGS dropped the size down to 128Kbytes.
07:33 jdurand Are you talking about libmarpa stand-alone or about c-marpaWrapper. c-marpaWrapper embeds libmarpa.
07:38 dvxd libmarpa standalone.  how small does c-marpaWrapper get?
07:41 jdurand Don't know - not much more a priori.
07:51 jdurand AFK
08:31 jdurand joined #marpa
08:33 jdurand Jeffrey, jus a remark: I noticed that R2.xs is using libmarpa ordering methods that start with an underscore (e.g. _marpa_o_or_node_and_node_count). Since R2.xs is "stable", IMHO it could be coherent to make'em not underscored
12:45 rgrinberg joined #marpa
12:51 rgrinberg joined #marpa
13:06 kaare_ joined #marpa
13:08 idiosyncrat_ joined #marpa
13:09 idiosyncrat_ jdurand: re marpa_r_clean() -- http://irclog.perlgeek.de/marpa/2016-06-18#i_12687922
13:10 idiosyncrat_ I am not sure I ever finished writing this method -- "projected" method might be a better term than "untested".
13:13 idiosyncrat_ I really don't know what I planned to do about events if I ever implemented marpa_r_clean -- probably it would change an previous Earley set without changing its events.
13:20 idiosyncrat_ jdurand re marpa_g_zwa_new() -- http://irclog.perlgeek.de/marpa/2016-06-18#i_12687930
13:20 idiosyncrat_ default_value must be 0 or 1
13:21 idiosyncrat_ It's a boolean.
13:23 idiosyncrat_ jdurand: Are you planned on using these methods?  In the doc I say "Their fate is uncertain. Users should regard these methods as unsupported." and that is truer now than when I wrote it.
13:24 idiosyncrat_ s/planned/planning/
13:27 idiosyncrat_ jdurand: re factoring_max -- http://irclog.perlgeek.de/marpa/2016-06-18#i_12688094
13:29 idiosyncrat_ Ambiguity is of two types factorings and symbolic choices -- http://search.cpan.org/~jkegl/Marpa-R2-3.000000/pod/ASF.pod#Symches_and_factorings
13:31 idiosyncrat_ Symches (symbolic choices) are limited to the number of rules and the difference is almost always of interest.
13:34 idiosyncrat_ The potential number of factorings is unlimited and can grow exponentially, but some applications doesn't care much (or at all) about factorings.
13:34 idiosyncrat_ By default the factorings are truncated when they would grow beyond 42.
13:34 idiosyncrat_ factoring_max allows you to change that number.
13:35 idiosyncrat_ Presumabily I didn't document it because I didn't think I'd tested it enought.
13:35 idiosyncrat_ s/enought/enough/
13:40 idiosyncrat_ jdurand -- re http://irclog.perlgeek.de/marpa/2016-06-18#i_12688281 -- _marpa_o_or_node_and_node_count
13:41 idiosyncrat_ _marpa_o_or_node_and_node_count is only used by other underscored methods, so AFAICT there's no interface violation.
13:44 idiosyncrat_ It is used indirectly by the ASF code, but that is documented as breaking the rules, and will be fixed someday.
14:43 jdurand joined #marpa
14:43 jdurand Re marpa_r_clean: fair enough, will drop that from c-marpaWrapper - I am not using it
14:44 jdurand Re marpa_g_zwa_new(): this was just a question up to now, and fair enough as well, I will not wrap it
14:45 jdurand Re factoring_max - ok, noted!
14:46 jdurand Re _marpa_o_or_node_and_node_count: "documented as breaking the rules" ?
15:00 idiosyncrat_ jdurand: best not to wrap any "untested" method, unless there is really some overriding reason to use it.
15:01 idiosyncrat_ re http://irclog.perlgeek.de/marpa/2016-06-18#i_12689166 -- _marpa_o_or_node_and_node_count
15:05 idiosyncrat_ https://github.com/jeffreykegler/Marpa--R2/blob/master/cpan/lib/Marpa/R2/ASF.pm#L30
15:06 idiosyncrat_ In the comments for ASF.pm, I note that it breaks the rule about using Marpa internal methods.
15:07 idiosyncrat_ The rule by the way, is that Marpa::R2 only uses external, documented Libmarpa methods, except for tracing, debugging, etc.
15:08 idiosyncrat_ So in this sense the ASF/ parse forests feature has the status of the tracing, debugging methods.
15:09 jdurand Ok - since I bundle explicitely libmarpa in marpaWrapper, I just have to track the eventual changes v.s. ASF - thanks
15:55 jdurand Jeffrey, in the doc for ASF, you say that pruning_traverser is called recursively - I do not see this recursivity in the code - perhaps in ASF.pm itself, but where?
16:40 idiosyncrat_ "It is called recursively." is used to say that "pruning_traverer" is intended to be called indirectly by the $asf->traverse() method, as it recursively examines (traverses) the nodes of the tree.
16:42 idiosyncrat_ The values of the glade is the return value of pruning_traverser() and it may be used by other calls to to pruning_traverser() in the same traversal, so pruning_traverser() does recurse, just indirectly.
16:56 jdurand oh ok I see - thanks
16:57 jdurand I am currently implementing the ASF in pure C - and noted your idea of using a bitvector in the ASF.pm comments - nevertheless my first implementation will be a basic perl->C translation
16:59 jdurand Btw I apologiez not participating in the R3 effort - my priorities have inverted since then - I am a bit tired of my amazing XML project which I want to "perfect" (from my point of vew), and definitely want to finish it - when finished this will be a great promotion of Marpa I believe
16:59 jdurand "to be "perfect""
17:05 jdurand Though I always backlog and appreciate very much to see rns expertise, that seems great -;
17:27 idiosyncrat_ jdurand: on Marpa::R3, I am, as usual, the main obstacle to progress. ;-)
17:28 idiosyncrat_ rns I suspect would like to do more but winds up waiting on my progress.
17:29 idiosyncrat_ In any case, I think it best people do what excites them rather than be handed duties.
17:29 idiosyncrat_ As a manager, I'd have to hand out duties, but I'd also have to pay you. :-)
17:29 idiosyncrat_ If people are doing what they like, they never burn out.
17:30 idiosyncrat_ btw, I've been sick the past few days, which has slowed me down.
18:43 jdurand Oh, hope you fill better now - I was wondering, this is off-topic - do you know a true portable hashmap library in ANSI C? I liked https://github.com/petewarden/c_hashmap - and dislike others saying this is portable when I see uintxxx_t typedefs, that are not ANSI-C -;
18:46 jdurand s/fill/feel/ grrrr
19:18 rns joined #marpa
19:23 rns jdurand: re http://irclog.perlgeek.de/marpa/2016-06-18#i_12689981 -- I was searching for the same thing one and found this list -- http://stackoverflow.com/a/6118603/4007818 -- the last one -- http://pokristensson.com/strmap.html -- claims ANSI C compliance.
19:23 rns s/one/once/
20:12 rns left #marpa
20:45 idiosyncrat_ If I needed a portable hashmap library at this point, I'd use Lua tables.
20:46 idiosyncrat_ AFK
21:38 dvxd joined #marpa
22:30 idiosyncrat_ joined #marpa

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary