Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2014-12-02

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:07 koo7 joined #marpa
00:08 koo5 joined #marpa
00:56 flaviu joined #marpa
01:04 jeffreykegler rns: What did you think of moving toward a test of Kollos on a JSON parser?
01:05 jeffreykegler I've done the pseudo-code for the sequence rules, but perhaps it's best that start coding the rewrite portion.
01:06 jeffreykegler If you are inclined to work with me on the JSON parser, we should start trying to put our various pieces together.
01:07 jeffreykegler Perhaps I should next write the Lua code which goes from the LUIF grammar to a list of Libmarpa rules, and symbols, ready to use for calling the methods which create a Lua grammar.
01:08 jeffreykegler rns: you're not online, but I assume you backlog.
01:15 jeffreykegler As an explanation, for those who are interested.  The reason for this approach is NOT because the world needs another JSON parser.
01:15 jeffreykegler It's that it makes a good starting point -- it's a small as you can get and still be anything like a real exercise for a Marpa parser ...
01:16 jeffreykegler and there's a good grammar, some great examples to benchmark against, a good likelihood of other open-source code we can leverage, etc., etc.
01:23 ronsavage Yes, a JSON parser, with various other implementations to compare against, makes good sense.
01:25 jeffreykegler Kollos will lose against some of them, because parsing JSON is 95% lexing, and some of them have hand-optimized lexers written in C ...
01:25 jeffreykegler Marpa::XS is a good example of that.
01:26 jeffreykegler But that makes it all the better as a benchmark, because we can reasonably take Marpa::XS as "fastest obtainable".
01:26 jeffreykegler * "fastest obtainable" -> "fastest attainable"
01:50 ronsavage Sure - it's not so much a race as a prototype. If it doesn't work, we have problems. If it does, we forge ahead.
02:00 Aria jeffreykegler: Adding the Adams paper to my collection!
02:01 Aria JSON ++
02:01 jeffreykegler Aria: thanks.
02:01 Aria Great choices for all those reasons exactly.
02:02 jeffreykegler The Wvyern paper cites Adams, but notes that there is "no stable parser" for Adams grammars.
02:03 jeffreykegler Adams describes some parsing strategies which apparently left the Wyvern folks not 100% satisfied.
02:05 jeffreykegler I haven't yet delved into the details, but they are all LR-based, so I can imagine why the Wyvern folks may have been less than 100% happy.
02:08 jeffreykegler AFK
05:01 koo7 can i use a grammar in two threads, nonconcurrently, just first from one, then another? been some time since i did something that involved threading, but i dont see how this wouldnt work
05:01 koo7 " Further, a Libmarpa time object can only be used safely in the same thread as its base grammar. This is because all time objects with the same base grammar share data from that base grammar. "
05:03 koo7 like, first define the grammar in one thread, then precompute and valuate in another
05:56 ronsavage joined #marpa
06:30 rns joined #marpa
06:34 rns jeffreykegler: re JSON parser — looks like a good idea.
06:35 rns I thought we can start from a SLIF JSON grammar, e.g. https://github.com/jeffreykegler/Marpa--​R2/blob/master/cpan/t/sl_json_ast.t#L220
06:40 rns Now that there is a design doc on sequence rules, I think I can start adding sequences to LUIF-to-Lua table (LHS = { {RHS}, { adverbs } }) translator to do a first cut and then you'd look into it to implement rewriting.
06:42 rns Also looks like LUIF-to-Lua translator will need character classes and single/double quote literals from SLIF.
06:45 rns Here is how I see what I can do — implement translation to lua table of just enough LUIF (sequences, literals, charclasses) for JSON so that you can do grammar rewriting for libmarpa and lexer — feedback is appreciated, as always.
06:46 rns Those are quick initial thoughts — by I'll be busy@work by this Friday morning, but I'll backlog and reply.
06:46 rns left #marpa
09:43 jeffreykegler joined #marpa
09:44 jeffreykegler koo7: You can't safely use that dual-thread strategy -- that's what the language you quote is trying to say.
09:45 jeffreykegler You can create two identical grammars in different threads, however. Perhaps that accomplishes what you want.
09:47 jeffreykegler rns: re http://irclog.perlgeek.de/​marpa/2014-12-02#i_9745451 -- Great!  Let me know when you're ready for me to start implementing the grammar rewriting.
09:48 jeffreykegler Also, could you point me to the LUIF-to-Lua code, so I can start familiarizing myself?
09:50 jeffreykegler The LUIF-to-Lua translator can start parsing character classes, but in this JSON parser I want to use an external lexer, so we would not actually use the character classes at this point.
09:53 jeffreykegler Using external lexing means that only the top (structural) layer of JSON is actually Kollos-powered, in a sense, but you have to start somewhere.
09:55 jeffreykegler As I see it, we'll add LUIF-internal lexing after we run the JSON benchmarks.
09:56 jeffreykegler In the meantime, I will write more pseudo-code, and study your LUIF-to_Lua translator.
10:05 lwa joined #marpa
10:06 jeffreykegler Here's my draft of our JSON LUIF grammar, for our first tests: https://github.com/jeffreykegler/ko​llos/blob/master/working/json.luif
10:06 jeffreykegler Notice all the lexical layer stuff is left out.
11:34 flaviu joined #marpa
11:36 flaviu joined #marpa
13:47 rns joined #marpa
13:50 rns jeffreykegler: re http://irclog.perlgeek.de/​marpa/2014-12-02#i_9746005 — sure, I've added the json grammar you provided to the test; it doesn't yet passes and I see that I've got the Lua table building logic all wrong so I'll have to give it more time this Friday and weekend.
13:51 rns LUIF-to-Lua translator code is in https://github.com/rns/MarpaX-Languages-Lua-AS​T/blob/master/lib/MarpaX/Languages/Lua/LUIF.pm
13:54 rns It works by adding LUIF grammar you've provided (which starts the module) to the Lua SLIF grammar, figuring out lexemes and unicorns for external lexing and adding handler subs for LUIF rules — this is done in extend() method of MarpaX::Languages::Lua::AST.
13:57 rns Then, when fmt() method of Lua::AST ses that an AST node has a handler (looking up in a hash of node_id's, i.e. LHSes of the LUIF grammar), the handler is called with the node as an argument and it has to return a string translated into Lua, which is added to the resulting string returned by fmt().
13:59 rns So, transpile() method of Lua::LUIF just calls fmt() method of Lua::AST.
14:00 rns LUIF handlers in Lua::LUIF are bnf2luatable() and do_grammarexp(), which handle LUIF BNF and g = grammar() ... end rules.
14:03 rns bnf_ast_traverse() method of Lua::LUIF takes LUIF AST and returns an intermediate form (array of alternatives, IIRC), which is stringified into  a lua table in the caller — bnf2luatable() method.
14:06 rns Hope it helps. When adding the Kollos json grammar, I had to change true and false to json_true and json_false to avoid clashing with Lua keywords. As I said, I'll need to rework the logic which builds lua table, will keep you informed.
14:06 rns left #marpa
14:14 koo7 joined #marpa
14:30 jluis joined #marpa
15:51 jluis joined #marpa
17:28 koo5 joined #marpa
18:20 koo7 joined #marpa
20:42 flaviu joined #marpa
21:15 flaviu joined #marpa
21:39 flaviu joined #marpa
22:09 jeffreykegler joined #marpa
22:10 jeffreykegler http://t.co/8vmx0oJZXA --  Mark Dominus's Higher Order Perl
22:10 jeffreykegler It's online for free, as Deyan just tweeted
22:11 jeffreykegler The chapter on parsing is about top-down parser, with a combinator approach, and IIRC does not even mention bottom-up parsing, never mind Earley's,
22:12 jeffreykegler but I've long recommended this book ...
22:12 jeffreykegler because it's introduction to the basics -- BNF, lexing, and the basics of the problem space that I was tackling ...
22:13 jeffreykegler are covered very well, and from a Perl viewpoint.
23:16 koo5 jeffreykegler, thanks for the answer about threading. Mind to satisfy my curiosity as on what makes the two-threaded use impossible? If you dont feel like, thats ok
23:38 koo6 joined #marpa
23:39 jeffreykegler koo5: What, me defensive? :-)
23:40 jeffreykegler But seriously, the reason is that I store a lot of what goes on with all objects in the grammar object from which they are derived.
23:41 jeffreykegler For example, it's a very convenient place to keep the error status, particularly for the sort of error which makes the whole inter-connected system of objects questionable.
23:42 jeffreykegler This works fine in a single thread, but it's non-constant data shared between objects, so all bets are off if you run in multiple threads.
23:43 jeffreykegler A fix wouldn't be hard -- just allow the cloning of grammar objects.  But
23:43 jeffreykegler 1.) it's almost as easy to re-create them from scratch.
23:43 jeffreykegler 2.) There's no real demand for it.
23:44 jeffreykegler Kollos, by the way, will probably allow the cloning of grammars.
23:44 koo6 the sort of error which makes the whole inter-connected system of objects questionable?
23:45 koo6 just trying to understand
23:45 jeffreykegler Internal fatal errors like "widget-gizmo mismatch" :-)
23:46 koo6 :)
23:46 jeffreykegler Basically, I'm talking about non-recoverable errors.
23:47 jeffreykegler If it's non-recoverable for a recognizer, it may well not be recoverable for the grammar either.
23:47 koo6 let me reword, how do two threads nonconcurrently calling something thread-safe pose a problem?
23:47 jeffreykegler "noncurrently"?
23:47 jeffreykegler Like two years apart? :-)
23:47 koo6 yes, like with a lock system
23:48 jeffreykegler Locks won't help
23:49 jeffreykegler In this case, all they do is synchronize your race conditions.
23:49 * koo6 scratches head
23:50 jeffreykegler For locks to work, the application has to have some idea of an "atomic transaction", which the locking is aware of.
23:52 jeffreykegler koo6: make sense?
23:52 koo6 absolutely not!
23:52 jeffreykegler OK.  Do the following exercise:
23:53 jeffreykegler Imagine a very simple race condition in your head -- or work it out on paper.
23:53 jeffreykegler Now have the two threads lock and work it out again.  What's the improvement?
23:53 jeffreykegler The answer is nothing.
23:54 jeffreykegler Locking may change the timings, and perhaps make the race condition less likely, but if it is not transaction-aware, it won't prevent it.
23:55 koo5 "A race condition occurs when two or more threads can access shared data and they try to change it at the same time"
23:55 jeffreykegler And if you're programming mission-critical stuff, making race conditions less likely during development makes your software not less dangerous, but more dangerous.
23:55 koo5 what if the two threads never even co-exist at any one time?
23:57 jeffreykegler koo5: recall there were race conditions before there were multi-processor systems.
23:57 koo5 hmhm
23:57 jeffreykegler In a single processor system, no access can take place strictly "at the same time".
23:58 jeffreykegler Yet, we programmers of days gone by, even with the limited hardware of the day, managed to write race conditions.
23:58 koo5 thats not the kind of time we are talking about, tho
23:58 jeffreykegler :-)

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary