Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2016-04-13

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:38 ronsavage joined #marpa
01:48 ilbot3 joined #marpa
01:48 Topic for #marpa is now Start here: http://savage.net.au/Marpa.html - Pastebin: http://scsys.co.uk:8002/marpa - Jeffrey's Marpa site: http://jeffreykegler.github.io/Marpa-web-site/ - IRC log: http://irclog.perlgeek.de/marpa/today
02:46 ronsavage joined #marpa
03:17 ronsavage joined #marpa
03:58 ronsavage joined #marpa
04:46 rns idiosyncrat: re http://irclog.perlgeek.de/m​arpa/2016-04-12#i_12325116 -- ok will take it.
04:47 rns re http://irclog.perlgeek.de/m​arpa/2016-04-12#i_12325121 -- good point, note taken.
04:47 rns re parallel -- need to think how to do it properly.
04:57 idiosyncrat_ rns: Thanks!
04:58 idiosyncrat_ We're down to 20 tests that need "rework", and we've been going after the hardest first, so I think we're well along.
04:59 idiosyncrat_ I'm going after "messy" cases, like those tests which are about debugging and diagnostics (which most modules don't test!), and also the Leo tests.
05:00 idiosyncrat_ I'll be glad to leave the parallel make issue for you to think about for a while.
05:01 idiosyncrat_ I've been holding off on all the make-oriented pull requests, for one reason or another.
05:01 idiosyncrat_ jdurand: I hope you'll have some time to comment on them.
05:02 idiosyncrat_ After the tests are converted to the SLIF, I will plunge straight into refactoring.
05:03 idiosyncrat_ rns: I think you and I can backport a lot of the interface improvements we'd plotted for Kollos into Marpa::R3
05:03 idiosyncrat_ The reason we did a fresh start with Kollos was that Marpa::R2 needed to be stable, and was a mess internally.
05:04 idiosyncrat_ With Marpa::R3 we can allow ourselves to break compatibility, and we'll clean up the mess (or a lot of it, anyway)
05:06 idiosyncrat_ Already in the tests, I can see that Marpa::R3 is easier and more fun to work with.
05:06 idiosyncrat_ Good night!  -- will backlog, as usual.
05:18 rns idiosyncrat: re http://irclog.perlgeek.de/m​arpa/2016-04-13#i_12327938 -- yes, holding off build stuff until tests are done is a good idea.
05:28 rns re http://irclog.perlgeek.de/m​arpa/2016-04-13#i_12327943 -- that would be great -- I was thinking about
05:28 rns cleaner lexeme and event syntax -- https://github.com/rns/kollos-luif-doc/issues/32 and
05:32 rns ASF-based evaluator with ability to do AST filtering by removing certain paths in ASFs as e.g. in http://www.bramvandersanden.com/p​ublications/sanden2014thesis.pdf
05:36 rns And even releasing THIF as a separate module with suitable examples for folks to write faster parsers with more manual work and possibly build interfaces on top of it.
05:38 rns But, first thing first -- tests and refactoring -- that's for sure.
05:50 rns which, in more practical terms, leads me basically to http://irclog.perlgeek.de/m​arpa/2016-04-12#i_12325116 :)
07:27 ronsavage joined #marpa
08:55 koo7 joined #marpa
11:37 Cheery joined #marpa
11:38 lucs joined #marpa
11:49 koo7 joined #marpa
12:56 rns idiosyncrat (and all): I saw 'Bad start position in slr->g1_lexeme_complete(): 0 message' while external scanning because of missing $recce->read( \$input, 0, 0 ) call before the first $recce->lexeme_complete() call.
12:58 rns Will appending 'perhaps you forgot to call $recce->read(\$input, 0, 0)?' to the above message or otherwise documenting what happens if such call is missing?
12:59 rns Because, frankly, figuring out the cause took some time for me. :)
13:07 rns As the input in Marpa starts at position 0, seeing 0 as bad start position was, well, surprising. :)
13:51 shadowpaste joined #marpa
14:15 idiosyncrat_ joined #marpa
14:16 idiosyncrat_ rns: re the SLIF and its non-support of token straddling ...
14:16 idiosyncrat_ as I said this feature proved far too arcane for my user base.
14:16 idiosyncrat_ Also, it's not clearly necessary or useful ...
14:19 idiosyncrat_ minus.t was perhaps the only use case I found for it, but look at what I did without it in sl_minus.t (which now passes all its tests):
14:19 idiosyncrat_ https://github.com/jeffreykegler/Marpa--R3/co​mmit/24d9e0ecfb5ba23fb055083a6a3e8a9149b7c7fd
14:20 idiosyncrat_ I think that new code the avoids using the token straddling feature may be simpler, clearer and more effective.
14:20 teatime joined #marpa
14:21 idiosyncrat_ And this was for a test that was designed to illustrate the use of token straddling.
14:23 idiosyncrat_ As context for the rest of the list, Libmarpa allows tokens to overlap, or straddle, each other.  The SLIF lets them be variable length, but every token starting at one position must have the same length, so there are no overlapping tokens under the SLIF.
14:23 rns Yes, I can see it now -- all the more I've just did the same thing porting randal.t :) Thanks for explaining.
14:42 rns Or, to be honest, not quite 'did', but 'trying to do' -- parse count and parse value tests still fail and the second test doesn't parse yet.
15:04 idiosyncrat_ rns: you could take randal.t to the same point that you did with minus.t, then kick it back to me.
15:05 idiosyncrat_ Some aspects are easier if you know what the test was trying to do, and that's easier for me to figure out.
15:08 rns idiosyncrat: that's what I'm tempted to do :) Still this time I understand what the test does (well, sort of) and thus have a question.
15:09 rns randal.t uses $recce->terminals_expected() to test for expected symbols (in NAIF terminology).
15:11 idiosyncrat_ ?
15:12 idiosyncrat_ Is there no equivalent in the SLIF?
15:12 rns Well, there is and even are ....
15:13 rns predicted events on G1 symbols and pre-lexeme (L0 symbol) events
15:13 rns ...
15:14 rns Now I use the former by defining symbol ::= symbol_lex setting predicted event on it and then symbol_lex ~ ...
15:15 idiosyncrat_ OK
15:15 rns I just thought pre-lexeme events are not equivalent to expected/predicted because they trigger when a lexeme is already matched in the input
15:15 rns not equivalent in this case, that is.
15:16 idiosyncrat_ Correct
15:17 idiosyncrat_ Prediction events are allowed for lexemes, btw
15:17 rns prediction in the sense that they trigger when lexeme is expected, but not matched?
15:19 idiosyncrat_ yes
15:19 rns So I can write lexeme ~ [/] and then event '^lexeme' = predicted lexeme ?
15:19 idiosyncrat_ I believe so.  That was the intent.
15:20 idiosyncrat_ http://search.cpan.org/~jkegl/Marpa-R2-3​.000000/pod/Event.pod#Prediction_events
15:20 rns Just read it, thanks a lot.
15:21 rns Ok, I'll convert my conversion of randal.t to using them and file a PR.
15:21 idiosyncrat_ That's why there are both pre-lexeme and prediction events for lexemes.
15:21 rns Yes.
15:21 idiosyncrat_ Sometimes you want an event before the lexeme, but only if it's actually there ...
15:22 idiosyncrat_ and sometimes you want a genuive prediction, which triggers when the lexeme *might* be there, before Marpa can know whether it will actually show up
15:34 rns Yep, just did and it worked beautifully and even removed an unproper nullable symbol in the grammar.
15:39 rns PR filed -- https://github.com/jeffreykegler/Marpa--R3/pull/14
15:40 rns I'll be able to take more test cases for conversion (well, initial setup to it in most cases) in a day, so let me know.
15:46 rns BTW, for this particular use case (matching lexemes inside one another) token straddling is arguably more elegant solution.
15:48 rns But I haven't yet seen how you'd do it si I well could (would be glad to, actually) be proven wrong.
15:48 rns s/si/so/
16:27 rns idiosyncrat: BTW, one use case I can imagine for token straddling are *very* smart comments -- which know how to end themselves at any valid language construct, even with delimiters like pattern match  -- // as in randal.t
16:32 rns So that they can be added as a statement alternative -- looks like that was the story behind randal.t, was it not?
17:52 kaare_ joined #marpa
18:02 kaare_ joined #marpa
18:08 kaare_ joined #marpa
18:41 lucs joined #marpa
19:40 maybekoo2 joined #marpa
19:53 Cheery_ joined #marpa
19:57 Cheery joined #marpa
19:58 idiosyncrat_ AFK
20:42 lucs_ joined #marpa
20:48 aredridel joined #marpa
21:34 Pursuit joined #marpa
21:49 lucs joined #marpa
21:54 maybekoo2 joined #marpa
23:05 idiosyncrat_ joined #marpa
23:07 ronsavage joined #marpa
23:07 ronsavage I've add an FAQ re prediction 'v' pre-lexeme events: http://savage.net.au/Perl-modul​es/html/marpa.faq/faq.html#q138
23:15 idiosyncrat_ ronsavage: Thanks!
23:16 idiosyncrat_ I'm hacking on the sl_randal.t test and it's been instructive.
23:17 idiosyncrat_ rns: I deleted the logic that tests $slr->terminals_expected vs. those generated by events -- IIRC correct when I added it to randal.t, I put it there because I didn't want to create a new test and randal.t was (so I thought) the closest fit.
23:17 idiosyncrat_ By the time I realized that was a bad idea, I had the test working so I left it there.
23:18 idiosyncrat_ Don't know what to do about $slr->terminals_expected vs. those generated by events -- we could create a new test file, and maybe our coverage elsewhere is now more than enough.
23:19 idiosyncrat_ Also, I'd been close to deciding to remove overlapping tokens from Libmarpa, but hacking sl_randal.t, I realize they *do* have a use.
23:20 idiosyncrat_ rns: do you want to find and put on this channel links to our latest thinking on the Kollos external interface?
23:20 idiosyncrat_ We're getting to the point where it's time to start talking new interface.
23:21 idiosyncrat_ Btw, for those with interface ideas, the best, if hardest, way to get your idea across is with a prototype.
23:22 idiosyncrat_ The SLIF grew out of a prototype by Peter Stuifzand ...
23:23 idiosyncrat_ and its two-layer lexing strategy was first implement by hobbs: Andrew Rodland.
23:23 idiosyncrat_ So prototypes have had radical effects on the design of Marpa interfaces in the past.
23:50 idiosyncrat_ === AATM: All Acceptable Tokens Match ===\
23:52 idiosyncrat_ A while ago someone (I apologize for forgetting who, and would be grateful if someone can remind me) asked "Why not have a lexer option to return *all* acceptable tokens regardless of length".
23:53 idiosyncrat_ I kind of brushed this idea aside at the time, as it seemed bizarre, not to mention that with Marpa::R2 becoming stable, it was unimplementable anyway.
23:53 idiosyncrat_ But I now realize it's the equivalent of allowing overlapping tokens, and would be the way to implement that feature of Libmarpa within the SLIF.
23:55 idiosyncrat_ An obstacle to implementing overlapping tokens had been that its details are difficult, if described formally: see http://jeffreykegler.github.io/Marpa-web-sit​e/libmarpa_api/stable/Earlemes.html#Earlemes
23:57 idiosyncrat_ And I could not think of any way of adding it to the SLIF, without adding many difficult paragraphs to the documentation, which is plenty challenging enough.
23:57 idiosyncrat_ But I think in context of an AATM option, that things from the point of view of the SLIF programmer "just work".
23:58 idiosyncrat_ ... and the whole thing becomes intuitive.

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary