Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2017-02-09

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:48 ilbot3 joined #marpa
02:48 Topic for #marpa is now Start here: http://savage.net.au/Marpa.html - Pastebin: http://scsys.co.uk:8002/marpa - Jeffrey's Marpa site: http://jeffreykegler.github.io/Marpa-web-site/ - IRC log: http://irclog.perlgeek.de/marpa/today
04:05 ronsavage joined #marpa
05:04 idiosyncrat joined #marpa
05:14 idiosyncrat ronsavage: re https://irclog.perlgeek.de/​marpa/2017-02-08#i_14068054 -- Thanks!
05:15 idiosyncrat I'm back online, and able to thank you.
05:16 idiosyncrat I got a glimpse of CPANtesters which looks as usual, all PASS except a "Dirty Dozen" from Andreas' sites
05:17 idiosyncrat This latest release is another refactoring release, but the last of its series (I hope)
05:17 idiosyncrat I will now move on to "eager" lexemes.
05:18 idiosyncrat As of this last release no Marpa grammar data is kept in XS -- it has all been moved, sometimes to Perl, more often into Lua code.
05:19 idiosyncrat (Actually very little data is still managed by XS, all associated by the SLIF recce.)
05:19 idiosyncrat The XS file has passed a milestone -- it is now less than half the size it was at its peak.
05:20 idiosyncrat It will get smaller yet.
05:21 idiosyncrat And the remaining XS code will be basically a sort of customized Lua interface, along the lines of Lua::API, etc.
05:47 idiosyncrat Good night!
07:59 sirdancealot joined #marpa
15:06 sirdancealot joined #marpa
15:15 kaare_ joined #marpa
16:39 idiosyncrat joined #marpa
16:52 kaare__ joined #marpa
17:48 idiosyncrat hobbs: Since I'm in the lexer, I've been thinking about AATM.
17:48 idiosyncrat The problem I have with it'
17:49 idiosyncrat its implementation is that the most obvious lexing logics rely on all recognized lexemes starting and ending at a single location.
17:49 idiosyncrat AATM preserves that property for start locations, but not end locations ...
17:49 idiosyncrat which raises a syncing issue.
17:50 idiosyncrat I'm hard put to think how to package AATM in a way that's useable.
17:50 idiosyncrat Libmarpa currently allows AATM, but AFAIK nobody has attempted anything like it.
17:50 hobbs yeah, my driver for it did work as far as I can tell, but at the very least it makes a ton of work
17:50 hobbs (for the computer)
17:51 idiosyncrat What was your target application?
17:51 idiosyncrat Forgive me, I haven't looked back at your talk recently.
17:52 hobbs the TAP parser (the same thing where I did the two-layer thing) also used an AATM lexer written in Perl
17:53 hobbs https://metacpan.org/source/ARODLAND/TAP-Sp​ec-Parser-0.10/lib/TAP/Spec/Parser.pm#L199 through 264, more or less
17:53 hobbs may be wildly incompatible with anything current :)
17:54 hobbs "MarpaX::Lex::Easy" was also an attempt to generalize it that never got beyond a single TRIAL release.
17:56 idiosyncrat I take it neither of these got far enough to conceptualize around the problem of syncing.  Or did they?
17:57 idiosyncrat Because it's easy enough for me to change the lexer to accept all the various length lexemes, but then which one (or which ones) does the app choose?
17:58 idiosyncrat And if the app chooses more then one and they differ in length, how do I track that for the app?
17:59 hobbs TAP::Spec::Parser works. That was my initial pleasant surprise, that it does.
18:00 hobbs I give the recognizer multiple ->alternative()s with different lengths, and poll it at each character position for what new tokens can possibly start at that position, and it appears to juggle everything correctly
18:01 hobbs in the end you may or may not have multiple parses, but that's already a known thing
18:02 idiosyncrat Is the upper layer forced to choose a single end location (== length)?
18:04 hobbs each token has a known length, but multiple tokens starting at the same location can have different lengths
18:04 idiosyncrat What does the upper layer (== app) do to keep track of where it is?
18:05 idiosyncrat To be specific, suppose I find lexemes <a>, <b> and <c>, of lengths 2, 3, and 42.
18:05 idiosyncrat Can the app accept all 3?
18:05 hobbs yes
18:06 hobbs for example in TAP::Spec::Parser, given the input "1..0" it will tentatively accept ONE_DOT_DOT (length 3), ONE_DOT_DOT_0 (length 4), and Positive_Integer (length 1) at that point
18:06 idiosyncrat So does it, in effect, start 3 threads at the point (conceptually, if not actual threads)?
18:07 hobbs yes, but it's not using anything except the Marpa R2 stuff that was documented at the time to do so
18:07 idiosyncrat Basically, threads done without threads. :-)
18:08 * idiosyncrat recalls that Earley's algorithm tracks multiple "threads", but uses tables instead of threading.
18:09 hobbs it just calls ->alternative('ONE_DOT_DOT', '1..', 3); ->alternative('ONE_DOT_DOT_0', '1..0', 4); ->alternative('Positive_Integer', 1, 1); ->earleme_complete;
18:09 hobbs and then 3 characters later, ->terminals_expected gives things that could appear after ONE_DOT_DOT; four characters later, ->terminals_expected gives things that could appear after ONE_DOT_DOT_0, etc.
18:10 idiosyncrat OK, so it's a character-by-character model.
18:10 hobbs yes
18:10 hobbs which is very brute force, I understand
18:10 sirdancealot joined #marpa
18:10 idiosyncrat Yes, it's hard to picture a lot of people doing that.
18:12 * idiosyncrat knows it's early to bring up efficiency, but also when the tracking is per-character rather than per-lexeme, all the overhead is per-character, and increases significantly.
18:13 hobbs you might be surprised. Look at the things on CPAN that use Parse::RecDescent because it was the easiest thing to use, the input isn't all that big, performance isn't all that big, performance isn't all that crucial, etc.
18:13 hobbs this still probably has a better worse case than P::RD
18:13 hobbs I'm not saying you *have* to cater to that, but that's the angle I'm coming from
18:15 idiosyncrat Interesting, I'll bear this in mind.
18:15 hobbs basically, if you have a switch that says "use this, and spend less time thinking about your lexer, at the cost of much worse performance; you can tune up your grammar and turn it off later for more speed", people will probably buy it
18:16 hobbs in line with the mantra of "make something work, then make it right, then make it fast"
18:16 idiosyncrat But the character-by-character model also seems to require more thought on the user's part, or do I get that wrong?
18:20 hobbs I don't think so. The extra thought is in the driver, not in the user input
18:22 hobbs the code in TAP::Spec::Parser drives Marpa in a complicated way, but it was also written before SLIF even existed
18:23 idiosyncrat By user I've been meaning the guy who writes the parser (who is the "user" from my point of view)
18:23 idiosyncrat Have you been meaning the user of the parser?
18:23 idiosyncrat That is, I've meant the user of Kollos, the parser generator, not the user of the generated parser.
18:24 hobbs no, I mean the parser writer
18:24 hobbs my hope would be that you would be able to access the same thing just by writing DSL and calling ->parse
18:24 idiosyncrat That would be cool.
18:24 idiosyncrat I'm having a very hard time thinking how I could deliver on it.
18:25 hobbs I'm probably not much help in that department
18:26 hobbs but if you want to pick my brain any further on how my old code works, I'll do my best there :)
18:28 hobbs and if it turns out to be impractical -- hey, I'm just making a suggestion :)
18:28 idiosyncrat If you feel inspired, a new example using the SLIF might make my synapses fire a little more readily.
18:28 hobbs I'll see what I can do. $WORK first.
18:29 idiosyncrat I've appreciate the suggestion -- as you know your work has inspired a lot of what's in Marpa::R2
18:32 idiosyncrat hobbs: you may want to try it in Marpa::R3 -- perhaps some of the new features will be inspiring.
18:37 ceridwen joined #marpa
19:00 sirdancealot joined #marpa
19:32 sirdancealot joined #marpa
21:23 ronsavage joined #marpa
22:59 sirdancealot joined #marpa

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary