Perl 6 - the future is here, just unevenly distributed

IRC log for #marpa, 2017-12-02

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:05 idiosyncrat joined #marpa
03:35 ilbot3 joined #marpa
03:35 Topic for #marpa is now Start here: http://savage.net.au/Marpa.html - Code paste/run: https://f.perlbot.pl/#marpa - Jeffrey's Marpa site: http://jeffreykegler.github.io/Marpa-web-site/ - IRC log: http://irclog.perlgeek.de/marpa/today - Youtube channel: https://www.youtube.com/channel/UCYKVfGBtfTqbs1JdYq-dc5g
03:44 idiosyncrat Demat!
06:15 ronsavage joined #marpa
19:06 Cheery hello
19:07 Cheery I feel I've figured out a nice way to implement indentation sensitivity into a programming language
19:08 Cheery one that allows the indentation to not interfere with rest of the parsing, and allows ambiguity to be recognized despite it.
19:11 Cheery to implement it, you add marks into the token stream.
19:12 Cheery each mark holds the indentation level of the line that continues it.
19:12 Cheery additionally each mark knows whether the indentation increased (+), decreased (-), or stood the same (=)
19:13 Cheery the (+) mark can generate indent -tokens.
19:14 Cheery the (=) mark must generate newline -token, if the last (+) token of the same indentation generated an indent token.
19:17 Cheery the (-) may close indent -tokens that have higher indent. It may also generate dedent tokens for them.
19:18 Cheery and the (-) must generate newline -token if the last (+) token of the same indentation generated an indent token.
19:22 Cheery to use this in naive manner, you'd have to split the parsing state at every indent
19:22 Cheery because the format of the stream changes depending on whether you take an indent -token or not.
19:23 Cheery but I think one can be smarter about this if he introduces a parsing element.
19:23 Cheery block(a)
19:24 Cheery generates a special rule with 'a' in it.
19:24 Cheery if there is indentation present, the parser may shift along with this rule
19:24 Cheery and it produces a special kind of an item.
19:25 Cheery for every indentation position we record a marker, the markers sit in a stack.
19:26 Cheery when newline occurs, we will try to ruby slipper shift with the matching (=) marker
19:27 Cheery if there's a block(a) in the stream collecting up stuff, with the same marker
19:27 Cheery then it will shift forward.
19:27 Cheery pushing a matching dedent token will equivalently cause it to reduce.
19:27 Cheery erm well.. complete
19:29 Cheery the nice thing about this is that I think it also works for LR parsing the similar way.
19:31 Cheery also, if the language does not use this feature then I think the parser with the mechanism will work in ordinary manner.
20:28 idiosyncrat joined #marpa
20:37 Cheery idiosyncrat: sup?
20:46 kaare_ joined #marpa
21:01 idiosyncrat Cheery: cool!  It'd be interesting to see an implementation, maybe even with a blog post.
21:02 Cheery idiosyncrat: I can step through and explain how it happens, with item sets.
21:04 Cheery lets say you got a grammar: S -> identifier block(A), A -> identifier
21:04 Cheery now you start with . S
21:05 Cheery you get S -> . identifier block(A)
21:05 Cheery shift with identifier
21:05 Cheery so into the next chart column you'll get
21:05 Cheery S -> identifier . block(A)
21:06 Cheery now to advance you will require an 'indent' token witha mark.
21:06 Cheery the mark represents indentation depth
21:06 Cheery and is unique for this indent token
21:07 Cheery so you shift with indent(a), where a is the new mark
21:08 Cheery you will get S -> identifier {}. block'(A)
21:08 Cheery you get A -> . identifier
21:08 Cheery shift with identifier
21:09 Cheery you'll get A -> identifier ., it reduces and you shift with A from previous column, the block'(A) will shift
21:10 Cheery so you get. S -> identifier{A}, newline(a) block'(A)
21:10 Cheery and of course. S -> identifier {A} . dedent(a)
21:11 Cheery shift with dedent(a) and your item completes.
21:12 Cheery hmm..
21:12 Cheery since this is this way.. I think that an explicit dedent token shift is not even required
21:12 idiosyncrat Cheery: btw, have you seen http://michaeldadams.org/papers/layout_parsing/LayoutParsing.pdf ?
21:14 Cheery idiosyncrat: I've seen this at least once
21:16 Cheery this seem to propose to solve the problem with grammar rewriting?
21:18 idiosyncrat Actually, I haven't read it in a while.  I left it because I had no time to work on the problem, and probably will not in anything like the near future.
21:19 idiosyncrat Almost certainly if you implement and/or write up your idea , first thing readers will ask is "How does it compare with the Adams approach?"
21:30 Cheery this seems quite simple as well.
21:30 Cheery it will require layout check on every token, and your every EIM needs additional column value.
21:32 Cheery I think that.. adam's approach is something like generalization in compared to mine.
21:35 Cheery in that syntax the above example is something like..
21:36 Cheery S -> identifier(=) B(>=)
21:36 Cheery B -> A(=) | B(=) A(=)
21:45 Cheery idiosyncrat: you think Adam's method can be used with marpa?
21:45 Cheery how about.. I'll implement it? :)
21:45 Cheery or at least try
21:46 * Cheery is still using his own implementation of marpa
21:49 Cheery idiosyncrat: so far I've been still using a rather simple python-like step to insert indent/dedent tokens into the stream when the parser expects them.
21:51 Cheery the problem with this approach is that it removes some of the nice advantages there are in having a language with true context-free-grammar.
21:53 Cheery it's like a step back into hand-written parsers.
21:53 idiosyncrat Cheery: I didn't get far enough thinking about the Adams paper to set it in the Marpa context -- I expect you could implement it in R2 using events, but whether there's a better way, I just didn't think out.
21:54 idiosyncrat Didn't Adams method allow automatic rewrite?  Internally Marpa::R2 uses rewriting and it's also fully context-free.
21:54 idiosyncrat Marpa::R3 will use even more internal rewriting.
21:54 idiosyncrat Good luck with your explorations!
21:57 Cheery idiosyncrat: good luck to you too. I also have an another fairly big problem with my language that I have to figure out
21:57 Cheery but I don't even have a clue how to start unraveling it
21:58 Cheery it has to do with the idea that I'd like to allow type promotions like there are in C
21:58 Cheery but in extensible manner
21:58 idiosyncrat Cheery: When I started Marpa 10+ years ago I have very clear ideas where I was going and how long it would take.  However most of those ideas were wrong. :-)
22:00 Cheery idiosyncrat: I have some of that too. It was simple when I was still trying to just make a nice dynamically typed language. Then I went to realise I need the static form of this language as well.
22:01 Cheery found a nice paper to subtyping, implemented it. Now I'm about almost finished with getting a SPIR-V output from the same code that I can run dynamically.
22:02 Cheery but the exploration into that subject made me realise lot of things.
22:03 Cheery one idea was this thing about numerical systems.
22:03 Cheery and lattices
22:04 Cheery if you allow anyone to do x + y, freeform operator overloading
22:05 Cheery in the best case they only do x + x or y + y overloads
22:06 Cheery in the worst case they will get 12 types, and make them all interact together in surprising ways. they'll have 12*12 method implementations to do this.
22:06 Cheery (144 methods)
22:08 Cheery the freeform operator overloading just shifts the burden of designing numerical systems
22:08 Cheery from the compiler&language writer to the user
22:08 Cheery into a position where the original problem is harder to solve
22:10 Cheery languages with operator overloading lack proper polymorphism of numerical systems.
22:11 Cheery an additional problem is that I'm not sure how to explain this well.
22:11 Cheery I could ramble about it for hours.

| Channels | #marpa index | Today | | Search | Google Search | Plain-Text | summary