Perl 6 - the future is here, just unevenly distributed

IRC log for #perl6, 2008-02-10

Perl 6 | Reference Documentation | Rakudo

| Channels | #perl6 index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:09 dalek joined #perl6
00:09 pugs_svnbot joined #perl6
00:09 Juerd joined #perl6
00:10 Juerd Okay, so feather *is* vulnerable. Anyone can haz root.
00:10 pugs_svnbot r19865 | jnthn++ | [spectests] Start to lay out some S12 tests, plus add a couple of very basic ones.
00:10 pugs_svnbot diff: http://dev.pugscode.org/changeset/19865
00:10 lambdabot Title: Changeset 19865 - Pugs - Trac
00:10 pugs_svn r19865 | jnthn++ | [spectests] Start to lay out some S12 tests, plus add a couple of very basic ones.
00:10 Juerd Exploit and die :)
00:14 jonathan1 joined #perl6
00:19 pugs_svn r19866 | jnthn++ | [spectest] Fix typo in methods tests.
00:19 pugs_svnbot r19866 | jnthn++ | [spectest] Fix typo in methods tests.
00:19 pugs_svnbot diff: http://dev.pugscode.org/changeset/19866
00:19 lambdabot Title: Changeset 19866 - Pugs - Trac
00:20 jnthn The bots! They're everywhere!
00:20 jferrero joined #perl6
00:23 rakudo_svn r25613 | jonathan++ | [rakudo] Start getting the S12-class spectests.
00:24 wolverian joined #perl6
00:29 thoughtp1lice joined #perl6
01:15 TimToady mncharity: for the bootstrap, yes, there's only one grammar; otherwise, no, there could be any number of different Perl grammars floating around.  The Perl grammar is just the base grammar from which many other grammars derive.
01:15 TimToady and STD is (mostly) written for the standard parser, not the bootstrap parser
01:17 TimToady even in the boostrap grammar, we can't assume that the first term is matched by expect_term, because it has to be mixed in with expecting statements in some spots
01:17 TimToady hence the $seen parameter
01:18 TimToady the grammar's structure is very much driven by the need to keep all alternatives out where the autolexer can see the longest tokens
01:18 TimToady so we have to make sure statements and terms are matched in the same | set
01:19 TimToady (either that, or make EXPR smart enough to report in advance what set of regex would match as a longest token; but this way seemed easier)
01:20 TimToady but all of the metholated methods are smart enough to report their longest token set
01:21 TimToady that's what I was playing with when I got bogged down with shoulder trouble
01:21 TimToady (and in fact I think metholate doesn't even put the gather/take code out there at the moment because I was just playing with the longest token harvesting)
01:27 TimToady well, okay, in the bootstrap grammar we could probably get away with <statement_control> || <expect_term>
01:27 TimToady but for real life it has to be |
01:28 TimToady BTW, the *really* fun part of generating the longest-token set is figuring out which hyperoperators are valid with getting into recursive tokens :)
01:28 TimToady s/with/without/
01:38 pasteling "TimToady" at 71.139.7.101 pasted "notional longest-token table for expecting a term (not well-quoted)" (790 lines, 20.2K) at http://sial.org/pbot/30248
01:39 TimToady I generated the table above for the fun of it several months ago
01:40 TimToady and it was generated "simply" by asking expect_term what it expected
01:41 TimToady the bits on the right of each pattern are the "fates" that get passed into rules that have already had their longest token matched externally
01:44 TimToady anyway, that whole thing turns into something like a tagged DFA
02:05 mncharity I'm still missing something.  Indeed when one has lots of Perl instances, and lots of Perl-like grammars inheriting from Perl, themselves with lots of instances... isn't distinguishing among them still the job of self() ?   why the introduction of an additional mechanism?
02:10 TimToady what additional mechanism?
02:12 TimToady can you point me at some suspect code?
02:16 TimToady or are you worried about the longest token matching itself?
02:17 TimToady or the fact that longest-token matching predetermines the dispatch of subrules?
02:17 TimToady I don't know what you're uneasy about
02:22 mncharity hi, let's see
02:24 mncharity I remain unclear on why EXPR is using $here.expect_term  rather than  self.expect_term.  since self() has the search state, of a specific grammar instance, of a specific grammar derived from Perl.   so why $here?
02:25 TimToady self is the search state at the start of the expression, it's not the search state in the middle of the expression
02:25 mncharity !
02:27 mncharity so self() has a strpos, and then some other things have their own copies of strpos?
02:28 TimToady All of the methods in the grammar basically work that way...if you look at Cursor.pm at something like _PLUSr you'll see the same thing happening
02:29 TimToady you get a self in at the top, but for the life of the method the current match state lives in $to
02:29 mncharity (I had been thinking  'entire search state' == 'grammar instance self() + +env vars + ...matches kicking around somehow'.  no?)
02:29 Lorn joined #perl6
02:30 TimToady yes, if you want to actually write methods that match things, you have to keep track of string positions exactly at each moment
02:30 TimToady and the string position changes at the end of every possible way a* can match
02:30 mncharity ok +anything in frame vars on the call stack.
02:30 mncharity hmm... /me ponders
02:31 TimToady anyway, Cursor.pm basically *works*, but it's just too slow in pugs without predeciding the longest tokens
02:31 TimToady which is why I was working on that part
02:32 mncharity (/me doesn't yet understand what exactly Cursor is doing, and its relationship to everything else)
02:33 mncharity (/me also doesn't yet understand the difficulty involved in "predicting the longest token".  RxHash.longest_token_match in prelude.rb was 3 lines.)
02:34 meppl good night
02:35 TimToady if you write a separate lexer with known tokens, yes, it's easy
02:35 TimToady we're not writing a separate lexer, we're getting the grammar to write its own lexer
02:35 justatheory joined #perl6
02:35 TimToady and we don't know the set of tokens in advance because we don't know the grammar in advance
02:36 TimToady all we know is that by the time we get to a set of | alternatives, if we know exactly what language we're in, we can cache that token set from then onward
02:36 TimToady for that language
02:36 mncharity redsix stored pos in a single StringScanner object.  which slid back and forth with the search and backtracking.  yare stored pos as a local() var, which p5 took care of unwinding on backtracking.  Cursor... I'm not yet clear on.  I started red_STD on the pos-in-StringScanner plus jiggling model.
02:36 justatheory joined #perl6
02:38 TimToady the cursor engine doesn't really do backtracking exactly, or rather, it doesn't do it by returning explicitly
02:38 TimToady instead, it relies on lazy lists of choice points, and you backtrack simply by taking the next gathered option
02:39 mncharity ah, ok
02:39 TimToady so the continuations are implicit to gather/take
02:39 TimToady makes it very clean, actually
02:39 TimToady and if you assume that gather/take is efficiently implemented, it won't waste any energy backtracking if it doesn't need to
02:40 TimToady obviously there's some setup/teardown for gather/take, but if you only ever use the first item in the option list, you never go back to the continuation
02:40 TimToady and evenutally it just gets GC'd
02:41 TimToady also obviously, if you can avoid taking the gather/take hit in the first place, that's even faster
02:41 TimToady so :ratchet will eventually use "andthen" instead
02:41 mncharity duh.  click.  so the STD stuff doesn't have explicit continuation information.  so that's all being maintained by Cursor-ness, which is getting folded into Match objects?
02:41 TimToady but I ran into troubles with that under pugs
02:42 mncharity hmm, so why isnt MatchContinuation a real class?
02:42 mncharity MatchCursor
02:43 TimToady where are you finding that?
02:43 mncharity s/doesn't have/isn't passing/
02:43 TimToady I used to pass a second parameter with the explicit continuation, then I decided it was simply part of the self along with the grammar
02:44 TimToady made the code a lot simpler
02:44 Auzon joined #perl6
02:45 mncharity re MatchCursorGrammar, no where, you mentioned that $here contained both Match and Cursor and Grammar info...
02:45 TimToady that's why when you metholate STD it turns 'grammar Perl' into 'grammar Perl is Cursor'
02:46 TimToady so all derivatives of the grammar know how to actually run a match
02:46 TimToady and it just comes through as self (and $here, $to, whatever)
02:47 mncharity One part I'm still missing is how self() in methods can be anything other than the instance object on which they are called, namely, a grammar instance.  so the concept that self() can have non-grammar instance stuff...?
02:47 TimToady but that means all these methods return one of these objects, and the actual match info is embedded in the history
02:47 TimToady when we make the transition back from method view to rule view, that's when it looks like a Match sub object again
02:48 * mncharity ponders
02:48 TimToady also, we have to set up $/ when calling user closers
02:48 TimToady closures
02:48 TimToady grammar Perl is Cursor;  Cursor carries all the moment-to-moment match state
02:49 mncharity comprendo
02:49 TimToady see all the has declarations at the start of Cursor.pm
02:49 TimToady and $.prior is the chain of prior cursurs that need to be visited to construct Match
02:50 mncharity bleep.  prior cursors?  one is stashing a stack of Grammars?
02:50 TimToady the reason .pos is separate from .from and .to is so that we can do backward matching for <before>
02:51 TimToady a list of successful matches, some of which contain bindings to names or $1
02:51 TimToady which have to be unwound if we backtrack, but that happens automatically by throwing away $here values
02:52 TimToady because the next thing you pull from a gather will have the previous state, and not any subsequent state
02:52 TimToady these are all immutable values, basically
02:52 mncharity the role (no pun intended) I've seen so far for Cursor so far is in "grammar Perl is Cursor;".  there are other uses of Cursor?
02:53 TimToady I don't necessarily envision any
02:53 TimToady but I am singularly lacking in vision at times :)
02:53 TimToady nevertheless, this applies to all pattern match in Perl 6, not just grammars
02:53 mncharity so a chain of prior Cursors is actually a chain of Perl grammar instances, no?
02:53 TimToady I want /a|b|c/ to automatically do DFA matching to the extent that it can
02:54 TimToady and that just falls out of longest token
02:54 TimToady yes, but the Perl part of the state object currently contains on $.ws_from and $.ws_to
02:55 TimToady mostly so we can cache if we already matched a whitespace here
02:56 mncharity (I'm still thinking of DFA'ness and all other optimizations as something a "sufficiently smart compiler" does transparently, at compile or run time, by groveling over the final grammar instance.  rather than an emergent property of how the grammar is implemented.)
02:56 mncharity (so all the DFA-etc remarks get "bleeped out" for me as "compiler optimization I don't have to worry about implementing)
02:56 mncharity (I'm not sure that's the right model or not:)
02:57 TimToady as with most tradeoffs in dynamic languages, doing it at the *last* possible moment conveys some big benefits
02:57 TimToady and the last possible moment is right when the | is first called
02:58 TimToady just as method dispatch is based on the class hierarchy and methods we see *right now*
02:58 TimToady this is just a slightly smart form of dispatch tuned for pattern matching
02:59 TimToady and in fact, ties into normal multiple dispatch also if the rules methods have extra args
02:59 mncharity re "Perl part of the state object", hmm...  "grammar Perl is Cursor;"  not  "class Grammar is Cursor;"?
02:59 TimToady sure, maybe someday
03:00 TimToady I just don't put in levels of indirection where they aren't needed yet, except where they are... :)
03:01 TimToady declarators like "grammar/class/module" are also just macros really
03:01 mncharity :).  so every time you say 'Cursor' above, you are really saying 'instance of Grammar'.  so prior is a list of instances of Grammar.
03:02 mncharity a Cursor is never instantiated, except as inherited by a Grammar.
03:02 TimToady yes, and note that it might even be a different grammar as we go back along the chain
03:02 mncharity s/as inherited/via inheritance into/
03:03 TimToady well, Cursor could be used by something other than grammar in theory
03:03 TimToady the "grammar" level is there more for humans than the computer currently
03:04 TimToady it's more like "parser" really
03:04 mncharity to what extent could grammar instances be immutable?  ie, what is handled by cloning a new instance, and what by assignment?
03:05 TimToady Cursor does everything by cloning
03:06 TimToady assignment just means you've changed a state that might not backtrack correctly unless you figure out how to back it out
03:07 mncharity (a few days ago, I thought of a Grammar as an abstract specification of a (CS) grammar.  so you analyzed and massaged it, and used something kind of separate to run it.)
03:07 TimToady (though doubtless there are places one can cheat on that...)
03:08 mncharity (then a day or two ago, I thought of a Grammar as mutable search state.  which got banged on by the search process.  and the original abstract grammar was obtained by reflection.)
03:09 mncharity (now Grammar is an instantaneous immutable state-of-search... and I find myself far from where I started, in a strange new land.)
03:10 drbean_ joined #perl6
03:11 mncharity "Cursor does everything by cloning" translates as "Grammar does everything by cloning", yes?  modulo any non-backtracked state which can be done via mutation.
03:12 pugs_svn r19867 | lwall++ | [metholate] put back bodies of matchers
03:12 pugs_svnbot r19867 | lwall++ | [metholate] put back bodies of matchers
03:12 pugs_svnbot diff: http://dev.pugscode.org/changeset/19867
03:12 lambdabot Title: Changeset 19867 - Pugs - Trac
03:13 mncharity Grammar as .y file, vs Grammar as .tab.c, vs Grammar as memory dump of running yacc.
03:14 TimToady I put back the more normal version of metholate so you can see how the actual metholated methods come out (don't forget to cheat first)
03:14 mncharity so now I'm trying to run down the yacc to shave it
03:16 mncharity and I'm having... forget what it's called... predator is confused which yak it is after...
03:17 mncharity "predator confusion"?
03:17 TimToady just pick on and gnaw on it :)
03:17 TimToady *one
03:19 * mncharity ponders menu
03:23 pugs_svn r19868 | lwall++ | [metholate] remove another temp hack
03:23 pugs_svnbot r19868 | lwall++ | [metholate] remove another temp hack
03:23 pugs_svnbot diff: http://dev.pugscode.org/changeset/19868
03:23 lambdabot Title: Changeset 19868 - Pugs - Trac
03:24 mncharity re gnaw one, the food reviews say...
03:27 mncharity Grammar as abstract CompSci-grammar spec makes for easy analysis, but is not trivial to get performance out of.  thus yare's "a method in a p5 grammar class, depending on its arguments, either returns a full Rule object, or is an express call into a cached optimized runtime emitted from it.
03:28 mncharity Grammar as a search, with method calls as simply steps in that search, is refreshingly simple after that.
03:28 TimToady well, I'm sure the output of metholate could make your eyes go crossed too.  :)
03:29 TimToady I'd suggest looking at a very simple rule and see how it looks after metholate
03:29 TimToady maybe I could paste one for you
03:31 mncharity Though I suppose that simplicity wane as one tries to integrate continuations and general backtracking.  Rather than "you can't go far in the p6 grammar without hitting a token, so you can ignore backtracking except in a few places, which can be overcome in detail with a bit of mess (maybe - a hypothesis)".
03:32 mncharity Grammar as immutable search state of the moment... I haven't tasted yet.  eyes still crossed.
03:33 pasteling "TimToady" at 71.139.7.101 pasted "metholated token ident" (26 lines, 599B) at http://sial.org/pbot/30249
03:33 TimToady the actual matching starts at .MATCHIFY
03:34 TimToady everything before that is deciding whether to autolex
03:35 TimToady we could in theory get rid of the map by knowing that the 'r' of STARr means "ratchet"
03:35 mncharity I'm not really familiar with p6 gather/take semantics, but ruby has nice fast blocks+yield - used for everything (iterating over arrays, etc).  so maybe a metholate_to_rb.
03:35 mncharity looking...
03:36 mncharity calm invoked on fate.  sigh.
03:36 TimToady the fate of '?' is called to look up the longest tokens of a subrule
03:37 TimToady so the autolexer in this case knows that if it needs to know what is in <alpha>, it can call the .alpha method with a fate of ? to fetch its longest tokens
03:38 TimToady something is still wrong in there with the "if not @try"
03:38 mncharity so <alpha>\w+  would peek for <alpha>\w on '?' ?
03:39 TimToady it would fetch <alpha>'s longest token set and tack \w+ on the end for its own set
03:41 TimToady the code that sets @try is currently commented out for pugsian and debugging reasons
03:42 TimToady for the moment the autolexer is actually using some yaml files of the regexen that metholate pokes down into /tmpyaml
03:43 TimToady anyway, that all works, or is a few tweaks from working again
03:44 TimToady it's the longest token matcher that I got bogged down on...
03:44 TimToady and I feel like if I got that working efficiently, all the rest of the pugs overhead wouldn't matter very much
03:45 TimToady (or whatever engine does the gather/take semantics)
03:45 TimToady ratchet would then just be a further optimization
03:45 mncharity re longest t m, i think i'm starting to see why that could be a headache on this Grammar model.  in the
03:47 TimToady well, the funny thing is I've already done the headachey part of extracting the current set of tokens on the fly at the last moment
03:47 TimToady I just have to turn it into something like a tagged DFA, which I can't do when my shoulder keeps me from sleeping at night...
03:47 TimToady me being more of an NFA guy...
03:48 mncharity Grammar is abstract model, it's just Rule objects' emit() methods chatting with each other, and then emit()ing whatever match code they want.  each Rule could have it's own complete implementation of the grammar, optimized for "what the grammar looks like entered via me".  even more than one,
03:49 mncharity one for capture ws, one for <?foo>, one for <foo>, one for <!foo>, etc.
03:50 TimToady a Rule can be entered via more than one grammar in the Perl model, so it has to keep those straight
03:50 mncharity so the longest t m is not an issue.  but the carpet gets pulled away from 'rules are like methods', they end up rather unlike methods, at least in the absence of a really nice mop.
03:51 TimToady the methods can still be called, but you can't know which order to call them without knowing the grammar
03:52 mncharity re entered, ah yes.  thus the "who am I called on" caching hair of yare.  so potentially n implementations, times m subclasses.  if one wishes.
03:55 TimToady that crossed with the problem of doing recursive descent when some of the descending is predetermined outside the normal call
03:55 mncharity at least in the Grammar as abstract model, the methods... are sort of hacks.  no, perhaps they could properly be Rule is Method objects.  but in the absence of a backend language which permits that, they are just ways to find (via reflection) and get at (fetchers for) "unrelated" Rule objects whose allocation then needs to be managed separately.
03:56 TimToady well, if pugs supported full-fledged method objects, this might be easier
03:57 TimToady but as it stands, I'm just passing extra parameters, and the "real" rule is what starts at the .MATCHIFY
03:57 TimToady also, I'm not entirely sure I have my story straight on alternations that are buried at deeper than the top level of a rule
03:58 mncharity re "crossed with the problem of ... some of the descending is predetermined outside the normal call", not sure I understand, but i don't think so?  the Grammar as abstract model seems a superset of the Grammar as runtime models.  they would simply be a particular choice of emit()ed runtime.
03:58 TimToady I'm probably okay as long as I can gen a unique identifier for the alternation for a cache id
04:00 TreyHarris joined #perl6
04:00 TimToady yes, but currently my emitter is metholate, I don't have any pugs code emitting pugs code
04:00 mncharity re buried alternations... yeah, while the regexp engine was a pain, redsix's "I'm just going to do full backtracking, and worry about ratchet stuff being required for a correct parse later" was somewhat more restful than the current "I think I can get by with fudging the backtracking like so... but I have a really really bad trackrecord of making such calls correctly".
04:00 TimToady the longest token matcher is called as a kind of interpreter over a set of regex
04:02 TimToady I basically said to myself, ratcheting is just an optimization, and if gather/take is right I don't really need it so much, and can ignore the optimization for now and feel good about having the right semantics
04:02 mncharity in retrospect at least, would you be happier with a p5-based STD.pm, directly runnable rather than the preprocess->pugs?
04:03 TimToady p5 doesn't do gather/take, alas
04:03 TimToady I could probably fudge something with coro
04:04 TimToady course, I do happen to know the two places in the grammar that require backtracking :)
04:04 mncharity alternate regex engine core?  choice point gathering isn't the only option...
04:04 mncharity oh, yeah, I meant to ask... which two? :)
04:05 TimToady I would tend not to trust any p5 regex engine not to run out of steam just when I need it
04:05 TimToady interpolating "$foo.bar.baz.whatever()" vs "$foo.bar.baz.whatever"
04:05 TimToady and deciding whether [= is the start of [=] or [=$iterator]
04:06 mncharity great.  tnx.  re steam,
04:06 mncharity any particular flavor of steam?
04:06 TimToady note that both spots are commented as backtracking in STD
04:07 TimToady actually, it might not have to backtrack on reduce ops if I get my longest tokens straight
04:08 TimToady well, typically reentrancy, and closures really being closures with their own pads
04:08 mncharity re comments, hmm, there's a third occurrence of "backtrack", associated with <suppose>.  not mentioned because currently not used?
04:08 mncharity (oh, there's is one <suppose>)
04:08 TimToady that's backtrack on panic, so kinda doesn't count
04:09 mncharity ok
04:09 TimToady it's mostly just an idea for really good error messages
04:09 mncharity righ
04:09 mncharity t
04:09 TimToady certainly not necessary for bootstrap
04:10 TimToady but if we noticed something looked suspicious, we just remember it and if the parse fails, see if it *might* have done better with a different guess
04:10 TimToady but it will fail the parse in any case
04:10 TimToady but the sort of thing p5 hardwires with "maybe you had a runaway string starting on line 432"
04:12 mncharity re steam, I guess performance or bug or don't have p6 runtime to transition to, or... that's what comes to mind.  sigh.  I didn't get warm fuzzies from the fact that moving subs around in the yare file could result in different behavior, tickling p5 bugs or not. :/  part of the reason the code is such a mess.  did _not_ want to refactor it without much better test suite.
04:13 TimToady if I could emulate backtracking, I think metholate could spit out a pretty decent chunk of Perl 5 code that doesn't rely on regex at all
04:15 TimToady doubt it would be very fast with all the OO though...
04:15 TimToady but again, might be fast enough with longest tm
04:16 pasteling "mncharity" at 76.24.29.201 pasted "a p5 bt api" (28 lines, 1.1K) at http://sial.org/pbot/30250
04:16 TimToady there is a tagged RE engine that p5 can use, but it was a little buggy
04:17 mncharity basically just return undef from a continuation to indicate match failure, and exceptions to fail() larger chunks of things.
04:18 TimToady bt stands for?
04:18 mncharity sorry, backtrack
04:19 mncharity red_STD is currently using the same approach.  false for failure, anything else (well, except for nil) on success.
04:20 mncharity so if one doesn't mind a bit of mess, one can just hand code it.  assuming continuations are subs.
04:20 TimToady exceptions can't backtrack down into a subrule unless you have real continuations
04:21 TimToady likely wouldn't matter for bootstrap
04:21 mncharity re hand code, that's whats going in the star,etc variants at the bottom of red_STD prelude.rb
04:21 TimToady but yes, the undef on failure mode was basically what the "andthen" was for
04:22 TimToady it actually interoperates okay with the gather/take model of backtracking too
04:22 TimToady (except under pugs :)
04:22 mncharity re exceptions... "up" to a subrule?  rules just wrap a try/catch around their bodies, and one can escape up the continuation chain when one fails into a rule commit.
04:23 thestarslookdown joined #perl6
04:24 TimToady in any case, you have to have some way of threading the continuations independently of the up-and-down of recursive descent
04:25 TimToady whether you use real continuations or some kind of stackless approach
04:25 mncharity ah, yes
04:26 TimToady but I will think about a p5-emitting metholate, thanks
04:27 TimToady as with the ruby approach, you still have to backtranslate the closures too
04:28 TimToady and things like EXPR
04:28 TimToady unless I go with some kind of hybrid p5/pugs approach
04:28 TimToady which might also work
04:28 mncharity yare and redsix are both explicit continuations.  red_STD is currently explicit continuations within a regex(), with "since there are token()s everywhere, maybe we can get by with having two separate calling api's, a limited use one for within a cluster of regex, within a boundary of tokens, and one for tokens, without an explicit continuation.".
04:28 TimToady (slowly:)
04:29 mncharity :/  sigh :)
04:30 TimToady or maybe I should just emit smop...
04:32 TimToady or just write a Perl parser in C.  I've done that before... :)
04:32 Auzon Only once? ;)
04:32 mncharity for a high level language, p6/p5/rb/python, there doesn't seem to be that much closure code to translate.  compared to the magnitude of other pain around.  so translated-closures + a backtrack story could target various things.  which gets one to "it *thinks* it parsed the file".
04:33 TimToady an AST beats a kick in the head
04:34 mncharity I've found the next step ends up being "but the only way to know if the parse is correct is to compile and run the test.  if it passes, I guess the parse was more likely correct than not, in general".  but maybe that's just because i didn't do "testing in detail" of individual syntactic constructs, so everything came out in integration phase.
04:35 TimToady well, I daresay any compiler that can produce an AST of STD.pm is close to something either very nice or very nasty
04:35 mncharity re AST, yeah, but eyeballing large volumes of ast for correctness... not fun.
04:35 mncharity lol
04:39 mncharity oh, so the point is, if you have to go on through compilation to test your parser, then you have the additional constrain of needing a target language with a p6 compiler/runtime.  which I guess means pugs, ruby, or... I'm not sure what the current p5 runtime looks like - if kp6 could magically parse all of t/ , how much of compile and run is done?  dunno.
04:41 mncharity but if one understands the language well enough to select the key microtests, then that kind of brute force testing is much less important.
04:42 TimToady mmm, I do love Blair's After Death sauce...
04:43 * mncharity googles for grade D utilitarian utopia
04:43 mncharity wrong channel.  Blair's After Death sauce?
04:43 * mncharity googles for Blair's After Death sauce
04:44 TimToady you'll feel like you're in a utilitarian utopia once the endorphins kick in :)
04:46 mncharity re sause - it's puzzling without a cultural context pointer... Massachussetts "extreme heat", or Arizona "extreme heat"?
04:47 TimToady it's about 15x tabasco sauce hot
04:48 TimToady not nearly as hot as eating a habanero directly, of course, with is about 150x
04:48 TimToady *which
04:48 TimToady but hot enough for me
04:49 mncharity :)
04:50 TimToady thing I notice about eating a habanero is that your face starts sweating profusely even before you can taste it.
04:51 mncharity now why does that make p6 development come to mind...?  maybe it's just the channel. :P
04:51 TimToady hey, my face has been sweating for 7 years now or so...
04:52 TimToady and I'll really be in trouble if Duke Nukem Forever actually comes out this year...
04:52 mncharity :)
04:53 TimToady on the other hand, I remind myself that Tolkien's sequel took 14 years to write...
04:53 TimToady and that did okay eventually
04:55 TimToady though I can't quite imagine someone throwing $300M on the table to make movies of Perl 6...
04:56 mncharity oky... /me tries to focus.  end-of-day approacheth.
04:56 TimToady amazing how that happens...
04:56 TimToady hope this helped
04:56 mncharity so I have to figure out what flavor of Grammar to use for red_STD.  lots-of-grammarlets, or fudge around $here.
04:57 mncharity re helped, oh, very much so.  clearer understanding, and a mind bending experience.  what more could one ask for?
04:58 mncharity more important that clearer understanding perhaps, the things I was stuck in puzzledment on got knocked down.  thanks:)
04:59 TimToady I think this will also help me get back into it after my bad-shoulder months, thanks!
04:59 mncharity :)
05:02 mncharity beyond red_STD... I just have this feeling that things are so close... redsix was running against t/... I was not unhappy with it... it was just the regex engine and p6 grammar and runtime (no sig/cap) which were a kludge. ;)  but since then there is yare and STD and the kp6-ruby ideas.  different languages, but those were the needed bits.
05:03 mncharity and kp6's ast.  the redsix ast was rather... traditionally non-minimalistic.
05:03 TimToady seems like we've prototyped most of the bits of it several different ways now
05:05 TimToady and I think there's a sense in which all these prototypes were needed for one reason or another
05:06 TimToady pugs itself is a wonderful way to bootstrap the test suite
05:06 TimToady even if it is never developed further
05:06 mncharity yeah.  past failure modes were...  hit missing bit; foundation you are building on let's you down; and ... something social... effort loses champion or community.  anything else?
05:08 mncharity long term, I actually really look forward to a haskell-based p6.  non-local deep transformation of code is something all the aspect oriented cruft can't buy you in a traditional language, but haskell gives you free (well, at little additional cost).
05:08 TimToady or mental model wasn't sufficiently rich for a viable design
05:08 Juerd <@TimToady> and I'll really be in trouble if Duke Nukem Forever actually comes out this year...
05:08 Juerd haha :)
05:08 * Juerd loves internet culture
05:08 penk joined #perl6
05:08 TimToady well, it gives it free to people with an IQ in excess of 200  :)
05:09 Juerd All four?
05:09 TimToady that seems about right
05:10 TimToady I only consider myself a 4-sigma event
05:10 TimToady and I find that's not quite good enough for Haskell  :)
05:10 TimToady well, maybe I was good enough when I was young enough to really be 4-sigma :)
05:11 Juerd Sigma is the symbol for standard deviation?
05:11 TimToady *nod*
05:11 Juerd context++
05:12 TimToady I just wish I even had above average Constitution to go with it...
05:13 mncharity why various prototype components of p6 haven't grown into a full p6: component insufficiently understood for successful implementation; hit missing component; foundation it was building on failed under the weight; effort lost champion/community.
05:14 mncharity how's that for a synthesis / cautionary check-list for the future?
05:14 TimToady did you see my "suffienctly rich" comment?
05:14 TimToady or however that's spelled?
05:14 mncharity looks...
05:15 mncharity "sufficiently rich for a viable design"... I parsed that as "component insufficiently understood for successful implementation", but was unsure I captured the nuance?
05:15 mncharity err, "mental model wasn't sufficiently rich for a viable design"
05:15 mncharity s/nuance/meaning/
05:15 Juerd Whose mental model though?
05:16 TimToady more like "too simple an approach to work on the complex problem"
05:16 mncharity ahh...
05:17 TimToady any approach using past technologies tends to presume the same limits as those technologies
05:17 TimToady and not design in the aspects necessary to transcend those limits
05:18 mncharity mncharity: why various prototype components of p6 haven't grown into a full p6: component insufficiently understood; implementation approach inadequate; hit missing component; foundation it was building on failed under the weight; effort lost champion/community.
05:18 mncharity ?
05:18 TimToady which is kind of the *why* of the other tools letting us down
05:18 mncharity not sure I buy that last part
05:19 TimToady well, it's not important to my mind; the only true failure is if everyone quits and nobody learns anything from it
05:20 TimToady "w...hich is, of course, mandatory in the long run" --Ozymandius
05:20 mncharity eg, if p5.10's reentrant regexps actually worked, rather than comming with "oh, but you better not do X after going through one, or a bug with crash you",... that's just a tool failure.  or I suppose an design failure to recognize "a usable reentrant engine doesn't exist yet".
05:21 TimToady then doubtless it will manifest some failure mode based on the fact that it uses a different runloop than normal Perl code... :)
05:24 TimToady or the very fact that p5 does *not*, in fact, do anything resembling DFA matching...
05:25 mncharity the problem with calling it a design failure is the slipper slope to "the choice of any tool is a design failure, because every tool is somehow broken".  not "not appropriate for task, but appropriate for task if it actually worked, but doesn't, quite enough".
05:25 mncharity *slippery
05:25 mncharity lol # re DFA
05:25 TimToady well, the design failure is trying to climb 582 rungs of the ladder in one hop, perhaps.  :)
05:26 cathyal joined #perl6
05:26 Juerd Or perhaps designing such a ladder in the first place :)
05:26 TimToady if I go with a p5 emitter for metholate, I may take another look at TRE (tagged re)
05:27 TimToady I looked at it a bit when I was thinking of using a p5 dfa from inside pugs
05:27 Juerd What is metholate?
05:27 TimToady that's a long story
05:27 Juerd Never mind then :)
05:27 TimToady short answer, a program that you do "cheat STD.pm | metholate | pugs" and it runs
05:28 Juerd Sounds rather magical
05:28 Juerd Does it finish pugs on the fly?
05:28 TimToady it turns p6 regexen into p6 code that uses no regexen, just method calls
05:28 mncharity re 582 rungs, not sure.  flying requires more prep and is riskier than walking, but for some terrain, anything else is, well, not insane, but perhaps even slower.
05:28 TimToady it has finished off pugs frequently
05:28 Juerd Oh, I think I saw that :)
05:29 TimToady yes, it's the worst way, except for all the others :)
05:29 mncharity s/slower/less attractive/
05:29 TimToady one could take a more deliberate evolutionary pace, but then we'd see the very same social ossification that p5 underwent
05:30 TimToady that is another failure mode
05:30 TimToady as spectacularly successful failure mode, to be sure...
05:31 TimToady *a
05:31 TimToady but the folks who think p6 is killing progress on p5 don't realize that we already saw it happening before 2000
05:32 Juerd They also don't realise that mostly, people who work on Perl 6 have lost interest in Perl 5 or were never Perl 5 contributors in the first place.
05:34 TimToady (often for the same (social) reasons in either case)
05:34 TimToady by which I don't mean to level blame at anyone in particular
05:34 mncharity I'd be really happy suggesting yare as a building block for p5 work, despite its mess, if noop code changes didn't tickle p5 buts, cause anon sub's to break, resulting in really puzzling time-sucking failure modes.  but I don't now if it's been tried on 5.10.  perhaps a day spent "refactor, and see if it keeps working" would not be misspent.
05:34 mncharity *bugs
05:34 TimToady it just became a very conservative culture
05:35 Juerd And if indeed many people went from P5 to P6, P6 would be done by now perhaps
05:35 TimToady maybe not, there's an optimal number of cooks in the kitchen at any given point
05:35 TimToady and we haven't been too far off that
05:36 mncharity if yare worked, basically all STD would need was p5 translations of the closures.
05:36 TimToady for bootstrap, yeah
05:37 TimToady and STD is by and large written with the assumption that | could be emulated with || for the boostrap
05:37 * mncharity listens to the clock tick in a sunny but deserted kitchen.  thinks throwing a kitchen party would be much nicer.
05:38 mncharity easy enough to do |.  may even have been done.  it's just adding : after all.
05:39 TimToady I mean the longest token part irrespective of the ordering of alternatives
05:39 TimToady which : doesn't have much to do with
05:39 TimToady it really needs a DFA (or non-bt NFA) to do that right
05:40 TimToady on the other hand, 5.10 does trie optimization, which is most of it
05:41 TimToady not sure if there's a way to turn off the ordering semantics tho
05:41 mncharity oh, right.  well, re "it really needs a DFA (or non-bt NFA) to do that right", you really mean "it really needs a DFA (or non-bt NFA) to do that FASTEST", yes?
05:41 mncharity wouldn't want to confuse "right" with "fast". ;)
05:41 TimToady no, I mean "correctly"
05:41 peeps[work] joined #perl6
05:41 mncharity explain?
05:41 TimToady unless you want to run all the alternatives, and then see which one was longest matching
05:41 * mncharity discovers another corner I don't quite understand...
05:41 TimToady in that sense it's just an optimization
05:42 mncharity re all the alternatives + longest, exactly.
05:42 TimToady a|bc|\w+ might match \w longer or shorter than bc
05:43 TimToady in the general case you have to run all legs of <foo>|<bar>|<baz>
05:43 TimToady DFA just runs them in parallel
05:45 TimToady and if <foo> <bar> or <baz> have side effects, then it's manifestly *incorrect* to run them willy nilly
05:45 TimToady you really want to know which leg contains the longest token before you commit to any side effects
05:45 TimToady that's what all this declarative/procedural language in S05 is all about
05:45 TimToady so in that sense you *can't* have correct semantics with a backtracking engine
05:46 TimToady at least, not with interruptable/resumably subrules
05:46 TimToady *without
05:46 * mncharity starts to worry, then chants "boot strap, boot strap, ..." and pictures a "mission creep" with red x circle plackard.
05:46 TimToady if each rule could do its longest token, and then suspend itself until told to go further, you could still get correct (albeit) slow behavior
05:47 TimToady with the metholate approach, we don't have to do the suspend part, just the resumable part :)
05:47 TimToady which is what all those @fate parameters are for
05:48 mncharity another advantage for resumeable escapes... but something for another day.  my smalltalk fu isn't up to it.
05:49 TimToady speaking of another day, I got up before 6:00 today, so I should probably totter off to bed.
05:49 mncharity not sure whether ruby callcc is up to it or not.
05:49 TimToady well, there's always scheme :P
05:49 mncharity righto.  been a very interesting afternoon/evening chat.  tnx
05:50 TimToady night &
06:12 pugs_svn r19869 | putter++ | [misc/red_STD]: very minor progress on EXPR.
06:13 pugs_svnbot r19869 | putter++ | [misc/red_STD]: very minor progress on EXPR.
06:13 pugs_svnbot diff: http://dev.pugscode.org/changeset/19869
06:13 lambdabot Title: Changeset 19869 - Pugs - Trac
06:15 mncharity I now understand what someone said about lots of bots.
06:16 mncharity that last commit was basically some pre-conversation edits.  haven't had to decide yet how to proceed.
06:16 mncharity g'night all
06:24 manfred joined #perl6
06:30 spinclad rats, finished backlogging 12 minutes late.
06:52 Southen joined #perl6
06:55 integral joined #perl6
06:56 spinclad joined #perl6
07:28 devogon joined #perl6
07:38 psapo joined #perl6
07:57 Aankhen`` joined #perl6
08:00 BinGOs joined #perl6
08:28 xda joined #perl6
08:39 Zygo joined #perl6
08:44 Zygo joined #perl6
08:46 iblechbot joined #perl6
08:48 monomorph joined #perl6
09:01 barney joined #perl6
09:23 jrockway > reverse . show $ (*) 6 9
09:23 lambdabot "45"
09:23 jrockway > reverse . show $ (*) 6 9 where 6 * 9 = 24
09:23 lambdabot Parse error at "where" (column 26)
10:15 pugs_svn r19870 | nwc10++ | Improve C compiler line number directives so that errors in the parts copied
10:15 pugs_svn r19870 | nwc10++ | verbatim from the original .sm0p files are reported as coming from the .sm0p
10:15 pugs_svn r19870 | nwc10++ | files, and generated code is reported as coming from the .c file.
10:15 pugs_svnbot r19870 | nwc10++ | Improve C compiler line number directives so that errors in the parts copied
10:15 pugs_svnbot r19870 | nwc10++ | verbatim from the original .sm0p files are reported as coming from the .sm0p
10:15 pugs_svnbot r19870 | nwc10++ | files, and generated code is reported as coming from the .c file.
10:15 pugs_svnbot diff: http://dev.pugscode.org/changeset/19870
10:15 lambdabot Title: Changeset 19870 - Pugs - Trac
10:26 Khisanth two svn bots?
10:31 diakopter hm
10:36 diakopter murdered
10:46 luqui joined #perl6
11:40 ruoso joined #perl6
11:45 pbuetow joined #perl6
12:07 blindfish joined #perl6
12:21 ilogger2 joined #perl6
12:22 manfred joined #perl6
12:52 pugs_svn r19871 | putter++ | [misc/red_STD] EXPR() attains "2+3*4".
12:56 DaGo joined #perl6
12:59 baest joined #perl6
13:06 mncharity joined #perl6
13:13 pugs_svn r19872 | putter++ | src/perl6/STD.pm: an EXPR() fix
13:17 mncharity Khisanth: re two bots, twice the karma :)
13:20 mncharity a tweak: why various prototype components of p6 haven't grown into a full p6: component insufficiently understood; implementation approach insufficiently powerful; hit missing component; foundation it was building on failed under the weight; effort lost champion/community.
13:25 chris2 joined #perl6
14:26 kanru joined #perl6
14:43 xinming joined #perl6
15:05 pbuetow joined #perl6
15:07 jhorwitz joined #perl6
15:09 pugs_svn r19873 | putter++ | [misc/red_STD] added read-eval-print-loop debugging aids, and some whitespace tokens.  blindly dropped .ws() calls into EXPR().
15:09 Limbic_Region joined #perl6
15:34 manfred joined #perl6
16:20 lypanov joined #perl6
16:42 alester joined #perl6
16:53 moritz_ is STD.pm supposed to be runnable by pugs, or to be correct Perl 6?
16:53 moritz_ I noticed @array[-1] in there
16:54 moritz_ which isn't valid Perl 6 anymore
16:55 [particle] it's supposed to be valid perl 6
16:56 wolverian fix! :)
16:56 buchetc joined #perl6
16:57 moritz_ low hanging fruit ;)
16:57 pugs_svn r19874 | moritz++ | [STD.pm]: @array[-1] is now @array[*-1]
16:57 moritz_ I guess it will kill metholate's output, because pugs doesn't like @array[*-1]
17:16 TimToady we can fix it with cheat or metholate, so go ahead
17:17 buchetc joined #perl6
17:20 pugs_svn r19875 | lwall++ | [cheat] translate [*-1] to [-1]
17:20 chris2 joined #perl6
17:54 meppl joined #perl6
18:05 pugs_svn r19876 | lwall++ | [t/spec/*.t] :todo whackage
18:07 justatheory joined #perl6
18:08 meppel-san joined #perl6
18:08 lypanov left #perl6
18:09 jferrero joined #perl6
18:09 devogon joined #perl6
18:19 pugs_svn r19877 | lwall++ | [STD] missing .ws after expect_infix
18:28 rindolf joined #perl6
18:28 njbartlett joined #perl6
18:29 Lorn joined #perl6
18:33 Alias_ joined #perl6
19:05 shlomif joined #perl6
19:08 thoughtpolice joined #perl6
19:11 DarkWolf84 joined #perl6
19:27 pmurias joined #perl6
19:28 clkao joined #perl6
19:30 pugs_svn r19878 | moritz++ | [STD] perlhints updates
19:30 moritz_ I've now seen STD and a few PGE grammars...
19:31 moritz_ and all quote literal text
19:31 moritz_ to improve readabilty
19:32 moritz_ shouldn't we change rules to require quoting of literal text, and allow subrule calls without <...>?
19:32 moritz_ and in regexes introduced with m/ ... / or mm/.../ etc. the current syntax could be used
19:36 obra_work joined #perl6
19:49 pmurias moritz_: i asked also asked for it, the answer is how do you pass parameters to a subrule?
19:49 pmurias s:1st/asked//
19:50 moritz_ pmurias: rule(@args)
19:50 moritz_ just like normal sub/method calls
19:51 moritz_ better huffman coding, more method <-> regex analogy
19:53 pmurias moritz_: it sort of conflicts with () as a grouping construct
19:53 moritz_ we need to think of some kind if disambiguation
19:54 moritz_ currently we have disambiguations syntax in the other direction
19:54 moritz_ rule comp_unit { :my $begin_compunit is context = 1; :my $endstmt        is context<rw> = -1; :my $endargs        is context<rw> = -1;
19:54 moritz_ <statementlist>
19:54 moritz_ ...
19:54 cognominal_ for (@argv {  given $_ { ... }   }   would deserve to be    forgiven @argv { ... }
19:55 cognominal_ would TimToady bless that :)
19:55 moritz_ cognominal_: 'for' sets $_ by default
19:55 moritz_ pugs: for <a b c> { $_.say }
19:55 exp_evalbot OUTPUT[a␤b␤c␤]
19:55 cognominal_ so I can skip the given
19:55 cognominal_ ??
19:55 wolverian pugs: .say for <a b c>
19:55 exp_evalbot OUTPUT[a␤b␤c␤]
19:55 moritz_ yes
19:55 wolverian you don't even need the $_ :)
19:56 cognominal_ so much for my pun
19:56 moritz_ cognominal_: as long as the pointy block has no signature
19:56 moritz_ cognominal_: you lost ;-)
19:57 pmurias does longest token search descend into subrules?
19:58 moritz_ dunno
19:58 cognominal_ so I can write   for (@argv) { when 'whatever' { }  when 'foo' {} }   ?
19:58 moritz_ yes
19:58 cognominal_ it makes sense
19:59 moritz_ 'given EXPR' just sets $_ to EXPR
19:59 moritz_ as an read only alias, iirc
19:59 moritz_ so anything that sets $_ instead is just fine
20:00 pugs_svn r19879 | moritz++ | [STD] more perlhints
20:00 cognominal_ we are so used to bad languages that the Perl6 natural way seems strange
20:00 cognominal_ we are so used to code that we don't know how to program
20:01 moritz_ aye
20:01 moritz_ funny, I found a p5ism today in TimToady's STD.pm code
20:02 cognominal_ :)
20:02 moritz_ so even the best of us aren't immune to history
20:03 cognominal_ I am happy the implementation takes so long so that TimToady can to the best for his second shot
20:03 cognominal_ perl4 was nice syntactically but perl5 is so ugly
20:04 cognominal_ s/to/do/
20:04 moritz_ sub calls with mandatory '&' are pretty?
20:04 cognominal_ making place to references wasted perl syntax
20:04 * moritz_ disagrees
20:05 cognominal_ I forgot about that
20:16 TimToady pmurias: yes, longest token descends into subrules, and even back out again if the entire subrule is pure
20:22 TimToady (that's why we don't put {*} into subtokens like <sigil>, in fact, so that they can be composed into longer tokens)
20:22 TimToady since {*} represents an unknown side effect
20:23 TimToady bbl &
20:23 eternaleye joined #perl6
20:23 clkao joined #perl6
20:24 clkao joined #perl6
20:42 cognominal_ prefix . implies an invocant of $_ or self?
20:42 moritz_ $_
20:43 cognominal_ ok
21:22 pugs_svn r19880 | moritz++ | [perlhints] updated README to new format
21:24 pugs_svn r19881 | moritz++ | [perlhints] remove data/, that information is now inlined into STD
21:24 Caelum joined #perl6
21:24 moritz_ the current state of perlhints is "needs more hacking"
21:24 moritz_ or to be blunt: "unusable"
21:47 pmurias moritz_: perlhints require STD to run on something to be truly usable
21:47 moritz_ pmurias: I know, but earlier versions had a very primitive stand alone mode
21:48 moritz_ pmurias: every description has a list of literals that can occur in that token
21:48 moritz_ pmurias: so ./perlhints * would find infix:<*>, postfix:<*> from regexes, whatever star etc.
21:48 moritz_ pmurias: not ideal, but better than nothing
21:49 moritz_ it will only work the way I intend it to do when TimToady defines a way to access the parse tree, and sombody implements that
21:52 pmurias moritz_: the perlhints should have a link to the spec
21:52 pmurias s/perlhints/hints
21:52 moritz_ pmurias: aye
21:52 moritz_ pmurias: feel free to add them
21:52 moritz_ ;)
21:53 moritz_ atm the specs aren't that linkable
21:53 moritz_ when I point you to S03, well, you've got a myriad of operators defined there
21:55 pmurias a smartlink style link would be usefull
21:55 polettix joined #perl6
21:56 pmurias sleep&
21:56 moritz_ good night
21:57 moritz_ ... and I'm alone again
21:57 moritz_ ... if you ignore all other 132 people and bots in this channel ;)
22:08 ebassi joined #perl6
22:16 pugs_svn r19882 | moritz++ | [perlhints] mostly unbroken now
22:16 pugs_svn r19883 | moritz++ | [STD] more perlhints tweaks
22:27 nerog joined #perl6
22:48 kingdong joined #perl6
22:48 kingdong left #perl6
22:51 Jedai joined #perl6
23:01 marmic joined #perl6
23:21 justatheory joined #perl6
23:36 LimbicRegion joined #perl6
23:53 lambdabot joined #perl6

| Channels | #perl6 index | Today | | Search | Google Search | Plain-Text | summary

Perl 6 | Reference Documentation | Rakudo