Perl 6 - the future is here, just unevenly distributed

IRC log for #perl6-dev, 2017-06-14

| Channels | #perl6-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:07 eveo japhb: well, if you're on latest rakudo, see if the module works on bdf2019. That's before the proc refactor.
00:07 * eveo goes to bed
00:16 japhb OK, I'll take a look
00:36 japhb .tell eveo Yes, Inline::Python seemed to build fine at Rakudo bdf2019 (it was as you guessed broken at nom HEAD)
00:36 yoleaux japhb: I'll pass your message to eveo.
00:44 BenGoldberg joined #perl6-dev
00:58 ugexe japhb: it fails based on the exit code
00:58 ugexe of running Build.pm
00:59 ugexe if you run zef with --debug it will show the command it uses to run the Build.pm, which you can run yourself without zef
02:32 ugexe windows failing since proc-using-proc-async merge https://ci.appveyor.com/project/moritz/rakudo/history
02:35 ugexe seems to run 20 tests, then gets dubious results on the remaining tests
03:27 ugexe trying to skip the tests and just install ends up failing during install-core-dist.pl with `Could not open (CompUnit::Repository::Staging). Failed to stat file: no such file or directory` (seems to think a CUR class name is a literal file path?)
03:31 ugexe https://gist.github.com/ugexe/fc2001addaa2bea27ad48e66c09c8373
04:07 yoleaux joined #perl6-dev
04:17 ugexe wonder if its line endings messing up parsing of the precomp dependency files
04:32 benchable6 joined #perl6-dev
05:02 astj_ joined #perl6-dev
06:07 [Tux] This is Rakudo version 2017.05-435-gaa9516be2 built on MoarVM version 2017.05-85-g21ee1a50
06:07 [Tux] csv-ip5xs        2.850
06:07 [Tux] test            13.233
06:07 [Tux] test-t           4.368 - 4.370csv-parser      12.688
06:34 Geth ¦ rakudo/js: 81390d1818 | pmurias++ | 2 files
06:34 Geth ¦ rakudo/js: [js] Implement nqp::p6clearpre, nqp::p6inpre, nqp::p6setpre
06:34 Geth ¦ rakudo/js: review: https://github.com/rakudo/rakudo/commit/81390d1818
07:12 pmurias joined #perl6-dev
07:12 pmurias are there any plans to go through the tests that are not in spectest.data and check if they are valid?
07:16 pmurias for example I found a S04-phasers/exit-in-check.t which check that END blocks are called when we exit at CHECK time (and has a note that the it used to test the opposite behavior and was changed in 2006 audreyt to test for Perl 5 compatible)
07:17 pmurias as CHECK is compile time and END runtime it seems that exit in CHECK time should avoid END blocks
07:25 lizmat joined #perl6-dev
07:30 Geth ¦ roast: 51662b9399 | pmurias++ | 2 files
07:30 Geth ¦ roast: Remove old tests we don't run that seem broken
07:30 Geth ¦ roast:
07:30 Geth ¦ roast: CHECK and BEGIN happen at compile time while END runs at run time.
07:30 Geth ¦ roast: If we exit in compile time it doesn't seem to make sense to have
07:30 Geth ¦ roast:  runtime END block run.
07:30 Geth ¦ roast: review: https://github.com/perl6/roast/commit/51662b9399
08:04 [TuxCM] joined #perl6-dev
09:10 jnthn morning, #perl6
09:14 ggoebel joined #perl6-dev
09:29 pmurias jnthn: morning
09:34 lizmat Files=1205, Tests=62036, 216 wallclock secs (13.18 usr  4.91 sys + 1315.71 cusr 130.26 csys = 1464.06 CPU)
09:48 Geth ¦ rakudo/nom: bd292225d4 | (Jonathan Worthington)++ | src/Perl6/Actions.nqp
09:48 Geth ¦ rakudo/nom: Slightly more aggressive return handler check.
09:48 Geth ¦ rakudo/nom:
09:48 Geth ¦ rakudo/nom: This makes more routines be compiled without the a return exception
09:48 Geth ¦ rakudo/nom: handler, which saves some instructions off their total code size.
09:48 Geth ¦ rakudo/nom: review: https://github.com/rakudo/rakudo/commit/bd292225d4
10:05 eveo japhb: so it's on Windows? Also, what was the output of the failure :/
10:05 yoleaux 00:36Z <japhb> eveo: Yes, Inline::Python seemed to build fine at Rakudo bdf2019 (it was as you guessed broken at nom HEAD)
10:05 eveo buggable: speed 6
10:05 buggable eveo, ▂▁▁▁▁▁ data for 2017-06-12–2017-06-14; range: 4.341s–4.427s; 1% faster
10:22 eveo Actually now I remember Proc::Async being really buggy on Windows last month, so maybe that's why something's busted on windows
10:23 * jnthn wonders why that'd happen, given in theory libuv abstracts away the differences :/
10:23 jnthn That said, proc was also busted in various ways
10:25 eveo C:\Users\zoffi>perl6 -e "await Proc::Async.new($*EXECUTABLE, q|-e|, q|say 'hi'|).start"
10:25 eveo ===SORRY!===
10:25 eveo Argument to "say" seems to be malformed
10:25 eveo at -e:1
10:25 eveo It's basically errors like that
10:25 jnthn :/
10:25 eveo ^ that's on 2017.04.3
10:25 jnthn Wonder if it's not sticking in quotes
10:25 jnthn Or some such
10:25 timotimo probably something like that
10:25 timotimo it has to cause the "say 'hi'" to become a single argument
10:26 timotimo it probably didn't succeed at that
10:26 timotimo there must be something like strace for windows where you can see what exact arguments it ended up passing
10:27 jnthn m: say 2.452 / 2.579
10:27 camelia rakudo-moar bd2922: OUTPUT: «0.950756␤»
10:32 Geth ¦ nqp: cabe421f02 | (Jonathan Worthington)++ | 2 files
10:32 Geth ¦ nqp: Start conveying decont context in QAST -> MAST.
10:32 Geth ¦ nqp:
10:32 Geth ¦ nqp: This means that we can, in various cases where we used to take a
10:32 Geth ¦ nqp: reference to a native lexical/attribute, now simply just get it
10:32 Geth ¦ nqp: boxed instead.
10:32 Geth ¦ nqp: review: https://github.com/perl6/nqp/commit/cabe421f02
10:34 Geth ¦ rakudo/nom: 8ff980e780 | (Jonathan Worthington)++ | 2 files
10:34 Geth ¦ rakudo/nom: Have decontrv op convey decont context.
10:34 Geth ¦ rakudo/nom:
10:34 Geth ¦ rakudo/nom: Includes an NQP bump to get a QAST->MAST compiler with support for
10:34 Geth ¦ rakudo/nom: this.
10:34 Geth ¦ rakudo/nom: review: https://github.com/rakudo/rakudo/commit/8ff980e780
10:35 jnthn These two improve the code we get in consume-line-chars
10:35 jnthn And probably in a bunch of other places
10:36 jnthn (By making us not spit out code to make a lexical reference)
10:38 jnthn Amazingly, that makes my read lines and add up the chars benchmark go from 182 GC runs to just 41
10:39 timotimo jeez, that's pretty darn good
10:40 jnthn Wins 5% off total time
10:43 jnthn m: say 2.452 / 1.134
10:43 camelia rakudo-moar bd2922: OUTPUT: «2.162257␤»
10:43 jnthn Still some more opts to find before within a factor of 2 of Perl 5
10:44 tadzik damn, cool
10:45 jnthn The comparison is, fwiw
10:45 jnthn perl 6: my $fh = open "longfile"; my $chars = 0; for $fh.lines { $chars = $chars + .chars }; $fh.close; say $chars
10:45 jnthn perl 5: open my $fh, "<:encoding(UTF-8)", "longfile"; my $chars = 0; while ($_ = <$fh>) { chomp; $chars = $chars + length($_) }; close $fh; print "$chars\n"
10:51 pmurias do we have any potential of of passing a handler_out_of_dynamic_scope flag to a lexical_handler_not_found_error handler?
10:56 pmurias jnthn: it doesn't seem to be used current at all, but not calculating it would just shave a few C level memory assignments
11:01 Geth ¦ rakudo/nom: 7edf9da6b6 | (Jonathan Worthington)++ | src/vm/moar/ops/perl6_ops.c
11:01 Geth ¦ rakudo/nom: Convey spesh facts better in p6decontrv.
11:01 Geth ¦ rakudo/nom:
11:01 Geth ¦ rakudo/nom: This allows spesh, when it knows (or has guarded on) the return type
11:01 Geth ¦ rakudo/nom: already, to toss the return type checks entirely. This also boots out
11:01 Geth ¦ rakudo/nom: the Nil checking and error handling basic blocks, shortening the
11:01 Geth ¦ rakudo/nom: specialized bytecode by a couple of basic blocks, or approximately
11:01 Geth ¦ rakudo/nom: 12 instructions, which will in turn help bring more things under the
11:01 Geth ¦ rakudo/nom: inline limit.
11:01 Geth ¦ rakudo/nom: review: https://github.com/rakudo/rakudo/commit/7edf9da6b6
11:01 jnthn pmurias: I put that in so we could give better errors at some point in the future
11:06 Geth ¦ rakudo/nom: 14d7571311 | (Elizabeth Mattijsen)++ | t/spectest.data
11:06 Geth ¦ rakudo/nom: Remove reference to tests removed from roast
11:06 Geth ¦ rakudo/nom: review: https://github.com/rakudo/rakudo/commit/14d7571311
11:07 jnthn Lunch now. Later I will take a look at spesh of optional args, because we're making a pretty bad job of ripping out a lot of code we really should be able to.
11:07 jnthn I *think* that'll bring consume-line-chars well under the inline size limit
11:11 jnthn hah, and also we're doing an inlining of .defined, which is good, and we're realizing based on the argument type that its result is constant, which is good, but then still managing to spit out code that boxes an integer only to truth test it :P
11:12 jnthn anyway, bbiab :)
11:12 lizmat jnthn
11:12 lizmat jnthn++   :-)
11:14 nine jnthn: I bet that's much more fun than debugging concurrency issues ;)
12:10 AlexDaniel joined #perl6-dev
12:14 * jnthn back
12:15 jnthn nine: It is, unless I have to debug a spesh issue ;)
12:30 eveo m: class Foo { method foo { say $++ } }; Foo.new.foo; Foo.new.foo;
12:30 camelia rakudo-moar 14d757: OUTPUT: «0␤1␤»
12:30 eveo A bit surprised by that... Sounds like using state vars in methods is setting yourself up for a lengthy bug hunt
12:31 moritz state vars do carry state around :-)
12:31 moritz that shouldn't be a surprise, really
12:32 eveo I guess
12:32 moritz there are ordinary lexical variables and there are attributes
12:32 moritz one for each desired behavior
12:32 eveo m: class Foo { method foo { -> { say $++ }() } }; Foo.new.foo; Foo.new.foo;
12:32 camelia rakudo-moar 14d757: OUTPUT: «0␤0␤»
12:32 eveo I guess in this case it's just a new block each time
12:33 moritz right; closure cloning happens when the outer lexical scope is entered
12:33 moritz closures are tricky. You may quote me on that :-)
12:33 eveo m: class Foo { method foo { (state $ = -> { say $++ })() } }; with Foo.new { .foo; .foo };  Foo.new.foo;
12:33 camelia rakudo-moar 14d757: OUTPUT: «0␤1␤2␤»
12:34 eveo moritz: you're good at closure cloning you say? Maybe fixing this bug will be easy for you: https://rt.perl.org/Ticket/Display.html?id=131548#ticket-history
12:35 moritz eveo: I don't think so, but I can take a look
12:36 eveo \o/
13:35 * lizmat needs to get a different IP number
13:42 lizmat joined #perl6-dev
13:50 * eveo starts a toast of HEAD on 64-core box
13:50 eveo oh yeah, big guns \o/
13:55 dogbert17_ m: sub foo () {$ = 42}; for ^2000000 { $ = foo }; say now - INIT now # IOninja
13:55 camelia rakudo-moar 14d757: OUTPUT: «Cannot assign to an immutable value␤  in block <unit> at <tmp> line 1␤␤»
13:56 eveo IOninnja what?
14:00 dogbert17_ a bug you reported, RT #130855
14:00 dogbert17_ in case jnthn wants to get into some hardcore spesh debugging :)
14:01 eveo Oh yeah, now we're talking: https://perl6.party/assets/pics/toaster/htop.png
14:04 ugexe re Proc::Async quoting: I found this old note above a proc/proc::async abstraction - https://github.com/ugexe/zef/blob/8e74fbc59ac8669e71610f0384a08384a3f9b030/lib/Zef/Roles/Hooking.pm6#L37-L39
14:14 lizmat joined #perl6-dev
14:42 eveo bah, google killed my VM minutes before it finished the re-testing of failures.
14:42 eveo This will have to do, I guess
14:42 eveo Still had hangs, so 64-core vs 24-core didn't really make a difference. Took ~50m for entire run in both cases.
14:44 eveo With a hack to count all the tests, even inside subtests, we have ~153374 tests (a bit inflateds, since stuff like `is_run` would count as more than one test)
14:44 eveo A lot fewer than I expected
14:44 eveo m: say "Only {153374-139011} tests are inside subtests"
14:44 camelia rakudo-moar 14d757: OUTPUT: «Only 14363 tests are inside subtests␤»
14:47 eveo CPU utlization chart from the toast run: https://perl6.party/assets/pics/toaster/cpu-utilization.png
14:49 eveo So ~10m to build moarvm/nqp/rakudo, 25m run to toast entire ecosystem. Then it waited for me to kill the hung install of one module, then another ~7m to re-test the ~250 failures
14:54 eveo 17 unsuccs: https://toast.perl6.party/
14:56 AlexDaniel joined #perl6-dev
14:57 * eveo can't tell if #perl6 got really busy recently, or just a few people keep spamming eval commands...
14:59 eveo I guess Building fail is what ugexe mentioned about exit code.
14:59 eveo 'cause run(:err, :out) always reported 0 for it
15:00 eveo Also, looks like only 2 modules from previous toast are still burnt :) Good job \o/
15:04 eveo A few of new burns are due to failed spawned commands, so seems like the reason they were Succ in earlier toasts is just because of the run(:err, :out) exitcode bug
15:20 * lizmat needs to get a different IP number again  :-(
15:24 AlexDaniel joined #perl6-dev
15:24 lizmat joined #perl6-dev
15:25 lizmat grrr   this IP number is no good either  :-(
15:28 lizmat joined #perl6-dev
15:31 AlexDani` joined #perl6-dev
15:34 mst lizmat: what problem are you having with IP addresses?
15:34 mst eveo: spot pricing?
15:35 lizmat mst: trying to get access to the n
15:35 lizmat NL name registry
15:35 lizmat it's been a while
15:35 mst and it hates certain IP addresses?
15:37 lizmat yeah
15:37 lizmat among other things
15:37 mst odd
15:37 lizmat I guess we missed a number of security upgrades
15:38 lizmat if it hadn't been for Xs4all no longer wanting to do our secondary DNS, I wouldn't be in this quagmire of rules and regulations  :-(
15:40 eveo mst: spot? what spot?
15:40 mst eveo: you said google ate the VM when you hadn't finished, I was assuming it was something like an EC2 spot instance where you can get outbid
15:40 eveo mst: oh, google compute engine... Yeah, I'm using pre-emptive instances they can kill at any time in exchange for making them a lot cheaper
15:41 mst right, same principle then
15:51 bisectable6 joined #perl6-dev
16:06 ugexe bisect: my $proc = run "ls", :out, :err; my $result = ?$proc; say $result
16:07 bisectable6 ugexe, On both starting points (old=2015.12 new=14d7571) the exit code is 1 and the output is identical as well
16:07 bisectable6 ugexe, Output on both points: «run is disallowed in restricted setting␤  in sub restricted at src/RESTRICTED.setting line 1␤  in sub run at src/RESTRICTED.setting line 14␤  in block <unit> at /tmp/ysne1TuTOW line 1␤»
16:07 benchable6 joined #perl6-dev
16:08 ugexe well anyways you can guess :)
16:09 eveo guess what?
16:09 ugexe something missed in Proc changes
16:09 eveo It now returns false?
16:09 eveo Ah
16:09 ugexe it would get sunk before
16:09 eveo s: Proc, 'Bool', \()
16:09 SourceBaby eveo, Sauce is at https://github.com/rakudo/rakudo/blob/14d7571/src/core/Mu.pm#L96
16:10 eveo s: Proc.new, 'Bool', \()
16:10 SourceBaby eveo, Sauce is at https://github.com/rakudo/rakudo/blob/14d7571/src/core/Proc.pm#L151
16:10 eveo ugexe: sunk where?
16:10 eveo at the end of run, I guess
16:11 eveo Wait, no, I'm not following.
16:11 eveo A sunk proc would explode.
16:11 ugexe yeah you're right, i thought there was more magic in that method
16:16 ugexe so probably need to add some awaits in methods that return exitcode related stuff?
16:19 BenGoldberg joined #perl6-dev
16:33 ugexe or self.result.exitcode
16:36 eveo I don't get how this works now.
16:36 jnthn Before it used, oddly enough, close_i to do the blocking. Except it I think got stuff messed up when both :err and :out were specified
16:36 eveo Proc is in terms of Proc::Async, but Proc::Async is in terms of Proc?
16:36 jnthn hah
16:37 jnthn Proc::Async just uses a Proc instance to hold the exit code
16:37 eveo Here's Proc::Async returning a Proc and the Proc we're running gets its status set from Proc::ASync's proc: https://github.com/rakudo/rakudo/blob/nom/src/core/Proc.pm#L138
16:37 eveo Ah
16:42 robertle joined #perl6-dev
16:42 jnthn I'll have a look on my Windows box tonight at what's going on there
16:43 ugexe not just windows tho
16:45 ugexe I have a couple bug reports for zef that stem from `return ?$proc` returning false when everything worked - both on OSX and I can reproduce on linux
16:46 ugexe the windows thing might stem from that, but seems like its really just misparsing a line from what I can tell
16:48 eveo I'm looking into the return ?$proc thing now
16:48 jnthn Were :in/:out/:err used?
16:48 eveo And it looks like our current behaviour is correct.
16:48 jnthn eveo: As in, you think we were wrong before and that wrong behavior was relied on?
16:49 eveo jnthn: I think Proc.Bool needs to throw if it's called before all the opened pipes were closed.
16:49 jnthn Did it do that before?
16:49 eveo here, .Bool returns False because $!exitcode is still at -1 because pipes weren't closed. -->  my $proc = run "ls", :out, :err; my $result = ?$proc; say $result
16:49 eveo Before exitcode was 0 if you didn't close pipes, IIRC
16:50 jnthn argh why would you even test that before using the output? ;P
16:50 eveo And we had a bug report with people saying that it's wrong, but the real problem was their programs weren't closing the pipes, so I think the proper solution is to throw in such cases
16:50 ugexe because there used to be other bugs that worked around where closing .out before .err (or at all) would cause issues
16:51 jnthn :/
16:51 eveo Oh, wait, default exitcode was -1 even before refactor
16:51 jnthn Yeah, I left the -1 in place
16:51 eveo Well, the attribute value, not what .exitcode returned
16:51 [TuxCM] joined #perl6-dev
16:51 ugexe that also works as a silent process
16:52 eveo cpan@perlbuild4~/CPANPRC/rakudo (nom)$ ./perl6 -e 'my $proc = run "ls", :out, :err; .err.close, .out.close with $proc; my $result = ?$proc; say $result'
16:52 eveo True
16:52 eveo So yeah, it works fine now.
16:53 eveo cpan@perlbuild4~/CPANPRC/rakudo (nom)$ ./perl6 -e 'my $proc = run "ls", :out, :err; .err.close with $proc; my $result = ?$proc; say $result'
16:53 eveo True
16:53 eveo hmmm... but the .out pipe is still open :\
16:53 * eveo re-reads the code
16:54 eveo Oh, nevermind. This build has my stupid patch in it.
16:54 * jnthn heads home
16:54 jnthn bbl
16:55 ugexe my $proc = run "ls", :out, :err; say $proc.Bool; sleep 5; say $proc.Bool # this just doesn't feel right, despite not closing the pipes
16:57 ugexe for :in yeah i get it
16:58 eveo That gives False False for both for me.
16:58 eveo I think we can inspect the promise when .sink/.Bool/.exitcode/.status are called and if it's finished, we can clean up the pipes and return right results.
16:59 eveo Or not
16:59 eveo Oh, we can leave the pipes as is and set the exit code
16:59 eveo and status
16:59 ugexe that was my gut feeling
17:00 * ugexe that zoffix would figure it out
17:00 eveo Gonna do that in ~5 hours, unless someone beats me to it
17:00 * eveo &
17:01 eveo maybe not even just inspect, but await even
17:01 * eveo &&
17:32 ugexe `perl6 -e 'run "xxx", :out, :err;'` is another related behavior change
17:35 TimToady so of *course* someone shoots up the town just before TPC, again, sigh...
17:36 TimToady .oO(it's the Wild, Wild East all over again...)
17:36 perlpilot what?  /me needs to watch some news it seems
17:36 perlpilot TimToady: At least they aren't shooting up the conference!
17:38 ugexe one of the other possible venues, milwaukee, also just had some shootings
17:51 eveo "again"? :)
17:51 eveo Must be Python programmers X)
17:52 AlexDaniel joined #perl6-dev
17:53 eveo m: with "abcd" { say gather for 0.. .chars-1 -> $i { for $i+1 .. .chars -> $j { .substr($i, $j).take } } }
17:53 camelia rakudo-moar 14d757: OUTPUT: «(a ab abc abcd bc bcd bcd cd cd d)␤»
17:53 eveo wonder if there's a shorter and nicer way to do that...
17:53 eveo the "subsequences" in a string
17:56 eveo m: with "abcd" { say gather for 0.. (.chars-1) -> $i { .substr($i, all $i .. .chars)».take } }
17:56 camelia rakudo-moar 14d757: OUTPUT: «( a ab abc abcd b bc bcd bcd cd cd cd d d)␤»
18:01 ugexe probably some way using array slices
18:03 perlpilot IF the order didn't matter, you could use .combinations  :)
18:04 perlpilot Though, the problem did make me wonder if there could be an enhancement to rotor that would take a coderef that would modify the steps as it went.
18:05 [Coke] eveo - do you want those dupes?
18:05 eveo gah, swap killed my box trying a way
18:05 eveo perlpilot: I don't want combinations tho. I want contiguous subsequences
18:06 eveo [Coke]: nope
18:07 jnthn Just pusehd a MoarVM patch to unbust precomp on Windows (it was indeed a bad arg quoting option)
18:07 eveo it don't matter much. I just saw Haskell has subsequences routine and wondered what it was
18:07 eveo what it would be in p6
18:08 perlpilot I suppose you could construct a regex with :overlap that would do it.
18:09 perlpilot (maybe)
18:10 b2gills m: say sort "abcd" ~~ m:ex/.+/
18:10 camelia rakudo-moar 14d757: OUTPUT: «(「a」 「ab」 「abc」 「abcd」 「b」 「bc」 「bcd」 「c」 「cd」 「d」)␤»
18:10 perlpilot yeah, :exhaustive is better  :)
18:10 eveo b2gills++
18:10 perlpilot b2gills++
18:12 perlpilot though, you'd have to do some sort of transform if your characters weren't in lexicographic order
18:12 eveo What do you mean?
18:12 eveo m: say "deadbeef" ~~ m:ex/.+/
18:12 camelia rakudo-moar 14d757: OUTPUT: «(「deadbeef」 「deadbee」 「deadbe」 「deadb」 「dead」 「dea」 「de」 「d」 「eadbeef」 「eadbee」 「eadbe」 「eadb」 「ead」 「ea」 「e」 「adbeef」 「adbee」 「adbe」 「adb」 「ad」 「a」 「dbeef」 「dbee…»
18:13 eveo Seems to work fine
18:13 b2gills m: say "abcd" ~~ m:ex/.+?/ # just make it non-greedy
18:13 camelia rakudo-moar 14d757: OUTPUT: «(「a」 「ab」 「abc」 「abcd」 「b」 「bc」 「bcd」 「c」 「cd」 「d」)␤»
18:13 perlpilot oh, so you don't care about the ordering of your subsequences
18:14 eveo Nah :)
18:14 perlpilot b2gills++ again.
18:14 perlpilot I apparently don't even have my brain all the way on today
18:20 dogbert17 joined #perl6-dev
18:20 [Coke] ah, nice. I had gotten as far as:
18:21 [Coke] <realizes that is giving the wrong ranges and backs away slowly>
18:22 dogbert17 [Coke]: I get a lot of strange spelling errors when I run make xtest in the docs repo. Is that normal?
18:22 dogbert17 xt/aspell.t ................ 2/309 # & ve 54 32: ver, vie, V, v, vec, veg, vet, Be, Ce, be, E, e, vex, VA, VI, Va, vi, vu, we, VD, VF, VG, VJ, VP, VT, Vt, vb, vm, vn, vs, vx, DE, Fe, GE, Ge, He, IE, Le, ME, Me, NE, Ne, OE, PE, Re, SE, Se, Te, Xe, ae, he, me, re, ye
18:22 dogbert17 xt/aspell.t ................ 6/309
18:22 dogbert17 # Failed test 'doc/Language/5to6-nutshell.pod6 has 1 spelling errors'
18:27 [Coke] m: sub a($a){(^$a X ^$a).grep({$_[0] < $_[1]})}; a(4).map({"asdf".substr(|$_)})
18:27 camelia rakudo-moar 14d757: ( no output )
18:28 [Coke] m: sub a($a){(^$a X ^$a).grep({$_[0] < $_[1]})}; say a(4).map({"asdf".substr(|$_)})
18:28 camelia rakudo-moar 14d757: OUTPUT: «(a as asd sd sdf df)␤»
18:28 [Coke] m: sub a($a){(^$a X ^$a).grep({$_[0] <= $_[1]})}; say a(4).map({"asdf".substr(|$_)})
18:28 camelia rakudo-moar 14d757: OUTPUT: «( a as asd s sd sdf df df f)␤»
18:28 [Coke] there. super complicated compared to the RE version. :)
18:28 [Coke] dogbert17: no. there is only one failure at the moment, and it has a ticket associated with it.
18:29 [Coke] ok 1 - doc/Language/5to6-nutshell.pod6 has no spelling errors
18:29 dogbert17 yeah, I made that commit, but I don't want to work on it until I understand why I get all these failures (you see one above)
18:30 [Coke] you have a local pending commit? put it in a branch or something, I can apply it locally and check.
18:31 dogbert17 I reverted the change to the pws file when I saw the failures, sadly that did not help. I must be missing something here
18:32 dogbert17 it seems to find errors which does not exist, very strange
18:35 dogbert17 interesting, if I run 'perl6 xt/aspell.t doc/Language/classtut.pod6' then no errors are found but if I run 'make xtest' then I get (among other things):
18:35 dogbert17 xt/aspell.t ................ 10/309 # & WHA 13 13: WHOA, WA, WHAM, WHAT, HA, WHO, WHY, YWHA, CHA, GHA, SHA, FHA, AHA
18:35 dogbert17 xt/aspell.t ................ 12/309
18:35 dogbert17 # Failed test 'doc/Language/classtut.pod6 has 1 spelling errors'
18:41 [Coke] what version of aspell do you have, and what dictionary?
18:42 dogbert17 @(#) International Ispell Version 3.1.20 (but really Aspell 0.60.7-20110707)
18:42 [Coke] I have Aspell 0.60.6.1 and aspell-dict-en (2017.01.22 from macports)
18:43 dogbert17 could it be /var/lib/dictionaries-common/aspell/aspell-en
18:44 * dogbert17 has never really used this program
18:45 [Coke] so, you currently have no local changes, and perl6 xt/aspell.t is failing?
18:46 dogbert17 yes
18:51 dogbert17 [Coke]: https://gist.github.com/dogbert17/f65d88b8fd952db97534b606b8c2bbc5
18:57 [Coke] note that it's complaining that 'brac' isn't a work, but that's never a word in that file.
18:57 [Coke] *word
19:00 dogbert17 yes, I saw that, strange indeed
19:03 geekosaur looks to me like it's dropping the last letter in all three cases
19:04 geekosaur second is probably WHAT, third is brace, first... given it's 5to6nutshell, I'd guess vec?
19:04 geekosaur so look for that word and the character immediately following
19:05 * geekosaur wonders if the numbers are line and character
19:06 * dogbert17 uses rakudobrew to switch back to 2016.12 and .....
19:06 dogbert17 the errors are gone
19:07 * dogbert17 errors out with 'This is Rakudo version 2017.05-439-g14d757131 built on MoarVM version 2017.05-85-g21ee1a504'
19:08 dogbert17 perhaps something for Zoffix to sink his teeth into
19:08 geekosaur ahah
19:08 eveo ?
19:08 dogbert17 eveo: backlog a little bit
19:08 geekosaur the first one is mishandling "you've"
19:09 * eveo mehs
19:09 geekosaur so I will guess xtest is leaving out some suffix processing option
19:11 * geekosaur did just notice a typo while looking at the second one
19:11 geekosaur "Perl 5, so it will not be detail  here. If you need to know the precedence and"
19:11 geekosaur "detail " -> "detailed"?
19:11 AlexDaniel is this from 5to6 docs?
19:12 geekosaur https://github.com/perl6/doc/blob/master/doc/Language/5to6-perlop.pod6#L22
19:12 AlexDaniel if you read any of these documents from top to bottom, you'll probably see many other issues…
19:12 geekosaur the fact that I spotted that immediately suggests so :)
19:14 [Coke] dogbert17: I'm using 2017.05-315-g160de7e6f atm.
19:14 [Coke] I'll upgrade later and try again. Also there's a doc ticket for something similar.
19:14 [Coke] https://github.com/perl6/doc/issues/975
19:15 [Coke] seems to be tied to a backslash there.
19:18 dogbert17 [Coke]: thx, will be interesting to hear what you find
19:19 perlpilot Is it a feature that the spell checker is checking code blocks too?  (I would have made it skip code)
19:19 perlpilot When I ran that test locally, I got failures in doc/Type/Supply.pod6 because of "GOOG" in a code example
19:27 [Coke] perlpilot: https://github.com/perl6/doc/issues/1375
19:27 [Coke] perlpilot: yes, because there are comments in code.
19:27 [Coke] it was a PITA, but now that have (HAD) a clean run, it's only stragglers like this that are an issue.
19:27 [Coke] if someone wants to fix #1375, that'd be awesome.
19:28 [Coke] and also, there's code in docs; class names, methods, etc. we have to whitelist those on the spellcheck anyway
19:52 stmuk_ joined #perl6-dev
20:00 dogbert17 the problem seems to occur when using :out from one Proc as :in for another proc. At that point the file sometimes gets cut short!
20:02 dogbert17 the last line of the cut 5to6-nutshell file looks like this '^A quick way to find the Perl 6 ve'
20:03 geekosaur glah. so what is copyng the data instead of dup-ing the fd?
20:06 dogbert17 geekosaur: the problem happens here: https://github.com/perl6/doc/blob/master/xt/aspell.t#L57
20:06 geekosaur no, it's triggered there. the actual problem is whatever is plumbing those together is doing it wrong
20:07 geekosaur i.e. the problem is in the implementation of run
20:07 dogbert17 could be, sometimes it works and sometimes it fails
20:09 geekosaur right, it'll depend on how much is waiting to be read at that point
20:10 dogbert17 I more or less changed the 'run aspell' line to 'run cat' instead and saved the output of several runs in order to compare.
20:10 geekosaur it should be just plumbing the file handles together at low level (fcntl(F_DUPFD)), instead it's copying whatever data is already available and the rest is lost
20:13 dogbert17 geekosaur: does this shed any light on what's happening: https://gist.github.com/dogbert17/c41f57896896d3111421f94c0d125d32
20:13 dogbert17 really afk &
20:14 geekosaur that shows it reading the current data and writing it to the other thing, yes
20:14 geekosaur which is exactly the wrong thing to do when plumbing filehandles
20:28 MasterDuke joined #perl6-dev
20:30 dogbert17 geekosaur: is wrong because things can go wrong or for performance reasons
20:30 geekosaur because if you are connecting handles together, you want to connect the *handles*, not the data
20:31 geekosaur say I spawn a long-running program and then want to connect its :out to another program's :in
20:31 geekosaur I want to connect the handle so the other program can keep reading as the long-running program generates output
20:32 geekosaur what it's actually doing, though, is copying whatever output happens to be available at the time you're connecting them, and writing it to the other program
20:32 geekosaur which is fairly useless
20:32 geekosaur so, what you want here is programA | programB
20:32 dogbert17 understood, thx for the explanation
20:33 geekosaur what run is doing is: programA >tmpfile & tail +0 tmpfile | programB
20:33 geekosaur which will terminate as soon as tail hits the end of tmpfile, whatever it happens to be at that time
20:33 geekosaur (note no -f on tail)
20:33 dogbert17 ahh
20:35 geekosaur anyway I would report this as a bug in the implementation of run
20:35 dogbert17 so in order for this to work, 'run' will have to be changed or [Coke] has to do some workaround in his code
20:36 geekosaur as I understand it, run is intended to work the way that code is using it (and as I described it)
20:36 geekosaur it's jst not implemented correctly for the :in(filehandle) case
20:37 dogbert17 aha, I should check RT, perhaps this has already been reported
20:37 geekosaur which should F_DUPFD the handle's file descriptor to the new process's fd 0 instead of copying data
20:40 dogbert17 I see that MasterDuke has already made a report, https://rt.perl.org/Public/Bug/Display.html?id=129291
20:41 dogbert17 at least I think he describes the same problem
20:41 geekosaur no, that's a different problem
20:42 geekosaur but it might be fixed as a side effect of fixing this, if the memory allocations in question are part of copying the data when it should be creating a pipe between the processes
20:47 japhb eveo: I'm not on Windows.  Haven't run a Windows box in ages, actually.  My problems were on Linux.
20:47 japhb eveo: If you still need a dump of the Inline::Python errors, I can do a fresh build attempt.
20:48 dogbert17 a quick valgrind doesn't uncover anything suspicious
20:48 eveo japhb: nah, it's fine. We've identified one of the problems that's likely the culprit.
20:48 eveo to be fixed soon
20:48 eveo as in 1-2 hours
20:49 geekosaur I'm making a rakudobug about the run() thing
20:50 japhb eveo: Ah, thank you kindly.
20:51 dogbert17 geekosaur++
20:56 japhb eveo: In https://perl6.party/assets/pics/toaster/htop.png why are some of the characters in '100.0%' red and others are yellow?
20:56 * eveo waits for geekosaur rakudobug
20:57 geekosaur just hit send
20:57 eveo geekosaur: what run thing? the exitcode?
20:57 geekosaur no
20:57 eveo Ah, got it
20:57 geekosaur :in($proc.out) copies whatever is currently readable from $proc.out instead of building a pipeline
20:58 eveo Is this a regression due to Proc refactor or has this been that way for ages, do you know?
20:58 geekosaur which currently means aspell.t in the docs repo is prone to spurious errors as it gets whatever happens to be available at that point instead of the full ouytput
20:58 geekosaur no idea
20:59 geekosaur I would suspect it's always been that way, though, as the implementation would have been fairly obviously different
21:00 geekosaur also my guess, from having poked at the moarvm <-> libuv interface before, is the current behavior is a placeholder because moarvm can't currently use the right libuv api
21:00 geekosaur but that's just a guess, I don't know at what level things are being done wrong
21:08 jnthn Bah, can't damn win
21:08 jnthn On Windows the plugging stuff together at fd level didn't work at all either :/
21:09 jnthn Thus the current attempt at portability
21:10 jnthn The only thing I've learned about I/O is you can never do the right thing :P
21:13 japhb jnthn: System calls in general are tricky to get right.  There are very particular reasons why the Unix APIs are the way they are, even though most of them seem quite weird at first glance.
21:14 jnthn japhb: Yeah, in this case I don't even really follow the issue though. In that, it's an asycnrhonous reader that, whenever it gets something, reacts by writing it to the input of the next thing.
21:16 japhb stdio buffering?
21:18 japhb (I'm wondering if the current copying method races at eof/exit of input program)
21:19 jnthn Ah, a write/close race sounds more possible
21:20 geekosaur [14 20:31:08] <geekosaur> say I spawn a long-running program and then want to connect its :out to another program's :in
21:20 geekosaur copying data at run() time of the second program will never handle this correctly
21:21 geekosaur you would have to run() the first thing, capture all the output to a file, then run() the second thing with input from that file
21:21 geekosaur there is no way to connect programs in a pipeline as currently implemented
21:22 jnthn geekosaur: I'm not disagreeing, I'm asking why.
21:22 geekosaur ?
21:22 geekosaur how long are you going to wait and/or copy stuff to the new program?
21:22 geekosaur [14 20:32:41] <geekosaur> so, what you want here is programA | programB
21:22 geekosaur [14 20:33:09] <geekosaur> what run is doing is: programA >tmpfile & tail +0 tmpfile | programB
21:22 geekosaur [14 20:33:36] <geekosaur> which will terminate as soon as tail hits the end of tmpfile, whatever it happens to be at that time
21:23 geekosaur bugffering will skew this even more, but no amount of buffering aside from block-until-programA-exits will fix this if you are going to implement it as a copy operation
21:24 geekosaur also I'd kinda expect perl 6 to be able to do pipes better than ms-dos 2 did
21:27 jnthn OK, my understanding of what run is doing doesn't match what you claim it's doing, so there's clearly something I don't understand correctly.
21:27 jnthn I don't think at this time of night I'm going to figure out what. Will re-read tomorrow.
21:28 geekosaur I hope run was not actually specced this way
21:28 MasterDuke m: my $p = run(:out, $*EXECUTABLE, "-e", "say qq|abc\n123|; sleep 1; say qq|def\n456|"); run(:out, :in($p.out), $*EXECUTABLE, "-e", q|say $*IN.read|).out.slurp(:close).say
21:28 camelia rakudo-moar 14d757: OUTPUT: «run is disallowed in restricted setting␤  in sub restricted at src/RESTRICTED.setting line 1␤  in sub run at src/RESTRICTED.setting line 14␤  in block <unit> at <tmp> line 1␤␤»
21:28 geekosaur because the currene behavior is not what I expect if I tell it to connect handles together
21:29 jnthn *sigh* the whole Proc thing kinda just happened
21:29 geekosaur if it is specced, thoug, very well, we are ms-dos 2
21:29 geekosaur I will wait for perl 6 to reinvent windows nt before expecting it to be able to create pipelines
21:31 geekosaur so as I read the Proc docs, I still expect it to create a pipe
21:31 jnthn Yes, it's a reasonable expectation
21:32 nebuchadnezzar joined #perl6-dev
21:34 ugexe before I think pipes might have worked on moar + linux, but thats it
21:34 ugexe definitely didnt work right on windows
21:34 geekosaur I think windows requires you to do things differently, yes
21:35 geekosaur a pipe is a virtual filesystem object, and you can't just dup() stuff together like you do on unix
21:35 geekosaur the first run() would have to know to allocate a pipe for the second one
21:35 geekosaur if I understand this correctly, that is
21:37 ugexe right, but so ideally does it "do things differently" on windows to make it work "right", or is it so fundamentally different that windows should just do what it does now (wait for first proc to finish)
21:37 * geekosaur goes digging
21:37 geekosaur it's *not* waiting
21:37 geekosaur that's part of the problem
21:37 geekosaur the wait version would be slow and annoying but work
21:37 geekosaur it's copying whatever happens to be available right then, and losing the resty
21:37 MasterDuke should what i tried above work? where work means print (essentially) the same thing as "abc\n123def456".encode
21:38 geekosaur MasterDuke, ideally it would. for that short an input, it might work as expected most of the time
21:38 geekosaur but sometimes it'll be truncated
21:38 MasterDuke if i use $*IN.lines i get all the output of the first run. if i do $*IN.read i get `Failed to write bytes to filehandle: Broken pipe`
21:39 geekosaur yes, that's another thing that will happen now
21:39 MasterDuke and only the first part of the output, i.e., "abc\n123".encode
21:41 MasterDuke i'm just thinking we should have some good tests in roast of what should work/happen, but i'm not 100% myself on how things should be
21:41 geekosaur because tje second run does not wait for the first to finish, and the pipe is not connected between them so data would keep flowing; whatever data happens to be available for the parent perl6 to read at the time it performs the second run() is what goes to the second run(), and the first run() will get a write error because the parent perl6 stopped reading and exited after doing the second run()
21:51 eveo m: sub foo ($x) { LEAVE { CATCH { default { say 70; False } }; $x and die; True }; say 42}; dd foo 0
21:51 camelia rakudo-moar 14d757: OUTPUT: «WARNINGS for <tmp>:␤Useless use of constant value True in sink context (line 1)␤42␤Bool::True␤»
21:52 * eveo would've thought LEAVE's value would be used for return.
21:57 * geekosaur wonders if pipes not working on windows is a libuv bug
21:58 jnthn eveo: No, that wouldn't make sense given there can be multiple of them and you often use them to clean up
21:58 eveo Ah cool
21:59 geekosaur actually I am looking at the low level windows stuff in msdn and it looks like it should work, with one caveat
21:59 jnthn sub f() { my $fh = open "foo"; LEAVE close $fh; $fh.slurp } # boring example of when you'd really not want what LEAVE returns as the return value :)
21:59 geekosaur (windows file handles are not inherited by default like they are on unix, so you need to set the inherit flag for pipes to work for IPC like this)
22:00 geekosaur but, this level of logic should be in libuv
22:01 eveo jnthn: when is @!post-spawn supposed to run?
22:01 eveo in Proc it closes stdin
22:02 eveo I mean a block is pushed onto it to close it
22:04 eveo maybe in !await-if-last-handle if it's the last handle...
22:05 eveo Ah, it already gets closed before the call.
22:06 eveo Well, I'm gonna shove it to run to where we wait for the proc to finish running and see if any stresstests explode
22:06 * eveo dines first
22:08 ugexe i dunno how valid this is, but https://gist.github.com/ugexe/118f826fa96ec131b711d23762dd1df6 sometimes shows the correct total and sometimes less
22:08 ugexe on the plus side it no longer deadlocks though :)
22:11 geekosaur "sometimes less" would be bad, but this should be a different issue
22:11 ugexe https://rt.perl.org/Public/Bug/Display.html?id=129882 # where i cribbed the example code from
22:27 jnthn eveo: After the $proc.start line
22:28 eveo jnthn: wouldn't that immediately close .stdin and prevent priting to .in? in `my $proc = run :in, "meows"; $proc.in.write: "foos"`
22:29 jnthn eveo: No, the :in case is handled in the elsif $in === True branch up above
22:29 geekosaur if this is what it sounds like, it's closing the parent's copy of the read side of the pipe for :in
22:30 geekosaur which the parent does not need; it's for the child to read
22:30 geekosaur the parent wants the write end
22:30 eveo jnthn: Ah, I see. Thanks.
22:30 BenGoldberg joined #perl6-dev
22:31 jnthn And it only delgates the .close-stdin to the underlying Proc::Async when $proc.in.close is called
22:31 jnthn I'm utterly confused how ugexe's gist is losing anything, given on-write correctly `await`s also
22:35 jnthn Anyway, sleep time
22:35 jnthn 'night
22:37 eveo night
22:41 ugexe well, it does only seem to happen when the buffer is above a certain size
22:41 ugexe so could be either the write or the read
22:47 BenGoldberg joined #perl6-dev
22:53 eveo Don't see anything wrong with pipes that would make that code wrong.
22:53 eveo Though found another rabbit hole:
22:53 eveo $ ./perl6 -e 'with Proc::Async.new: «"$*EXECUTABLE" -e "say 42"» { (await .start).exitcode.say }'
22:53 eveo 42
22:53 eveo 1
22:57 BenGoldberg m: dd run("echo")
22:57 camelia rakudo-moar 14d757: OUTPUT: «run is disallowed in restricted setting␤  in sub restricted at src/RESTRICTED.setting line 1␤  in sub run at src/RESTRICTED.setting line 14␤  in block <unit> at <tmp> line 1␤␤»
22:58 BenGoldberg m: dd Proc.new("echo 42")
22:58 camelia rakudo-moar 14d757: OUTPUT: «Proc is disallowed in restricted setting␤  in sub restricted at src/RESTRICTED.setting line 1␤  in method new at src/RESTRICTED.setting line 32␤  in block <unit> at <tmp> line 1␤␤»
22:59 eveo It's pretty scare to have this huge breakage and not a single failing stresstest :/
22:59 eveo *scary
23:00 eveo Which ultimately is the result of no-test coding.
23:01 eveo Even with crappy TDD you cover most bases... like... exit code -_-
23:05 BenGoldberg I notice on <https://docs.perl6.org/type/Proc>, methods in, out, err aren't documented, except for the example code.  They also don't show up in the searchbox.
23:06 BenGoldberg I wonder how horrible it would be to add a new requirement to every rakudo class and method: Either it must be explicitly marked as internal, or it must have documentation.
23:06 geekosaur they're documented unhelpfully
23:06 geekosaur "$in, $out and $err are..."
23:06 eveo "Unknown QAST node type NQPMu" ... I think I'm going the wrong way :)
23:06 geekosaur well, the documentation is helpful, just not findable
23:08 BenGoldberg If we generated docs from the code, then everything not marked as internal would have some sort of findable doc :)
23:09 cognominal joined #perl6-dev
23:11 eveo ugh this is all falling appart.
23:11 eveo The whole Proc::Async returns a Proc, which is implemented in terms of Proc::Async is backfiring
23:13 BenGoldberg c: Proc::Async, 'start'
23:13 committable6 BenGoldberg, ¦Proc::Async: «Cannot find this revision (did you mean “ccd2bcc”?)» ¦: «Cannot find this revision (did you mean “all”?)»
23:14 BenGoldberg Undercover, Proc::Async, 'start'
23:16 eveo cover: Proc::Async, 'start'
23:17 eveo Undercover: helpo
23:17 eveo Undercover: help
23:17 eveo stupid robot
23:17 Undercover eveo, Failed to figure out coverage status on http://perl6.WTF/SETTING__src_core_Proc_Async_pm.coverage.html#L210
23:17 Undercover eveo, Use cover: trigger with args to give to sourcery sub. e.g. cover: Int, 'base'. See http://modules.perl6.org/dist/CoreHackers::Sourcery
23:27 eveo This is unfun now
23:28 eveo bastard just doesn't want to be awaited on :/
23:31 BenGoldberg joined #perl6-dev
23:37 AlexDaniel joined #perl6-dev
23:45 eveo ehehe. Called a method on the wrong thing and was catching exception about that
23:56 AlexDaniel “the problem seems to occur when using :out from one Proc as :in for another proc. At that point the file sometimes gets cut short!”
23:56 AlexDaniel is it a new problem?
23:56 AlexDaniel I think whateverables were suffering from something like this also…

| Channels | #perl6-dev index | Today | | Search | Google Search | Plain-Text | summary