Perl 6 - the future is here, just unevenly distributed

IRC log for #perl6-dev, 2016-09-14

| Channels | #perl6-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:07 MasterDuke joined #perl6-dev
01:35 dalek rakudo/nom: 61a71f5 | MasterDuke17++ | src/core/metaops.pm:
01:35 dalek rakudo/nom: Fix RT #118223
01:35 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/61a71f5ad3
01:35 dalek rakudo/nom: beec02a | (Zoffix Znet)++ | src/core/metaops.pm:
01:35 dalek rakudo/nom: Merge pull request #859 from MasterDuke17/nom
01:35 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=118223
01:35 dalek rakudo/nom:
01:35 dalek rakudo/nom: Fix RT #118223
01:35 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=118223
01:35 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/beec02a6fa
01:48 ilbot3 joined #perl6-dev
01:48 Topic for #perl6-dev is now Perl 6 language and compiler development 2.0 | Logs at http://irclog.perlgeek.de/perl6-dev/today
02:49 travis-ci joined #perl6-dev
02:49 travis-ci Rakudo build failed. Zoffix Znet 'Merge pull request #859 from MasterDuke17/nom
02:49 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159764045 https://github.com/rakudo/rakudo/compare/626a2220e719...beec02a6fa69
02:49 travis-ci left #perl6-dev
02:49 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
05:58 [Tux] This is Rakudo version 2016.08.1-184-gbeec02a built on MoarVM version 2016.08-43-g3d04391
05:58 [Tux] csv-ip5xs       10.830
05:58 [Tux] test            16.131
05:58 [Tux] test-t           6.838
05:58 [Tux] csv-parser      17.228
06:06 nine mst: I really thought so. But a quick test shows me wrong. I have no idea how this works and where the INIT block is called. But I guess, I should just be glad. This opens a lot of possibilities :)
06:47 dalek rakudo/nom: 5f2e96b | lizmat++ | src/core/Channel.pm:
06:47 dalek rakudo/nom: Make Channel.elems return a Failure
06:47 dalek rakudo/nom:
06:47 dalek rakudo/nom: Rather than silently eating the currently available values on the
06:47 dalek rakudo/nom: Channel and then returning the number of values eaten.
06:47 dalek rakudo/nom:
06:47 dalek rakudo/nom: Inspired by discussion started at
06:47 dalek rakudo/nom:
06:47 dalek rakudo/nom:   http://irclog.perlgeek.de/perl6/2016-09-14#i_13206149
06:47 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/5f2e96b8c0
06:48 brrt joined #perl6-dev
07:25 |Tux| joined #perl6-dev
07:42 travis-ci joined #perl6-dev
07:42 travis-ci Rakudo build failed. Elizabeth Mattijsen 'Make Channel.elems return a Failure
07:42 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159797732 https://github.com/rakudo/rakudo/compare/beec02a6fa69...5f2e96b8c046
07:42 travis-ci left #perl6-dev
07:42 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
08:00 |Tux| joined #perl6-dev
08:05 RabidGravy joined #perl6-dev
08:58 jnthn morning, #perl6-dev
09:10 brrt good morning, jnthn
09:53 jnthn Well, there's the first bug of the day hunted down...
09:54 jnthn RT #129249 fwiw
09:54 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=129249
09:55 jnthn Turns out we mis-built the QAST tree, causing an explosion in the thingy that makes NFAs out of regexes
09:59 DrForr Uh. Looking at the report...
09:59 jnthn m: use Test; my %what = foo => 42, bar => 43; my $m = "foo3bar4" ~~ /$<cat>=@(%what.keys)  4/; is $m, "bar4"; is $m<cat>, "bar"
09:59 camelia rakudo-moar 5f2e96: OUTPUT«===SORRY!===␤Cannot find method 'rxtype' on object of type QAST::Op␤»
10:00 DrForr Yeah, looking now. I'm right now worried more aobut what's going to change the NQPMatch output.
10:01 Zoffix hm
10:01 DrForr *about
10:02 Zoffix Those travis failures are because I fixed the harness yesterday, so now it's correctly detecting the failing test suite.
10:10 dalek rakudo/nom: c3b044d | jnthn++ | src/Perl6/Actions.nqp:
10:10 dalek rakudo/nom: Correct code-gen of /$<x>=@(...)/ constructs.
10:10 dalek rakudo/nom:
10:10 dalek rakudo/nom: An argument intended for INTERPOLATE was placed too high up in the
10:10 dalek rakudo/nom: tree. This not only meant it would not be passed, but confused the
10:10 dalek rakudo/nom: NFA builder, which then exploded (which is why compilation of the
10:10 dalek rakudo/nom: above construct failed). This fixes the code-gen, making things
10:10 dalek rakudo/nom: happy again.
10:10 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/c3b044df16
10:10 dalek roast: 25168ae | jnthn++ | S05-capture/alias.t:
10:10 dalek roast: Test to cover RT #129249.
10:10 dalek roast: review: https://github.com/perl6/roast/commit/25168ae7cf
10:10 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=129249
10:11 dalek rakudo/nom: 135bba9 | (Zoffix Znet)++ | .travis.yml:
10:11 dalek rakudo/nom: Ignore JVM failures
10:11 dalek rakudo/nom:
10:11 dalek rakudo/nom: Rakudo's test suite has been failing for a while on JVM, but this went on silent on Travis because the test harness was erroneously indicating success with its exit code. The harness was fixed in 626a2220e7195 and now we're getting hammered on IRC with JVM test suit failures.
10:11 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/135bba9a86
10:12 Zoffix ^ I muted JVM failures for now. make test; needs to start passing again and right now it seems all of NativeCall tests fail right away with "dubious" result.
10:43 dalek rakudo/nom: 63e6b81 | (Zoffix Znet)++ | t/04-nativecall/05-arrays.t:
10:43 dalek rakudo/nom: Remove trailing whitespace
10:43 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/63e6b815aa
10:47 dalek rakudo/nom: cc75c82 | (Zoffix Znet)++ | / (2 files):
10:47 dalek rakudo/nom: Refactor dispatch
10:47 dalek rakudo/nom:
10:47 dalek rakudo/nom: Since the slurpy just shuttles values to non-slurpy variant and
10:47 dalek rakudo/nom: can handle the cases for both multies, we do not need any multies
10:47 dalek rakudo/nom: at all.
10:47 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/cc75c825f9
11:04 travis-ci joined #perl6-dev
11:04 travis-ci Rakudo build errored. Jonathan Worthington 'Correct code-gen of /$<x>=@(...)/ constructs.
11:04 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159833750 https://github.com/rakudo/rakudo/compare/5f2e96b8c046...c3b044df1695
11:04 travis-ci left #perl6-dev
11:04 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
11:36 lizmat Zoffix: re 63e6b815aa , there is a reason for having 2 candidates
11:37 lizmat calling a method with @a with a signature of *@a is 6x slower than when if the signature is @a
11:38 travis-ci joined #perl6-dev
11:38 travis-ci Rakudo build failed. Zoffix Znet 'Ignore JVM failures
11:38 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159833874 https://github.com/rakudo/rakudo/compare/c3b044df1695...135bba9a8643
11:38 travis-ci left #perl6-dev
11:38 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
11:39 lizmat Zoffix: whereas the overhead of shuttling from the *@a to the @a is only 16% extra, which the JIT will probably OSR anyway
11:39 lizmat when called sufficiently often enough
11:42 lizmat in fact: if this is all about handling the empty array / no parameters case, we probably want an extra candidate
12:05 dalek rakudo/nom: 838c001 | lizmat++ | lib/NativeCall/Types.pm6:
12:05 dalek rakudo/nom: Optimize CArray.new some more
12:05 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/838c0011ac
12:05 lizmat Zoffix: ^^^ feel free to revert if you think I missed the point  :-)
12:06 * lizmat goes back to translating
12:07 MetaZoffix joined #perl6-dev
12:08 MetaZoffix lizmat++ thanks for fixing it. I'll keep in mind the cost of slurpies from now on.
12:27 travis-ci joined #perl6-dev
12:27 travis-ci Rakudo build failed. Zoffix Znet 'Remove trailing whitespace'
12:27 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159839919 https://github.com/rakudo/rakudo/compare/135bba9a8643...63e6b815aa66
12:27 travis-ci left #perl6-dev
12:27 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
12:31 MetaZoffix Hm. Why does it still report JVM failures? :/
12:35 MetaZoffix Weird. If I go to https://github.com/rakudo/rakudo/blob/nom/.travis.yml I see my changes in the file (... I think...) but my commit doesn't show up in history :/
12:36 dalek rakudo/nom: 4f0a551 | (Zoffix Znet)++ | .travis.yml:
12:36 dalek rakudo/nom: Remove -os from allow_failures for JVM
12:36 dalek rakudo/nom:
12:36 dalek rakudo/nom: To replicate the version in fcf27ae07eb411d0, as the current one doesn't appear
12:36 dalek rakudo/nom: to work to mute the failures.
12:36 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/4f0a551b88
12:54 mst nine: oh good. now you see why I reacted so horrified at the original misunderstood version ;)
13:04 travis-ci joined #perl6-dev
13:04 travis-ci Rakudo build failed. Zoffix Znet 'Refactor dispatch
13:04 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159840556 https://github.com/rakudo/rakudo/compare/63e6b815aa66...cc75c825f90b
13:04 travis-ci left #perl6-dev
13:04 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
13:13 dalek nqp: f960976 | jnthn++ | tools/build/MOAR_REVISION:
13:13 dalek nqp: Bump MOAR_REVISION for EVAL leak fix.
13:13 dalek nqp: review: https://github.com/perl6/nqp/commit/f960976428
13:14 dalek rakudo/nom: 43b4f3d | jnthn++ | tools/build/NQP_REVISION:
13:14 dalek rakudo/nom: Get MoarVM with EVAL leak fix.
13:14 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/43b4f3d02f
13:17 * lizmat pulls, builds and spectests
13:17 jnthn That fixes RT #129161
13:17 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=129161
13:17 MetaZoffix Oh, sweet. jnthn++ :)
13:18 jnthn It's kinda hard to write a test. I pondered exposing memory use stats, but even then it's still kinda hard in the sense that VMs take a while finding their memory ceiling "sweet spot"
13:18 jnthn Anyway, the fix is in for now. moar-ha++ for helping find it. :-)
13:18 lizmat jnthn: perhaps test on a side-effect of the leak, like contexts hanging around when they shouldn't ?
13:19 jnthn Well, it'd be an interesting test then in so far as the best way to write it would be by taking a heap snapshot and looking at it. :-)
13:19 jnthn But yeah, that'd be the good way to do it :)
13:20 jnthn It's a bit of a yak shave though...
13:21 jnthn NeuralAnomaly: status
13:21 NeuralAnomaly jnthn, [✘] Next release will be in 2 days and 15 hours. Since last release, there are 49 new still-open tickets (0 unreviewed and 0 blockers) and 10 unreviewed commits. See http://perl6.fail/release/stats for details
13:21 jnthn 0 blockers. Yay :)
13:21 brrt \o/
13:27 MasterDuke_ joined #perl6-dev
13:31 MasterDuke_ lizmat: how expensive are where clauses in multis?
13:31 lizmat MasterDuke_: very, as they disallow optimisation
13:32 MasterDuke_ it struck me after looking at your recent optimization to CArray.new that the "if @values.elems" could be implemented as "multi method new(@values where *.elems) {...}" and "multi method new(@values) { nqp::create(self) }"
13:32 lizmat every call needs to check all the candidates, run the where condition code to see if there's a match
13:32 lizmat yeah, that would be expensive
13:32 MasterDuke_ i suspected that would be the case
13:33 lizmat anyway, that is just handling the case where you would do my @a; CArray.new(@a)   aka, sending an empty array
13:33 lizmat the CArray.new() case is handled by the first candidate
13:34 MasterDuke_ yeah, i was just really thinking of that as an example (it triggered the thought)
13:35 MasterDuke_ the where clause needs to be run even if there's a tighter? (forgetting the precise word) multi?
13:36 lizmat MasterDuke: how can it be sure it is tighter without checking ?
13:37 MetaZoffix .oO( narrower )
13:38 MasterDuke_ well, i'd consider "multi nethod new() {...}" narrower than "multi method new(@values where {any code whatsoever})"
13:38 MasterDuke_ MetaZoffix++ that's the word i was looking for
13:39 lizmat MasterDuke_: true, that would work
13:42 MasterDuke_ obviously the implementation of checking candidates and such makes a difference, but at least it sort of seems in a high-level way, that the total amount of "logic" is the same
13:43 MasterDuke_ between "multi method new(@values) { if @values.elems {...} else { nqp::create(self) }" and "multi method new(@values where *.elems) {...}; multi method new(@values) { nqp::create(self) }"
13:43 MetaZoffix The one with .where would check .elems twice, because we use that value.
13:43 MetaZoffix for $n
13:44 MasterDuke_ ooh, can you do "where *.elems -> $n"? because that would be cool
13:44 moritz yes
13:44 travis-ci joined #perl6-dev
13:44 travis-ci Rakudo build failed. Elizabeth Mattijsen 'Optimize CArray.new some more'
13:44 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159854364 https://github.com/rakudo/rakudo/compare/cc75c825f90b...838c0011acdf
13:44 travis-ci left #perl6-dev
13:45 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
13:45 MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where my $n = .elems ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
13:45 camelia rakudo-moar 43b4f3: OUTPUT«none␤|␤slurpy␤|␤0 pos␤|␤3 pos␤»
13:45 MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where say('CHECK!') && my $n = .elems ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
13:45 camelia rakudo-moar 43b4f3: OUTPUT«none␤|␤slurpy␤|␤CHECK!␤CHECK!␤0 pos␤|␤CHECK!␤CHECK!␤3 pos␤»
13:46 MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where .elems -> $n ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
13:46 camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------> { say "slurpy" }; multi x(@ where .elems⏏ -> $n ) { say "$n pos" }; x; say '|'; x␤…»
13:46 MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where *.elems -> $n ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
13:46 camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------>  say "slurpy" }; multi x(@ where *.elems⏏ -> $n ) { say "$n pos" }; x; say '|'; x␤…»
13:46 MetaZoffix moritz: yes?
13:46 MetaZoffix Doesn't seem to work
13:47 moritz MetaZoffix: oh, in a signature
13:47 moritz MetaZoffix: I missed that part, sorry
13:47 moritz in mainline code it should work
13:47 moritz m: where 42 -> $n { say $n }
13:48 camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------> where 42⏏ -> $n { say $n }␤    expecting any of:␤        infix␤        infix s…»
13:48 moritz wait, what's the new if defined thingy called?
13:48 * moritz totally out of the loop
13:48 MetaZoffix with
13:48 lizmat pretty sure the scope of a sub and the scope of the code executed in a .where are different
13:48 moritz m: whith 42 -> $n { say $n }
13:48 camelia rakudo-moar 43b4f3: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unexpected block in infix position (missing statement control word before the expression?)␤at <tmp>:1␤------> whith 42⏏ -> $n { say $n }␤    expecting any of:␤        infix␤        infix s…»
13:48 lizmat *with
13:48 moritz m: with 42 -> $n { say $n }
13:48 camelia rakudo-moar 43b4f3: OUTPUT«42␤»
13:48 moritz helps to spell it correctly :-)
13:48 MetaZoffix lizmat: but the above code shows the scope is visible.
13:49 lizmat ah?
13:49 MetaZoffix And I know from IRC::Client that regex matches keep the $<captures>
13:49 MetaZoffix m: multi x () { say "none" }; multi x(*@) { say "slurpy" }; multi x(@ where say('CHECK!') && my $n = .elems ) { say "$n pos" }; x; say '|'; x 1, 2, 3; say '|';x []; say '|';x [1, 2, 3]
13:49 camelia rakudo-moar 43b4f3: OUTPUT«none␤|␤slurpy␤|␤CHECK!␤CHECK!␤0 pos␤|␤CHECK!␤CHECK!␤3 pos␤»
13:49 MetaZoffix The "3 pos" is $n = .elems
13:49 lizmat hmmm... TIL  :-)
13:49 moritz thunks don't introduce a new scope
13:50 moritz m: sub f(@x where my $n = @x.elems) { say $n }; f <a b>
13:50 camelia rakudo-moar 43b4f3: OUTPUT«2␤»
13:50 moritz MetaZoffix: ^^ better? :-)
13:50 moritz m: sub f(@x where my $n = @x.elems) { say $n }; f []
13:50 camelia rakudo-moar 43b4f3: OUTPUT«0␤»
13:50 moritz huh, why didn't that fail to dispatch?
13:50 lizmat because it's not a multi ?
13:50 jnthn Precedence
13:51 moritz m: sub f(@x where (my $n = @x.elems)) { say $n }; f []
13:51 camelia rakudo-moar 43b4f3: OUTPUT«0␤»
13:51 jnthn m: sub f(@x where (my $n = @x.elems)) { dd @x }; f []
13:51 camelia rakudo-moar 43b4f3: OUTPUT«[]␤»
13:51 moritz lizmat: where should be checked for nonly's too, no?
13:51 jnthn Hmm
13:51 jnthn But I'm a bit surprised by that
13:51 moritz m: sub f(@x where (my $n := @x.elems)) { dd @x }; f []
13:51 camelia rakudo-moar 43b4f3: OUTPUT«[]␤»
13:51 MetaZoffix m: sub f(@x where (my $n = .elems) != 0) { dd @x }; f []
13:51 camelia rakudo-moar 43b4f3: OUTPUT«Constraint type check failed for parameter '@x'␤  in sub f at <tmp> line 1␤  in block <unit> at <tmp> line 1␤␤»
13:52 MetaZoffix It's using smatmatch
13:52 MetaZoffix m: say so 0 ~~ []
13:52 camelia rakudo-moar 43b4f3: OUTPUT«False␤»
13:52 jnthn Oh, right
13:52 MetaZoffix m: say so [] ~~ 0
13:52 camelia rakudo-moar 43b4f3: OUTPUT«True␤»
13:52 moritz aaah
13:52 lizmat aaah   TILA
13:52 jnthn m: sub f(@x where (so my $n = .elems) != 0) { dd @x }; f []
13:52 camelia rakudo-moar 43b4f3: OUTPUT«Constraint type check failed for parameter '@x'␤  in sub f at <tmp> line 1␤  in block <unit> at <tmp> line 1␤␤»
13:53 jnthn :)
13:53 moritz m: sub f(@x where so my $n := @x.elems) { say $n; dd @x }; f [42]; f []
13:53 camelia rakudo-moar 43b4f3: OUTPUT«1␤[42]␤Constraint type check failed for parameter '@x'␤  in sub f at <tmp> line 1␤  in block <unit> at <tmp> line 1␤␤»
13:56 lizmat something different: shouldn't this be a compile time error?
13:56 lizmat my @a[3]; @a[3] = "foo"
13:56 lizmat m: my @a[3]; @a[3] = "foo"
13:56 camelia rakudo-moar 43b4f3: OUTPUT«Index 3 for dimension 1 out of range (must be 0..2)␤  in block <unit> at <tmp> line 1␤␤»
13:56 lizmat I guess this is LHF in the optimizer  :-)
13:59 MasterDuke_ moritz++ jnthn++ MetaZoffix++ lizmat++ (Perl 6)++ (where clause in multi-dispatch)++
13:59 MasterDuke_ very cool
14:00 MasterDuke_ would it ever be possible to make it fast?
14:01 jnthn There's various ways we could make it faster
14:01 jnthn Caching a narrowed-down candidate list rather than doing the full slow-path dispatch would be one way.
14:04 MasterDuke_ that sounds sort of like a MoarVM thing?
14:05 jnthn I don't think it'd need MoarVM changes
14:05 jnthn Since MoarVM doesn't care exactly what's in the cache, it just invokes it
14:06 MasterDuke_ interesting, rakudo populates the cache?
14:06 jnthn So it should be OK if the multi-dispatcher code (written in NQP) decided to poke a closure in there
14:06 jnthn Yeah
14:06 jnthn MoarVM/JVM don't know Perl 6's multi-dispatch algorithm
14:07 jnthn (Which is nice, 'cus we only have to maintain it in one place :))
14:09 MasterDuke_ where is the multi-dispatcher code? (mostly idle curiosity, but you new know...)
14:09 MasterDuke_ *never
14:10 jnthn BOOTSTRAP.nqp
14:11 jnthn (src/Perl6/Metamodel/)
14:16 MasterDuke_ thanks
14:24 travis-ci joined #perl6-dev
14:24 travis-ci Rakudo build passed. Zoffix Znet 'Remove -os from allow_failures for JVM
14:24 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159860651 https://github.com/rakudo/rakudo/compare/838c0011acdf...4f0a551b8847
14:24 travis-ci left #perl6-dev
14:25 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
14:26 MetaZoffix heh
14:26 MetaZoffix Ah, right makes sense, 'cause JVM failures are still there
14:36 RabidGravy joined #perl6-dev
14:48 MetaZoffix New blog post: "Perl 6 Core Hacking: Wrong Address; Return To Sender": http://perl6.party/post/Perl-6-Core-Hacking-Wrong-Address-Return-To-Sender
15:01 jnthn So, here is my rough thinking on improving our encodings handling: https://gist.github.com/jnthn/4b2ce730fe7b505d9104d157291de572
15:01 lizmat afk&
15:01 lizmat will read when back :-)
15:02 jnthn :)
15:03 jnthn Feedback welcome; will take a short break and bbi15 or so :)
15:08 travis-ci joined #perl6-dev
15:08 travis-ci Rakudo build failed. Jonathan Worthington 'Get MoarVM with EVAL leak fix.'
15:08 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159870352 https://github.com/rakudo/rakudo/compare/4f0a551b8847...43b4f3d02faf
15:08 travis-ci left #perl6-dev
15:09 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
15:26 MetaZoffix joined #perl6-dev
15:30 MetaZoffix jnthn: the $incomplete in consume-line-chars() isn't ideal IMO... If I if I want to signal that no more bytes are coming, then I pass ":incomplete". But that can also be interpreted by person reading the code that the decoding is `incomplete` and there *are* more bytes coming. Maybe :no-more-bytes or :done or :end or something is a more clear name.
15:31 MetaZoffix (FWIW, there's a typo in method name set-line-separaters )
15:32 MetaZoffix The last Travis failure is a real failure, but it looks like just a flopper in t/04-nativecall/13-union.t
15:34 robertle joined #perl6-dev
15:39 jnthn MetaZoffix: Maybe :eof would be better
15:39 MetaZoffix sounds good
15:41 jnthn Tweaked that and the spello
15:43 MetaZoffix jnthn++
15:44 jnthn Thanks for looking at it :)
15:48 nine Encoding.aliases really could provide a default implementation ;)
15:48 jnthn The empty list? :)
15:49 nine yes
15:49 jnthn Point. Changed.
15:51 MasterDuke_ minor, but "provide ore flexibility"
15:51 jnthn d'oh
15:52 jnthn Fixed it for the sake of other readers (don't expect that text to make it anywhere but this gist, though)
15:52 nine jnthn: so what about encoding/decoding errors?
15:52 jnthn Well, encoding should take a replacement
15:52 jnthn We already handle that, just need to work it into the API
15:52 jnthn Without that, it's a straight exception.
15:53 jnthn For decoding, it's always an exception at present.
15:54 jnthn (Which is a valid way to go per http://www.unicode.org/faq/utf_bom.html#gen8)
15:55 nine jnthn: https://metacpan.org/pod/Encode#Handling-Malformed-Data is probably very much inspired by real world use cases
15:55 jnthn Might be nice to also offer the replacement char functionality while we're doing this
15:57 nine I guess if I want to check if a bunch of bytes is decodable UTF-8 I just .decode and catch exceptions?
15:57 jnthn If you want to only decode valid UTF-8, then yes.
15:57 * nine has had to deal with Maybe-UTF-8 far too often...
15:58 jnthn I think looking at that, we can reasonably provide the replacement char strategy too (and let the user pick that)
15:59 nine In Perl 5 I've written utf8::upgrade($content) unless utf8::decode($content) quite a few times
16:01 nine Though utf8::decode($value) unless utf8::is_utf8($value); is more common and I'm glad, we don't need it in Perl 6 :)
16:03 jnthn :)
16:05 * timotimo returns from the autobahn
16:05 mst nine: "this appears to be encoded in WTF8"
16:11 AlexDaniel joined #perl6-dev
16:14 dogbert17 joined #perl6-dev
16:15 jnthn nine: Added some stuff on replacements.
16:17 dogbert17 .seen MetaZoffix
16:17 yoleaux2 I saw MetaZoffix 15:43Z in #perl6-dev: <MetaZoffix> jnthn++
16:18 dogbert17 Zoffix: The spectest which fails consistently in Linux (Mint) 32 bit is t/spec/S32-io/socket-accept-and-working-threads.t
16:19 jnthn Odd...how does it fail?
16:19 dogbert17 jnthn: https://gist.github.com/dogbert17/b9e4a8e6502e279fb37e7d8b13c19ca3
16:20 timotimo that fails on my machine, too. it just consumes 100% of each of my 4 cores until i kill it
16:20 jnthn wtf
16:21 * jnthn runs it in a loop for a while
16:21 jnthn Does it always fail in the same place for the two of you?
16:22 jnthn Just did 10 runs and it passed every time
16:22 dogbert17 I'm on 32 bit
16:22 jnthn I don't immediately see how that'd be a singificant difference.
16:23 dogbert17 If always fails a while after having printed 'ok 5 - Server responded (4)'
16:23 timotimo i'll fetch newest everything and give it a go
16:25 jnthn uh, wowzer
16:25 dogbert17 found something?
16:26 jnthn https://gist.github.com/jnthn/ddc28fba5043a183a6964a626c4efaf4
16:27 jnthn See the maxresident on that!
16:27 jnthn If that's in KB then it really does try to allocate 2GB
16:27 awwaiid joined #perl6-dev
16:27 dogbert17 I only have 1.5 gigs in my vm
16:28 dogbert17 just ran it in perl6-valgrind-m, big explosion :)
16:28 timotimo yeah, rakudo omnomnoms ram sometimes :)
16:28 jnthn 16GB in mine :)
16:28 jnthn Yes, but that's a rather insane amount :)
16:29 dogbert17 sometime I get 'can't allocate ... bytes' or the sjort message 'killed'
16:29 timotimo oh, it finished
16:30 dogbert17 if it actually uses ~2 gigs that would at least explain my failures
16:30 jnthn Killed is probably the OOM killer
16:31 timotimo 2139212maxresident
16:32 dogbert17 I could restart my vm with say 2.5 gigs an see what happens
16:32 timotimo perf says it spends by far the most time in multi_cache_find_callsite_args and multi_cache_add
16:33 b2gills could it be that gc doesn't happen the for loop?
16:33 timotimo if there's no gc safe point, that could happen
16:34 timotimo with valgrind on it seems like the server responds with '' instead of "don't hang up"
16:35 jnthn Yeah, noticed that
16:35 jnthn Hm, it does say no GC runs
16:35 timotimo hm. with a failed test "server responded (4)", will it ever ok the "started work was completed (4)"
16:35 jnthn In the profiler
16:36 jnthn timotimo: Depends how it failed
16:36 timotimo "got ''" is probably not enough info to figure out how exactly it failed?
16:36 jnthn No
16:36 timotimo in the profiler? isn't this async and thus uses multiple threads and as such the profiler output is likely useless? :)
16:37 timotimo i see 6% total time spent in run_gc
16:37 dogbert17 ran with perl6-valgrind-m, don't know if the out put is useful: https://gist.github.com/dogbert17/1c2820b548a0d768033e389664718da7
16:38 jnthn timotimo: Yeah, I think the --profile output is bogus
16:38 jnthn Looking into --profile=heap now
16:38 jnthn Yeah, that has 5 snapshots
16:38 timotimo probably not useful, dogbert17
16:38 jnthn 1 which will have come at the end, so it did 4 in the normal course of execution
16:38 jnthn So it's not that it's never doing GC
16:38 timotimo first of all, if you don't add --full-cleanup to the arguments moar receives, you'll get a whole lot of stuff extra
16:39 timotimo but then the program was killed in the middle of doing business
16:39 dogbert17 yes it was, a sad state of affairs :(
16:40 jnthn > snapshot 3
16:40 jnthn Loading that snapshot. Carry on...
16:40 jnthn > summary
16:40 jnthn Total heap size:              40,090,806 bytes
16:40 jnthn That doesn't seem to acocunt for it too much :)
16:41 Ven_ joined #perl6-dev
16:42 timotimo OK, so we have memory allocated that's not being counted in the reprs?
16:42 jnthn Something like
16:42 jnthn crap
16:42 jnthn Tried a run with massif and it was killed :/
16:43 jnthn Right near the end though
16:43 * jnthn tries toning it down to 3 loop iterations
16:52 * timotimo isn't getting anything sensible from adding a probe on malloc to perf
16:53 jnthn 99.18% (425,735,041B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
16:53 jnthn ->73.45% (315,284,808B) 0x4FB37BB: MVM_fixed_size_alloc (in /home/jnthn/dev/MoarVM/install/lib/libmoar.so)
16:53 jnthn | ->58.39% (250,639,520B) 0x4FE9C79: MVM_multi_cache_add (in /home/jnthn/dev/MoarVM/install/lib/libmoar.so)
16:53 jnthn A huge volume of allocation comes from adding to the multi cache too?!
16:54 timotimo hah
16:54 timotimo clearly we're doing cache invalidation wrong
16:54 * jnthn #define MVM_MULTICACHE_BIG_PROFILE 1 to see what it says
16:56 jnthn Multi cache for new reached 2048 entries
16:56 jnthn Multi cache for new reached 4096 entries
16:56 jnthn ...
16:57 * dogbert17 feels useless, should I RT in the meantime?
16:59 jnthn timotimo: Curiously, MVMMultiCache does implement unmanaged_size...
17:08 robertle joined #perl6-dev
17:10 dalek Heuristic branch merge: pushed 81 commits to rakudo/better-test-pm6 by zoffixznet
17:11 Woodi jnthn: maybe encoding/decoding work can have some nice, userproof buffering subsystem ? probably part of streaming. also, if plan is to always read files :binary then user-implemented decoding can do eg. unpacking .jpeg into array of pixels. even if now it is just for text maybe one day it will be universal. also2: gist is about always returning Blob but serialization is similiar to encoding maybe it can
17:11 Woodi be do this way ?
17:28 cygx joined #perl6-dev
17:28 cygx o/
17:28 * cygx throws https://gist.github.com/cygx/7d792e09b182a1a79b1dac82663dfddb into the ring again
17:28 timotimo Woodi: that doesn't make much sense. you're better off working with incoming Buf directly
17:28 cygx it didn't get any comments the last time I did :(
17:29 timotimo since jpeg (especially) isn't a stream-oriented format
17:30 timotimo Woodi: don't forget that if you want to do image loading in general, you have to handle many different bit depths, floating point vs integers, multiple layers, alpha-transparency, metadata, ...
17:31 jnthn timotimo: OK, think I found it. Turns out it's a confusion between flag indexes and arg buffer indexe
17:32 timotimo so it keeps adding the same thing over and over?
17:32 jnthn Yeah :)
17:32 jnthn 'cus the arg index is truncated
17:32 timotimo heh.
17:33 jnthn Just needs 1 bit to fix :)
17:33 timotimo so every darn call ends up giving us a new node in the tree
17:33 jnthn yup!
17:33 timotimo does the tree get longer and longer and more and more lopsided?
17:33 jnthn Longer and longer...but the nodes are unreachable
17:33 jnthn And the reason it's such an incredible memory blow-up is that it's a concurrent data structure, so frees at safe points.
17:34 jnthn (Which come, but not before we've built thousands of copies of the cache)
17:34 timotimo ah, right
17:35 timotimo which reminds me of that other bug again where we had a tight loop of calls(?) and we ended up allocating loads of stack frames (at least i think it was that) and never reached a safe point in that tight loop
17:38 cygx joined #perl6-dev
17:40 jnthn The test runs a bunch faster with that fixed too :P
17:41 jnthn cygx: Something along those lines looks like a reasonable direction.
17:42 jnthn Though I'd prefer a design without the runtime mixins...
17:43 Woodi btw. I remember some commit saying that "no allocations on variable access" (in some case). why access to some memory location needs to allocate things ?
17:46 jnthn Normally lazy allocation
17:46 jnthn dogbert17: Spectesting a fix at the moment
17:49 timotimo m: my @foo; my $a := @foo[10]; my $b := @foo[20]; my $c = @foo[30]; say @foo.perl; ($a, $b, $c) = (1, 2, 3); say @foo.perl
17:49 camelia rakudo-moar 43b4f3: OUTPUT«[]␤[Any, Any, Any, Any, Any, Any, Any, Any, Any, Any, 1, Any, Any, Any, Any, Any, Any, Any, Any, Any, 2]␤»
17:49 timotimo Woodi: here's an example where access will allocate
17:50 dogbert17 jnthn++, I'm very impressed
17:51 timotimo and we keep some lexical variables nulled out until they are first accessed
17:51 timotimo then a Scalar gets created, for example
17:53 dalek rakudo/better-test-pm6: 9fcdaff | (Zoffix Znet)++ | lib/Test.pm6:
17:53 dalek rakudo/better-test-pm6: Add rough design notes
17:53 dalek rakudo/better-test-pm6: review: https://github.com/rakudo/rakudo/commit/9fcdafff9b
17:53 dalek nqp: 630fade | jnthn++ | tools/build/MOAR_REVISION:
17:53 dalek nqp: Bump MOAR_REVISION for multi-cache fix.
17:53 dalek nqp: review: https://github.com/perl6/nqp/commit/630fade28f
17:54 dalek rakudo/nom: a2ae6cd | jnthn++ | tools/build/NQP_REVISION:
17:54 dalek rakudo/nom: Bump NQP_REVISION for multi cache fix.
17:54 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/a2ae6cd474
17:54 jnthn dogbert17: That should do it; let me know if you run into any more issues with that test file.
17:55 dogbert17 jnthn: thx a bunch, I'll try it out asap
17:55 Ven_ joined #perl6-dev
17:56 timotimo wow
17:57 timotimo that test is now so fast that its total time amounts to 4 times starting up
18:00 dalek rakudo/nom: 0cf7128 | jnthn++ | docs/ChangeLog:
18:00 dalek rakudo/nom: Add today's fixes to the ChangeLog.
18:00 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/0cf71280fa
18:02 jnthn Enough for now :)
18:09 Ven_ joined #perl6-dev
18:24 FROGGS joined #perl6-dev
18:37 Woodi got it
18:38 dogbert17 jnthn: the test worked on my system as well. It looks as if memory usage is down as well :-)
18:38 Ven_ joined #perl6-dev
18:42 timotimo i wonder when that bug impacted other workloads
18:43 timotimo not only was it causing a bunch of memory churn, but it also meant we had to go without a multi dispatch cache for anything with too many named parameters
19:03 dogbert17 joined #perl6-dev
19:32 jnthn timotimo: Quite possibly. I'm glad we found/fixed it, anyways.
19:36 travis-ci joined #perl6-dev
19:36 travis-ci Rakudo build passed. Jonathan Worthington 'Bump NQP_REVISION for multi cache fix.'
19:36 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159951909 https://github.com/rakudo/rakudo/compare/43b4f3d02faf...a2ae6cd4744a
19:36 travis-ci left #perl6-dev
19:37 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
19:40 jnthn Man, even when I succeed the bots think I failed :P
19:47 timotimo ;(
19:47 timotimo they are just jealous
20:12 dogbert17 which optimization level is the default for perl6? When starting perl6 I can set --optimize=num, but wat is the default?
20:31 timotimo i think for normal stuff it's 2, for the core setting it's 3
20:33 dogbert17 timotimo: thx
20:34 dogbert17 timotimo: btw, have you ever seen this, happens sometimes when running t/spec/S32-num/power.rakudo.moar. https://gist.github.com/dogbert17/ca96cc9962d5e20199f6d3e71850f262
20:37 timotimo something didn't get sunk
20:38 timotimo normally, we rely on calls to .sink on results of stuff to be generated by code-gen to make failures throw, or otherwise handle them
20:38 timotimo so when it doesn't get its "handled" flag set to 1 before the GC kills it, we output a big warning
20:38 timotimo and that warning also contains the error "inside" the failure
20:39 timotimo it's interesting that the backtrace is inside the optimizer; it must have been compile-time evaluated?
20:40 dogbert17 it does not happen very often
20:41 timotimo right, it depends on the GC to be run at least twice between the Failure being created and the program ending
20:41 dogbert17 aha, so it's nothing to worry about then?
20:41 timotimo not 100% sure
20:42 timotimo it might be interesting to figure out which piece of code causes that failure to be generated
20:42 timotimo maybe the optimizer wants to learn about Failure objects when doing compile-time evaluation. perhaps it already knows to throw them away, but doesn't know to set them to "handled"
20:46 dogbert17 apart from that message I only have one test fail left, then 'make spectest' would run cleanly on my system
20:46 dogbert17 you and jnthn fixed the most serious failure, thx
20:48 dogbert17 the last test in t/spec/S32-num/power.rakudo.moar is the only one which still 'fails'
20:48 travis-ci joined #perl6-dev
20:48 travis-ci Rakudo build failed. Jonathan Worthington 'Add today's fixes to the ChangeLog.'
20:48 travis-ci https://travis-ci.org/rakudo/rakudo/builds/159953729 https://github.com/rakudo/rakudo/compare/a2ae6cd4744a...0cf71280faa9
20:48 travis-ci left #perl6-dev
20:49 buggable [travis build above] ☠ Did not recognize some failures. Check results manually
21:05 Zoffix t/04-nativecall/13-union.t flop again
21:05 Zoffix As for the previous... There *are* failures in JVM, which buggable sees, but travis ignores. So travis says passed
21:06 Zoffix left #perl6-dev
21:12 lizmat jnthn: re your encoding gist:
21:12 lizmat I would probably have written the encode method as:
21:13 lizmat method encode(Str:D $to-encode, *%options) returns Blob {
21:13 lizmat %options<replacement> = .NFC with %options<replacement>;
21:14 lizmat or is there a reason you want to have it as a separate named ?
21:16 jnthn lizmat: A little shorter, felt more natural...no strong reason. :)
21:17 jnthn Though .=NFC with %options<replacement>; probably works... :)
21:18 jnthn I was more interested in comments on the API tbh ;-)
21:19 jnthn timotimo: fwiw, we do detect Failure in the constant folder, but it's possible we fail to disarm them
21:19 lizmat need to sleep on that for a night, it's a lot to take in  :-)
21:19 jnthn :-)
21:19 jnthn fwiw, the API of the streaming decoder isn't really "new"
21:20 jnthn It's largely a slight Perl 6-ification of the decode stream API inside of MoarVM that's been serving us well for a while now :)
21:20 jnthn (So in that sense it's a tested design.)
21:23 timotimo failure to disarm would be what causes this symptom i'd say
21:24 lizmat jnthn: slightly worried about performance
21:24 lizmat what are your thoughts about that ?
21:24 jnthn Performance of what in particular?
21:25 jnthn Note that for the currently supported encodings, the implementations of this API will delegate directly to the existing VM implemetation.
21:26 jnthn So really we're just pulling coordination of things up to Perl 6 level
21:26 jnthn Not hot lops
21:26 jnthn *loops
21:27 jnthn timotimo: If you're interested to investigate, then grepping for Failure in Optimizer.nqp will probably show up the appropriate point :)
21:30 timotimo ah, yes, right.
21:31 timotimo i could have a quick look
21:32 lizmat jnthn: ah, ok, yes  :-)
21:33 timotimo yeah, easy peasy
21:43 lizmat sleep&
21:45 jnthn 'night
21:45 dalek rakudo/nom: 7c48b90 | timotimo++ | src/Perl6/Optimizer.nqp:
21:45 dalek rakudo/nom: prevent optimizer from causing unhandled failures in GC
21:45 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/7c48b90b01
21:52 timotimo dogbert17: ^- this commit fixes the spurious Failure objects in the GC with backtraces into the optimizer
21:52 timotimo actually, the fix wants to look a tiny bit different
21:52 dalek rakudo/nom: 2c95f74 | timotimo++ | src/Perl6/Optimizer.nqp:
21:52 dalek rakudo/nom: don't want to call .Bool on undefined Failure objects
21:52 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/2c95f74903
21:59 dogbert17 timotimo: many thanks, man you're fast :-)
22:00 jnthn :)
22:00 timotimo would have been even faster if i had looked at it immediately, ond not waited for jnthn to poke me :)
22:00 jnthn timotimo++
22:00 timotimo jnthn++
22:02 dogbert17 jnthn++, timotimo++
22:06 dogbert17 well, it's time for me to hit the sack, good night and thanks for the awesome help
22:09 timotimo gnite dogbert!
22:12 timotimo i might also go to bed soon
22:14 Zoffix joined #perl6-dev
22:15 Zoffix I think I spotted a race condition when outputting to STDOUT and then immediately to STDERR: https://gist.github.com/zoffixznet/5bf9a6d0b7ee5c6489ed3002c8875a06
22:15 Zoffix That's our current Test.pm. Notice how the first run the two `not ok`s are together, but in the next run they're separated.
22:16 Zoffix There's no async code in Test.pm. Is this 'cause we're using libuv's async out stuff?
22:16 Zoffix Should I RT, or is this stuff well-known?
22:17 timotimo http://rosettacode.org/wiki/Chat_server#Perl_6 - the bug this refers to has been fixed and this task can now be fixed up
22:17 timotimo yeah, that's a race condition. you can only get them in the right order if you're quite fast i guess?
22:18 Zoffix OK :0
22:18 Zoffix *:)
22:18 timotimo https://stackoverflow.com/questions/4497817/save-stdout-stderr-and-stdoutstderr-synchronously - shortest google search gives us this
22:18 Zoffix Doesn't matter much here, as the STDERR is just for humans
22:18 timotimo which i'm now reading
22:31 dalek rakudo/better-test-pm6: 4fb849c | (Zoffix Znet)++ | / (5 files):
22:31 dalek rakudo/better-test-pm6: Implement ok/nok and all of the supporting infrastructure
22:31 dalek rakudo/better-test-pm6: review: https://github.com/rakudo/rakudo/commit/4fb849cfb8
23:19 travis-ci joined #perl6-dev
23:19 travis-ci Rakudo build passed. Timo Paulssen 'prevent optimizer from causing unhandled failures in GC'
23:19 travis-ci https://travis-ci.org/rakudo/rakudo/builds/160011326 https://github.com/rakudo/rakudo/compare/0cf71280faa9...7c48b90b013c
23:19 travis-ci left #perl6-dev
23:19 buggable [travis build above] ☠ Did not recognize some failures. Check results manually

| Channels | #perl6-dev index | Today | | Search | Google Search | Plain-Text | summary