Perl 6 - the future is here, just unevenly distributed

IRC log for #perl6-dev, 2017-11-11

| Channels | #perl6-dev index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 BenGoldberg joined #perl6-dev
01:49 MasterDuke joined #perl6-dev
01:59 dugword joined #perl6-dev
02:04 MasterDuke_ joined #perl6-dev
02:57 ilbot3 joined #perl6-dev
02:57 Topic for #perl6-dev is now Perl 6 language and compiler development 2.0 | Logs at http://irclog.perlgeek.de/perl6-dev/today | For toolchain/installation stuff see #perl6-toolchain | For MoarVM see #moarvm
03:27 ufobat_ joined #perl6-dev
05:22 Ben_Goldberg joined #perl6-dev
05:29 dugword joined #perl6-dev
10:36 geospeck joined #perl6-dev
10:37 dugword joined #perl6-dev
11:50 geospeck joined #perl6-dev
12:01 Geth ¦ rakudo: 345fbf5a06 | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6
12:01 Geth ¦ rakudo: More Telemetry tweaks and fixes
12:01 Geth ¦ rakudo:
12:01 Geth ¦ rakudo: - make sure Telemetry object correctly roundtrip
12:01 Geth ¦ rakudo: - no longer try to make sense of Telemetry:U - Telemetry:U, just die
12:01 Geth ¦ rakudo: - add some more return types to add optimization opportunities
12:01 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/345fbf5a06
12:01 lizmat Files=1229, Tests=75833, 323 wallclock secs (14.74 usr  5.29 sys + 2171.38 cusr 214.53 csys = 2405.94 CPU)
12:20 Geth ¦ rakudo: b80d486c7e | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6
12:20 Geth ¦ rakudo: Handle empty Telemetry reports better
12:20 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/b80d486c7e
12:22 Geth ¦ rakudo: ba49b34329 | (Elizabeth Mattijsen)++ | t/07-telemetry/01-basic.t
12:22 Geth ¦ rakudo: Add some more basic Telemetry tests
12:22 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/ba49b34329
12:26 * nine actually uses a push mover. Though frankly his girlfriend does most of the mowing.
12:57 sjn joined #perl6-dev
13:12 Geth ¦ rakudo: fe799a9888 | (Elizabeth Mattijsen)++ | src/core/ThreadPoolScheduler.pm
13:12 Geth ¦ rakudo: Handle thread exhaustion more gracefully
13:12 Geth ¦ rakudo:
13:12 Geth ¦ rakudo: - leave more CPU to the threads that *are* doing stuff
13:12 Geth ¦ rakudo: - add $!exhausted attribute to ThreadPoolScheduler
13:12 Geth ¦ rakudo: - this gets set if creating a new thread failed
13:12 Geth ¦ rakudo: - which then inhibits trying to add any more workers
13:12 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/fe799a9888
13:13 tbrowder .tell Zoffixi added a blurb about kebab-case on the p6 wiki, please critique and correct as necessary <https://en.wikipedia.org/wiki/Perl_6>
13:13 yoleaux tbrowder: I'll pass your message to Zoffixi.
13:14 tbrowder .tell Zoffix
13:14 yoleaux tbrowder: I don't know what you want me to say to Zoffix.
13:14 tbrowder .tell Zoffix i  added a blurb about kebab-case on the p6 wiki, please critique and correct as necessary
13:14 yoleaux tbrowder: I'll pass your message to Zoffix.
13:37 dugword joined #perl6-dev
13:57 timotimo m: Bag.new.pick(1)
13:57 camelia rakudo-moar fe799a988: OUTPUT: «(signal SEGV)»
13:57 timotimo oh?
13:58 timotimo ah, moarvm not bumped
14:00 AlexDaniel joined #perl6-dev
14:04 robertle joined #perl6-dev
14:18 travis-ci joined #perl6-dev
14:18 travis-ci Rakudo build failed. Elizabeth Mattijsen 'Add some more basic Telemetry tests'
14:18 travis-ci https://travis-ci.org/rakudo/rakudo/builds/300587018 https://github.com/rakudo/rakudo/compare/b80d486c7e78...ba49b34329bb
14:18 travis-ci left #perl6-dev
14:18 buggable [travis build above] ☠ Did not recognize some failures. Check results manually.
14:21 MasterDuke_ /home/travis/build/rakudo/rakudo/install/bin/moar --libpath=src/vm/moar/stage0 src/vm/moar/stage0/nqp.moarvm --bootstrap --module-path=gen/moar/stage1 --setting=NULL --no-regex-lib --target=mbc --output=gen/moar/stage1/NQPCORE.setting.moarvm gen/moar/stage1/NQPCORE.setting
14:21 MasterDuke_ Segmentation fault (core dumped)
14:22 MasterDuke_ only on one job
14:41 Geth ¦ rakudo: ufobat++ created pull request #1246: RFC: IO::Path.resolve for windows 2nd attempt
14:41 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/pull/1246
14:41 synopsebot RAKUDO#1246 [open]: https://github.com/rakudo/rakudo/pull/1246 RFC: IO::Path.resolve for windows 2nd attempt
14:54 dogbert17 joined #perl6-dev
15:31 ugexe Does fe799a assume that once exhausted there can -never- be another thread created?
15:35 timotimo yes
15:36 ugexe Why? Cannot the thread use go down at some point and thus allow more workers to spawn?
15:37 geospeck joined #perl6-dev
15:38 timotimo what do you mean by "thread use"?
15:40 ugexe Well, what does exhausted mean? And is it truely a state that cannot be changed once reached?
15:40 timotimo the common failure case is "too many open files"
15:41 ugexe yes, but what if 100 threads suddenly complete their task and there are now more files that can be opened
15:41 timotimo then you don't need to add another thread because 100 threads just became free
15:42 ugexe hmmm I’m not sure this adds up right to me but I can’t really say why so just ignore me
15:42 timotimo i can see why you're worried about this
15:43 timotimo adding more threads is one mechanism we use to dislodge deadlocks that could be solved by finishing up some more work tasks
15:43 timotimo in that situation the whole program (or maybe just big parts of it) are at a standstill anyway
15:57 ugexe Wouldn’t hours manual thread creation be gimped by this?
15:57 ugexe hours / bouts of
15:57 ugexe or rather the scheduler gimped by bouts of manual thread creation
15:58 timotimo yeah, Don't Do That :)
15:58 timotimo manual thread creation should hardly ever be used
16:02 ugexe Yes but if you want to be smarter than the schedule you should be able to (without supplying a different scheduler)
16:05 ugexe this just seems like some line that, once crossed in the program, will change its running characteristics for the remainder of its runtime. But this may be a case of I prefer the program blowing up and telling me it’s broke than trying to work around my situation
16:06 ugexe so a naive compliment to this might reset the exhausted flag if I successfully create a thread manually.
16:12 timotimo that would require Thread to know about the Scheduler in question, that's weird
16:12 timotimo i wouldn't be against just preventing threads from being created with exponential backoff timing
16:14 timotimo i suspect before this commit it would have tried to build new threads 100 times a second orwhatever the interval for the supervisor is
16:14 timotimo that's clearly Bad
16:14 timotimo bbl
16:20 ugexe yes not a real solution, but more trying to explain the behavior i expect
16:22 geospeck joined #perl6-dev
16:44 geospeck joined #perl6-dev
17:02 AlexDaniel releasable6: next
17:02 releasable6 AlexDaniel, Next release in 7 days and ≈1 hour. No blockers. 0 out of 155 commits logged
17:02 releasable6 AlexDaniel, Details: https://gist.github.com/d8161556014814006fd1d8e5af051797
17:15 releasable6 joined #perl6-dev
18:20 BenGoldberg joined #perl6-dev
18:26 lizmat timotimo ugexe I'm thinking about only trying to have another try at creating a new Thread once a second (aka once every 100 supervisor loops)
18:33 lizmat ugexe: please note that in the current setup, a Thread === a worker, *not* a task
18:34 lizmat once started, a Thread will never stop (in the current setup)
18:34 lizmat it will just block for another task to appear as soon as it runs out of tasks
18:35 dugword joined #perl6-dev
18:38 Geth ¦ rakudo: fe1f8632c1 | (Elizabeth Mattijsen)++ | src/core/Baggy.pm
18:38 Geth ¦ rakudo: Fix segfault on Bag.new.pick(1)
18:38 Geth ¦ rakudo:
18:38 Geth ¦ rakudo: Although it would be interesting to see how that was possible to begin with.
18:38 Geth ¦ rakudo: Problem was, it should have selected the empty iterator, but there was no
18:38 Geth ¦ rakudo: check on an empty Baggy in there yet.  This commit basically adds that.
18:38 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/fe1f8632c1
18:40 timotimo lizmat: with a newer moarvm it doesn't segfault, instead it throws an exception
18:40 jnthn lizmat: Pretty sure timotimo fixed the segv
18:40 jnthn heh :)
18:40 lizmat ah, ok, good
18:41 ugexe i see. so is the goal to allow the workers to change to another type of worker? or does work stealing make this pointless?
18:43 timotimo stealing allows workers that are bored to steal from other workers that are overloaded
18:43 timotimo i haven't looked at the code to see if it crosses worker types or not
18:43 timotimo but i think it does, at least in some directions
18:43 ugexe otherwise I question what this means for the performance profile of an application when worker/affinity differences are caused by what happened in the past
18:44 ugexe e.g. i run the application one time and get 9/1, and another time I get 4/5
18:44 lizmat although this implementation is much more advanced than before
18:44 lizmat it's still pretty simple in its assumptions
18:44 ugexe which could be problematic for anything long running
18:44 timotimo your system should allow you to get far more than 10 threads
18:44 lizmat 1. whenever a worker pool seems to need another worker, we add one
18:45 lizmat I've now added:
18:45 lizmat 2. if we've run out of low level system threads, don't bother trying to add them
18:46 jnthn https://github.com/rakudo/rakudo/commit/fe799a9888 introduces a regression
18:46 jnthn prod-affinity-workers should be run every time
18:47 jnthn And should not be inside of the loop
18:47 lizmat but I didn't move them into the loop ?
18:47 lizmat they were always there ?
18:47 jnthn It was moved into unless $!exhausted {
18:47 lizmat that;'s not a loop ?
18:47 jnthn https://github.com/rakudo/rakudo/commit/fe799a9888#diff-6d7bfa05d538ae828eff330472ec119fR545
18:47 jnthn oh shit
18:47 jnthn should not be inside of the *unless
18:48 jnthn Sorry
18:48 jnthn That was confusing
18:48 lizmat you mean to say that you always need to prod ?
18:48 jnthn Yes
18:48 lizmat ok, will do
18:48 jnthn prod doesn't start new threads
18:48 jnthn It just steals work from a stuck affinity queue into the general queue
18:48 jnthn To try and break deadlocks
18:48 lizmat ok
18:49 lizmat still, my change was spectest clean  :-)
18:49 jnthn If there is a test that'd catch this, I think it'd be a stresstest
18:49 jnthn I also wonder a little if there's a more robust way to detect the failure to start thread exception than by text
18:50 lizmat well, then it should throw something else than an X::AdHoc  :-)
18:50 jnthn Yeah, we could wrap it up in Thread.start I guess :)
18:50 timotimo maybe make the catch much tighter so the only thing it's catching for is ... yes
18:51 timotimo or that
18:51 jnthn In the meantime, I'm meant to cook some dinner :)
18:51 timotimo me bbl, too
18:52 lizmat jnthn timotimo but if we wrap it in Thread.start, how will the supervisor every know ?
18:53 jnthn Thread.start is where the exception is coming from today, I think?
18:53 jnthn Well, the nqp::startthread op inside of it, at least
18:53 timotimo right, put a catch around only that, then throw a proper typed exception
18:53 jnthn https://github.com/rakudo/rakudo/commit/fe799a9888#diff-6d7bfa05d538ae828eff330472ec119fR280
18:53 jnthn Yes, what timotimo said is what I meant by wrap
18:54 geospeck joined #perl6-dev
18:54 lizmat ok, gotcha
18:55 lizmat jnthn: I assume you mean nqp::newthread, right ?
18:55 jnthn I forget the exact op names
18:56 jnthn There's one that creates the object and one that runs it
18:56 lizmat yeah, newthread it is then :0)
18:56 lizmat $!vm_thread := nqp::newthread(
18:56 jnthn https://github.com/rakudo/rakudo/blob/nom/src/core/Thread.pm#L40
18:57 jnthn I think that one (nqp::threadrun) may be the one that throws the exception you're catching to day, rather than nqp::newthread
18:57 jnthn Not 100% certain though
18:57 lizmat ok, will check
18:57 jnthn But certainly a lot of the setup work actually happens in threadrun
18:57 jnthn And newthread just creates a handle object
18:58 jnthn Looking at the code, https://github.com/MoarVM/MoarVM/blob/master/src/core/threads.c#L30 might throw
19:00 lizmat so that would be nqp::newthread
19:00 jnthn Yup
19:00 jnthn Really cooking :)
19:00 jnthn bbl
19:00 Geth ¦ rakudo: 57374490ae | (Elizabeth Mattijsen)++ | src/core/ThreadPoolScheduler.pm
19:00 Geth ¦ rakudo: Try creating a new Thread once a second if exhausted
19:00 Geth ¦ rakudo:
19:00 Geth ¦ rakudo: Instead of completely not try anymore.  Also, always prod the affinity
19:00 Geth ¦ rakudo: workers, even if thread creation will not work.
19:00 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/57374490ae
19:14 MasterDuke AlexDaniel: you ran the toaster last release, right? have you done a new one for the upcoming release?
19:28 AlexDaniel MasterDuke: not yet
19:29 AlexDaniel MasterDuke: but see https://toast.perl6.party/
19:29 AlexDaniel MasterDuke: Zoffix++ did run it a few days ago
19:30 MasterDuke ah, good
19:32 AlexDaniel MasterDuke: now I'm curious. Why?
19:33 MasterDuke just curious myself. lot fewer changes this month, hoped there was less fallout
19:34 AlexDaniel yeah, this release should be much quieter :)
19:35 AlexDaniel but we have a whole week to do disruptive changes!
19:42 Geth ¦ rakudo: 14fbb5e76d | (Elizabeth Mattijsen)++ | 3 files
19:42 Geth ¦ rakudo: Add typed error for thread creation exhaustion
19:42 Geth ¦ rakudo:
19:42 Geth ¦ rakudo: - and make sure the supervisor knows about it
19:42 Geth ¦ rakudo: - bikeshedding on the new X::Exhausted exception may start
19:42 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/14fbb5e76d
19:45 ugexe it should be X::hausted
19:47 lizmat and mauve of course  :-)
19:52 Zoffix Don't see https://toast.perl6.party/ It's outdated. The change causing most of that fallout was reverted.
19:52 yoleaux 13:14Z <tbrowder> Zoffix: i  added a blurb about kebab-case on the p6 wiki, please critique and correct as necessary
19:57 bartolin while looking for some failures on the jvm backend where a wrong multi candidate is selected, i stubled above this inconsistency:
19:57 bartolin m: multi sub foo(Int $bar is rw) { }; foo(42)  ## X::Multi::NoMatch
19:57 camelia rakudo-moar 14fbb5e76: OUTPUT: «Cannot resolve caller foo(Int); the following candidates␤match the type but require mutable arguments:␤    (Int $bar is rw)␤  in block <unit> at <tmp> line 1␤␤»
19:57 bartolin m: multi sub foo(int $bar is rw) { }; foo(42)  ## X::TypeCheck::Argument+{X::Comp}
19:57 camelia rakudo-moar 14fbb5e76: OUTPUT: «5===SORRY!5=== Error while compiling <tmp>␤Calling foo(Int) will never work with any of these multi signatures:␤    (int $bar is rw)␤at <tmp>:1␤------> 3multi sub foo(int $bar is rw) { }; 7⏏5foo(42)  ## X::TypeCheck::Argument+{X::C␤»
19:58 bartolin with this change the first evaluation becomes a compile time error, too: https://github.com/usev6/rakudo/commit/db077336c5
19:58 bartolin but we have some tests that expect a X::Multi::NoMatch there.
20:00 bartolin i'd like to get some feedback whether the suggested patch makes sense. (at least it fixes a problem on the jvm)
20:02 MasterDuke catching thing at compile time is good
20:04 bartolin the patch that catches the native case went in shortly before christmas: https://github.com/rakudo/rakudo/commit/c96deddc2a
20:06 bartolin maybe jnthn can remember why the check for $literal was not applied for the non-native case ...
20:07 Geth ¦ rakudo: 3e4ccce9b8 | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6
20:07 Geth ¦ rakudo: Tweak the T:I:Usage.preamble + allow util% in R_R_COLUMNS
20:07 Geth ¦ rakudo:
20:07 Geth ¦ rakudo: - also show final memory size for easy observation
20:07 Geth ¦ rakudo: - make sure you can specify a column name with % in RAKUDO_REPORT_COLUMNS
20:07 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/3e4ccce9b8
20:23 Geth ¦ rakudo: 6959349e7f | (Elizabeth Mattijsen)++ | src/core/ThreadPoolScheduler.pm
20:23 Geth ¦ rakudo: Move variable definitions out of the supervisor loop
20:23 Geth ¦ rakudo:
20:23 Geth ¦ rakudo: - this does not seem to have a measurable effect on CPU
20:23 Geth ¦ rakudo: - but saves about 3M of memory on a 10 second run
20:23 Geth ¦ rakudo:   - assuming this is not a leak, it's just about less GC churn
20:23 Geth ¦ rakudo:   - but we cannot tell since we cannot profile this cause its threading
20:23 Geth ¦ rakudo: - wish there was a nqp::vmstat op for things like number of GC runs
20:23 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/6959349e7f
20:38 lizmat afk for some time&
21:50 Geth ¦ rakudo: 86e9f44aec | (Elizabeth Mattijsen)++ | lib/Telemetry.pm6
21:50 Geth ¦ rakudo: Telemetry::Instrument.columns -> default-columns
21:50 Geth ¦ rakudo:
21:50 Geth ¦ rakudo: - this is a better name for what it returns
21:50 Geth ¦ rakudo: - also add a .columns method that returns all columns in alphabetical order
21:50 Geth ¦ rakudo: review: https://github.com/rakudo/rakudo/commit/86e9f44aec
21:52 lizmat good night, #perl6-dev!
22:33 dugword joined #perl6-dev
22:56 bisectable6 joined #perl6-dev

| Channels | #perl6-dev index | Today | | Search | Google Search | Plain-Text | summary