Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-02-15

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:02 zakharyas joined #moarvm
00:04 yoleaux2 joined #moarvm
00:06 timotimo i'm getting super tired super fast
00:09 timotimo maybe i'll sign out soon
00:09 samcv kk
00:10 timotimo i'm definitely relieved that these frames are actually jitted
00:10 timotimo have you had a chance to find the spesh output in the log for these inner for loop frames?
00:10 samcv no
00:10 samcv i can't really tell what's going on in the spesh log
00:11 timotimo the graph spesh tool helps you make sense of the superstructure of the BBs
00:11 samcv Of 52159936 specialized or JIT-compiled frames, there were 3216089 deoptimizations (that's 6.17% of all optimized frames).
00:11 samcv is that a lot of depopts or normal
00:13 samcv so i guess on my computer it takes 16 seconds to sort all the cp. though idk if it was slowed down by doing my str @sorted = %points.keys.sort....
00:13 samcv the native str type or possibly the array did it too, idk
00:13 samcv what would be the fastest way to assign it to something.
00:14 samcv i mean obv a scalar but. i know arrays can cause reification right
00:14 timotimo that's all right
00:14 timotimo arrays will itemize each element at least
00:15 timotimo i.e. put things into scalar containers
00:15 samcv lol still taking 70.7900297 seconds to run that section of code
00:15 timotimo not native arrays of course
00:15 samcv was about 79 for me before
00:15 timotimo what did you change?
00:15 samcv i just did the sorting outside of it
00:15 timotimo ah
00:15 samcv so didn't gain 16 seconds
00:15 samcv but gained like 9
00:16 timotimo well, you probably sort more than once all in all
00:16 timotimo it has like 70 entries?
00:16 samcv no it has hundreds of thousands
00:16 samcv i sorted twice at least in that file
00:16 samcv i think i may sort in another file
00:16 timotimo the tab with the profile in it crshed :)
00:17 samcv just say you want to wait
00:17 samcv and it will eventually work
00:17 samcv it's just slow
00:17 samcv or did it fully crash?
00:17 timotimo no, it full on crashed
00:17 timotimo had to reload
00:17 timotimo i do know about the telling it to wait, i've used the profiler for a while :P
00:18 timotimo it does that for almost all workloads ;(
00:18 timotimo there's a qt/c++ program that can interpret the json output from the profiler, but it's feature-incomplete
00:18 timotimo and what's even worse is that the json parser it uses refuses to load files above a random size
00:18 timotimo which we routinely surpass
00:19 timotimo but it'd probably make more sense to base a gui on the SQL output instead
00:19 timotimo what i meant with "like 70 entries" is the sort
00:19 timotimo MERGESORT-REIFIED-LIST-AS
00:20 timotimo i see a sort with 14 entries, one with 205 entries, one with 106 and another with 8
00:20 timotimo so more than i thought, but not as many as you said, so you were looking at something else?
00:21 * timotimo relocates closer to bed
00:37 timotimo any questions before i dip into dreamland?
00:38 samcv let me read
00:39 samcv oh some of the sorts are sorting things of not very many elements
00:39 samcv though some of the modules do some sort which i can remedy
00:40 samcv no that's it i think, you can sleep
00:41 timotimo have you calculated how many msec per entry the for loops take?
00:41 samcv which ones, the point index ones
00:41 samcv with the stuff i took out of the other sub?
00:41 samcv i have not
00:43 timotimo that'd interest me
00:43 samcv how do i find that
00:43 timotimo just a rough estimate would be enough
00:44 samcv let me see
00:45 timotimo just opening the profile again myself
00:45 timotimo so, the inner loop, right?
00:45 timotimo m: say "spent 20136
00:45 camelia rakudo-moar b51a55: OUTPUT«===SORRY!=== Error while compiling <tmp>␤Unable to parse expression in double quotes; couldn't find final '"' ␤at <tmp>:1␤------> say "spent 20136⏏<EOL>␤    expecting any of:␤        argument list␤        double quotes␤        â€¦Â»
00:46 timotimo durrr
00:46 timotimo m: say "spent { 20136 / 29792280 }ms per iteration"
00:46 camelia rakudo-moar b51a55: OUTPUT«spent 0.00067588ms per iteration␤»
00:46 samcv i haven't pushed the latest changes in removing the sort
00:46 samcv computing now
00:48 samcv Finished computing all rows, took 67.3072336 for 354670 elems. Now creating the final unduplicated version.
00:49 samcv so 0.0001889s per iteration
00:49 timotimo so, i notice method Bool is taking up a surprising amount of time
00:49 samcv 0.188908 ms
00:50 samcv would this call bool nqp::if(nqp::atkey(points-point, $prop), '1', '0')
00:50 samcv that would hold a boolean value
00:50 samcv points-point{$prop} = True/False
00:51 samcv also what is calling the Bool
00:51 timotimo oh, there's definitely a bool object in there?
00:52 samcv yes it's either True or false
00:52 samcv should i change it to nqp::atkey(points-point, $prop) ?? '1' !! '0'
00:53 timotimo maybe try =:= True
00:53 timotimo the profile crashed again :\
00:54 samcv but what if it's set to 1 on accident
00:54 samcv lol.
00:54 samcv then it wouldn't give the right value
00:54 timotimo do you call &so or prefix:<?> or prefix:<!> or something somewhere hot?
00:54 samcv no
00:54 samcv i don't think
00:54 timotimo hmm
00:54 timotimo anyway, the spesh for the innermost loop looks okay
00:55 timotimo there's some lexical lookups that might be expensive perhaps?
00:56 timotimo and of course the always-beloved setlex a lexical and then immediately getlex it again
00:56 samcv timotimo, check it out again
00:56 samcv and get my latest changes
00:56 samcv s/check out/pull
00:57 timotimo right
00:58 timotimo now it bites us in the butt that we don't generate a map for perf and friends to understand
00:58 timotimo so they can't introspect our jitted code at all :(
00:59 timotimo otherwise we could figure out what pieces of the jitted code are hot and what pieces are not so hot
01:01 timotimo is atkey2 small? it gets called every time, and calling stuff can be costly
01:01 timotimo perhaps you can inline it manually and see if that helps?
01:01 samcv it just calls atkey( atkey( ))
01:01 samcv is in now inlining it?
01:03 samcv let me try putting it back to how i had it before
01:04 timotimo if it's so small, it's probably costly to call it each time
01:05 timotimo let me run it again to see if it gets inlined
01:05 timotimo i'm not sure why it would do that now as compared to before, though? did you change something inside the inner loop?
01:05 samcv ok it may make it 2s faster
01:05 samcv no
01:05 timotimo how do you figure?
01:06 samcv it went from  67s to 65s
01:06 timotimo oh
01:06 samcv when i put back the code i previously removed
01:06 timotimo well, that's underwhelming
01:06 samcv yeah
01:06 timotimo oh, those are the comments all over the place?
01:06 samcv yea
01:06 samcv 782
01:07 samcv timotimo, just pushed it with the code as it was week ago or so
01:07 timotimo i'm not sure i understand how that can be equivalent to the code below it?
01:07 samcv see changes in the last commit
01:09 timotimo you could try what happens if you turn the for loop into a while loop and manually shifting the array (need to clone) or an iterator over it (don't need to clone)
01:09 timotimo because then we might have the chance that the inner loop gets inlined into the outer loop
01:09 timotimo getting rid of a few million frame invocations
01:10 samcv both for loops into while loops?
01:10 timotimo and then see if you can get it to stash the object reference to these lexicals you defined at the beginning of the sub into the inner loop
01:10 timotimo try only the inner one first
01:10 samcv ok
01:11 timotimo i'll go sleep for reals now :)
01:11 timotimo good luck!
01:11 samcv night
01:11 samcv timotimo, should i use an actual iterator object?
01:11 timotimo night
01:11 samcv or just like a c for loop or something
01:11 samcv except in perl 6 obv
01:12 timotimo i was thinking nqp::iterator over the $!reified of the array
01:12 timotimo see the core setting for how to cheat the $!reified out of an Array/List object
01:12 samcv in $!reified
01:12 samcv err
01:12 samcv src/core/Rakudo/Iterator.pm
01:12 timotimo but doing a c-style for loop and atpos might also be worth a try
01:13 timotimo a loop loop
01:13 samcv yeah
01:13 samcv nqp::getattr(list,List,'$!reified')
01:13 samcv like that?
01:13 timotimo actually ... you can use a normal rakudo iterator and .pull-one
01:13 timotimo that's how
01:13 samcv @array.iterator?
01:14 samcv doesn't that automatically happen with a for loop?
01:14 timotimo i ... think so
01:14 timotimo i never had to do this
01:14 timotimo yes, but the for loop ends up calling the sub you give it a bunch of times
01:14 samcv what is the 'that's how' refering to. your message the line previous?
01:14 samcv or what i asked
01:14 timotimo the getattr code
01:14 samcv ok
01:14 timotimo that's what i meant
01:14 samcv kk
01:14 timotimo anyway, with a while loop we may have the inner block embedded directly into the outer block and only a thin piece of code before and after to make the looping happen
01:15 timotimo rather than having to have sink-all call the sub every time
01:15 samcv so do i need to get an iterator each time?
01:15 samcv for each cp?
01:15 samcv nqp::getattr(list,List,'$!reified') i mean
01:15 timotimo 'fraid so
01:15 timotimo how many times does the inner loop run per codepoint?
01:16 samcv like
01:16 samcv 20
01:16 timotimo oh
01:16 timotimo for some reason i had thought it'd be more often
01:16 samcv well more than 20
01:16 samcv depends how many props you are building
01:16 samcv may be 35tm not 100% sure
01:16 timotimo in that case you'd probably have to turn the outer loop into such a while loop, too
01:16 timotimo and yeah, there'd have to be a new iterator every time
01:17 samcv and so how do i get the next thing out of the iterator?
01:17 samcv do i use nqp or perl 6
01:17 timotimo for the iterator class you linked to in Iterator.pm, you'd use .pull-one until it returns IterationEnd
01:17 samcv my $iter = nqp::getattr(@list,List,'$!reified'); so that i need the pull one
01:17 samcv ok
01:17 timotimo you could look at the implementation of .sink-all and friends
01:18 timotimo with $!reified, you'll get a low-level nqp list
01:18 samcv oh ok
01:18 samcv so i need to use nqp. i can do that
01:18 timotimo with @list.iterator you'd get the thing with .pull-one and such
01:18 samcv ok
01:18 timotimo you have a choice between nqp and perl6 stuff here
01:18 samcv nqp sounds fine
01:18 timotimo OK
01:19 timotimo i'll leave now, otherwise i'd end up staying up all night
01:19 samcv night
01:19 timotimo o/
01:19 samcv uh how do i tell if iterator has more
01:20 samcv with nqp::elems? or?
01:36 samcv if this went properly it took 46 instead of 60 seconds making the inner loop a while loop
01:36 samcv cloning the iterator for each cp
01:37 samcv nice it did work fine
01:37 samcv 30% speedup. not too shabby
01:42 lizmat joined #moarvm
01:45 MasterDuke joined #moarvm
01:49 samcv nice now at 38 instead of 46
01:50 MasterDuke nice. you figured out how to use nqp::iterator i assume?
01:51 samcv nope
01:51 samcv but sorta i suppose
01:51 samcv i had to change it from my str @blah; to just @blah though
01:51 samcv not sure how to iterate a native array
01:51 samcv maybe have to clone the object and then shift it
01:51 samcv not sure if that will be faster
01:51 samcv but i can try
01:52 MasterDuke nqp: my $l =: nqp::list("ab", "cd"); my $i := nqp::iterator($l); while $i { say(nqp::shift($i)) }
01:52 camelia nqp-moarvm: OUTPUT«Assignment ("=") not supported in NQP, use ":=" instead at line 2, near ": nqp::lis"␤   at gen/moar/stage2/NQPHLL.nqp:765  (/home/camelia/rakudo-m-inst-1/share/nqp/lib/NQPHLL.moarvm:panic)␤ from <unknown>:1  (/home/camelia/rakudo-m-inst-1/share/nqp/lib/nqp.moa…»
01:52 MasterDuke nqp: my $l := nqp::list("ab", "cd"); my $i := nqp::iterator($l); while $i { say(nqp::shift($i)) }
01:52 camelia nqp-moarvm: OUTPUT«ab␤cd␤»
01:53 samcv let me try a native array and cloning the whole array instead of cloning the iterator
01:53 samcv m: my str @array = 'test', '123'; use nqp; say nqp::iterator(@array)
01:53 camelia rakudo-moar b51a55: OUTPUT«Cannot find method 'gist': no method cache and no .^find_method␤  in block <unit> at <tmp> line 1␤␤»
01:53 samcv m: my str @array = 'test', '123'; use nqp; my $item := nqp::iterator(@array); say nqp::elems($item)
01:53 camelia rakudo-moar b51a55: OUTPUT«This representation (VMIter) does not support elems (for type BOOTIter)␤  in block <unit> at <tmp> line 1␤␤»
01:54 samcv vmiter isn't in nqp docs
01:56 samcv m: my str @array = 'test', '123'; use nqp; my $item := nqp::clone(@array); say nqp::elems($item)
01:56 camelia rakudo-moar b51a55: OUTPUT«2␤»
01:58 MasterDuke m: my @array = "test", "123"; use nqp; my $item := nqp::iterator(nqp::getattr(@array,List,q|$!reified|)); nqp::while($item, say(nqp::shift($item)))
01:58 camelia rakudo-moar b51a55: OUTPUT«test␤123␤»
01:58 samcv so i'll see which is faster
01:58 samcv cause i pulled out the nqp iterator and cloned that every time
01:58 samcv i will compare to cloning the native array and shifting that instead of shifting the iterator
01:59 samcv (not calling getattr each time, only calling it once and cloning the object that returned)
02:00 MasterDuke you're comparing against atpos?
02:01 samcv no
02:01 samcv comparing against using nqp iterator
02:01 samcv shifting the iterator each time
02:03 samcv m: my str @array = 'test', '123'; use nqp; my $item := nqp::clone(@array); say nqp::shaft($item)
02:03 camelia rakudo-moar b51a55: OUTPUT«===SORRY!===␤No registered operation handler for 'shaft'␤»
02:03 samcv m: my str @array = 'test', '123'; use nqp; my $item := nqp::clone(@array); say nqp::shift($item)
02:03 camelia rakudo-moar b51a55: OUTPUT«MVMArray: shift expected string register␤  in block <unit> at <tmp> line 1␤␤»
02:03 samcv argh
02:03 samcv m: my str @array = 'test', '123'; use nqp; my $item := nqp::clone(@array); say nqp::shift(@array)
02:03 camelia rakudo-moar b51a55: OUTPUT«MVMArray: shift expected string register␤  in block <unit> at <tmp> line 1␤␤»
02:04 samcv m: my str @array = 'test', '123'; use nqp; my $item := nqp::clone(@array); say nqp::shift_s(@array)
02:04 camelia rakudo-moar b51a55: OUTPUT«test␤»
02:04 samcv \o/
02:07 MasterDuke do you want to shift @array or $item?
02:08 samcv well item, since i'm cloning it
02:08 samcv but it works fine for $item
02:08 samcv ok not sure if it's faster using a native array or not
02:08 samcv well it maybe it's 41s now, maybe it was 45 before, not sure
02:11 samcv ok 38.44655769 before. so doesn't seem to make it faster
02:12 samcv slower and or it's just being different between runs
02:13 MasterDuke oh well
02:13 samcv but i clone the nqp iterator so that probbaly pretty fast
02:14 samcv 38 instead of 73 is good
02:14 MasterDuke almost half
02:14 samcv yep :)
02:49 ilbot3 joined #moarvm
02:49 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:04 zostay_ joined #moarvm
03:06 avarab joined #moarvm
03:06 avarab joined #moarvm
03:08 geekosaur joined #moarvm
03:19 MasterDuke joined #moarvm
03:50 agentzh joined #moarvm
06:25 samcv not sure why but my program goes from 42 to like 60 seconds if i change from a perl 6 while loop to a nqp while loop
06:25 samcv all the code inside the while loop is nqp
06:25 samcv maybe it's not getting JIT'd hmm
06:26 samcv or maybe something else
06:30 samcv wow i'm down to 166 seconds
06:31 samcv for a build without the names
06:40 brrt joined #moarvm
07:13 stmuk_ joined #moarvm
07:23 Geth ¦ MoarVM/even-moar-jit: 4e857e9690 | (Bart Wiegmans)++ | 2 files
07:23 Geth ¦ MoarVM/even-moar-jit: Refactor spill_register
07:23 Geth ¦ MoarVM/even-moar-jit:
07:23 Geth ¦ MoarVM/even-moar-jit: Because it is also necessary to be able to spill a specific
07:23 Geth ¦ MoarVM/even-moar-jit: register (rather than just any), I need to reuse the logic for spilling
07:23 Geth ¦ MoarVM/even-moar-jit: a live range, which has been extracted. The logic to find a spillable
07:23 Geth ¦ MoarVM/even-moar-jit: position and to select the 'optimal' live range for spilling has also
07:23 Geth ¦ MoarVM/even-moar-jit: been split out because they can develop independently.
07:23 Geth ¦ MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/4e857e9690
07:25 brrt next up; indirect-mode two-ref fixing
07:48 brrt joined #moarvm
07:58 brrt good *, by the way :-)
08:01 samcv good (
08:01 samcv err *
08:02 brrt :-)
09:57 jnthn morning o/
10:00 brrt moarning jnthn
10:01 timotimo o/
10:02 dogbert11 o/
10:02 timotimo i'm really glad the iterator and while-loop suggestion gave good results
10:03 dogbert11 jnthn: so what's on the agenda today?
10:04 jnthn Well, at the moment just looking over recent RT tickets to see if any particularly merit my attention :)
10:05 samcv morning
10:05 jnthn o/
10:05 samcv me too timotimo
10:05 samcv for some reason the perl 6 while loop is fairly faster than nqp while loop
10:05 timotimo that's weird indeed
10:06 samcv and that's changing nothing at all except turning while nqp::elems($iter) into nqp::while(nqp::elems($iter
10:06 samcv yeah
10:08 timotimo i could compare the spesh logs
10:08 timotimo a more proper spesh diff tool would really be nice to have *cough*
10:10 timotimo i pulled and the outer loop is still a for loop
10:11 brrt perl6s 'blocks are function calls' dogma is really costly in loops
10:12 timotimo just backlogging right now, samcv, you said todd could /{3 - 2}/, and that "it evaluates to True"
10:12 samcv uh. i didn't say that
10:12 timotimo but raw code blocks in regex have no effect on the matching by themselves; you'd need <?{ }> or <!{ }> for that
10:13 timotimo oh? let me re-read
10:13 samcv i was giving an example of how it evalutes things
10:13 samcv ah ok
10:13 samcv well i was pointing out bracketns in perl 6 cause the code in it to be executed
10:14 dogbert11 jnthn, timotimo: apart from MVM_string_repeat the function collapse_strands is used here as well https://github.com/MoarVM/MoarVM/blob/master/src/strings/ops.c#L397
10:14 dogbert11 do you see any dangers therein?
10:15 jnthn Will take a look soon :)
10:15 timotimo no danger there, AFAICT
10:16 dogbert11 timotimo: sounds promising
10:17 timotimo i'm surprised nobody suggested .contains($Thingie) rather than ~~ /$Thingie/
10:17 jnthn I think it's codepaths with allocate_strands that want a close look
10:17 timotimo the difference in speed should be about a gazillion times
10:20 timotimo all 6 occurrences of allocate_strands look safe upon first glance
10:21 samcv gazillion
10:21 timotimo in scientific terms, yes
10:24 jnthn Been through all the allocate_strands also
10:24 timotimo i don't really know how to do it, but at some point a regex like that should definitely be compiled to just compile to .contains
10:26 dogbert11 and hopefully samcv hasn't stumbled on any more nasty bugs ..
10:26 samcv heh
10:26 samcv not yet
10:26 Geth ¦ MoarVM: 7b2e3d943c | (Jonathan Worthington)++ | src/strings/ops.c
10:26 Geth ¦ MoarVM: Factor out repeated code.
10:26 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/7b2e3d943c
10:27 jnthn Currently trying to figure out https://rt.perl.org/Ticket/Display.html?id=130692 fwiw
10:31 jnthn Ah. Argh.
10:31 jnthn Wondered how soon we'd run into this fun...
10:31 timotimo uh oh
10:54 jnthn The basic problem is that various things in Supply internals use locks
10:54 jnthn Which isn't automatically a problem.
10:55 jnthn But when we do an await somewhere in such a call stack, the lock "moves"
10:55 jnthn To another thread on resume
10:55 timotimo oh, and our locks are re-entrant
10:55 jnthn Right
10:55 timotimo and so when a different thread resumes and unlocks, it thinks that one doesn't own the lock
10:55 timotimo and explosion happens
10:55 jnthn Yup
10:56 timotimo but we also can't just unlock when we continuation outwards
10:56 jnthn MoarVM catches that, so no overly bad explosion at VM level
10:56 jnthn Right
10:56 timotimo so ... we need something "more" than reentrant
10:56 timotimo where the owner isn't just GetCurrentThread, but ... GetOwnerObject
10:56 jnthn Well, the immediate fix is to install an awaiter in Lock.protect
10:56 jnthn That forces a real blocking
10:57 timotimo OK, so when you're holding a lock, you can't actually switch ot another thread
10:57 timotimo right
10:58 jnthn Thing is, that'll hurt anyone who does reasonable things like an await in a supply block
10:58 timotimo aye, that's not so great
10:58 jnthn We kinda need a Lock::Awaitable
10:58 dogbert11 sounds difficult
10:58 timotimo can the awaiter that belongs to the lock semi-undo the lock (like, set a regular variable to 1, unlock the lock) and then await via the outer awaiter?
10:59 jnthn timotimo: Not sure that's a too good idea
10:59 jnthn A Lock::Awaitable basically would work like:
10:59 timotimo yeah, i see some problems with it already
10:59 jnthn await $alock.lock();
11:00 jnthn Which will either complete immediately if you get it
11:00 jnthn Or suspend you until you can have it
11:01 jnthn (In a pool-efficient way)
11:01 timotimo right
11:02 samcv that sounds great
11:02 jnthn We might also want to replace the mechanism in supply/react blocks with this
11:03 jnthn As I'm not entirely happy with the contention handling there at the moment
11:03 jnthn If we have two producers
11:04 timotimo we need punny names for our inspection tools. i suggest "Inspeshtor Gadjit"
11:04 jnthn And the supply block has tapped them both
11:04 jnthn When the first will emit a value
11:05 jnthn Then it notes the fact it's "working" in the supply block by putting itself onto a queue and then running itself
11:05 jnthn Well, running the code in the appropriate whenever
11:05 timotimo that's the current implementation, right?
11:05 jnthn Yes
11:05 jnthn If the second supply then emits a value while the first is being processed, then it sees it can't process it now
11:05 timotimo ISTR how it does the queueing its own work stuff
11:05 jnthn So it just sticks it into the queue and goes on its own way
11:06 timotimo so the queue can grow pretty big, yeah?
11:06 jnthn When the first emitter is done, it sees new work arrived and decides to help out with it
11:06 timotimo that's something that you'd call "backpressure", or a lack thereof?
11:06 jnthn So yeah, there's two things here
11:06 jnthn First, there's no backpressure on the second emitter. Depending on the nature of it we might not get any anyway of course.
11:06 timotimo oh, so when you emit, you might end up with your emitting thread doing work for hours before it can on its own resume
11:07 jnthn But also, yes, you might end up doing a lot more work than you expected
11:07 jnthn I mean, it's implicit that emit is a blocking operation for the sender
11:07 jnthn But it might be surprising that you end up doing somebody elses work too :)
11:07 timotimo right
11:07 timotimo on your thread, no less
11:07 jnthn Yup
11:07 timotimo but we don't really offer thread local storage
11:07 jnthn Before we used real locks
11:07 timotimo except maybe when you nativecall
11:08 jnthn But that ran into a deadlock in some cases
11:10 jnthn 5f1249e843b in Rakudo did that change fwiw
11:12 jnthn Also pondering https://rt.perl.org/Ticket/Display.html?id=130716 in this
11:13 timotimo ah, that one
11:22 jnthn Thing is, the way to do cleanup is a CLOSE block
11:23 jnthn Except we never get change to run it, because the supply block's main body is always holding the "lock", as it were
11:38 jnthn timotimo: Did your last/redo in whenever stuff get merged?
11:40 Geth ¦ MoarVM: 5538029892 | (Daniel Green)++ | src/strings/ops.c
11:40 Geth ¦ MoarVM: Del unused var, don't call MVM_string_graphs twice
11:40 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/5538029892
11:40 Geth ¦ MoarVM: 542baec756 | (Jonathan Worthington)++ | src/strings/ops.c
11:40 Geth ¦ MoarVM: Merge pull request #537 from MasterDuke17/minor_optimization_to_MVM_string_indexing_optimized
11:40 Geth ¦ MoarVM:
11:40 Geth ¦ MoarVM: Minor optimization to MVM_string_indexing_optimized
11:40 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/542baec756
11:43 jnthn Ah, no, it wasn't
11:47 samcv do we need to flush stdout to proc async?
11:49 jnthn samcv: What are you observing that's making you ask?
11:49 samcv https://gist.github.com/7bbb850655bcc3fa27e16846a911b4a6
11:49 samcv i'm getting Broken from this
11:50 dogbert11 timotimo, jnthn: is this gist 'informative' or is a case of 'something was clobbered at an earlier stage'? https://gist.github.com/dogbert17/d3d735210c1fd43615b99f968eb135a9
11:50 samcv broken for the call to .print
11:50 jnthn The :r isn't needed (and ignored) fwiw
11:52 samcv maybe it's the tap not working
11:52 jnthn "This process is not opened for write"...huh
11:54 jnthn Wait, does echo even accept stuff on stdin?
11:54 jnthn echo foo | echo at the shell does nothing
11:54 jnthn Except print an empty line
11:54 samcv oh
11:54 samcv yeah it seems to work with cat
11:55 samcv but not with my program hmm
11:55 jnthn Was gonna say, :w can't be entirley busted 'cus I'm using it in $dayjob code running dozens of times a day
11:56 samcv i'm using scanf also tried gets
11:56 samcv and it doesn't respond back. but running from terminal is fine
11:57 samcv can it not do relative paths? though i'd think it'd give an error-
11:58 samcv not getting stdout even when i make it show a message on starting it
11:58 samcv this should work right my $proc = Proc::Async.new('./build/bitfield', :w, :r);
11:58 jnthn The :r is redundant as mentioned earlier
11:58 samcv yeah but that changes nothing
11:58 jnthn Otherwise, don't see anthing wrong there
11:58 jnthn *anything
11:59 samcv wtf
12:10 jnthn lunch; bbi30
12:12 samcv i gotta go to bed. night all
12:12 moritz \o
12:37 * jnthn back
12:49 dogbert11 jnthn: was the gist crap, i.e. should I valgrind/ASAN the code?
13:15 jnthn dogbert11: No, it makes sense
13:15 jnthn I know how to fix that one :)
13:17 jnthn dogbert11: What code is it from, ooc?
13:17 jnthn Something small enough we can add it as a spectest?
13:19 jnthn Currently looking at a fragility in Supplier::Preserving
13:24 dogbert11 jnthn: MoarVM #458 and RT #128833
13:24 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=128833
13:26 dogbert11 I believe that the RT bug was two problems from the beginning but you fixed the first some time ago
13:27 dogbert11 it's still a bit difficult to reproduce (for me that is) however it became easier when I learned how to autmatically (re)run a program in gdb until it crashes
13:34 jnthn :)
13:50 jnthn Yay, I think I've fixed the Supplier::Preserving bug
13:50 jnthn Language lesson now
13:50 jnthn Will turn that into a spectest, then have a reliable fix for japhb++
13:50 jnthn bbi1h
14:35 dogbert11 .oO
14:44 Ven joined #moarvm
15:33 japhb joined #moarvm
15:39 jnthn m: https://gist.github.com/jnthn/a4b1dee678638041e791c63e4cd5c5c8
15:39 camelia rakudo-moar b51a55: OUTPUT«not ok 1 - Received expected messages (0)␤␤# Failed test 'Received expected messages (0)'␤# at <tmp> line 25␤# expected: '1 2 3 4 5'␤#      got: '35'␤not ok 2 - Received expected messages (1)␤␤# Failed test 'Received expected messages (1)'…»
15:43 timotimo you mixed tabs and spaces! ;)
15:53 jnthn wat?
15:54 jnthn *sigh*
15:54 jnthn Why can't everyone just use spaces :P
15:54 jnthn And why does /\t in vim not find them
15:55 jnthn Oh, :retab I guess should sort it out
16:21 timotimo :set tabexpand or :set notabexpand
16:35 japhb joined #moarvm
16:39 jnthn timotimo: I think if you have :set paste, those don't deal with it?
16:40 timotimo if you have :set paste, it won't do anything at all, IIUC
16:40 timotimo so if you're pasting something with tabs, you'll get tab characters
16:41 timotimo even if you :set tabe
16:41 timotimo no, it's expandtab
16:41 timotimo :set et
16:41 jnthn Yeah, that's what I did
16:41 jnthn So original musta had tabs :)
16:42 timotimo likely
16:58 * jnthn tests a fix for the issue dogbert11++ golfed/gisted earlier today
16:58 jnthn Well, best I can, anyway
16:58 jnthn Got it running in a loop in gdb
16:59 Geth ¦ MoarVM/even-moar-jit: f7aef159f7 | (Bart Wiegmans)++ | 2 files
16:59 Geth ¦ MoarVM/even-moar-jit: Special case handling of ARGLIST
16:59 Geth ¦ MoarVM/even-moar-jit:
16:59 Geth ¦ MoarVM/even-moar-jit: ARGLIST can create more references than can fit in the register set and
16:59 Geth ¦ MoarVM/even-moar-jit: register assignment to an ARGLIST reference will not work. So
16:59 Geth ¦ MoarVM/even-moar-jit: special-case those.
16:59 Geth ¦ MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/f7aef159f7
17:00 timotimo one more step towards function calls!
17:01 jnthn \o/
17:16 jnthn dogbert11: I pushed the fix for the thing you gisted earlier; feel free to give it a try out
17:18 jnthn m: for ^10 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:18 camelia rakudo-moar cf500e: ( no output )
17:18 jnthn m: for ^100 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:18 camelia rakudo-moar cf500e: OUTPUT«(signal )»
17:19 jnthn aha
17:19 jnthn but...which signal...
17:19 jnthn m: for ^100 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:19 camelia rakudo-moar cf500e: ( no output )
17:19 jnthn m: for ^100 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:19 camelia rakudo-moar cf500e: ( no output )
17:19 jnthn m: for ^100 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:19 camelia rakudo-moar cf500e: OUTPUT«(signal )»
17:19 dogbert17 jnthn: will test
17:19 jnthn OK, not just resoruce consumption
17:19 jnthn dogbert17: Could you try out the thing I just sent to the evalbot on a build without it?
17:20 jnthn without the fix, that is
17:20 jnthn commitable: 2017.01 for ^100 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:20 jnthn ah, not here
17:20 dogbert17 ok
17:21 dogbert17 SEGV
17:21 jnthn Seems that it'll be a decent test to cover it though
17:21 jnthn Nice :)
17:21 timotimo i can ask them to get the *able bots in here
17:21 dogbert17 so you got it, impressive
17:21 timotimo also, we could try to figure out why camelia doesn't understand what signal that is
17:22 dogbert17 SEGV seems to be in the same place as the original bug
17:23 dogbert17 =================================================================
17:23 dogbert17 ==12090== ERROR: AddressSanitizer: heap-use-after-free on address 0x90f14fdc at pc 0xb6061c56 bp 0x9c2dd828 sp 0x9c2dd81c
17:23 dogbert17 READ of size 4 at 0x90f14fdc thread T16
17:23 dogbert17 #0 0xb6061c55 in bind_key /home/dogbert/repos/rakudo/nqp/MoarVM/src/6model/reprs/MVMHash.c:98
17:26 jnthn cool
17:26 dogbert17 jnthn: that means that you can close at least one RT and a MoarVM issue
17:29 jnthn You got the IDs of those two handy?
17:29 dogbert17 MoarVM #458 and RT #128833
17:29 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=128833
17:30 dogbert17 it seem as if this wasn't a bug in MoarVM after all
17:30 timotimo the "bug" is that hashes aren't threadsafe
17:31 jnthn We shouldn't SEGV, but yeah, code assuming they are threadsafe is wrong
17:31 timotimo jnthn: got a better place to put the vmhealth implementation than moar.c?
17:32 jnthn Add a new file for it, perhaps?
17:32 timotimo ugh, so much work :)
17:33 dogbert17 :)
17:34 dogbert17 there are now only ~30 RT tickets marked SEGV. I don't think all of them are valid anymore
17:35 jnthn RT closed
17:35 dogbert17 cool
17:36 dogbert17 should I close the MoarVM one then?
17:36 jnthn Just did ;0
17:37 dogbert17 :)
17:37 * timotimo turns "number of threads" into "number of threads in each state"
17:38 mst timotimo: and make sure you give texas extra
17:38 jnthn Uff, tired...guess I'll have to save the next issues I want to look into until tomorrow
17:38 timotimo of course
17:38 dogbert17 maybe it's time for dinner?
17:39 jnthn Something like that :)
17:39 dogbert17 should MoarVM be bumped or should we wait until after this months release?
17:43 timotimo it gets bumped for the release anyway
17:43 timotimo at least we've done it like that forever
17:43 timotimo if we didn't want something for "in" the release, we'd put it into a branch for the time being
17:44 camelia joined #moarvm
17:45 jnthn Odd, https://rt.perl.org/Ticket/Display.html?id=130690  notes that it's sensitive to MVM_SPESH_DISABLE
17:46 jnthn Oh...was about to say it seems to go away under just MVM_JIT_DISABLE, but it showed up just after I was gonna write it :P
17:46 IOninja m: for ^400 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:46 camelia rakudo-moar 62bd30: ( no output )
17:46 IOninja m: for ^400 { my \g = EVAL 'grammar { token TOP { <a> | <b> }; token a { \d | \s }; token b { a | b } }'; await (start g.parse('b')) xx 8 }
17:46 camelia rakudo-moar 62bd30: ( no output )
17:47 IOninja m: use NativeCall; sub raise(int32) is native { * }; raise 11
17:47 camelia rakudo-moar 62bd30: OUTPUT«(signal SEGV)»
17:47 IOninja There. Fixed forever.
17:47 jnthn IOninja++
17:47 timotimo i'm also going to want to give out the number of threads marked as "blocked for gc", right? for vmhealth, i mean
17:48 jnthn Could do, yeah
17:49 jnthn OK, will continue with this bug tomorrow...
17:49 jnthn Rest; bbiab
17:49 timotimo good rest!
18:17 Geth joined #moarvm
18:20 ggoebel joined #moarvm
19:06 timotimo i'm not sure if i should be polling the GC Status for threads that aren't in stage 3 (running iirc)
20:14 agentzh joined #moarvm
20:21 agentzh joined #moarvm
20:27 jnthn timotimo: At some point they cease to have a ->tc
20:28 timotimo it'll get nulled out properly, though?
20:33 jnthn Yes
20:33 dogbert17 should this snippet recurse infinitely? class SQLString { }; my $stringy = Str.^find_method("Stringy"); my $handler = $stringy.wrap(method () { SQLString.new(:str(callsame)) }); say "foo".Stringy
20:33 dogbert17 timotimo has seen this before :)
20:35 dogbert17 if it should recurse/hang then we should be able to close https://github.com/MoarVM/MoarVM/issues/412 since jnthn has fixed the callsame bug which caused it to crash
20:35 jnthn m: class SQLString { }; my $stringy = Str.^find_method("Stringy"); my $handler = $stringy.wrap(method () { SQLString.new(:str(callsame)) }); say "foo".Stringy
20:36 camelia rakudo-moar a62882: OUTPUT«(timeout)»
20:37 jnthn m: class SQLString { }; my $stringy = Str.^find_method("Stringy"); my $handler = $stringy.wrap(method () { SQLString.new(:str(callsame)) }); say "foo".WHAT
20:37 camelia rakudo-moar a62882: OUTPUT«(Str)␤»
20:37 jnthn m: class SQLString { }; my $stringy = Str.^find_method("Stringy"); my $handler = $stringy.wrap(method () { SQLString.new(:str(callsame)) }); say "foo".Stringy.WHAT
20:37 camelia rakudo-moar a62882: OUTPUT«(SQLString)␤»
20:37 jnthn m: class SQLString { }; my $stringy = Str.^find_method("Stringy"); my $handler = $stringy.wrap(method () { SQLString.new(:str(callsame)) }); say "foo".Stringy.Str
20:37 camelia rakudo-moar a62882: OUTPUT«SQLString<59372912>␤»
20:37 jnthn m: class SQLString { }; my $stringy = Str.^find_method("Stringy"); my $handler = $stringy.wrap(method () { SQLString.new(:str(callsame)) }); say "foo".Stringy.gist
20:37 jnthn hah
20:38 jnthn Well, I guess .gist calls .Stringy on something
20:38 camelia rakudo-moar a62882: OUTPUT«(timeout)»
20:38 jnthn Which is reasonable I guess
20:39 dogbert17 and the SEGV is gone
20:39 jnthn Wrapping Str.Stringy probably counts as playing with fire
20:39 jnthn If the SEGV is gone then it's certinaly not a moar bug any more :)
20:40 dogbert17 but I can't close it, perhaps nine should do that
20:40 jnthn I think I'd need more persuading that it's a Rakudo bug
20:40 jnthn I mean, for starters SQLString doesn't even do the Stringy role
20:44 dogbert17 I'm not going to try to do that :)
20:47 dogbert17 but there are several MoarVM issues open that I believe have been solved already
20:53 timotimo food time \o/
21:48 dogbert17 kabooom: https://gist.github.com/dogbert17/c1f9fe869322ae7eefbf6c6422570694
21:51 jnthn Innerestin'
21:53 dogbert17 it's the code from https://rt.perl.org/Public/Bug/Display.html?id=130494 run with a 64k nursery and GC_DEBUG=1
22:03 * jnthn wonders where this bug will end up being
22:03 jnthn It's...odd
22:03 jnthn It almost looks like a double-release of an object
22:03 jnthn Or double-gc_free of it
22:10 jnthn Phew, nearly got my grant report done
22:21 dogbert17 I have the strand and gc-fixes in 'my' moarvm, do you think thay might be involved in which case I can remove them
22:34 MasterDuke joined #moarvm
22:34 timotimo i wonder if there's any circumstances where we share decodestream across objects
22:34 timotimo or clone the insides of a decodestream?
22:34 timotimo i.e. the pointer there
22:37 MasterDuke timotimo: have you seen this? is it a legit problem? https://gist.github.com/MasterDuke17/5f07124670c43c819c6e230f388b51d4
22:45 timotimo probably no problem
22:46 timotimo it happens basically every single time we precomp
22:47 MasterDuke causes the test to fail
22:47 MasterDuke to be more precise, the test fails occasionally. when i run it in valgrind, i get that output occasionally, i assumed they were related
22:48 timotimo oh?
22:48 timotimo maybe i should look closely into it
23:13 timotimo what do i have to run to get that exact one, MasterDuke
23:13 timotimo oh
23:13 timotimo there it is
23:24 timotimo for those all the "program did not hang" tests all fail all the time
23:25 timotimo is that because of valgrind slowing things down?
23:27 timotimo i think you can only get that error if you have precompilation happening
23:31 timotimo and it's potentially not important what process you're running as long as there's some precomp
23:32 timotimo ah, interesting
23:34 timotimo (i said interesting because i looked at the right line in a different file)
23:38 timotimo i don't get it. i changed the offending allocation to calloc and it still says uninitialized memory?
23:38 timotimo maybe it's memcpying some uninitialized bytes into it or something
23:41 timotimo it seems to be inside the serialized blob

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary