Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-02-23

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:11 pyrimidine joined #moarvm
00:58 pyrimidine joined #moarvm
03:37 pyrimidine joined #moarvm
04:27 agentzh joined #moarvm
05:28 pyrimidine joined #moarvm
05:33 pyrimidine joined #moarvm
06:34 pyrimidine joined #moarvm
07:04 agentzh joined #moarvm
07:31 Geth ¦ MoarVM/even-moar-jit: 919ee4d8a8 | (Bart Wiegmans)++ | 2 files
07:31 Geth ¦ MoarVM/even-moar-jit: Pseudoimplement toposort for ARGLIST
07:31 Geth ¦ MoarVM/even-moar-jit:
07:31 Geth ¦ MoarVM/even-moar-jit: Also change API of prepare_arglist to take both ARGLIST and CALL
07:31 Geth ¦ MoarVM/even-moar-jit: tiles and panic if they're not adjacent. They are now due to the
07:31 Geth ¦ MoarVM/even-moar-jit: way the tiling process works, but if they wouldn't be then we'd at
07:31 Geth ¦ MoarVM/even-moar-jit: least detect it.
07:31 Geth ¦ MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/919ee4d8a8
08:21 zakharyas joined #moarvm
08:37 * moritz points out that "arglist" is a word in German :-)
08:37 moritz roughly translates to deceitfulness
08:39 nwc10 the lies we tell computers...
08:39 nwc10 that's going to amuse me every time I remmeber it
08:39 nwc10 (thanks)
08:41 jnthn And that's exactly why parameters should be validated :P
08:48 brrt joined #moarvm
08:48 brrt good *, #moarvm
09:12 agentzh joined #moarvm
09:13 * lizmat points out "arglistig" is a Dutch word
09:14 lizmat but oddly enough, "arglist" isn't
09:45 brrt well, that surprises me
09:45 brrt i'm not even sure what that means
09:56 brrt do you know what it means?
10:10 dogbert17 MasterDuke: for me it fails with the message 'MoarVM panic: Could not spawn thread: errorcode -11'. A quick check with strace gives 'mmap2(NULL, 8392704, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = -1 ENOMEM (Cannot allocate memory)'
10:12 timotimo we don't really have to panic there, do we?
10:13 jnthn If we don't, it might be that we've got a bunch of cleanup to do
10:13 jnthn 'cus presumably we've prepped stuff for the thread
10:15 dogbert17 timotimo, jnthn: if you have had your morning coffee the offending program can be found here: https://github.com/rbastic/CounterMutexSingleton/blob/master/CounterMutexSingleton.pl6
10:16 timotimo yeah, uh, 100000 threads is a bit optimistic
10:17 jnthn o.O
10:17 dogbert17 wat, in Erlang that's a piece of cake :)
10:17 timotimo but yeah, it should definitely fail earlier when it tries to epoll_create and gets EMFILE
10:18 dogbert17 it's RT #123434 btw
10:18 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=123434
10:19 timotimo i'll work on a more proper error here
10:20 timotimo i can run it with 565 threads
10:20 dogbert17 timotimo: should we continue looking at the coverity stuff later?
10:20 timotimo 566 threads and it asplode
10:21 dogbert17 I can't create that many :(
10:21 brrt hmm, that exact many? how sensitive is that to MVM_JIT_DISABLE, MVM_SPESH_DISABLE etc?
10:24 timotimo disable spesh and it fails already at 526
10:24 brrt okay, so it's not due to speshing
10:27 timotimo well, speshing doesn't open many file descriptors, does it?
10:27 timotimo timo@schmand /tmp [139]> env MVM_SPESH_DISABLE=1 perl6 CounterMutexSingleton.pl6 --num_threads=526
10:27 timotimo MoarVM panic: ThreadContext: could not create a new libuv loop!
10:27 timotimo how's that?
10:27 timotimo much better than a segfault, no?
10:27 timotimo next step might be to put in proper cleanup for this case and throwing a catchable exception
10:28 Geth ¦ MoarVM: c5379702f0 | (Timo Paulssen)++ | src/core/threadcontext.c
10:28 Geth ¦ MoarVM: panic when new TC can't create an uv_loop
10:28 Geth ¦ MoarVM:
10:28 Geth ¦ MoarVM: next step might be to put in proper cleanup for this case and
10:28 Geth ¦ MoarVM: throwing a catchable exception. until now it segfaulted.
10:28 Geth ¦ MoarVM:
10:28 Geth ¦ MoarVM: cf RT #123434
10:28 synopsebot6 Link:  https://rt.perl.org/rt3//Public/Bug/Display.html?id=123434
10:28 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c5379702f0
10:31 dogbert17 oh, another SEGV gone
10:32 brrt timotimo: why should throwing an adhoc exception not work?
10:34 timotimo *shrug*, it probably should
10:34 jnthn Where are you when you throw it?
10:36 timotimo in a BUILD method :P
10:37 timotimo called by Thread.start
10:37 brrt uhuh; that's perl code running on some mvm runloop, isn't it?
10:37 jnthn But not actually on the started thread yet?
10:38 brrt in which case, throwing adhoc should be capturable by the creating thread
10:38 timotimo not on the started thread yet. no
10:38 timotimo it asplodes while trying to create the thread
10:38 jnthn ah, OK, then should work out :)
10:38 timotimo i.e. MVM_thread_new is trying to MVM_tc_create, but fails
10:38 timotimo thing is, we also MVM_thread_new in main.c and when something requests the eventloop to exist
10:38 timotimo that'll surely asplode a bit more mysteriously whenever someone manages to hit that case in the wild
10:39 timotimo oh, but
10:40 timotimo when we create the thread for the main thread, it'll just grab uv_default_loop
10:40 timotimo surely that'll never be null ... :) :)
10:45 timotimo timo@schmand /tmp> env MVM_SPESH_DISABLE=1 perl6 CounterMutexSingleton.pl6 --num_threads=526
10:45 timotimo Could not create a new Thread.
10:45 timotimo in sub MAIN at CounterMutexSingleton.pl6 line 79
10:45 timotimo in block <unit> at CounterMutexSingleton.pl6 line 71
10:45 dogbert11 nice
10:46 jnthn Beats SEGV :)
10:46 timotimo by a large margin
10:48 Geth ¦ MoarVM: acb20d7f7e | (Timo Paulssen)++ | 3 files
10:48 Geth ¦ MoarVM: tc_create can now return NULL, throw in thread_new
10:48 Geth ¦ MoarVM:
10:48 Geth ¦ MoarVM: also panic in moar.c if tc_create returned NULL.
10:48 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/acb20d7f7e
10:52 timotimo interesting. it'll throw the exception way, way, way earlier than it did before
10:53 timotimo (with earlier i mean when the num_threads is much lower)
11:01 timotimo i'll also want to output the uv error here
11:09 timotimo (but how ...)
11:10 timotimo i'll actually put the exception throw into tc_create ...
11:11 timotimo now how do i sanely figure out if we're in the "the instance hasn't even been set up yet" state ...
11:11 timotimo well, the whole instance struct is still nulled at that point, but i'm not sure it's sane to rely on that
11:11 timotimo oh! in that case instance->main_thread isn't true, so we're fine
11:11 timotimo cool.
11:12 timotimo argh :D
11:12 timotimo haha
11:13 timotimo we don't have the tc that's responsible for creating the thread. so throw adhoc with the tc that's in scope is definitely wrong, but i can't just go ahead and guess which one to choose
11:14 timotimo oh C, it's so fun to program in you sometimes
11:15 timotimo oh! MVM_tc_create isn't MVM_PUBLIC!
11:15 timotimo praise the lord
11:16 timotimo timo@schmand /tmp> perl6 CounterMutexSingleton.pl6 --num_threads=253
11:16 timotimo Could not create a new Thread: too many open files
11:16 timotimo in sub MAIN at CounterMutexSingleton.pl6 line 76
11:16 timotimo in block <unit> at CounterMutexSingleton.pl6 line 71
11:17 Geth ¦ MoarVM: b0dc2bf64e | (Timo Paulssen)++ | 4 files
11:17 Geth ¦ MoarVM: throw from tc_create so we have the error code
11:17 Geth ¦ MoarVM:
11:17 Geth ¦ MoarVM: also use that error code to give a proper error message.
11:17 Geth ¦ MoarVM: requires tc_create to have the "parent" tc passed, but it's
11:17 Geth ¦ MoarVM: not MVM_PUBLIC, so we can just change the API here.
11:17 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b0dc2bf64e
11:19 dogbert17 you're on a roll :-)
11:20 timotimo nah, all of that should have been a single commit, really ;)
11:20 brrt timotimo++
11:20 dogbert17 indeed timotimo++
11:22 timotimo replied to the ticket again
11:22 timotimo and now AFK for a little bit
11:23 timotimo someone can go a head and write a test for this :)
11:23 timotimo dies_ok ... loop { Thread.new ... } or something like that
11:24 dogbert17 what happens if someone has unlimited fd's
11:24 timotimo unlimited, eh?
11:24 timotimo i want to see the machine that has an unlimited wordsize
11:25 dogbert17 :)
11:25 timotimo it looks like num_threads = 253 (the first one it crashes with?) gets up to an fd of 1021
11:25 timotimo oh, i was wrong
11:26 timotimo immediately after that it uses pipe2 which allocates 1022 and 1023
11:26 timotimo ah, ok, it makes one for the epoll, two for pipe2, and another one for eventfd2
12:10 dogbert17 is the function 'uninline' in src/spesh/deopt.c used? There are at least three coverity complaints wrt that function (135422, 135424 and 135436).
12:11 timotimo yes, it is very used
12:11 timotimo it's also maximum scary
12:11 jnthn Yes, it's used and it's the scariest part of MoarVM IMHO :)
12:11 jnthn I spotted the complaints, but didn't have the concentration to see how real they are
12:11 dogbert17 using several uninitialized vars makes it even scarier or does my eyes deceive me?
12:12 timotimo might be best to null out everything you see there :)
12:12 jnthn We can do that for safety of course :)
12:12 timotimo or ... look very closely at the trace that allows the uninitialized values to be used
12:12 timotimo and try to figure out if those traces are actually possible in real-world data
12:12 * dogbert17 should probably refrain from touching that code then
12:12 jnthn It's possible that for <reasons> the traces it detects can't occur in the rewal world.
12:13 jnthn bbiab
12:13 timotimo you mean the real world. *anything* can occur in the rewal world, however.
12:13 timotimo the rewal world is a real scary place
12:13 timotimo good lunch, jnthn :)
12:13 dogbert17 the complaints look real enough for my untrained eyes
12:14 nwc10 we found this with coverity and the perl 5 regex engine - coverity (legitimately) thought that something might be possible, but the control flow meant that it couldn't happen
12:14 nwc10 (that doesn't mean that the control flow was clear, obvious or bug free. Although these cases the relevant unit of wetware did have a good audit)
12:14 brrt uninline is the stuff of nightmares. it does work though
12:14 moritz there are several uses of variables guarded with an if (last_uninlined)
12:15 dogbert17 aha, so there is no danger then, perhaps we should follow the old adage 'if it works ...'
12:15 moritz and when last_uninlined is set, those variables are also set
12:16 moritz so if coverity complains about last_res_res, last_res_type and last_return_deopt_idx, they are likely false positives
12:16 timotimo oh snap
12:16 nwc10 I wouldn't say "no danger". Far from it
12:17 nwc10 but things like ASAN and a lot of abusive tests seem to be useful
12:17 timotimo no danger, william robinson
12:17 nwc10 and even more so, fuzzers
12:17 timotimo aye
12:17 nwc10 American fuzzy lop seems to be particularly cool
12:17 moritz designing fuzzing to hit deopt/uninline sounds like fun :-)
12:17 dogbert17 perhaps I should move on to another complaint then :)
12:18 moritz first have to to create a hot loop for the otpimmization to happen, then break an assumption in made earlier
12:21 timotimo rawr is also very interesting
12:21 timotimo no, not rawr, grr.
12:21 timotimo https://blog.trailofbits.com/2016/11/02/shin-grr-make-fuzzing-fast-again/
12:33 pyrimidine joined #moarvm
12:35 brrt mixing-in-at-runtime should be a good way to do it
12:35 brrt maybe somthing like:
12:36 brrt role Foo { method Str("foo") }; sub make-foo(Int $i) { if $i > 500 { $i * 3 } else { $i but Foo } }; for ^1000 { make-foo().Str.say; }
12:37 brrt otoh, not sure if that will actually inline anything...
12:38 timotimo don't you mean to reverse the > there?
12:38 timotimo so that it starts mixing in stuff late, rather than early?
12:38 brrt oh, yeah
12:38 brrt you're right
12:39 brrt probably should throw in something that would inline
12:39 timotimo right, our operators on native ints seem to do that
12:39 timotimo but you migh thave to use --optimize=0 to prevent the static optimizer from doing htat
12:39 timotimo that*
12:55 timotimo with 90000 threads when stracing i see it do some stuff, then count down from 2000 and calling close on a bunch of fds
12:55 timotimo then doing stuff again
12:55 timotimo then closing stuff
12:56 jnthn The static inliner will only do stuff on natively typed subs these days, fwiw
12:57 timotimo right. i suggested to use native int stuff first, then warned to --optimize=0 for that
13:07 timotimo i have no idea how far it's come yet, but that script is running without segfault apparently
13:13 timotimo this is really gonna take a while
13:13 timotimo and i'% about to leave my desktop for the rest of the day
13:13 timotimo i didn't start the script in a tmux or anything
13:13 timotimo guess i'll restart it
13:13 agentzh joined #moarvm
14:04 brrt joined #moarvm
15:06 brrt joined #moarvm
15:23 brrt joined #moarvm
15:34 pyrimidine joined #moarvm
15:52 pyrimidine joined #moarvm
16:35 pyrimidine joined #moarvm
16:40 ggoebel joined #moarvm
17:26 pyrimidine joined #moarvm
17:33 pyrimidine joined #moarvm
17:50 agentzh joined #moarvm
17:53 pyrimidine joined #moarvm
17:56 harrow joined #moarvm
19:38 Geth ¦ MoarVM: MasterDuke17++ created pull request #540: Change MVM_string_equal_at_ignore_case to use fc
19:38 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/540
19:55 pyrimidine joined #moarvm
20:37 ggoebel joined #moarvm
21:56 pyrimidine joined #moarvm
22:01 pyrimidine joined #moarvm
22:09 pyrimidine joined #moarvm
23:22 Geth joined #moarvm
23:40 diakopter .u hurtrealbad
23:40 yoleaux2 No characters found
23:42 diakopter :hurtrealbad:
23:42 diakopter emoji paste fail

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary