Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-04-07

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 samcv hopefully my PR looks sane
00:01 samcv timotimo, how is appveyor coming?
00:02 timotimo well, it runs, but i get mysterious nqp failures that i didn't investigate further yet
00:02 samcv that's okay
00:02 samcv if i do a pr will appveyor run on it?
00:02 timotimo can has the json document that asplodes there? maybe only the part where it asplodes, though?
00:02 samcv kk
00:02 samcv yea
00:03 timotimo i hit "new build" so we can get a fresh nqp test run
00:03 timotimo it should be clean, right?
00:05 samcv here https://a.uguu.se/4wdPsjslTP7s_100cache.xz
00:05 samcv what should be clean?
00:06 timotimo "make test" in nqp
00:06 samcv yeah
00:06 samcv show me if it's not
00:06 timotimo well, it wasn't on appveyor 3 days ago :)
00:07 timotimo https://ci.appveyor.com/project/timo/moarvm/build/job/6aklp6uom38mbshc - you can watch the progress here
00:07 timotimo it's just cloning nqp
00:09 timotimo t\nqp/019-file-ops.t                 (Wstat: 0 Tests: 104 Failed: 0)
00:09 timotimo Parse errors: Bad plan.  You planned 107 tests but ran 104.
00:09 timotimo ok 104 - reading with nqp::readlinefh stuff written by nqp::writefh
00:09 timotimo Failed 3/107 subtests
00:09 timotimo (less 23 skipped subtests: 81 okay)
00:09 timotimo what were you doing with that cache file? because i can run the script just fine and don't get an error
00:10 timotimo um
00:10 timotimo there's totally a \ that doesn't belong in there :P
00:10 timotimo look at how the first line ends and the second line begins in that file
00:10 timotimo i don't think it should be like that
00:13 timotimo same failure mode on x86, btw
00:14 timotimo the thing is
00:15 timotimo the last test that succeeds in there ... is also the last test in the file
00:15 timotimo so the plan is wrong, right?
00:15 timotimo but i have the same code on my machine and it ends up with the right number of tests
00:15 timotimo so we're improperly skipping something somewhere?
00:16 timotimo aha
00:16 timotimo there's a skip there that says 6, but the "else" branch of that has 9 tests in it
00:16 timotimo so that's probably the culprit?
00:19 samcv uhm
00:19 samcv use JSON::Fast; spurt("otherCache", from-json(to-json(slurp("scache"))
00:19 samcv so
00:19 samcv it should work cause i'm converting something to json then back
00:19 timotimo oh, it's a long-ass string, rather than split at \0
00:20 timotimo ok, i did that differently
00:20 timotimo let's see
00:20 samcv oh wait. no. wait hold on. i think i pasted wrong
00:20 samcv use JSON::Fast; spurt("otherCache", from-json(to-json(slurp("100cache").split("\0"))).join("\0"))
00:26 timotimo that exact code works for me
00:26 timotimo did you make sure to install the latest JSON::Fast with zef, or alternatively used -I on the git checkout's lib folder?
00:27 timotimo https://ci.appveyor.com/project/timo/moarvm
00:27 timotimo look, it green
00:30 MasterDuke samcv: is your PR faster?
00:30 samcv a tiny bit yes
00:31 samcv it is measurable
00:31 MasterDuke nice
00:31 timotimo oh nice trick
00:32 samcv see stats https://gist.github.com/samcv/035ead8a920e5f61d01808c9293d324e
00:32 samcv that's with uhm. use JSON::Fast; spurt("otherCache", from-json(to-json(slurp("100cache").split("\0"))).join("\0"))
00:33 samcv which is 1/100 of the original one
00:33 samcv $cache.substr(0, $cache.chars/100)
00:33 samcv ^that's how i made it
00:33 samcv if timotimo is wondering
00:33 timotimo ah, thanks
00:33 samcv yay green
00:33 timotimo yes yes very green
00:34 timotimo i don't know if we can sign up to travis with a github organization or something
00:34 timotimo not travis
00:34 timotimo appveyor
00:34 samcv tell it to do PR
00:34 timotimo smh
00:34 timotimo i can't
00:34 samcv :\
00:34 timotimo i don't have access to the github hooks on the moarvm repo
00:34 timotimo i can't even make it build commits to master
00:34 timotimo i can, however, click "new build" every time someone asks me to
00:34 samcv i am disappointed
00:34 samcv :)
00:34 samcv hahaha
00:35 timotimo except when i'm asleep, or AFK, or something
00:35 timotimo i won't give anybody a pager number for me so they can wake me the middle of my sleep to run an appveyor build :D
00:35 samcv lol.
00:35 samcv uhm i want one ran
00:35 timotimo okay, i would, but only if they give me tons of money
00:35 samcv on my repo
00:35 samcv i'm sure i could pay somebody in india much less money to do it
00:35 timotimo oh, on your repo? gotta figure out how that works
00:36 samcv i don't *think* mcsvc has memmem but it's possible. worst case we can always make our own
00:36 samcv or just use whatever it was we did before
00:37 timotimo um ... i don't even know how to tell appveyor to build something else tbh :(
00:37 timotimo however!
00:37 timotimo it's really easy to set up your own!
00:37 samcv https://github.com/samcv/MoarVM/commit/88994712163518f8e6dbd1f0dd90671eea0af3aa but yeah
00:37 samcv we could just ifdef that part and only have it build on linux or something idk
00:37 samcv or bsd
00:38 samcv though i haven't benched it on mac os x yet
00:38 samcv you have mac right timotimo ?
00:38 samcv can you check for me
00:38 samcv or is that somebody else. i don't know
00:38 samcv it's 2.2x faster under worst case for me though with the change on linux with glibc
00:38 timotimo no, i'm on linux, i don't own a mac at all
00:38 timotimo lizmat has a mac
00:38 samcv cool
00:39 samcv i can't imagine it would be *slower* than what we do now
00:39 timotimo you might call 'er ... lizmac!
00:39 * samcv dies
00:39 samcv i'm hungry
00:39 timotimo i hope your grant gets accepted soon enough so you can eat!
00:39 samcv haha
00:40 timotimo i haven't written a comment on it yet ... i always find it hard to write something that doesn't sound thoroughly weird
00:40 timotimo already had that problem with jnthn's proposals, and also with pmurias' one
00:40 timotimo and i think also with brrt's proposal?
00:45 timotimo anyway, bedtime!
00:56 samcv night :)
01:46 samcv timotimo, hopefully this check works, I think it should, so far have not encountered it when running with roast https://github.com/samcv/MoarVM/blob/14935bcbfe6865dda74b1d51f4c77969a12672af/src/strings/ops.c#L236
01:48 ilbot3 joined #moarvm
01:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
01:56 Geth ¦ MoarVM: samcv++ created pull request #574: Use memmem in string index. Uses Knuth-Morris-Pratt on glibc 2.2x+ faster
01:56 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/574
04:57 JimmyZ samcv: re MVN, it's typo :)
05:20 domidumont joined #moarvm
05:27 domidumont joined #moarvm
05:31 brrt joined #moarvm
06:05 domidumont joined #moarvm
06:13 samcv heh
06:14 samcv MVN: Moar virtual normalizer. heh since it's MVN_unicode_normalizer_form
06:21 brrt ohai samcv
06:21 samcv hey brrt  :)
06:22 samcv it's your morning right brrt ?
06:22 brrt how are things?
06:22 brrt yes
06:22 samcv good, got some work done today
06:22 brrt i just figured out i nuked objdump
06:22 samcv nice
06:22 brrt well, and now i need it :-P
06:22 samcv got regex in perl 6 2.2x faster
06:22 brrt because somehow i screwed up something in the LET to DO translation
06:22 brrt oh, very nice
06:22 samcv https://github.com/MoarVM/MoarVM/pull/574 using memmem. though it's only fast on glibc platforms
06:22 brrt all regex
06:22 brrt ?
06:22 samcv well
06:23 samcv if they are both 32bit or both 8bit needle/haystack
06:23 brrt hmm, i think there are some feature macro's you can use
06:23 samcv also need to get some pointer subtraction and division checkd by somebody else though
06:24 samcv can read the full text of my PR https://github.com/MoarVM/MoarVM/pull/574 explains pretty well all conceirns things i need comments on
06:39 brrt joined #moarvm
06:43 brrt sure, no problem
06:46 brrt memmem is a gnu extension, though
06:46 samcv yeah. bsd has it too (and mac os x)
06:46 brrt and windows?
06:46 samcv windows probably doesn't. but we can always ifdef it and just have it do the same path we do now
06:47 brrt that'd be wise
06:47 samcv since if the strings aren't both the same storage type we have to do that on any platform
06:47 samcv also please tell me what #ifdef's to use
06:47 samcv what do we use on MoarVM for that? or do i need to ask jnthn
06:47 brrt let me think about that for a bit
06:48 brrt i recall having a decent set of ifdefs….
06:48 samcv i guess i could grep all the ifdefs
06:48 samcv but i think bsd memmem is probably going to be faster than what we currently do, even though it's not fancy and super great like the glibc one
06:48 brrt anyway, even-moar-jit has a definition MVM_JIT_PLATFORM_POSIX vs MVM_JIT_PLATFORM_WIN32
06:48 brrt but that's not going to help you much
06:48 samcv and if it *is* a fair bit faster, we can just implement a basic memmem for windows or something
06:49 brrt anyway, what i wanted to say is
06:49 brrt we have 'isolated' the platform-specific bits into src/platform
06:49 samcv we could always just have memmem on windows that is our own function
06:50 brrt exactly
06:50 samcv yeah
06:50 brrt although i'd wrap it in an MVM_memmem in that case
06:50 samcv that would be the most neat way to do it
06:50 samcv and then ifdef it?
06:50 brrt aye
06:50 samcv k
06:51 brrt iirc the macro _MSC_VER for the microsoft C compiler
06:51 samcv if we use gcc on windows do we have glibc?
06:51 brrt .. not sure
06:52 samcv idk how windows environment works... maybe only if we build in cygwin?
06:52 brrt what mingw has
06:52 brrt or cygwin, yes
07:07 samcv i mean we could have a MVM_memmem with a 5th argument, which is in size_t, which denotes what offset the match is needed at
07:08 brrt not sure if i grok
07:08 samcv like for 32bit strings can supply it with sizeof(MVMGrapheme32) and it can know to check if it's a multiple of that thing
07:08 brrt aha
07:08 samcv ok. so memmem replies back with memory addresses
07:08 brrt uhuh
07:08 samcv and 8 bits vs 32 bits, and it's possible but rare it could find one inside another
07:08 samcv well the likelyhood goes down exponentialy if the needle is longer than 1
07:09 samcv but regardless. it should be checked at least. and would be a useful argument i ncase we have a magic futuristic memmem which does it for us and we don't need to continue again
07:10 brrt right, i see
07:10 brrt that could be something, yes
07:11 samcv see my comment on the code on the PR. and hopefully somebody can double check that it does what i think. pretty sure it does but
07:11 samcv pointer addition subtraction etc can have weird things
07:12 brrt oh, i see
07:12 brrt just cast to (char*), and subtract
07:12 brrt iirc that will (officially) give you a ptrdiff_t
07:13 samcv i think it got mad at me doing that
07:13 samcv when i did the division
07:13 samcv well. the %
07:13 brrt o.O
07:13 samcv did not like me checking remainder of a char *
07:14 samcv and wouldn't compile
07:14 domidumont joined #moarvm
07:15 brrt okay, then… let me check
07:16 samcv go ahead and try it yourself, maybe i did something weird in addition to that
07:17 Geth ¦ MoarVM: 4dce22fcee | (Samantha McVey)++ | 3 files
07:17 Geth ¦ MoarVM: Fix typo MVM_unicode_normalizer_form was MVN (typo)
07:17 Geth ¦ MoarVM:
07:17 Geth ¦ MoarVM: MVN_unicode_normalizer_form should have been MVM_unicode_normalizer_form.
07:17 Geth ¦ MoarVM: Fix the typo in all references to this function.
07:17 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/4dce22fcee
07:19 brrt https://gist.github.com/anonymous/fa28475fc1cc16f4014393bb8fd48aad <- this just works
07:19 brrt that said
07:19 brrt you can check for within-boundaryness simply with
07:21 brrt (((char*)mm_return_32) - ((char*)haystack->body.storage.blob32) & 3) == 0
07:21 brrt well, that's if you're on the 4 byte boundary,  if you're within the boundary you're nonzero
07:21 brrt that said
07:22 brrt you probably shouldn't throw in that block
07:22 brrt i mean
07:22 brrt hmmmm
07:22 brrt memmen wants to find the *first* occurrence, doesn't it?
07:23 brrt so you know by definition that there isn't an earlier match
07:23 brrt so you can restart the search at the next grapheme boundary
07:36 domidumont joined #moarvm
07:37 brrt samcv: left my comment on the PR
07:45 samcv thx for comments :-)
07:46 brrt yw :-)
07:46 samcv yeah exactly. see my comment i just posted in replyp
07:46 brrt okay, i can live with a memmem-like with an extra argument, but as a rule, you should probably call it something else than memmem
07:47 brrt since memmem has no such argument
07:47 brrt memfind, maybe
07:47 brrt mempos
07:47 samcv ohalso it seems the speed gains are similar on bsd as on glibc
07:47 samcv i had somebody bench it
07:47 samcv 2x faster
07:47 brrt \o/
07:49 samcv let me check the source for memmem on bsd. i know the glibc would do memchr for short needles
07:49 samcv https://github.com/st3fan/osx-10.9/blob/master/Libc-997.1.1/string/FreeBSD/memmem.c looks like bsd does that too
07:49 samcv which is how i was benching it. so makes sense it'd be similar for that
07:50 samcv are bsd licenses compatible with artistic 2.0?
07:51 samcv bsd one is much more self contained than glibc one. so could be very easy for us to use for msvc
07:52 brrt i'm… not sure, and to be on the safe side, i'd not agree with blanket inclusion
07:54 samcv libuv is MIT
07:57 brrt we compile-and-link libuv as a strictly separate library, though
07:57 brrt anyway, it might well be ok
07:57 brrt i can't judge
07:58 samcv i'm sure i can find a gpl one or just code our own if it's an issue
07:59 samcv our UTF-8 decoder included in MVM source is MIT
08:02 brrt alright, alright :-)
08:02 brrt not a GPL one, we're on the artistic license
08:02 samcv well it's important to check though
08:02 samcv yeah artistic 2
08:14 brrt :-)
08:14 samcv brrt, looks like 25% faster under worstcase on mac
08:14 samcv but single char searches are 2x faster
08:14 brrt awesome
08:14 samcv so 25% to 50% better and 100%better (2x faster) in easiest case for the memmem
08:21 samcv brrt, i'm thinking of making MVM_mempos and making so many arguments
08:22 samcv well. adding on also uh. starting position
08:22 samcv or something
08:22 samcv so it becomes easy to use it from other functions
08:23 brrt hmm, you could do that, but it's not really necessary, the formulation base_ptr + start_position will do what you mean just fine
08:23 samcv you can just give it the pointers to the objects, say what position to start from in haystack (will be in units of block size)
08:23 samcv true
08:23 samcv but then you have to subtract to get the haystack length
08:23 brrt you could also call it mempos_at_boundary or something like that
08:23 samcv as well
08:24 brrt yes
08:24 samcv at_boundary sounds nice
08:25 brrt mempos(void*base, size_t base_elems, void *needle, size_t needle_elems, size_t elem_size)
08:25 brrt or something like that
08:25 brrt i mean, hack away :-)
08:47 domidumont joined #moarvm
09:03 samcv ok well i sort of have it. but i have a bug on one line
09:03 domidumont joined #moarvm
09:05 samcv return ((char *)mm_addr - (char *) haystack)/block_size; # works fine for MVMGrapheme32 but is the wrong number for MVMGrapheme8
09:06 samcv btw the mm_addr i checked and it's the same on the new function vs how the old one was. so the only difference is in this subtraction/division line
09:06 samcv before i had:
09:06 samcv (MVMGrapheme8*)mm_return_8 -  haystack->body.storage.blob_8;
09:06 samcv (MVMGrapheme32*)mm_return_32 - haystack->body.storage.blob_32;
09:07 samcv for each of the two types. and that works perfectly. but the division line is not working argh
09:07 brrt hmm, that's a bit odd
09:08 samcv yeah
09:10 samcv i pushed my work if you can take a quick look. https://github.com/MoarVM/MoarVM/pull/574/files i'm gonna go to bed soonish
09:11 timotimo samcv: we could have a quick fail if the needle can't fit in 8bit but the haystack already is; though i don't think it'd be very common
09:11 samcv how can the needle not fit in 8bit
09:12 samcv you mean 0 wide?
09:12 samcv memmem already handles that
09:14 timotimo no, i mean, if the haystack is known to contain only ascii, and the needle has some esoteric unicode characters in it, we can automatically fail, no?
09:14 samcv oh
09:14 samcv but how do we know the needle has that. i guess if it's a small needle?
09:14 samcv so we'd have to see what's in the needle
09:15 samcv .o(so we can pretend perl6 has super fast regex by testing searching unicode needles of an ascii string)
09:15 samcv we can be faster than perl 5!
09:15 samcv it will be great
09:15 timotimo i find it worrysome that you claim "most" regex get faster from that patch
09:16 samcv why?
09:16 timotimo we only use indexat when we have a scan followed by a literal
09:16 samcv well i have tested only literals
09:16 timotimo i.e. when you start your regex with a character class, or when it's ignoremark or ignorecase ...
09:17 samcv i mean most literal regex
09:17 samcv that have the same content type
09:17 samcv sorry i really meant... most indexing
09:17 samcv more than 50%
09:17 timotimo "anytime it is regex between an 8bit string and 8bit string or 32bit string and 32bit string."
09:17 timotimo this tripped me up a bit
09:18 timotimo did you try tux' benchmark with your commits?
09:18 samcv nope
09:19 samcv was going to but i forgot how
09:20 timotimo i'll do it now
09:20 samcv kk
09:20 samcv just don't check out the most recent commit
09:20 samcv the one before most recent on that branch
09:22 timotimo i have the one before the "nonworking" one
09:22 samcv perfect
09:24 timotimo something's not right here
09:25 timotimo all i can see it do in strace is read chunks from the hello.csv
09:26 timotimo but it's really slow
09:26 samcv ok
09:26 timotimo how does tux run it in like 5 seconds?
09:26 samcv it is really slow
09:26 timotimo it's already been 3 minutes
09:26 samcv idk...
09:26 samcv i made the file smaller
09:26 timotimo oh, perhaps the /tmp on tux' machine is a tmpfs
09:26 samcv mine is tmpfs. is yours?
09:27 timotimo oh, it is
09:29 samcv but it is slow. it's not just you
09:29 samcv unless everybody does it wrong. idk
09:29 samcv i assumed the README.speed just didn't have the number of iterations it writes the csv file as he actually does
09:29 timotimo head -30 /tmp/hello.csv | perl6 $t.pl >/dev/null 2>&1
09:29 timotimo that's what test.sh does
09:30 timotimo except the output tux always gets is - i think - 5000?
09:30 samcv idk
09:30 samcv i didn't run the .sh
09:31 samcv or did i...
09:31 timotimo with 1000 instead of 30, i get only 1.18 seconds in total
09:31 timotimo ok now i'll drop your branch and see how it changes
09:32 samcv ok
09:32 brrt joined #moarvm
09:33 timotimo not seeing a difference tbh
09:34 timotimo i'll check in gdb if it reaches your code
09:34 lizmat fwiw, Text::CSV uses as little regexen as possible
09:35 timotimo if it uses nqp::index or similar it ought to get a speedup, too
09:35 lizmat one notable case is Str.split(<a b c>) rather tagn Str.split( /a|b|c/ )
09:35 lizmat *than
09:36 samcv ah
09:36 samcv yeah maybe it doesn't hit that code
09:36 lizmat or rather: Str.split($a,$b,$c) rather than Str.split( /$a|$b|$c/ )
09:36 lizmat I would be surprised  :-)
09:36 timotimo it'd be neat if we could do memmem but with more than one needle
09:36 samcv well at least other code will be faster :)
09:37 timotimo i've theorized about that in the past, but i never got anything written, not even a POC
09:37 samcv yes
09:37 samcv that would be great
09:37 lizmat implementing fixed string multi-needle split was one of the things that made Text::CSV almost 2x faster at one point
09:38 lizmat and yes, that uses nqp::index
09:38 samcv idk can the kruth morris-pratt be adapted for alternates
09:39 samcv timotimo, memmem with strandsssssssss
09:39 samcv that would be neat
09:40 timotimo hm, but when we apply a regex to a string, we use indexingoptimized on that string first
09:40 samcv yeah
09:40 samcv to collapse it right
09:40 timotimo so no strands when we hit your code
09:40 timotimo yeah
09:40 samcv yeah i know that
09:51 brrt1 joined #moarvm
10:37 samcv going to get to bed now. goodnight
10:44 timotimo goodnight!
12:05 AlexDaniel joined #moarvm
14:21 brrt joined #moarvm
14:38 brrt joined #moarvm
15:08 Geth ¦ MoarVM/even-moar-jit: 1d16e823c7 | (Bart Wiegmans)++ | 2 files
15:08 Geth ¦ MoarVM/even-moar-jit: Add DISCARD node
15:08 Geth ¦ MoarVM/even-moar-jit:
15:08 Geth ¦ MoarVM/even-moar-jit: In order to make e.g. DO nodes whose child nodes yield values tileable
15:08 Geth ¦ MoarVM/even-moar-jit: (as in the new LET-to-DO translation) we add a discard node to enforce
15:08 Geth ¦ MoarVM/even-moar-jit: void context on those nodes.
15:08 Geth ¦ MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/1d16e823c7
15:08 Geth ¦ MoarVM/even-moar-jit: 01aa8f4546 | (Bart Wiegmans)++ | 2 files
15:08 Geth ¦ MoarVM/even-moar-jit: Graph log: parameters in node name
15:08 Geth ¦ MoarVM/even-moar-jit:
15:08 Geth ¦ MoarVM/even-moar-jit: This is a more compact display than the previous one. Now all node
15:08 Geth ¦ MoarVM/even-moar-jit: parameters are logged in the node name, rather than as separate nodes,
15:08 Geth ¦ MoarVM/even-moar-jit: which emphasizes the (active) node structure better, I think.
15:08 Geth ¦ MoarVM/even-moar-jit: review: https://github.com/MoarVM/MoarVM/commit/01aa8f4546
15:09 brrt i'm thinking there is probably some bug in the compilation of DO nodes
15:10 brrt because the let->do conversion code is actually using it and well, stuff ain't going so well
16:26 cog_ joined #moarvm
16:31 domidumont joined #moarvm
17:08 AlexDaniel joined #moarvm
17:50 synopsebot6 joined #moarvm
18:58 spebern joined #moarvm
19:14 Geth ¦ MoarVM: 9653dc6aa1 | (Jonathan Worthington)++ | src/6model/reprs/MVMHash.c
19:14 Geth ¦ MoarVM: Implement serialize/deserialize in VMHash REPR.
19:14 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/9653dc6aa1
19:28 Voldenet joined #moarvm
19:28 Voldenet joined #moarvm
19:29 timotimo i'm surprised we never needed that until now?!?
19:37 samcv got a build failure on windows https://gist.github.com/samcv/0b2abb9da0fa6b6613c965ca27d32975#file-moar-L198
19:37 samcv somehow it found gcc which i didn't want to use
19:38 samcv not sure what installed it
19:38 samcv git? strawberry perl?
19:38 samcv ah looks like strawberry perl did
19:39 lizmat timotimo: well, I know some places where we can put BEGIN blocks in the setting now  :-)
19:40 lizmat hmmm... won't the JVM need something like that as well then?
19:40 lizmat jnthn: ^^^
19:41 samcv maybe i didn't actually get the c compiler?
19:52 geekosaur looks like strawberry perl is built to use its own gcc install
19:53 samcv yeah
19:54 samcv trying again to get the right msvc thing installed. think should work this time since i went the full thing, not just some compilier which turned out to only have visual basic and C#
19:57 nwc10 jnthn: ASAN barfage. First in a month or so: http://paste.scsys.co.uk/557802
20:19 jnthn lizmat: Yeah, it probably will. Though at least it's fixed in one place now.
20:19 jnthn samcv: I always used ActivePerl when doing MSVC builds, fwiw
20:20 jnthn nwc10: ooh, that's an intersting one
20:20 jnthn Too tired to fix it tonight
20:20 jnthn But it points very well at the problem
20:20 jnthn What were you running?
20:25 samcv ok using the developer prompt now. things are in path now
20:25 samcv ok now it seems to be ok
20:25 samcv gonna nmake now
20:29 nwc10 jnthn: make m-spectest6
20:29 nwc10 and I did think/hope that it pointed straight at a problem, as the two stack traces converged on adjacent lines
20:42 timotimo we're not allowed to enter gc from spesh, right?
20:42 timotimo that's why that crashed?
20:43 jnthn Right
20:44 jnthn I thought I'd caught all those spots but apparently I missed one
20:44 timotimo how should we deal with this for a proper fix?
20:45 jnthn I'm too tired to know
20:45 jnthn It involves inlining, so it's not liable to be simple
20:45 jnthn Though it may be that we can catch that this would happen in the fixup
20:45 jnthn And refuse to inline
20:46 jnthn That's probably the conservative solution
20:46 timotimo ah, sure
20:46 timotimo another way would be to just abort the whole spesh process and try again later
20:46 jnthn That's not likely to change the fact that the wval in question wasn't deserialized yet
20:46 jnthn It's probably code paths like
20:47 timotimo because then we're probably going to have something else having kicked off gc for us
20:47 jnthn die X::Blah.new unless $condition-that's-usually-tree;
20:47 jnthn *true
20:47 jnthn So we never end up accessing the wval for X::Blah and it ain't deserialized
20:47 timotimo right
20:47 timotimo is the proper fix to never allocate when inside spesh, or just never actually gc?
20:48 jnthn Well, actually the problem is that we acquire a mutex on the SC
20:48 jnthn And that acquisition can enter the GC
20:48 jnthn Because it's a block point and thus a safe point
20:48 jnthn An alternative is to teach spesh to be GC safe
20:50 jnthn But that's probably something for spesh2 :)
20:51 timotimo mhh
20:55 samcv cool. ok. got it compiling. now got undefined external symbol memmem cool
20:56 timotimo cool cool
20:58 timotimo i can probably hack in the "refuse this if we're in spesh" thing there
20:58 Geth joined #moarvm
21:03 timotimo we don't really have an "abort" mechanism in place yet, so i'll have to put a bunch of return values everywhere
21:06 timotimo ho-hum. at the point where we "fix_wval", we've alreday done a whole lot of stuff, not sure if it can still properly abort here or if i should add a bit to the first pass
21:10 samcv jnthn, it's cool with you if we use the memmem from libc aka freebsd memmem
21:12 samcv nice got it to compile with memmem on windows
21:14 timotimo i can put the check in much earlier when we're actually checking whether an inline will be possible
21:14 timotimo so there's already mechanisms in place to refuse to inline
21:16 jnthn timotimo: yeah, there's a validation step
21:16 jnthn samcv: Yeah, I only had chance to quickly glance what you're doing, but it sounds sensible. :)
21:17 jnthn So long as we don't bust the build on any of the common platforms :)
21:17 samcv ok cool
21:18 samcv yeah moarvm is working fine on windows atm :) with libc's memmem
21:18 samcv compiling nqp and printing all my debug messages. so it's def working... hehe
21:18 samcv spew of text
21:18 samcv as long as it compiles i guess
21:18 samcv then will run the tests
21:19 timotimo nwc10: will you be able to test a patch when i push it?
21:28 timotimo oh jnthn
21:28 timotimo can you set up my appveyor so it gets kicked off when moarvm gets a commit or a pull request?
21:29 samcv that would be nice
21:29 timotimo i'm not sure what you'll need
21:30 jnthn timotimo: Ask me tomorrow, if you would :)
21:30 timotimo OK
21:32 samcv cool nqp spectest pass on msvc.
21:33 timotimo jnthn: i put a little fprintf into the code where we jump to not_inlinable when a wval is dangerous, and holy hell does it come up often
21:34 timotimo hm
21:34 timotimo my code might be wrong, it doesn't ever hit the else branch here
21:34 timotimo let's see ...
21:36 timotimo yes, mea culpa
21:37 timotimo ok, cool. it finds a whole lot of harmless wvals and also a few dangerous ones
21:41 Geth ¦ MoarVM/refuse_dangerous_inlines: 2aa5c36df3 | (Timo Paulssen)++ | 3 files
21:41 Geth ¦ MoarVM/refuse_dangerous_inlines: refuse inlines when it could cause GC inside spesh
21:41 Geth ¦ MoarVM/refuse_dangerous_inlines:
21:41 Geth ¦ MoarVM/refuse_dangerous_inlines: nwc10++ was able to trigger this.
21:41 Geth ¦ MoarVM/refuse_dangerous_inlines: review: https://github.com/MoarVM/MoarVM/commit/2aa5c36df3
21:45 timotimo it's kinda sad that an inline that was refused because of this won't ever be re-instated again later on
21:45 timotimo i've already wished for being able to spesh frames multiple times in the past
21:46 timotimo maybe that's along the same lines
22:33 samcv ok it's now much prettier
22:34 samcv https://github.com/MoarVM/MoarVM/pull/574/files#diff-09b6d1e275ae06e53fd2eb3e85f2b959R1 added a MVM_memmem function in platform so we can handle the platform specific stuff there
22:35 samcv gonna test the build on windows now, should be good still hopefully
22:48 timotimo want me to walk you through setting up appveyor with a moarvm repo?
22:51 samcv maybe. i guess.
22:51 samcv but yeah i tested it on windows and it all works now on both windows and linux
22:51 samcv after some experimenting
22:56 timotimo my neck is tensing up and the last time i got that i felt like throwing up for more than one whole day :<
22:58 timotimo tomorrow is going to be fun, in the Dwarf Fortress sense of the word :|

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary