Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-01-28

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:14 pyrimidine joined #moarvm
01:27 samcv well what was the switch checking
01:27 samcv every single value of the numbers?
01:27 samcv like 0-9?
01:28 MasterDuke samcv: https://gist.github.com/MasterDuke17/e14fe534c3d348c975da200a98abdbb9 this is the code i used to create the switch
01:39 samcv MasterDuke, yeah you need to group it by digits
01:40 MasterDuke by digits?
01:40 samcv uhm
01:40 samcv m: say "0".ord.base(2); say "9".ord.base(2)
01:40 camelia rakudo-moar cfae23: OUTPUT«110000␤111001␤»
01:42 samcv maybe i need to think about this a little more
01:43 samcv uhm but if you look by hex
01:43 samcv m: say '0'.base(16).say; '9'.base(16).say
01:43 camelia rakudo-moar cfae23: OUTPUT«No such method 'base' for invocant of type 'Str'␤  in block <unit> at <tmp> line 1␤␤»
01:43 samcv m: say '0'.ord.base(16).say; '9'.ord.base(16).say
01:43 camelia rakudo-moar cfae23: OUTPUT«30␤True␤39␤»
01:44 samcv so you want to switch ignoring the last hex digit
01:44 samcv so you can combine all of each set of 0-9 into one 'case'
01:46 MasterDuke more like the pre-existing if statements you mean?
01:47 samcv yeah i guess if is more optimized than i thought i guess
01:47 samcv but we can just generate all the if statements maybe
01:48 MasterDuke for the Unicode Nd digits?
01:48 samcv yeAH
01:49 MasterDuke well, that might be nice, but i'm way more interested in making the non-Unicode case faster
01:49 samcv yes but
01:49 MasterDuke that's going to be the 99.99% use case
01:49 samcv if we consolidate them together don't we have to do less checks?
01:50 samcv and not process each point twice?
01:50 MasterDuke twice?
01:51 samcv uhm let me look at the function again, sorry
01:51 samcv also that switch would be very slow because they're not in order
01:51 samcv the numbers are totally out of order so it would be slow
01:52 MasterDuke well, the ascii digits are first
01:53 samcv yeah but it needs a really weird jump table
01:55 samcv also that table doesn't have everything in it, it's missing a lot
01:55 samcv you are only adding Nd general category
01:55 MasterDuke i guess i could extract the ascii digits out into their own switch first
01:56 MasterDuke well, isn't that the same as the existing code?
01:56 MasterDuke (the Nd general category thing)
01:56 samcv uh
01:56 samcv idk
01:56 samcv what function is it in MVM again?
02:00 MasterDuke https://github.com/MoarVM/MoarVM/blob/master/src/core/coerce.c#L352
02:02 samcv it does look like it checks Nd
02:02 samcv wtf why is it checking that :(
02:02 MasterDuke what would it check instead?
02:02 samcv and it looks up the property code...
02:02 samcv yeah this is gonna be slow
02:03 samcv Numeric_Type > 0
02:03 MasterDuke ah, but we don't want to do Nl or No
02:04 samcv you want things with numteric values right?
02:04 MasterDuke no, just digits
02:04 samcv define digits
02:04 samcv like <:Digit> ?
02:04 MasterDuke because unlike the highlander, there can be more than one
02:05 samcv why general category Nd though
02:05 samcv i mean sure it's the category for numeric digits
02:06 samcv you just want only things from 0-9?
02:06 samcv in value?
02:06 MasterDuke so the string "12۳4" should give the value 1234
02:06 samcv of course
02:06 samcv idk what you mean by digit. do you want all unicode cp's that have a value?
02:07 samcv only ones from value 0-9?
02:07 MasterDuke but "1Ⅷ" is invalid
02:07 timotimo then say "decimal" :)
02:07 yoleaux2 27 Jan 2017 13:54Z <samcv> timotimo: so I have added some indexes to names.c for every other codepoint
02:07 MasterDuke Ⅷ by itself is valid
02:07 samcv yeah. i had no clue wtf you meant lol
02:07 MasterDuke timotimo++
02:07 timotimo samcv: you wanted me to try to make lookup from codepoint to name work with your changes?
02:08 pyrimidine joined #moarvm
02:08 samcv so you want Numeric_Type=Decimal then
02:08 samcv that is what you want
02:08 MasterDuke :shrug:
02:08 MasterDuke all i know is .uniprop eq 'Nd' is what i want
02:09 samcv you sure
02:09 samcv i really don't think
02:09 MasterDuke yeah, that's what TimToady has said
02:09 samcv that is what you want. though it may have full overlap
02:09 samcv looking up the property value is gonne be faster than getting the property code for general category
02:09 samcv and THEN string comparing that it is 'Nd'
02:09 samcv gonna be slow
02:09 samcv and it'll have to do that every loop
02:10 MasterDuke true, but only if there is in fact a Nd char in the string
02:11 Geth joined #moarvm
02:11 samcv why though. you want decimal digits right?
02:11 MasterDuke and i'm way less concerned about speed in that case. sure it should be fast, but the usual ascii case is of most concern
02:11 samcv but it checks them together?
02:11 samcv exactly. it has to check that every time and it'll be slow. that's what i'm trying to say
02:11 samcv and so by fixing that it will be faster going through that ifelse loop
02:11 MasterDuke ah, yes, the switch does
02:12 samcv you mean the if else if else?
02:13 MasterDuke the if else will short circuit on ascii chars and not do the slow lookup
02:13 samcv yes
02:13 samcv i guess so
02:13 timotimo samcv: i didn't have time today and time will be sparse tomorrow as well .. but feel free to leave a few more comments with some extra detail in the backlog
02:14 samcv kk
02:14 samcv i did leave comments in backlog?
02:14 samcv np tho
02:14 timotimo a little bit
02:14 timotimo but i don't exactly know what i'm supposed to try to make work :)
02:15 samcv ok the consume a string function needs to be able to skip a specified number of strings
02:15 samcv and it also needs to be able to center itself and start from the first 0
02:16 samcv because i may give it some position that has the first base40 number not be a 0
02:16 samcv and it needs to skip to the 0, and then skip a specified number of 0's further, if i tell it to skip 1
02:16 samcv before returning. if that makes sense. i have a table that holds values of every other cp's location
02:16 * timotimo disappears already
02:16 samcv as we had talked about
02:17 samcv so right now it just consumes a string and assumes it will be started at the beginning, but i need it to go to the beginning of the next name
02:18 samcv so if it sees 600, 255, 0 < three 'base40' numbers i just made up
02:18 samcv it has to skip to the 0, then grab a string
02:18 samcv or however many further down i request
02:21 MasterDuke i just tried extracting the ascii values out into their own switch before the larger one. it was a little faster than the combined one, but still slower than the existing code
02:21 samcv what are you using to bench? can i see
02:23 MasterDuke perf stat -B ./nqp-m -e 'my int $i := 0; my int $s := 0; while ++$i < 1_000_000 { $s := $s + nqp::radix2(10, "1234567890123456789", 0, 0)[0] }'
02:24 MasterDuke i added a radix2 nqp op that calls a radix2 moar function to make it easier to compare
02:27 MasterDuke one thing that shoud help with speed is getting nqp::radix speshed and JITted (neither of which i know how to do)
02:27 ggoebel joined #moarvm
02:28 MasterDuke i just have all my changed locally, are you interested enough for me to push them to my moarvm fork on github?
02:28 samcv well i sped it up 50%
02:28 samcv by implementing that thing i said
02:28 samcv for numbers that are non-ascii
02:30 MasterDuke cool
02:30 samcv :)
02:31 samcv and can make it even faster when we don't need to call atoi
02:32 samcv and just have the values be integer like i have in my rewrite for UCD
02:34 MasterDuke could it use MVM_unicode_codepoint_get_property_int() instead of MVM_unicode_codepoint_get_property_cstr()?
02:42 samcv well if the property were integer it could
02:42 samcv but atm it's just an array of strings
02:46 samcv but yeah that's one of the many things trying to fix in the rewrite
02:46 MasterDuke hm, yeah, just removing the atoi and using _int() instead of _cstr() gives 36 for ۳
02:48 ilbot3 joined #moarvm
02:48 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:12 MasterDuke ah, just subtracting 33 makes _int() work, and is faster
03:46 samcv lol subtracting 33
03:47 samcv except they're not in order :P
03:47 samcv goes NaN, -1, 0, 1, 3, 2, 5, 7, 4, 11, 9
03:48 samcv err wait the numeric_value might be in order. numeric_value_numerator def isn't
03:48 MasterDuke heh, yeah, just checked. the first couple i checked worked, but it's definitely not correct
03:48 samcv also there's 1.5 in between 1 and 2
03:48 samcv yep
03:49 samcv XD
03:51 samcv MasterDuke, we also use ascii strings for storing the decomposition values
03:51 samcv m: 'ā'.NFD.say
03:51 camelia rakudo-moar c6e37e: OUTPUT«NFD:0x<0061 0304>␤»
03:51 MasterDuke can they be reordered to work? or is that something the unicode spec defines
03:52 samcv idk there's so many things that need fixing tho, i already have it working in my rewrite though
03:52 samcv numeric value's being integer values that is
03:52 samcv and not strings
03:52 MasterDuke ok, i won't play around with that part anymore
03:52 samcv not sure how to make the ascii faster tbh tho
03:54 MasterDuke TimToady mentioned python has some very fast code for them that he thought might be worth stealing
03:54 MasterDuke i know almost nothing about python or where to look for that code, but maybe you do
03:54 samcv don't know much about python either
03:55 samcv you mean for Nd's? or just for atoi in general
03:57 MasterDuke https://irclog.perlgeek.de/perl6-dev/2017-01-23#i_13970619
04:41 samcv hmm looks like we support fullwidth letters even though unicode doesn't list them as AHex digits
05:24 MasterDuke well, we support more than hex characters (when radix is > 16)
05:24 MasterDuke sleep &
05:25 samcv how does that relate to fullwidth letters?
05:26 samcv night
05:58 pyrimidine joined #moarvm
07:12 domidumont joined #moarvm
07:19 domidumont joined #moarvm
09:19 pyrimidine joined #moarvm
09:48 Ven joined #moarvm
10:06 Ven joined #moarvm
11:36 pyrimidine joined #moarvm
11:50 Ven joined #moarvm
12:29 MasterDuke samcv: i mean when the radix is >16 we allow letters higher than the hex digits (e.g., radix == 36, a-z are allowed letters/digits/characters, not just the a-f of hex)
12:35 domidumont joined #moarvm
12:38 pyrimidine joined #moarvm
12:55 Ven joined #moarvm
13:17 MasterDuke joined #moarvm
13:45 domidumont joined #moarvm
14:35 pyrimidine joined #moarvm
14:37 Ven joined #moarvm
14:42 pyrimidi_ joined #moarvm
14:46 Ven joined #moarvm
15:18 pyrimidine joined #moarvm
16:37 pyrimidine joined #moarvm
17:23 pyrimidine joined #moarvm
17:38 Geth joined #moarvm
17:39 pyrimidine joined #moarvm
17:55 pyrimidine joined #moarvm
18:29 pyrimidine joined #moarvm
18:34 Ven joined #moarvm
18:40 MasterDuke how do you make an nqp:: op speshable/speshed/(this word has lost all meaning to me)?
18:44 jnthn That's a property of MoarVM ops rather than nqp:: ops.
18:44 jnthn And beyond that, it depends what you want to do with it, exactly
18:45 jnthn src/spesh/ is where stuff happens, src/spesh/graph.h is a good point to start reading
18:45 MasterDuke well, MVM_radix is just interpreted, how would we get it speshed
18:46 jnthn Which op is it?
18:47 jnthn I'm trying to figure out what you're wanting to do :)
18:47 MasterDuke just radix i believe
18:47 jnthn It may be that we don't JIT it
18:48 jnthn Though it's a fairly large C function so the best we're going to machine, I suspect, is to JIT it into a C function call. That'd still prevent us having to interpret frames with it in.
18:48 MasterDuke i thought things had to be speshable before they could be JITted
18:48 jnthn src/jit/graph.c is the place to look for that
18:48 jnthn Not really.
18:48 jnthn spesh = specialization, it mostly means producing devirtualized versions of code
18:49 jnthn There are various ops where there's no interesting things to do in terms of specialization, but we can still have calls to their implementation JITted into machine code
18:49 MasterDuke same with the coerce_* ops
18:50 jnthn case MVM_OP_radix_I: return MVM_bigint_radix;
18:50 jnthn That's already in src/jit/graph.c
18:50 jnthn As are
18:50 jnthn case MVM_OP_coerce_ni:
18:50 jnthn case MVM_OP_coerce_in:
18:50 jnthn And a bunch more
18:50 MasterDuke but not regular radix
18:51 jnthn Ah, indeed
18:51 jnthn That just is a function call to MVM_radix
18:52 jnthn So should be quite easy to add the required bits to src/jit/graph.c
18:52 jnthn Shouldn't need edits beyond to that one file
18:52 jnthn And should look like radix_I does
18:52 jnthn Though the exact number of arguments may differ
18:52 MasterDuke cool. i'll give that a try
19:04 timotimo if the argument to radix is statically (i.e. at spesh time) known to be a constant number, we could dispatch to an optimized version that handles that exact radix, i.e. base 10
19:04 MasterDuke hm, a profile of nqp::radix_I shows 100% interpreted frames and no speshed or JITted
19:05 timotimo frames not getting speshed can have a whole host of reasons
19:05 timotimo among other things not being run often enough, but that's unlikely if you're building a benchmark that calls it often
19:05 MasterDuke this was my test: my int $i := 0; my $s := 0; my $t := nqp::knowhow().new_type(:name("TestBigInt"), :repr("P6bigint")); while ++$i < 1_000_000 { $s := $s + nqp::radix_I(10, "12345678", 0, 0, $t)[0] }
19:06 timotimo jitting, on the other hand, can be prevented by a single op not being implemented in the jit
19:07 timotimo you can figure out if a frame gets prevented like that by outputting a jitlog (i.e. MVM_JIT_LOG=foo.txt) and searching for "BAIL:"
19:08 MasterDuke there were a couple: `BAIL: op <throwpayloadlex>`
19:15 MasterDuke copying the radix_I case in src/jit/graph.c and adopting it for radix compiles. my benchmark still runs, but nothing seems to have changed
19:15 timotimo i might have tried to build that in the past
19:15 timotimo how many cases did you copy?
19:15 timotimo you need at least three, i think
19:16 MasterDuke ah, right
19:17 MasterDuke fixed the other place, still the same
19:17 MasterDuke i gotta afk for a while, but i'll play around some more later
19:18 timotimo well, if the frame with radix in it already doesn't get speshed, it won't have a chance to get jitted in the first place
19:22 pyrimidine joined #moarvm
19:47 pyrimidine joined #moarvm
21:02 MasterDuke timotimo: it looks like pretty simple/straightforward code to me (my test code that is). any reason it wouldn't get speshed?
21:04 timotimo *shrug*, i usually don't know myself when i have my own code
21:07 domidumont joined #moarvm
21:20 MasterDuke jnthn: got any ideas about why i see radix_I (and my newly added to src/jit/graph.c radix) just as interpreted, not speshed or JITted in the test code i pasted above?
21:39 MasterDuke joined #moarvm
21:44 pyrimidine joined #moarvm
22:21 jnthn MasterDuke: I'd expect OSR to be happening there. Not clear why it isn't.
22:22 MasterDuke a profile definitely says no OSR
22:25 MasterDuke i also ran the same code with my system nqp (so not using my modified moarvm at all), same results
22:26 jnthn *nod*
22:26 jnthn Yeah, I get that here too
22:48 agentzh joined #moarvm
22:48 agentzh hi guys
22:49 jnthn o/
22:49 agentzh i've run into a performance issue in the rakudo compilation time where the --profile-compile report shows that get() at SETTING::src/core/IO/Handle.pm:128 takes most of the Exclusive Time (88%)
22:49 jnthn o.O
22:49 agentzh anyone willing to do me a favor to optimize that thing away? :)
22:49 jnthn What's the call count?
22:49 agentzh jnthn: is it the Entries column in the routines report?
22:49 jnthn Yeah
22:49 agentzh 557
22:50 jnthn What on earth is it reading...
22:50 agentzh it's a pretty large perl 6 project
22:50 jnthn Hmm
22:50 jnthn I wonder if it's pre-comp database files
22:50 agentzh 6K LOC excluding empty lines and comment lines.
22:50 MasterDuke FIWI, i just perf recorded reading a 1m line file
22:50 MasterDuke 43.92%  moar     libmoar.so          [.] MVM_string_utf8_decodestream
22:50 MasterDuke 23.08%  moar     libmoar.so          [.] find_separator.isra.6
22:50 agentzh or 8K LOC if including empty lines and comment lines.
22:51 agentzh MasterDuke: is that an artificial p6 file?
22:51 agentzh mine is a real thing.
22:51 agentzh ported from a working perl 5 project.
22:52 jnthn Oh, one other thing
22:52 jnthn That's blocking I/O
22:52 MasterDuke it wasn't source, i was just doing an nqp::readlinefh through it
22:52 jnthn And it might be reading from a pipe
22:52 jnthn I think precomp spawns subprocesses
22:53 jnthn And reads from stdout to get info about the results of that process
22:53 agentzh mine is 46 .pm6 files for 46 compilation units on the file system.
22:53 jnthn So it's possible that what you're actually seeing there is we spend 88% of time waiting on a spawned process to do its thing
22:53 agentzh the report is generated from a partial compilation.
22:54 agentzh otherwise the resulting profiling report is too big to render in my web browser.
22:54 agentzh the whole things takes 30+ sec to compile on my side.
22:54 jnthn Yes, but does that trigger compilation?
22:54 jnthn uh, to be more clear
22:54 agentzh while the report i'm referring to is just a 6 ~ 13 sec partial compilation.
22:54 jnthn Does it trigger compilation of modules?
22:54 agentzh already slow enough for me to cry :)
22:55 jnthn :(
22:55 agentzh jnthn: it triggers a few dependent modules to recompile.
22:55 agentzh even though those modules do not change at all.
22:55 agentzh nor is the API of the edited file.
22:55 MasterDuke agentzh: you're on linux?
22:55 agentzh MasterDuke: yes.
22:56 MasterDuke can you run `perf record -g --call-graph dwarf <whatever command you run to compile here>`?
22:56 jnthn Could also look at the RAKUDO_MODULES_DEBUG (iirc) output to see what exactly it's doing
22:56 agentzh assuming that the profiling report is correct, the IO::Handle::get() thing looks like a very low hanging fruit?
22:56 agentzh unfortunately i don't have the skillset to optimize that thing myself. so i'm asking for help here :)
22:56 agentzh it's really a showstopper for me.
22:56 agentzh imagine a 6 ~ 13 sec delay everytime i make a single edit.
22:56 agentzh feeling like hacking on a machine sitting on the moon.
22:57 agentzh *grin*
22:57 agentzh MasterDuke: trying
22:57 MasterDuke and then when that's done, `perf report --call-graph=none --no-children`
22:58 zakharyas joined #moarvm
22:58 agentzh [ perf record: Woken up 1949 times to write data ] [ perf record: Captured and wrote 487.745 MB perf.data (60689 samples) ]
22:59 agentzh running the 2nd cmd.
22:59 agentzh https://gist.github.com/agentzh/c760f72509ddaee63fdc20380ffae504
23:00 MasterDuke huh
23:00 agentzh need more?
23:01 jnthn agentzh: It triggers recompilation of modules even if nothing in them or their dependencies changed?
23:01 jnthn If so that sounds decidedly odd
23:02 agentzh jnthn: it triggers recompilation of modules depending directly and indirectly on the edited module.
23:02 agentzh using today's rakduo git nom.
23:02 jnthn That's right.
23:02 agentzh it would be great if it can check if the API changes in the dependency module.
23:02 jnthn hehehe
23:02 agentzh since i seldom or never change the module API.
23:03 jnthn Oh, if only Perl 6 was such a simple language that was reasonable to actually do :)
23:03 agentzh solely compiling a single CU is fast enough to bear.
23:03 agentzh but i have to run the tests.
23:03 agentzh it's all about test driven dev :)
23:03 jnthn *nod*
23:03 jnthn I think this really boils down to "the compiler needs to be faster"
23:03 jnthn Which I totally agree with
23:03 agentzh the 7sec ~ 13sec delay is driving me nuts :)
23:04 jnthn I managed to know 20% off CORE.setting, which is a huge compilation unit, a couple of weeks back. TimToady++ is working on improving parse time.
23:04 agentzh good to see TimToady is still hacking on rakudo himself :)
23:05 agentzh MasterDuke: anything useful found in my report?
23:05 jnthn I think the .get thing is a false lead, though. I expect it's blocking in .get waiting to read the results of the precompilation process that it spawned, and all the work is in the subprocess.
23:05 jnthn So much as I'd love an easy win...I fear there ain't one here. :(
23:05 agentzh the profiling is measuring wallclocks instead of CPU ticks?
23:05 jnthn Yes
23:06 agentzh jnthn: i see.
23:06 TimToady I wonder if we could treat 'need' as less of a dep than 'use'...
23:06 agentzh TimToady: i was actually wondering if need can help :)
23:06 TimToady it does restrict you to the OO interface, to the first approximation
23:07 agentzh TimToady: that would be sweet :)
23:07 TimToady but I doubt the precomp treats it as less likely to influence the downstream parse
23:08 * agentzh is hoping p6 comes with something like the C header files.
23:08 MasterDuke agentzh: i don't think so. there doesn't appear to be file reading moarvm functions there, so i'll defer to jnthn's explanation
23:09 agentzh MasterDuke: got it.
23:09 agentzh merging all the CUs into a single p6 file helps a bit.
23:09 jnthn If it only helps a bit, then that pretty much incriminates compilation time...
23:10 TimToady that's essentially what we do with compiling the setting, where we deal with a 60-second turnaround or so
23:10 TimToady so we have plenty of motivation to get the compiler faster our own selves :)
23:10 jnthn Yeah, that's the one I knocked the 20% off recently... :)
23:11 jnthn Now we just need to find another dozen ways to do that ;)
23:11 TimToady well, most of the ways I know are likelier to knock 5% or 10% off, but still
23:11 jnthn *nod*
23:11 TimToady it's about time for me to test how our dynvar overhead is again, too
23:11 jnthn It'd be kinda nice if we could set of parallel precompilation of dependencies
23:12 jnthn Trouble is, a `use` can switch out the language
23:12 TimToady we keep adding them, and the dynvar cache was already overstressed
23:12 jnthn Meaning it can cause the `use` after it to compile differently
23:12 TimToady we could say that 'need' promised not to depend on anything OO in the subsequent parse
23:12 jnthn Guess that doesn't prevent us from speculating though
23:13 jnthn After all, MoarVM speculates happily all over the place and just deopts if it guessed wrong :)
23:13 jnthn .oO( "just" deopts... )
23:15 TimToady we could also try to get a better handle on which modules are actively trying to modify the parse of their users
23:15 jnthn There's also that
23:15 TimToady agentzh: are you defining any of your own operators?  That's known to slow parsing way down at the moment.
23:16 agentzh with all the CUs merged into a single .p6 file, it now always takes 7.2 sec.
23:16 agentzh i'm amazed to see it's as fast as a typical partial compilation.
23:16 jnthn It's kinda silly in 2016 to be doing stuff that prevents us from parallelizing compilation though :)
23:16 agentzh when always compiling everything in the project.
23:16 jnthn Uh, in 2017 too
23:17 agentzh TimToady: nope. the only thing i'm redefining is .Str(), which does not count as my own operators, right?
23:17 jnthn Nope
23:18 agentzh jnthn: yeah, my box has 8 logical CPU cores :)
23:18 agentzh jnthn: i also tried to do parallel recompilation myself via a makefile.
23:18 agentzh jnthn: but oddly enough it does not make things faster.
23:19 agentzh jnthn: the module later compiled still recompile its dependencies even though those deps are just manually recompiled right before.
23:19 TimToady that seems a bit like a bug
23:19 agentzh TimToady: i tried both perl6 /path/to/file.pm6 and perl6 -c /path/to/file.pm6 in my makefile commands.
23:20 agentzh the whole make -j8 time is still 30+ sec.
23:20 agentzh like before without -j8
23:20 TimToady something to ask nine about over in #perl6-dev when he's awake
23:20 agentzh seems like rakudo does not test timestamps correctly.
23:21 brokenchicken IIRC we don't use timestamps because they're not precise enough.
23:21 jnthn afaik, it doesn't precomp entrypoints
23:21 jnthn Only dependencies
23:22 jnthn So you might have had more luck with perl6 -e 'use Module::To::Compile'
23:23 agentzh oh, good to know
23:23 agentzh will try.
23:31 agentzh okay, i tried again, even without perl6 -e 'use xxx', make -j8 seems to be working now.
23:31 agentzh maybe i messed thing up the last time i tried make -j8.
23:31 agentzh now it takes 6.8s to compile everything.
23:31 MasterDuke jnthn: nqp-m -e 'my $f := nqp::open("small_compile.sql", "r");my str $l; while $l := nqp::readlinefh($f) { }', where small_compile.sql is 1m lines, also has no speshed or JITted frames
23:31 agentzh oh, wait, i need to wipe off .precomp...
23:32 agentzh forgot that one.
23:32 jnthn MasterDuke: What about if you do the same code with perl6-m ?
23:33 agentzh okay, with .precomp wiped out, make -j8 takes 31.9 sec to compile everything.
23:33 agentzh the same as a single compilation of the root from scratch.
23:33 MasterDuke ah, that's 33% speshed
23:34 MasterDuke but was 60ms slower
23:35 agentzh brokenchicken: sorry to hear that.
23:35 MasterDuke 1 OSR
23:36 agentzh hmm, seems like i have to give up the rakudo route for now due to the compilation time issue.
23:36 agentzh tried everything already.
23:37 agentzh will jump directly to my perl 6 dialect compiler targeting luajit which will soon support linking and partial compilation.
23:38 agentzh my plan was to use rakudo as the intermediate reference impl.
23:40 TimToady jnthn: it appears that dynvar lookup is still around 5% compiling the setting
23:40 MasterDuke agentzh: ah right, you work at cloudflare don't you? i've read some really good blog posts from them
23:40 agentzh MasterDuke: i used to. i quit CF 3 months ago.
23:40 agentzh MasterDuke: now i'm setting up my own company, OpenResty Inc.
23:41 MasterDuke TimToady: if you're interested, here's a --profile-compile of a recent rakudo build, sorted by exclusive time. https://gist.github.com/MasterDuke17/7723025658833476d5f312805c3274b5
23:41 MasterDuke agentzh: cool
23:41 jnthn TimToady: $*ACTIONS would be a nice one to be rid of :)
23:42 MasterDuke TimToady: and here's a perf record. https://gist.github.com/MasterDuke17/aff44a22a3a22d3ce0bf476d1d0ba537
23:42 agentzh MasterDuke: maybe generating a flame graph is more intuitive to find places to optimize?
23:42 agentzh at least some times.
23:43 MasterDuke TimToady: and here's a list of the names and counts of what find_symbol is called with. https://gist.github.com/MasterDuke17/a3b0018f427dc15c36d7f4c4b884946b
23:43 agentzh and an inverted flame graph can give you the same and also more info than the exclusive sort.
23:43 agentzh *exclusive time
23:44 MasterDuke TimToady: and here are the names passed to MATCH (and their counts). https://gist.github.com/MasterDuke17/ad31d008085e7f10891ac243e02f67bb
23:45 MasterDuke agentzh: i'm not a viz/ui person at all. and i think trying to do that in a web browser would still choke on the amount of data a profile of the rakudo build creates
23:46 MasterDuke but i agree that a flame graph would be good somehow
23:48 TimToady jnthn: $*WANTEDOUTERBLOCK actually has a bit more time, for fewer calls; a better cache and maybe some interning will help there, but yeah, ACTIONS, LANG, and PRAGMAS shouldn't really be using dynvars in the first place
23:51 pyrimidine joined #moarvm
23:52 TimToady MasterDuke: your files crash my ff tab
23:53 TimToady I did see your text version earlier
23:55 MasterDuke TimToady: whoops. and, text version?
23:56 TimToady well, I saw something that showed find_symbol was hot
23:57 MasterDuke ah, this? https://irclog.perlgeek.de/moarvm/2017-01-28#i_14005549
23:58 jnthn find_symbol I can probably figure out something to help with
23:58 * TimToady notes that it's probably not a coincidence that find_symbol calls $*WANTEDOUTERBLOCK
23:59 jnthn Oh.
23:59 jnthn o.O
23:59 TimToady which is really only used during the interspersed "second" pass
23:59 TimToady so there are a lot of not-there's the rest of the time
23:59 MasterDuke a little while ago i tried adding a find_symbol2 that didn't take an array, since the vast majority of the time it's just a single element in the array

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary