Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-09-12

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:29 committable6 joined #moarvm
00:30 coverable6 joined #moarvm
00:30 bloatable6 joined #moarvm
00:30 releasable6 joined #moarvm
00:30 bisectable6 joined #moarvm
00:30 nativecallable6 joined #moarvm
01:06 nwc10 joined #moarvm
01:10 bloatable6 joined #moarvm
01:10 unicodable6 joined #moarvm
01:10 quotable6 joined #moarvm
01:10 evalable6 joined #moarvm
01:10 committable6 joined #moarvm
01:10 benchable6 joined #moarvm
01:10 bisectable6 joined #moarvm
01:10 releasable6 joined #moarvm
01:10 greppable6 joined #moarvm
01:10 nativecallable6 joined #moarvm
01:10 squashable6 joined #moarvm
01:10 statisfiable6 joined #moarvm
01:11 coverable6 joined #moarvm
01:56 ilbot3 joined #moarvm
01:56 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
02:47 MasterDuke i just now realized the value inlined into MVMP6bigintBody is only 32bits (i'm assuming because we have 64bits to work with, and that's the largest single type less than 64)
02:48 MasterDuke and then the other 32bits is used as a flag
02:49 MasterDuke would it be worth the trouble to break it down further, e.g., make it a 32bit + 16bit value and a 16bit flag?
02:50 MasterDuke so we get more possible values inlined, but the code the get/set them is more complex
03:25 solarbunny joined #moarvm
03:27 AlexDaniel joined #moarvm
05:39 domidumont joined #moarvm
05:46 domidumont joined #moarvm
06:06 brrt joined #moarvm
06:06 brrt good * #moarvm
06:16 brrt MasterDuke: i feel like, ironically, there is not going to be a lot of values that are between 2^32-1 and 2^48-1 that are actually going to be used
06:17 brrt so you'd a huge amount of complexity for relatively little benefit
06:37 domidumont joined #moarvm
07:12 brrt joined #moarvm
07:18 zakharyas joined #moarvm
07:50 samcv good * brrt
07:52 brrt good hi samcv
07:52 samcv :-)
07:53 HTTP_____GK1wmSU joined #moarvm
07:53 leont joined #moarvm
07:57 brrt i was writing up the documentation for the register allocator recently. did you know i implemented two versions  before the current one?
07:59 zakharyas joined #moarvm
08:09 samcv no i didn't
08:09 samcv did those help you write the current iteration?
08:20 zakharyas1 joined #moarvm
08:25 brrt i guess, ultimatley, yes
08:25 brrt i learned a bunch of things
08:25 brrt for one thing, the current viteration assigns registers to instructions at the same time that it allocates the register for the value
08:26 brrt that way, any instruction always has the allocated register for the values it uses; if we need to spill a register, we split the live range of that value into 'atomic' units
08:27 brrt for which we allocate a new register once they become live
08:27 brrt (atomic meaning, they can't be 'split' themselves, because they are started directly before the using instruction, so it never makes sense to spill them)
08:28 zakharyas joined #moarvm
08:43 samcv ah i see
08:43 samcv in general things get better as you redesign them :)
08:44 brrt yes :-)
08:44 samcv i know i had many different iterations of my collation algorithm. some things you only learn when implementing really. but as long as you make progress that is what counts. though it is frustrating things not working as you'd want at first
08:45 samcv "making sure the values goes in the right place" as it were. i think you said similar to something like that a few months ago
08:46 samcv the first collation iteration was simplistic and couldn't handle sequences. but was helpful as it let me lay the foundations for how we would configure sort order and how it'd relate to the keys. the 1? 2nd iteration of the full type implementation. wow it like *mostly* worked but the code scared me
08:47 samcv way too many branches that made it hard to follow, and i thought prone to programming errors as i could hardly tell which place it went except for having put a ridiculous amount of logging in
08:47 samcv never have i used so many printf's :) as it's a pretty complicated thing. it'd dump the stack at the end. i'd be like. wait. why isn't that right...
08:47 samcv and gdb was rarely helpful except in rare cases
08:49 samcv and then i rewrote it again, and that time it was much simpler, but my test failures shot wayyyy up. then went down again and it stayed much neater
08:49 samcv i mean it's still a lot of moving parts but i can describe what all the code does and why it's there :)
08:55 brrt hehe, that's pretty much how i feel about the register allocator as well
08:56 domidumont joined #moarvm
09:04 jnthn MasterDuke: The 32-bit limit isn't primarily about storage, but about calculation: as most operations on 2 32-bit values won't overflow a 64-bit integer, we can do them in at 64-bit width and then see if the result needs to be promoted to a bigint or not. There's no portable way to do overflow detection based on flags, which is what'd limit some other strategy here.
09:06 jnthn samcv++ brrt++ # not settling on the first design, but hunting down a better one
09:09 samcv jnthn, i wanted to have our prepend detection in the normalizer instruct the synthetic creation if it needed to do extra checks, but that would have involved tons of code everywhere... so we have one property check for each synthetic being created so that's not too bad
09:10 samcv may eventually have some MVM_MIN_PREPEND and MVM_MAX_PREPEND so we can do some bound checks before checking the property. or even since the number of them isn't that much i don't know it could be faster to check bounds and if they match check against the like 10 that exist with some if conditionals
09:12 samcv not totally sure. but i went the way of doing that for the uca checks, where it checks if something is in a certain block or not. though haven't yet optimized it at all to do a bounds check before checking more heavily. in any case it doesn't seem to slow it down much. wonder if compiler does optimizations like that? if none of the conditionals checks have side effects
09:13 zakharyas joined #moarvm
09:13 samcv well it only does that for things that don't return a value for the MVM collation values
09:14 jnthn C compilers do all kinds of smarts. :)
09:15 samcv oh i can also fix casechanges too
09:15 nine I notice how amazing they are every time I need to run a --debug moar ;)
09:15 samcv m: "\c[arabic number sign]a".uc.say
09:15 camelia rakudo-moar 760530: OUTPUT: «؀a␤»
09:16 Geth ¦ MoarVM: f8a639e205 | (Samantha McVey)++ | 3 files
09:16 Geth ¦ MoarVM: Fix ignoremark with Prepend characters and ordbaseat op
09:16 Geth ¦ MoarVM:
09:16 Geth ¦ MoarVM: Previously ordbaseat would return the first codepoint in the
09:16 Geth ¦ MoarVM: Synthetic instead of the correct base character. Now, when we
09:16 Geth ¦ MoarVM: add a Synthetic, we check if the first codepoint is Prepend, and
09:16 Geth ¦ MoarVM: if so we keep going until we find the last Prepend character.
09:16 Geth ¦ MoarVM:
09:16 Geth ¦ MoarVM: This builds on the recent change so that synthetics are a single
09:16 Geth ¦ MoarVM: array instead of storing the base codepoint separately (3bd371f1).
09:16 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/f8a639e205
09:16 Geth ¦ MoarVM: db3102c452 | (Samantha McVey)++ | src/strings/nfg.c
09:16 Geth ¦ MoarVM: For degenerate Synth's with Prepend and Extend set base cp to 1st cp
09:16 Geth ¦ MoarVM:
09:16 Geth ¦ MoarVM: If we have a degenerate that starts with Prepend and follows with Extend,
09:16 Geth ¦ MoarVM: this commit sets it so that its base character will be the first one,
09:16 Geth ¦ MoarVM: instead of the one following the last Prepend. I also set up a cache so we
09:16 Geth ¦ MoarVM: don't have to repeatedly get the property values repeatedly in case the user
09:16 Geth ¦ MoarVM: has the same codepoint repeating a large number of times.
09:16 Geth ¦ MoarVM: <…commit message has 9 more lines…>
09:16 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/db3102c452
10:03 samcv is it at all beneficial jnthn to declare something const if you don't need the value to stay the same between invocations
10:04 samcv i see that in some places of the code. i mean for variables inside of functions
10:14 jnthn It may in some cases let the optimizer do a better job, but to me `const` everywhere quickly becomes a lot of clutter.
10:14 jnthn Modern compilers are good at alias analysis, so can figure out a lot of "what might change", "what won't change", etc.
10:15 jnthn Especially on local variables
10:22 zakharyas joined #moarvm
11:08 timotimo joined #moarvm
11:08 MasterDuke i've updated https://gist.github.com/MasterDuke17/9bd82ae1823f271f7e83d00210817b32 with a diff of what my MVM_fixed_size_realloc(_at_safepoint)? looks like now. anybody have comments?
11:09 lizmat samcv: is it correct there are no roast tests yet for .collate ?
11:09 timotimo it's just missing support for the fsa debug mode
11:10 MasterDuke yeah, i haven't looked at what the other function do with that yet
11:10 samcv lizmat, well i just got it merged in (the full thing) but there will be shortly
11:10 samcv uhm but it's still under use experimental though. not sure what the policy is for that lizmat
11:10 lizmat samcv: is there a reason we have a method collate in List that is identical to the one in Any ?
11:11 MasterDuke timotimo: but it looks reasonable otherwise?
11:11 lizmat good question: we have tests for pack/unpack in roast, but I guess that's a historical artefact
11:12 Skarsnik joined #moarvm
11:12 samcv maybe we need an experimental folder? until it graduates? dunno
11:13 samcv i could put it in there. or in rakudo
11:13 samcv or just put it under the proper S or whatever
11:13 samcv the only thing possibly not stable might be $*COLLATION. otherwise it's ready for use
11:14 samcv i forget what TimToady mentioned we might want to change for that eventually
11:14 samcv he had some comments on that
11:14 lizmat I was thinking that .collate should allow named params, like :!secondary, :!tertiary, which would then set $*COLLATION in that scope
11:15 samcv i could hide accesing $*COLLATION elements with experimental?
11:15 samcv hm
11:15 samcv yeah that sounds like a good idea
11:16 samcv but yeah the reason tit was in both files is because i had planned on making the list one have more in it so it'd be faster than calling sort with a different sub
11:17 samcv we would get the value of $*COLLATION and save it in a variable as a native int and have some raw unicmp function that has extra arguments for options
11:17 samcv lizmat, how do you feel about me removing experimental from .collate?
11:18 lizmat I would be ok with it
11:19 samcv ok cool
11:19 samcv i'll keep $*COLLATION experimental though. but let the function work without exp
11:20 timotimo MasterDuke: i think it looks okay. worst case, you'll find out when you use it in MVMArray and valgrind or asan complain :)
11:21 lizmat m: use experimental :collation; say "Ə" unicmp "E"  # samcv: what would needed to have that say Same ?
11:21 camelia rakudo-moar 760530: OUTPUT: «More␤»
11:22 lizmat if ever ?
11:22 brrt jnhtn: i'm sometimes using const to declare that stuff can live in the rodata section of the process and be shared and stuff
11:22 brrt jnthn ^
11:23 brrt i read that this was important somewhere on the internets so i believed it
11:23 brrt but that's for staticly compiled data though
11:24 MasterDuke timotimo: heh, you mean getting some C code to compile doesn't guarantee it'll work perfectly? then why would anyone use such a language?!
11:24 samcv m: for <PRIMARY SECONDARY TERTIARY> { my $a = "Ə"; say $a, ' ', $a.uniprop("MVM_COLLATION_$_") }
11:24 camelia rakudo-moar 760530: OUTPUT: «Ə 7460␤Ə 33␤Ə 9␤»
11:25 samcv m: for <PRIMARY SECONDARY TERTIARY> { my $a = "E"; say $a, ' ', $a.uniprop("MVM_COLLATION_$_") }
11:25 camelia rakudo-moar 760530: OUTPUT: «E 7441␤E 33␤E 9␤»
11:25 samcv well you'd have to disable primary and disable quaternary
11:25 samcv quat breaks ties by cp
11:25 timotimo MasterDuke: feel free to rewrite moarvm completely in rust or something :)
11:26 samcv MasterDuke, hush! if it compiles it works!
11:28 Skarsnik hm do you think I can get a moarvm function addr in NC (rakudo side)?
11:30 jnthn brrt: Yes, my answer was about local variables
11:31 MasterDuke ha. i have a friend who's always going on about how much he loves Rust. i'm happy people are using it for critical stuff, i don't have any particular interest though
11:31 jnthn Skarsnik: Probably doing the usual trick of "load a symbol in this process" would work
11:31 jnthn I think just `is native` or `is native(Nil)` or some such
11:32 jnthn lunch &
11:33 Skarsnik Yeah but I want the addr of a function to pass it as a pointer (in a cstruct) to pass it to C code
11:33 Skarsnik like give Moar alloc and free function to a lib x)
11:34 timotimo might be able to use cglobal with int64 for that
11:34 Skarsnik Oh right cglobal give a symbol addr?
11:35 * jnthn really hopes code doing that won't make it into the module ecosystem
11:35 timotimo oh, addr
11:35 timotimo d'oh, it wouldn't do that
11:36 timotimo it'd give you the first 64 bits of the function instead
11:36 timotimo well, you can still nativecall into dlsym itself
11:36 Skarsnik m: use NativeCall; my $mv_free := cglobal('MVM_free', size_t, '')
11:36 camelia rakudo-moar 760530: ( no output )
11:36 Skarsnik m: use NativeCall; my $mv_free := cglobal('MVM_free', size_t, ''); say $mv_free;
11:36 camelia rakudo-moar 760530: OUTPUT: «Cannot unbox a type object (NativeCall::Types::size_t) to a str.␤  in block  at /home/camelia/rakudo-m-inst-1/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 517␤  in block <unit> at <tmp> line 1␤␤»
11:36 Skarsnik what
11:37 timotimo it returned a null for you
11:37 Skarsnik Oh yeah, I don't like these unbox error x)
11:38 Skarsnik it's MVM_Free ?
11:38 timotimo no, MVM_free
11:38 Skarsnik m: use NativeCall; my $mv_free := cglobal('', 'MVM_free', size_t); say $mv_free;
11:38 camelia rakudo-moar 760530: OUTPUT: «Must specify something as a path: did you mean '.' for the current directory?␤  in sub guess_library_name at /home/camelia/rakudo-m-inst-1/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 231␤  in block  at /home/camelia…»
11:38 Skarsnik m: use NativeCall; my $mv_free := cglobal(Any, 'MVM_free', size_t); say $mv_free;
11:38 camelia rakudo-moar 760530: OUTPUT: «Cannot locate symbol 'MVM_free' in native library ''␤  in block  at /home/camelia/rakudo-m-inst-1/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 517␤  in block <unit> at <tmp> line 1␤␤»
11:40 Skarsnik m: use NativeCall; my $errno := cglobal(Any, 'errno', int32); say $errno;
11:40 camelia rakudo-moar 760530: OUTPUT: «0␤»
11:41 Skarsnik hm it's weird it can't find a symbol of libmoar
11:46 timotimo joined #moarvm
12:01 MasterDuke brrt: there are definitely deminishing returns, but just `perl6 -e ''` makes 4 calls to MVM_bigint_to_str with an actually-big int (plus 46 more with an actually-small int). and the value returned by nqp::objectid would fit in 2^48-1
12:02 MasterDuke but jnthn++ pointed out it would be pretty difficult to implement
12:03 nine MasterDuke: 50 calls doesn't sound like all that much?
12:04 brrt hmmm
12:04 MasterDuke to do nothing?
12:04 brrt i think i see where you are going
12:05 MasterDuke i don't think it's the single most expensive function you can call in moarvm or anything like that, i was just hoping it was one of the easier ones to optimize
12:06 brrt we could, with aggressive memory management, make those objectids reliably fit in 48 bits
12:06 brrt stressing reliable :-)
12:06 MasterDuke i noticed a heaptrack report for a test script said MVM_bigint_to_str was one of the top causes of temporary allocations
12:06 nine Taking a step back, there's actually nothing _requiring_ an objectid to be a pointer, is there?
12:07 brrt nine: no
12:07 brrt we could have atomic sequence numbers
12:07 brrt or
12:07 brrt thread-local sequence numbers
12:07 nine Was just gonna suggest the latter :)
12:07 brrt with, say, the upper 16 bits for the thread that allocated it, and the lower 48 for the sequence number itself
12:08 brrt if someone ever runs out of one of those, well, i think we should say we are succesful
12:09 nine Or just give each thread a range of 24 bits or so and have it allocate a new block of numbers when it runs out. Only block allocation would have to lock something or use atomics.
12:10 nine But I guess we're a long way from nqp::objectid becoming a bottleneck :)
12:11 [Coke] (experimental) isn't the forthcoming 6.d a good point to make things not experimental?
12:12 [Coke] (samcv jnthn lizmat ^^)
12:12 [Coke] also mebbe zoffix
12:14 MasterDuke nine: nqp::objectid (and MVM_bigint_to_str) definitely aren't bottlenecks, but a lot of the times i ask about such things it's because they look like something simple enough i might be able to work on them
12:14 lizmat well, if 2017.10 is going to be 6.d, then just after the release of 2017.09 would be the right time, no ?
12:15 [Coke] ... it is?
12:15 [Coke] I didn't realize we were committed to that timing. If so, then sure, though, yes.
12:15 lizmat well, I dunno, I think we all agree we want 6.d rather sooner than later
12:16 lizmat and no, no formal commitment has been made...  I'm just putting a bit of pressure on it here  :-)
12:16 [Coke] I haven't seen any work on roast cleanup. is the plan to take 'whatever's in master' ?
12:17 [Coke] anyway, if it's tied to a spec bump, +1 from me.
12:28 jnthn Well, in good news for 6.d I do now have some resources to try and carry the non-blocking await a bit further forward
12:28 jnthn Together with making the scheduler smarter
12:29 [Coke] \o/
12:32 jnthn I'd really, really like to see .hyper and .race sorted out this side of 6.d also
12:34 timotimo <3
12:34 Skarsnik nine, maybe we can start a NC apiv2 or whatever it can be called x)
12:40 MasterDuke (.hyper, .race).race(*++)
12:53 MasterDuke is there any reason not to use the FSA for (nearly) everything in VMArray.c? is that what jnthn means here? https://irclog.perlgeek.de/moarvm/2017-09-11#i_15146218
12:54 timotimo when we already easily have the current allocation size on hand it's probably good to use the FSA
12:54 timotimo otherwise, don't
12:54 timotimo is my approximation
12:59 MasterDuke hm. definitely not quite fully understanding what he meant/is involved with using FSA. can i keep the existing functionality pretty much identical, just swapping in FSA functions, and then implement what he said?
13:00 jnthn MasterDuke: Approximately yes
13:00 jnthn But
13:00 jnthn You need a flag bit somewhere (perhaps in the object header)
13:00 jnthn That indicates whether or not the FSA is being used
13:00 jnthn Since sometimes we populate a buffer from memory we were given by, say, libuv
13:01 jnthn And we have to do a normal MVM_free of it in that case
13:01 jnthn I'd kinda like to switch all of those cases to MultiDimArray in the future
13:01 MasterDuke ah, was trying to figure out why that flag was needed
13:01 jnthn And have them fixed size and immutable
13:01 jnthn But that's a bigger step
13:03 MasterDuke so what would need to be changed to read that flag?
13:04 jnthn Everything that might allocate/free the buffer
13:05 MasterDuke done by implementing the check in VMArray's gc_free?
13:05 jnthn There it'd be needed
13:05 jnthn But also in the resize operation
13:06 MasterDuke and copy_to i assume?
13:06 jnthn Well, there we have a choice
13:06 jnthn We could always use the FSA in the destination array
13:06 jnthn And we don't touch the source array
13:06 jnthn Just read it
13:06 MasterDuke right
13:06 jnthn And the flag won't be copied automatically
13:07 jnthn So I don't know that needs to check/set it
13:08 MasterDuke cool. many thanks, everyone here is very helpful
13:08 jnthn From memory, unshift may also need a look
13:09 jnthn Maybe or maybe not splice, but I think splice isn't yet optimized enough to need it
14:20 stmuk joined #moarvm
15:26 brrt joined #moarvm
15:40 Skarsnik_ joined #moarvm
16:00 Geth ¦ MoarVM: 49b90b99db | (Samantha McVey)++ | src/strings/nfg.c
16:00 Geth ¦ MoarVM: Fix case change when base cp isn't the first cp in synthetic
16:00 Geth ¦ MoarVM:
16:00 Geth ¦ MoarVM: Now the base codepoint will change case even if there are Prepend
16:00 Geth ¦ MoarVM: codepoints in front of it. Now updated to use our updated Synthetic
16:00 Geth ¦ MoarVM: representation.
16:00 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/49b90b99db
16:00 dogbert17 joined #moarvm
16:13 buggable joined #moarvm
16:13 huggable joined #moarvm
16:13 Geth_ joined #moarvm
16:13 ZofBot joined #moarvm
16:14 SourceBaby joined #moarvm
17:21 leont joined #moarvm
18:40 domidumont joined #moarvm
18:51 domidumont joined #moarvm
19:04 dogbert17 joined #moarvm
19:38 Skarsnik_ joined #moarvm
20:19 domidumont joined #moarvm
21:21 zakharyas joined #moarvm
21:48 dogbert17 joined #moarvm
23:03 bisectable6 joined #moarvm
23:37 MasterDuke joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary