Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-09-09

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 geekosaur joined #moarvm
00:12 zakharyas joined #moarvm
01:01 ugexe why is it not necessary to MVMROOT async_type here? https://github.com/MoarVM/MoarVM/blob/0441e075b4f89f029492e156a54b02c8b3af8114/src/io/filewatchers.c#L109
01:55 ilbot3 joined #moarvm
01:55 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:33 dogbert2 joined #moarvm
04:29 geekosaur joined #moarvm
04:41 geekosaur joined #moarvm
04:45 travis-ci joined #moarvm
04:45 travis-ci MoarVM build errored. Jonathan Worthington 'Merge pull request #679 from ugexe/patch-1
04:45 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/273397039 https://github.com/MoarVM/MoarVM/compare/e86428d45e9b...32322f393365
04:45 travis-ci left #moarvm
07:05 JimmyZ ugexe: when: MVMOject *a; a function may trigger GC, use a. and then you need MVMROOT(a) arround 'a function may trigger GC'
07:06 JimmyZ ugexe: in that exmaple, async_type is not used after 'a function may trigger GC'
07:07 JimmyZ ugexe: 'a' must be NULL or MVMObject
07:09 JimmyZ s/GC, use/GC; use/
07:19 robertle joined #moarvm
07:31 domidumont joined #moarvm
07:38 domidumont joined #moarvm
08:08 BooK joined #moarvm
09:10 robertle joined #moarvm
10:39 Skarsnik joined #moarvm
10:51 Skarsnik_ joined #moarvm
11:11 dogbert2 joined #moarvm
11:17 leont joined #moarvm
11:34 dogbert2 Good afternoon #moarvm
11:34 dogbert2 I might have asked this before but if I have then I have forgotten the answer
11:36 dogbert2 there's an --optimize switch in Perl 6, what kind of optimization does it actually do? I guess it has nothing to do with either Spesh or the JIT.
11:39 jnthn Some lexical -> local lowering, some scope flattening, various peephole-y rewrites (like if $a == 1 | 2 { } into something cheaper using a temporary and ||), and inlining of basic native operators
11:40 Skarsnik hm this get called to free the memory? https://github.com/MoarVM/MoarVM/blob/master/src/6model/reprs/CArray.c#L121
11:41 jnthn Yes
11:41 dogbert2 jnthn: thx, so it deals with stuff which does not need any runtime analysis
11:41 jnthn Of note, it's called by the GC when the object has become unreachable
11:42 jnthn gc_free is for top-level objects, gc_cleanup is when it was embedded into another object
11:42 jnthn dogbert2: Correct. And we try not to put too much there, because for quick scripts without any hot loops in them it's mostly wasted effort.
11:43 jnthn See slides from my SPW talk on deopt for a little more about that
11:43 BooK jnthn: regarding https://github.com/MoarVM/MoarVM/issues/683 I'm shying away from working on a patch because of the probable signature change needed
11:44 Skarsnik I am curious because valgrind repport a leak on the expand function in this, this seems true since it always allocate stuff for the child_objs in the body but they will not get free if body->managed is false
11:44 travis-ci joined #moarvm
11:44 travis-ci MoarVM build errored. Jonathan Worthington 'Merge pull request #671 from duke-m/patch-1
11:44 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/273397310 https://github.com/MoarVM/MoarVM/compare/32322f393365...1e735a899891
11:44 travis-ci left #moarvm
11:46 dogbert2 jnthn: many thanks
11:46 timotimo oh, i have been meaning to look into whether the junction optimization is currently working
11:47 Skarsnik I am not even sure expanding a C managed array is a good idea in the first place x)
11:47 jnthn BooK: Signature change to a MoarVM op?
11:48 timotimo aha, it's broken. as i feared.
11:48 jnthn BooK: One option is to introduce a new op (take one of the oldest DEPRECATED_n)
11:48 timotimo dogbert2: you can look in rakudo's repo under Perl6/Optimizer.nqp (and there's also a regex optimizer)
11:49 jnthn BooK: And map the NQP one to the new one, and then we can deprecate/remove the old one later
11:49 jnthn Ah, I forgot the regex optimizer :)
11:54 timotimo the regex optimizer ought to do a whole lot more work, as we can't do much there in a dynamic optimizer
11:57 Skarsnik hm what gc_mark does? register memory to the gc?
11:57 timotimo it registers pointers
11:57 timotimo because the GC has to update pointers when the target objects move
11:58 timotimo but the pointers also keep other objects alive
11:59 timotimo ah, ok. == handles Mu, so i'm not allowed to do the static optimization of unfolding the junction for 'em
12:01 dogbert2 timotimo: do you know if --optimize=3 is the default?
12:02 jnthn No, 2 is
12:02 Skarsnik Ok, so there is memory lost in CArray. If I understand correctly : Since it does not know his own size, it will call expand (via at-pos) when going through it (when using a C for loop). This will allocate object for the perl6 repr of the content? when the Carray come from a C call it does not allocate the C memory for it. But when a gc_free is called on the array it only remove the perl6 obj if Moar has the hold of the Array (memory wise). So the perl6
12:02 Skarsnik obj created by expand are lost when it's managed by the lib
12:02 timotimo 3 is set explicitly for the core setting
12:03 dogbert2 so it (--optimize=3) shouldn't be used for 'normal' p6 scripts then
12:03 timotimo you can use it if you want to
12:04 jnthn Skarsnik: Sort of. The memory that points to the Perl 6 objects created by expand would be lost, I think.
12:04 dogbert2 I bet there's a downside somewhere :)
12:04 jnthn Skarsnik: The objects themselves would not be marked, however, and so could be collected
12:04 timotimo it looks like 2 and 3 are currently equivalent
12:04 Skarsnik and I am not sure how to fix that?
12:04 jnthn Skarsnik: Since our GC is about reachability, not ref-counts
12:05 jnthn Well, if it's just as you described it, would the fix not just be to unconditionally free child_objs?
12:05 jnthn Just move it outside of if managed
12:06 jnthn iirc it's just a piece of C memory that points to other MoarVM objects, and the GC will take care of those objects
12:06 Skarsnik The issue I am not sure what these obj are. like if they contains other stuff that came from the C lib?
12:07 jnthn Maybe, but the array you'd be freeing, that I agree we seem to be leaking, is *this* object's list of pointers to the other objects
12:07 jnthn We'd not be changing the lifetime of the other objects by freeing that array
12:07 dogbert2 speaking of optimization, is it still the case that array indexing is less optimized than it could be?
12:08 jnthn Since it's just a piece of malloc'd memory, not something that our GC tracks
12:08 jnthn dogbert2: Yes
12:08 timotimo yeah, i was able to get much better performance from .AT-POS than from .[]
12:08 timotimo even in the simplest case
12:08 jnthn Inlining should make the difference about go away, but until we get the derived type specializations and then spend some more time tweaking and tuning...
12:09 dogbert2 a lot more hard work then ..
12:10 timotimo jnthn: that's when we see that a part of the type tuple is static and another part is megamorphic?
12:11 Skarsnik jnthn, Thanks, I am not used to working with a GC ^^
12:12 jnthn timotimo: Yes
12:13 timotimo ugh. i'll have to be much more clever in finding candidates for multis that handle Mu
12:13 timotimo i'm not sure if i can properly reuse the "find me a multi that matches" code
12:13 Skarsnik jnthn, spesh seems to lost lot of memory according to http://www.nyo.fr/~skarsnik/tmp/valgrind.txt x)
12:14 jnthn Skarsnik: Hard to say
12:16 jnthn It is running in the background, and when the process is torn down we don't try and sync up with the spesh thread and get it to clean up
12:16 jnthn So worth a look but it could just be that that's the working memory it happens to have at the time of shutdown
12:17 jnthn Hm, the later parts of it do look a lot more suspect though
12:17 MasterDuke somewhat related, here's a heaptrack summary of a script that essentially just calls Proc::Async.new 5 times, capturing the output of 'git rev-list <options>': https://imgur.com/zD2CfJm
12:18 Skarsnik dang this heaptrack thingy look interesting
12:19 MasterDuke some more info is in a gist linked from https://github.com/MoarVM/MoarVM/issues/680
12:19 jnthn Yeah, looking more closely it seems spesh is losing something
12:19 MasterDuke (i ran the script with fewer iteration than in the gist for the heaptrack record)
12:20 MasterDuke oh, that heaptrack record may have had MVM_SPESH_DISABLE=1 set
12:23 MasterDuke here's without that: https://imgur.com/lhqm6HS
12:24 timotimo cool, my fix works
12:24 timotimo now to spec test
12:25 timotimo i bet this has been broken for months now :S
12:26 timotimo ever since all these ops got a Mu $?, Mu $? signature
12:26 Skarsnik the NQPMu stuff?
12:26 timotimo no
12:26 timotimo &infix:<==> has a proto sub &infix:<==>(Mu $?, Mu $?) is pure { * }
12:28 timotimo the junction optimizer opt can't statically do anything if the sub accepts Mu, because then it wouldn't autothread
12:28 timotimo i just changed the opt to only look at candidates if the sub is a proto sub
12:34 Ven`` joined #moarvm
12:37 domidumont joined #moarvm
12:52 Skarsnik if someone want to push these: http://www.nyo.fr/~skarsnik/tmp/various-nc-stuff.diff ~~
13:15 travis-ci joined #moarvm
13:15 travis-ci MoarVM build canceled. Jonathan Worthington 'Merge pull request #679 from ugexe/patch-1
13:15 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/273397039 https://github.com/MoarVM/MoarVM/compare/e86428d45e9b...32322f393365
13:15 travis-ci left #moarvm
13:27 Geth ¦ MoarVM: Skarsnik++ created pull request #684: Fix a leak in CArray repr.
13:27 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/684
13:27 leont joined #moarvm
13:27 Skarsnik this edit file directly on github is handy ^^
13:34 timotimo yup!
13:49 travis-ci joined #moarvm
13:49 travis-ci MoarVM build canceled. Jonathan Worthington 'Merge pull request #671 from duke-m/patch-1
13:49 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/273397310 https://github.com/MoarVM/MoarVM/compare/32322f393365...1e735a899891
13:49 travis-ci left #moarvm
13:55 AlexDaniel handy, by doesn't sign your commits (obviously)
13:55 AlexDaniel same with PR merges
13:55 AlexDaniel but*
13:58 timotimo i haven't done any commit signing :S
14:07 AlexDaniel timotimo: you set it up once and that's it :)
14:08 AlexDaniel if you sign your mails already then I guess you don't even have to create a gpg key…
14:26 timotimo i used to have a good gpg key with lots of signatures
15:02 dogbert2 if a program runs slower with the current release, compared to say 2017.07, is callgrind the tool to use in order to figure out why?
15:47 robertle joined #moarvm
15:55 zakharyas joined #moarvm
16:00 BooK so regarding https://github.com/MoarVM/MoarVM/issues/683 jnthn said I could introduce a new op using an oldest deprecated one, and I didn't get fully what that meant
16:00 BooK if someone is interested in that issue, and willing to take me by the hand implement my proposed change, I'd be delighted
16:02 BooK think of it as an investment ;-)
16:05 Skarsnik BooK, for this issue, resolve_blabbla is not exposed at nqp/rakudo level
16:08 Skarsnik it's only exist for connect and listen (and another one maybe) maybe looking at https://github.com/MoarVM/MoarVM/pull/659 can help you figure this stuff. What is probably missing is getting the sa-family attribute at Moar level
16:13 BooK Skarsnik: no, it's not, but the extra argument needs to be passed from Perl6
16:14 Skarsnik passing the io:s:inet obj could probably be more useful x)
16:15 BooK regarding that 659 change, I was surprised it was needed. after all, a network connection is defined/identified by these four parameters
16:16 BooK Skarsnik: can it be passed down as is to the vm layer?
16:16 Skarsnik other wise it became ridiculous nqp::connect($host, $port, $source-host, $source-port, $dest-sa, $source-sa)
16:16 BooK sa is sockaddr?'
16:17 Skarsnik sa-family I mean
16:17 BooK well, there's only one socket, so indeed it could just be the full object
16:17 Skarsnik or whatever the name at Rakudo level x)
16:17 BooK yeah, family is a socket attribute, so just one's needed
16:18 BooK but it's true that all these args are related to the socket, so maybe just pass them all at once
16:18 Skarsnik yeah I am dumb. sa-family must be the same on both side anyways
16:18 BooK otoh, a port of 0 is not the final version of it, it's just saying "I don't care, let the system pick it"
16:18 Skarsnik Yes
16:18 BooK so it's not the actual value for the socket once it's created
16:23 Skarsnik but passing the objet could be more useful, like to set some attribute, like port choosen, ip of the host or stuff like this
16:23 BooK is it what your getport does? getting the port chosen?
16:24 Skarsnik getport? i think it's another pr, not from me ^^
16:24 BooK oh sorry
16:24 Skarsnik but I think that what it does
16:24 BooK so it wouldn't be needed anymore
16:25 BooK the ops that seem to need to be modified to receive the socket obj are: socket, connect, bindsock
16:25 BooK not sure about accept
16:25 Skarsnik I think io::s:inet was rushed because I think the async version is prefered x)
16:26 BooK but at that level there sure must not be that different
16:26 BooK in the end, it's still the same syscall that creates the socket
16:26 Skarsnik No, async socket use libuv
16:26 Skarsnik very different code
16:28 BooK ah ok
16:30 BooK so I've tried to follow bindsock, and I think that led to bind_sk, and then to MVM_io_bind is that right?
16:30 BooK (from nqp to moarvm)
16:31 BooK and so all of those have to change is MVM_io_bind is to receive the IO::Socket::INET object from Perl6
16:33 BooK is GET_REG(cur_op, 0).o an object?
16:33 Skarsnik I am just not sure at all nqp accept fully build Perl 6 object. should ask in perl-dev maybe x)
16:34 Skarsnik Try to see if there is an op that take an object?
16:36 BooK in moar?
16:37 Skarsnik dunno, in nqp or Rakudo?
16:38 timotimo um, having an IO::Socket::INET object from perl6 seems like a very bad idea
16:38 timotimo we have nqp::socket for a reason
16:39 timotimo you should be using that object to set low-level attributes and such
16:39 BooK ok
16:39 travis-ci joined #moarvm
16:39 travis-ci MoarVM build canceled. Jonathan Worthington 'Merge pull request #631 from MasterDuke17/smarter_on_alloc
16:39 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/273399542 https://github.com/MoarVM/MoarVM/compare/1e735a899891...e70f8972b27c
16:39 travis-ci left #moarvm
16:40 BooK timotimo: I only ever looked at moar yesterday, so I'm pretty much don't know anything
16:40 BooK hence my call for some hand-holding
16:42 timotimo right
16:42 timotimo in the IO::Socket::INET code you can see the $PIO and $!PIO variables being passed to nqp ops
16:43 BooK yes
16:43 timotimo that's where the real communication with moarvm happens
16:43 BooK ok
16:44 BooK my $PIO := nqp::socket($.listening ?? 10 !! 0); # so nqp::socket returns a new nqp/moarvm socket object?
16:44 timotimo MVMOSHandle is one interesting thing to look at, in moarvm's 6model/reprs folder
16:44 timotimo that's right
16:45 timotimo look for MVM_io_socket_create in the source to see how it's created
16:45 timotimo in syncsocket.c
16:45 BooK and $!PIO is a private attribute that keeps that lowlevel socket object
16:45 BooK ?
16:45 timotimo yes
16:46 BooK so shouldn't all this family stuff we're talking about already be in the low-level socket?
16:47 timotimo you might want to give MVMIOSyncSocketData a field to hold the address family or something
16:47 timotimo well, that lowlevel socket object is still one layer above the regular socket api you might know
16:47 timotimo you'll see the MVMIOSyncSocketData has a member called "handle", that's the one the OS wants to have
16:48 timotimo though perhaps putting the address family into the struct is unnecessary
16:48 timotimo depends on what OS-level functions want to have the address family value
16:48 timotimo ok, gtg
16:50 BooK it's all in MVM_io_resolve_host_name I think
16:50 BooK the family is needed to give better hints to getaddrinfo
16:53 Skarsnik But this function only get the thread context and the hostname/port ^^
16:59 BooK and MVMIOSyncSocketData is built on the already existing socket, I think
17:02 BooK Skarsnik: also, changing MVM_io_resolve_host_name is probably not a big issue
17:02 Skarsnik yeah it's only used in Moar
17:02 BooK socket_connect gets a MVMOSHandle
17:03 Skarsnik that sucks
17:03 BooK does it?
17:04 Skarsnik well if it does not get the syncsockdatastruct or whatever is the nqp::socket objet you have to change the nqp instruction
17:05 BooK it's defined here https://github.com/MoarVM/MoarVM/blob/master/src/6model/reprs/MVMOSHandle.h#L13-L16
17:06 Skarsnik oh nvm it's MVMIOSyncSocketData *data = (MVMIOSyncSocketData *)h->body.data;
17:06 domidumont joined #moarvm
17:06 BooK right
17:06 BooK so is there anything in MVMOSHandleBody / MVMIOSyncSocketData we can use?
17:07 BooK gtg but I'll definitely be back :-)
17:08 Skarsnik ^^
17:19 ugexe JimmyZ: so you don't have to root things that are passed -to- the function, only things used after it?
17:21 JimmyZ ugexe: yes
17:23 ugexe JimmyZ: thanks for explaining
17:23 JimmyZ ugexe: the function will MVMROOT it inside if it has something gc trigger inside
17:25 JimmyZ ugexe: which will used after another gc trigger function
17:53 Skarsnik_ joined #moarvm
18:43 ggoebel joined #moarvm
20:38 Skarsnik I wonder if someone ran the whole spec test in valgrind x)
20:41 timotimo that'd take a week or so
20:42 timotimo not to mention you'd get some test failures from tests that expect threads to generally execute at the same time
20:42 Skarsnik I mean each separate test file
20:42 timotimo me, too
20:42 Skarsnik I kinda want to try now
20:42 Skarsnik I mean tomorrow
20:50 timotimo you should start it now so it can run while you're sleeping
20:52 Skarsnik there is already a process running with efence that take 100% cpu and 70% of the 4Gb of ram x)
20:53 timotimo OK
20:54 timotimo poor you with 4 gigs of ram
20:54 timotimo i know what that feels like now
20:54 timotimo (it's why i have 8 gigs of zram)
20:55 leont Yeah, I have 8 in my laptop, and sometimes still run out of it
20:56 leont 16 in my work laptop, so I can actually run VMs
21:03 timotimo i used to have 12, then i b0rked it
21:05 Skarsnik I have 8Gb on my desktop
21:05 Skarsnik I use my vps to run long thing
21:06 timotimo ah
21:10 Skarsnik the issue I have chrome that take like 4gb alone xD
21:10 timotimo yeah
21:10 timotimo i also suffer from that
21:10 timotimo and if i open thunderbird in addition, bam.
21:11 Skarsnik I often have to start a 3d game to have him release memory ^^
22:03 dogbert2 joined #moarvm
22:04 zakharyas joined #moarvm
22:17 zakharyas joined #moarvm
23:09 geekosaur joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary