Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-09-07

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:13 lizmat joined #moarvm
01:55 ilbot3 joined #moarvm
01:55 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:15 Geth ¦ MoarVM: ugexe++ created pull request #679: Mark GC blocked when acquiring mutex
03:15 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/pull/679
04:19 lizmat_ joined #moarvm
04:21 nwc10 joined #moarvm
04:21 samcv joined #moarvm
04:24 nativecallable6 joined #moarvm
04:24 greppable6 joined #moarvm
04:24 releasable6 joined #moarvm
04:24 bisectable6 joined #moarvm
04:41 ilmari[m] joined #moarvm
06:47 brrt joined #moarvm
06:47 leont joined #moarvm
06:48 brrt good hi #moarvm
06:51 leont Morning!
06:51 samcv morning!
06:51 samcv good hi brrt :-)
06:52 samcv okie dokie
06:52 brrt morning samcv, leont
06:53 samcv brrt, i was trying to set up the KMP string index so that it would search a partial needle length, so we have max pattern length
06:54 samcv and if it found that pattern, it would then check if the substring of what's left of the needle matched the rest of the haystack. but i was running into issues. maybe i had the numbers off. but it was very frustrating and i gave up after a few hours
06:54 samcv though 8192 is bigger than most needles probably are. and the jump table uses 16bit integers instead of using 32 bit before
06:54 brrt uhuh
06:55 samcv i doubled the jump table limit and doubled the max pattern length
06:55 samcv few days ao
06:55 samcv i'll probably revisit it eventually
06:55 brrt no rush :-)
06:55 samcv 8192 needle graphemes is all anyone will ever need! - Bill Gates
06:57 samcv i think if i revisit it, i may at least for testing purposes make a new function that calls the KMP index function instead of trying to build it into the loop. that should make it easier to debug what's going on until i have the numbers right
07:23 Geth ¦ MoarVM/master: 5 commits pushed by (Samantha McVey)++
07:23 Geth ¦ MoarVM/master: c2fc14dd7b | Optimize MVM_string_gi_move_to
07:23 Geth ¦ MoarVM/master: 23d613e31d | Fix bug in gi_move_to if not starting at 0
07:23 Geth ¦ MoarVM/master: b2770e2783 | Use grapheme iterator cached for ignorecase/ignoremark index ops
07:23 Geth ¦ MoarVM/master: a418dcb8ce | Only use grapheme iter cached if the Haystack is a strand for index*
07:23 Geth ¦ MoarVM/master: 65933cc6bc | Merge pull request #677 from samcv/ignorecase_ignoremark_cached
07:23 Geth ¦ MoarVM/master: review: https://github.com/MoarVM/MoarVM/compare/de6dceda81...65933cc6bc
07:24 samcv yay merges
07:50 geekosaur joined #moarvm
08:11 zakharyas joined #moarvm
08:24 robertle joined #moarvm
08:48 jnthn morning o/
08:54 timotimo morning jnthn
08:54 jnthn src/6model/reprs/MVMStaticFrameSpesh.c: In function ‘describe_refs’:
08:54 jnthn src/6model/reprs/MVMStaticFrameSpesh.c:124:5: warning: implicit declaration of function ‘MVM_spesh_arg_guard_gc_describe’ [-Wimplicit-function-declaration]
08:54 jnthn Seems new
08:55 timotimo oh, let me see
08:55 timotimo i wonder why i didn't see that
08:55 timotimo oh, it does output on my machine, so i was just negligient :|
08:57 Geth ¦ MoarVM: c78477cd0a | (Timo Paulssen)++ | src/spesh/arg_guard.h
08:57 Geth ¦ MoarVM: put missing forward declaration of arg_guard_gc_describe in
08:57 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/c78477cd0a
09:03 * timotimo is reading the old heaptracker posts again
09:07 timotimo i don't think i quite grokked how exactly to make the heap snapshot contain entries that don't actually have a proper object to go with them, i.e. not sure how to properly replicate the effects of process_object; is it the "set_type_index" call that makes a difference?
09:08 jnthn What does "not a proper object" mean?
09:08 jnthn There are different kinds of entries
09:08 jnthn Some of them are synthetic things
09:08 timotimo yeah, i want some of those :)
09:08 zakharyas joined #moarvm
09:09 timotimo the spesh arg guard doesn't have a repr for example
09:10 jnthn Ah, and you want something cleverer for it
09:11 jnthn I'd just add another synthetic
09:11 jnthn And add it to the enum in the viewer
09:11 timotimo OK, let's see about that
09:11 timotimo that's the big switch/case in process_workitem?
09:12 timotimo workitems*
09:12 jnthn Yeah
09:24 timotimo ah, set_ref_from is the magic thing that makes all of this work
09:44 timotimo oh, there's no api to put a COL_KIND-annotated thing into the heapsnapshot's work list? so there'd have to be an unbroken chain from the root-of-all-roots to every spesh arg guard if i want to use the special col kind?
09:46 timotimo except if get_workitem_idx learns how to spot a SpeshArgGuard
09:47 timotimo which it can't because no REPR or similar
09:48 timotimo i *can* have a special col kind for StaticFrameSpesh and put knowledge about its internals into heapsnapshot.c
09:53 timotimo i don't like this a lot :/
09:57 jnthn tbh I'd just put Spesh Arg Gurd as the key and not try to represent the structure of which arg guard has what in the snapshot
09:58 timotimo oh, so it'd just directly hang off of the MVMStaticFrameSpesh
09:58 timotimo i think that's the way it is right now
09:58 timotimo i still haven't actually looked at a heap snapshot enough to *find* such an entry to see if it even works :D
10:04 timotimo haha, i want that "ignore inter-generational roots" patch now :D
10:05 timotimo https://gist.github.com/timo/800d375d4eeb01e4b0c6db87c98af185 - not terrible
10:08 timotimo nine: sorry, i hid the patch to have incidents away a bit. it now lives in the main repo in a branch "incidents"
10:15 timotimo AFK
10:16 brrt are spesh guard tree thingies ever modified?
10:16 brrt i suppose they could be
10:17 jnthn A guard tree is immutable; updates copy/tweak/install
10:17 jnthn So that reads of it can be done lock free
10:18 brrt aha
10:19 brrt well, i'm asking because i know you mentioned JITting the guard tree at some point
10:19 brrt which i think should be doable
10:19 jnthn Guess we'd take the same approach
10:20 brrt and i was wondering what kind of lifecycle that'd have
10:20 jnthn Similar, I guess, though we might decide not to bother throwing away previous versions if they're just tacked on to the end of a candidate's JITting
10:21 brrt this reminds me of markmont's work. if we did have a open tmpflie handle that we would map, we could relatively easily map new pages out of that, and that might make the whole mmap/mprotect dance a little cheaper
10:21 brrt oh, that is possible too, of course
10:21 brrt we could add a section for it
10:22 brrt and a label at the start of that
10:22 brrt that would seem to be nearly optimal to me
10:45 ggoebel joined #moarvm
10:46 timotimo don't we have to change a little bit of the tree rather than just tacking new parts onto the end?
10:46 timotimo or did you mean to just concatenate it to the existing page?
11:04 brrt no, you can concatenate however you compile the *current* tree to the last thing you specialize
11:05 brrt so if i add a new specialization, i write a new tree, i compile that new tree after the JITcode for my last specialization, and then update the pointers to look at that
11:06 timotimo we can probably re-use parts from earlier that we can just jump to and get the right result from anyway
11:07 brrt either i'm not parsing you correctly, or it doesn't seem like a good idea
11:07 timotimo right, probably a premature optimization anyway
11:11 timotimo i can probably use atomics in the slowest part of the "calculate incident edges" code
11:16 timotimo Total objects:                555,773
11:16 timotimo Total references:             2,115,698
11:16 timotimo ^- takes 10 seconds to calculate the reverses
11:28 Skarsnik joined #moarvm
11:28 timotimo i do this in three steps: first i count up the number of incoming refs for every collectable, then i go through the list of collectables and give each a start and end point into a so-far empty references list, and finally i go through and put the from of every reference into the to's from-list
11:29 timotimo the last step is achieved using an array of "cursors", one for each collectable, that lets me increment after writing one entry
11:32 timotimo the count-up at the beginning takes 2.4s, the assignment one takes about 0.9, then 6.3s for the final assignment
11:33 timotimo i'd say the first part can also be done with two workers in parallel using atomic ops
11:34 Skarsnik timotimo, add stuff to query the profiler at run time or stuff like that x)
11:34 timotimo i assume that the access pattern for ++ing the "incomint edge count" will go sufficiently distinctly from the first and second half of the thingie
11:34 timotimo Skarsnik: the heap profiler you mean?
11:34 timotimo turning it on and off during a running program?
11:34 timotimo if you're willing to have the proram cooperate, you can do that with nqp ops :)
11:35 Skarsnik dunno, being able to take snapshot during an execution could be useful
11:38 timotimo you can literally nqp::mvmstartprofile(nqp::hash('kind', 'heap', 'path', '/tmp/myheapsnapshot')); nqp::mvmendprofile();
11:38 timotimo since it forces a GC before it finishes up
11:38 Skarsnik ohhh
11:44 dogbert17 cool error message of the day: "MoarVM panic: native callback ran on thread unknown to MoarVM"
11:45 Skarsnik hm
11:45 dogbert17 it's from t/04-nativecall/21-callback-other-thread.t
11:46 Skarsnik actuallt that a good question. what happen to callback given that the C code can run in his own thread
11:46 timotimo yeah moarvm can't just run on a thread that doesn't have stuff set up for it
11:47 timotimo maybe in the future we could ask for another thread to spawn a threadcontext and such for us and wait for that to happen
11:55 Skarsnik hm
11:56 Skarsnik not sure how that work well if you write/read from stuff owned by the C thread
11:56 zakharyas joined #moarvm
12:16 jdv79 left #moarvm
12:19 jnthn dogbert17: Hm, odd...wonder how the thread is unknown to it
12:30 dogbert17 jnthn: yes, difficult to reproduce as well, grr
12:32 timotimo so, i was wondering if i should apply for a tpf grant for some perl6-related work i'd like to do
12:33 timotimo including web UI for the heap snapshot profiler, a new frontend (maybe TUI, maybe web ui) for the regular profiler, ...
12:35 timotimo the next call for applications is probably coming up very soon, it says "the first week of september" which has now passed
12:35 timotimo will have passed tomorrow*
12:36 lizmat timotimo: please go for it!
12:37 jnthn ++timotimo
12:37 timotimo oops, i did not expect actual enthusiasm like this :)
12:37 [Coke] timotimo: as the person who wrote that, I meant... the first ... full week? plus it's still the 7th, the first actual week isn't done yet. :P
12:38 timotimo right
12:38 [Coke] (it's coming very soon. :P)
12:38 brrt timotimo++ go do it
12:39 dogbert17 timotimo++ go go go
12:39 nwc10 timotimo: in theory you have a timezone advantage over TPF :-)
12:39 [Coke] nwc10: *thbbthp*
12:39 geekosaur joined #moarvm
13:21 timotimo it'd probably be nice to have a single GUI that understands heap profiles, instrumented profiles, and a telemeh timeline, as well as make spesh and jit logs simpler to navigate
13:22 brrt instrumentation is good in general
13:24 timotimo by "instrumented profile" i just mean the first profiler we got
13:46 timotimo https://docs.google.com/document/d/102h_Nlpq4ED-PpJVfy7kggp5LYmpORhsB3E1eeZ3yfI/edit?usp=sharing  -  here i've started my grant application text; could someone look at it and tell me if the tasks outlined in the synopsis are a reasonably small and achievable chunk of work?
13:47 timotimo brrt: ? :)
13:49 brrt oh, sure, but i'm not known for having my work reasonably small and achievable ?
13:49 timotimo heh.
13:49 brrt so i'm not the judge of that
14:05 timotimo i realize now that the heap snapshot profiler and the regular profiler are mutually exclusive, as you'd get loads and loads of profiler-related objects in the heap profiler
14:05 timotimo and the profiler also keeps some objects alive for longer than they would be normally
14:12 jnthn Yeah, I don't think you could sanely use the two together really
14:12 jnthn Not as they currently stand
14:13 Geth ¦ MoarVM: e86428d45e | (Jonathan Worthington)++ | src/spesh/inline.c
14:13 Geth ¦ MoarVM: Don't try duplicate unreachable handlers on inline
14:13 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e86428d45e
14:13 Skarsnik timotimo, maybe you can add in the goals : documentation/tutorial about that
14:15 timotimo Skarsnik: if i can up the requested amount by another 1k for documentation ... :D :D
14:16 jnthn timotimo: Looking quite good so far
14:28 timotimo How dare they ask for details and a proposed schedule
14:28 timotimo don't they know how the real world works :P
14:30 dogbert17 hmm, t/spec/S10-packages/precompilation.t fails on MoarVM HEAD
14:30 dogbert17 not ok 51 - precompiled module constants get updated on change
14:34 timotimo i might go for angular again for this web app
14:36 timotimo angular 4 this time; though angular 5 is supposedly coming out this month or something
14:40 timotimo good look, npm. getting an infinite loop of EACCESS for a folder it wasn't able to create for some reason
14:42 dogbert17 ==1977== Conditional jump or move depends on uninitialised value(s)
14:42 dogbert17 ==1977==    at 0x41960CD: MVM_string_gi_move_to (iter.h:68)
14:42 dogbert17 ==1977==    by 0x419A47B: MVM_string_index (ops.c:433)
14:42 dogbert17 ==1977==    by 0x41A225C: MVM_string_split (ops.c:1569)
14:42 dogbert17 ==1977==    by 0x40EB2F5: MVM_interp_run (interp.c:1637)
14:51 dogbert17 I get the above when I run './perl6-valgrind-m t/spec/S32-str/words.t' among others
15:01 dogbert17 the error seems to have been introduced with https://github.com/MoarVM/MoarVM/commit/a418dcb8ceb0aaca14dee986eeebb36715699f5c
15:04 timotimo i wonder if angular is a good fit at all
15:05 * jnthn prefers react/redux fwiw
15:05 timotimo me, too. but the patent clause in their license is not cool
15:05 jnthn Oh?
15:05 * jnthn didn't hear about that before
15:06 timotimo now i don't claim i understand it
15:06 timotimo https://github.com/facebook/react/blob/master/PATENTS
15:10 ilmari it basically means nobdy using react can sue facebook over _any_ patent
15:11 jnthn Well, they can, it just means their license to use react is terminated as a consequence
15:11 jnthn So they'd have to stop doing so
15:12 brrt joined #moarvm
15:14 brrt timotimo, i like most of your proposal.
15:14 brrt i'm wondering what you mean by 'framework for embedding domain knowledge'
15:14 brrt also, i'm a tad bit worried about relying on the super adhoc nature of the JIT log
15:14 timotimo well, you know how i immediately spot BOOTSTRAP.nqp:1234 as "oh shit, we're doing multi resolution without the cache"?
15:15 brrt i did not know that
15:15 timotimo find_best_dispatchee is in there
15:15 brrt uhuh
15:15 timotimo there's also bind_one_param and ... i think just "bind", too
15:15 timotimo i think they both usually show up as "<anon>" or whatever; just the line number
15:16 timotimo for those the most important thing is to point the user at what exact piece of code causes many calls to these
15:16 timotimo but the user wouldn't know to even try, unless the tool somehow points them at it
15:16 jnthn Yeah, some "intelligence" to point out common pitfalls would be really neat where we can do it
15:18 timotimo i wonder how stable the line number of those functions are :D
15:20 brrt hehe
15:20 brrt well, i see what you mean now
15:20 timotimo i think i'd prefer react/redux, too
15:21 timotimo brrt: also "hey these functions have BAIs in them for param_*_* ops, we can point the user at the signature and tell them what makes the jit unhappy
15:21 timotimo "
15:21 timotimo except of course if the jit fully implements all these param ops before the tool is finished :)
15:22 brrt well, yeah, stuff has to be a bit robust
15:22 brrt or not
15:22 brrt i don't know
15:23 timotimo i'm not sure what you mean
15:24 brrt i mean, for instance, that even-moar-jit dumps graphviz of expression trees in the jit log
15:24 timotimo right
15:24 brrt i mean, jnthn has done a pretty nice job of making the spesh log a consistent format
15:24 timotimo i don't find them very understandable, btw :S
15:24 brrt i have not done the same job with the jit log
15:24 timotimo right
15:24 brrt the graphviz dumps?
15:25 timotimo yeah, even after rendering them
15:25 brrt can't say i blame you
15:25 timotimo fwiw, making the jit log more useful can surely be part of the grant
15:25 brrt i do, but then agian, i'm not exactly the target audience
15:25 brrt or i am
15:25 brrt but my use is not super representative of perl6 developers
15:25 brrt uhuh, but it does increase the scope tremendously
15:26 timotimo how, i just have to write a few printfs! :D :D
15:26 brrt :-)
15:27 timotimo fwiw, it'd be neat if i could include x86 disassembly for the really savvy users
15:27 timotimo maybe after the grant
15:28 brrt i'm wondering if there is a smarter way to do that than a directory dump
15:28 brrt maybe writing a tarball
15:32 nine What's wrong with a directory dump?
15:32 timotimo immediately write the output of objdump rather than the raw bytes?
15:33 timotimo then it could go into thel og
15:37 brrt a directory dump is messy
15:37 brrt and to identify the files, we need a separate file
15:38 brrt not necessary if you write a single tarball, and you can use a tmpfile if needs be
15:38 brrt so, for instance
15:39 brrt you could setup the jit dump file to be a tempfile right before forking and execing moar
15:39 brrt objdump and pipelining is less than awesome i think, but i could see ways to use it
15:39 timotimo set it to be a named pipe :P
15:39 brrt or that
15:47 timotimo the redux dev said in one of his talks that redux is also popular in the angular community
15:49 brrt what even is redux
15:50 timotimo here's a slide with 16 lines of code that are "basically all redux does"
15:50 timotimo do you know what react.js does?
15:52 nine no
15:52 brrt hacker news tells me it is evil facebook code
15:53 timotimo OK, so basically™ it lets you write render functions that turn a state (any object or object hierarchy) into some html stuff by plopping together components
15:53 timotimo then react takes your results, computes a diff for the html, and applies that
15:53 timotimo (or in the case of react native i guess there is no html involved or something?)
15:53 timotimo redux is used to handle the state stuff
15:53 jnthn Basically they are "let's do FP" instead of screwing around with a DOM :)
15:54 timotimo quite.
15:54 jnthn Which I found quite pleasant
15:54 timotimo redux asks you to express user interaction (and outside influences like results form get requests or websockets or what have you) as simple "action" objects, which are suppose to be simple javascript objects, i.e. what you could dump as json
15:55 timotimo you write a reducer function that takes a state and an action and transforms the state accordingly, and redux asks you to not modify the previous state
15:55 * jnthn heads home; bbl
15:55 timotimo and all of these things can be composed nicely
15:56 timotimo the first talk i saw also had a nice time-traveling debugger thingie where you can turn on and off actions from "the past" and it'll give you a proper result, and also hot load your code and re-apply all the actions
16:04 nine Reminds me of the JS framework I created for work a couple years ago for a very dynamic user interface. That is centered around a global data structure representing the app's state. The UI can register event handlers for this data structure.
16:05 nine State changes have to be expressed in terms of create/update/delete events on the data structure. The UI reacts to those and only to those. User events must only trigger data events (those create/update/inserts) and the UI must only react directly to data events.
16:06 nine Sounds restrictive at first but this lets us store all events in a log in the data base. Re-loading a page just replays the existing events. And, since it's all basic create/update/delete any language that can read JSON (i.e. all of them) can recreate the data structure from the event log.
16:06 nine This includes PostgreSQL itself. So we have database triggers that react to events and update the server side state.
16:09 nine All in about 500 LOC including UI transactions and offline support, so it will send the events to the server whenever there's a network connection.
16:09 timotimo cool
16:13 ugexe joined #moarvm
16:14 nine It was very much inspired by masak++'s talk at a twin city Perl workshop ages ago :)
16:15 timotimo is there a recording online?
16:18 nine I don't think so :/ Anyway, there's still this: http://masak.org/carl/ddd-cqrs-wiki/faq#command-query-responsibility-segregation
16:34 dogbert2 joined #moarvm
16:37 robertle joined #moarvm
16:46 leont joined #moarvm
17:02 nine Turning nativecallinvokejit into a real sounds easy at first. The real trickyness is finding out in which layer to add the support. The arg handling ops do not even exist in QAST. Call handling seems to be split between QASTOperationsMAST and mast/compiler.c
17:33 Skarsnik nine, you just jit the whole process to create the c call?
17:35 timotimo well, most of it
17:40 zakharyas joined #moarvm
18:01 timotimo redux-saga seems neat
18:07 Skarsnik sound like an anime title
18:26 jnthn nine: I'd probably figure out a way to tweak it into https://github.com/MoarVM/MoarVM/blob/master/lib/MAST/Nodes.nqp#L540
18:26 jnthn nine: Since that code path will handle the whole callsite business and spitting out the prepargs etc.
19:08 zakharyas joined #moarvm
19:30 timotimo with non-globally-unique CUIDs the inline debug output will sometimes not be very helpful :)
19:33 AlexDaniel joined #moarvm
19:35 coverable6 joined #moarvm
19:36 timotimo especially with so many routines having "" as their name :)
19:37 bloatable6 joined #moarvm
19:46 timotimo oh damn, these filenames are loooooong
19:49 timotimo ah, it's the path to the bytecode file, not the filename of the source
19:55 timotimo i wonder if it'd be nice to have the filename and line number from the most recent annotation as well
20:06 MasterDuke timotimo++ for grant work
22:19 geekosaur joined #moarvm
22:34 mtj_ joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary