Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2015-11-02

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:08 colomon joined #moarvm
00:16 tokuhirom joined #moarvm
01:00 tokuhirom_h joined #moarvm
03:22 tokuhirom joined #moarvm
07:44 nwc10 good *, #moarvm
07:56 colomon joined #moarvm
08:09 kjs_ joined #moarvm
08:35 zakharyas joined #moarvm
09:28 timotimo nwc10: i only looked at the very newest commit in there, so i didn't notice all the other savings
09:28 timotimo nwc10: do you have a before-everything vs now comparison for that branch? of the core setting, i mean
09:28 timotimo retupmoca: that's a known jitbug ;(
09:29 timotimo it's about miscompiling statements that start with "return" and have named parameters somewhere in them
09:30 nwc10 about 11M down to 10.5M
09:30 timotimo and ... damn, gitlab is slow right now :o
09:30 timotimo oh, that's not bad
09:31 timotimo it seems like you've gone through the whole serialization systematically, so i assume no more easy savings are to be had?
09:31 nwc10 There's one thing I asked jnthn about privately
09:32 nwc10 and the frame section might have some win still to have
09:32 nwc10 and serialisation contexts are still a big proportion of what's written
09:32 nwc10 but, basically, nothing easy I can spot
09:32 timotimo "serialization contexts"?
09:33 nwc10 SC id, and SC index
09:33 timotimo ah
09:33 timotimo yeah, i'd like our SC ids to not contain a complete timestamp
09:33 timotimo or perhaps we could just encode the timestamp as hexadecimal instead of decimal and get a little win from that
09:33 timotimo cuuids, i mean
09:34 timotimo ... or why not refer to the SC ids by index into a SC id table or the string heap ...
09:37 timotimo i see here that making string references half as big if they fit into the packed loc area makes the core setting 0.5 MB smaller already; does that mean all the other commits make almost no difference to the core setting?
10:04 nwc10 in which case, my numbers might be out - I thought I was comparing completely before with completely after
10:04 timotimo perhaps
10:04 nwc10 I can't think *too* hard on this - we have a current hard problem to solve (or bodge round) at work
10:04 timotimo understood
10:05 nwc10 I *thought* that it >100K from the stable changes
10:05 timotimo did you find any indication of the lazy deserialization logic not being used currently? or was everything all right before your work started?
10:16 nwc10 I can't answer either question usefully.
10:16 nwc10 I know that the lazy logic is being used at times
10:16 timotimo right, fair enough :)
10:16 nwc10 and that if I mess things up, it's used less than before
10:17 nwc10 I can't tell you if it's used everywhere it should be
10:17 timotimo we don't have any tool or metric to figure out if things are all right, then?
10:17 nwc10 none that I'm aware of
10:19 timotimo maybe i'll think about that at some point; not in the right headspace at the moment
10:35 jnthn nwc10: ooh, that "don't store things with \r in as Latin-1, but UTF-8" is a lovely solution :)
10:41 nwc10 jnthn: good. it's sort of a hack, but probably a "hack" in the good sense
10:43 nwc10 it avoids any more flags or runtime complexity
10:43 nwc10 ("compile time" is not "run time" in my head)
10:46 timotimo compiler-run-time, eh?
10:46 jnthn Before I worked on Perl 6, I thought I knew what compile time and runtime were :P
10:49 * jnthn figures he'll take on the #moarvm backlog first :)
10:51 nwc10 you have enough coffee ready?
10:52 jnthn Just made a fresh one :)
10:55 leont joined #moarvm
11:06 dalek MoarVM: a8898a7 | nicholas++ | src/6model/serialization.c:
11:06 dalek MoarVM: Note that deserialize_method_cache_lazy() is cruel and unforgiving.
11:06 dalek MoarVM:
11:06 dalek MoarVM: As the comment said, deserialize_method_cache_lazy() stashes what we need to
11:06 dalek MoarVM: eserialize the method cache lazily later, and then skips over it.
11:06 dalek MoarVM:
11:06 dalek MoarVM: As the comment now adds:
11:06 jnthn nwc10++
11:07 jnthn 11,532,280 CORE.setting.moarvm # Before
11:07 jnthn 10,950,714 CORE.setting.moarvm # After
11:07 nwc10 "OMG I killed dalek."
11:07 dalek joined #moarvm
11:07 jnthn m: say 10.95 / 11.53
11:07 camelia rakudo-moar 7c5390: OUTPUT«0.949696␤»
11:07 nwc10 thanks for checking it over and applying it
11:07 nwc10 problem is that other folks keep implemetning stuff, and making the setting bigger
11:08 nwc10 I think we should also just use these: https://en.wikipedia.org/wiki/One_instruction_set_computer
11:08 nwc10 although maybe that would need an even bigger "setting" to be useful
11:09 nwc10 jnthn: I think that the changes only make the (new) NQP bootstrap code about 9K smaller.
11:09 nwc10 (but that wasn't really the point)
11:10 timotimo nwc10: how big is a bootstrap right before your changes?
11:11 nwc10 my generated nqp.moarvm is currently 852287
11:11 nwc10 whilst stage0 is 845178, that's not a proper comparison, because it's not the same source code compiled
11:12 nwc10 anywa, about 852287 + 9K before. IIRC :-)
11:12 jnthn Indeed, I think the bootstrap will be a little bigger this time
11:12 jnthn 'cus NQP has gotten a little bigger in terms of amount of code
11:16 jnthn Phew, while 20 new tickets landed in RT over the weekend, most of them are JVM one
11:16 jnthn *ones
11:19 timotimo i didn't expect the glr to explode the jvm backend this much :(
11:20 timotimo ... is an easy thing to say for someone who did next to nothing %)
11:20 jnthn It woulda been worse if it had added more VM-specific code rather than removing a good amount of VM-specific code
11:22 dalek MoarVM: ee70ddc | jnthn++ | src/mast/compiler.c:
11:22 dalek MoarVM: Force \r-containing strings to be UTF-8 stored.
11:22 dalek MoarVM:
11:22 dalek MoarVM: Retaining the "already in NFG" invariant for the Latin-1 optimization.
11:22 dalek MoarVM: nwc10++ for suggesting this simple solution.
11:22 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ee70ddcca7
11:24 timotimo that's true
11:46 tokuhirom_h joined #moarvm
11:50 dalek MoarVM: 789fd4e | jnthn++ | / (6 files):
11:50 dalek MoarVM: Stub a couple of line-based reading ops.
11:50 dalek MoarVM:
11:50 dalek MoarVM: setinputlineseps_fh will allow for setting mutliple separators, so we
11:50 dalek MoarVM: can look for both \r\n (which will be 1 grapheme) and \n.
11:50 dalek MoarVM: readlinechomp_fh will allow us to just construct a string without the
11:50 dalek MoarVM: line separator at VM level, to avoid having to then chomp it off at
11:50 dalek MoarVM: Perl 6 level.
11:50 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/789fd4e248
11:52 jnthn Ah, duh, looks like my breaking line number reporting in the \r\n -> 1 graph thing was just a silly thinko
11:57 ilmari is tools/windows1252_cp_gen.p6 supposed to be kept in sync with src/strings/windows1252.c?
11:57 ilmari or should it be rewritten to only modify the parts that are actually autogenerated?
11:58 ilmari it's currently a year out of sync
11:58 timotimo how often does that codepage change?
11:58 jnthn ilmari: I'd just toss the script tbh
11:58 jnthn Because...what timotimo said :)
11:58 timotimo oh, the code changes, not the source
11:59 jnthn The script was probably more fun for whoever did it than wrting out a translation table by hand
11:59 ilmari it generates the whole file, not just the tables and binary search code
11:59 jnthn Right
11:59 jnthn But I mean, the tables and binary search code won't change
12:00 ilmari true
12:00 jnthn 'cus the definition of windows-1252 ain't going to change
12:00 nwc10 "are you sure?" :-/
12:00 * ilmari ignores it for now
12:00 jnthn I'd just delete the script
12:00 nwc10 I thought that MS did add some things to windows-1252
12:00 jnthn To avoid the confusion
12:00 nwc10 a decade ago
12:00 * ilmari is looking at adding optional replacement character support to the various encode functions
12:01 nwc10 indeed they did: https://en.wikipedia.org/wiki/Windows-1252#History
12:01 nwc10 until 18 years ago
12:02 nwc10 any excuse to avoid lunch!
12:02 nwc10 sorry, slow. just realised what time it is
12:03 dalek MoarVM: d7150f9 | jnthn++ | tools/windows1252_cp_gen.p6:
12:03 dalek MoarVM: Toss windows 1252 code-gen script.
12:03 dalek MoarVM:
12:03 dalek MoarVM: The source file it generated has gotten way ahead of the script, and
12:03 dalek MoarVM: it's quite unlikely we'll need to regenerate the boring mapping part
12:03 dalek MoarVM: in the future (and certainly not regularly). ilmari++ for spotting
12:03 dalek MoarVM: the outdated script.
12:03 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/d7150f9389
12:23 itz_stmuk joined #moarvm
12:25 JimmyZ jnthn: could you review https://github.com/MoarVM/MoarVM/pull/286 ? :)
12:27 jnthn JimmyZ: Will do it later this week, when I'm looking more closely at I/O things (including dealing with the using sync handles between threads stuff)
12:29 JimmyZ jnthn: thanks :)
12:30 timotimo jnthn: is our solution for "using sync handles between threads" going to be "stop using libuv for synchronous stuff"?
12:30 dalek MoarVM: 8b806b1 | jnthn++ | src/ (7 files):
12:30 dalek MoarVM: Refactor I/O API for multiple separator support.
12:30 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/8b806b1f6e
12:30 jnthn timotimo: Yeah
12:31 jnthn At least, that's the most expedient solution I have :)
12:32 JimmyZ +1 :)
12:32 timotimo if i want to get a little head-start on it ... is socket stuff on windows the same api as on linux?
12:33 timotimo i seem to recall having to use SDL_net way back when to get bsd socket like API on windows
12:33 jnthn It's not miles off, but not the same
12:34 timotimo but using that API is the right way to go? bsd sockets on all platforms except windows, where we'll be using winsock?
12:34 jnthn timotimo: Aye, believe so
12:34 timotimo m: start { say $*IN.get }
12:34 camelia rakudo-moar c5c19a: ( no output )
12:34 jnthn I wanted to take a look at UDP along the way
12:34 timotimo m: await start { say $*IN.get }
12:34 camelia rakudo-moar c5c19a: OUTPUT«Céad slán ag sléibhte maorga Chontae Dhún na nGall␤»
12:34 timotimo that'd be lovely
12:35 jnthn I don't think that one works out on Windows fwiw
12:35 timotimo but i'd rather get some work done at #work
12:35 timotimo i recall at some point moar was complaining when using $*IN from a different thread
12:35 jnthn Tried to read() on a socket from outside its originating thread in block <unit> at -e:1
12:35 jnthn Does on Windows
12:35 timotimo that's the one
12:36 timotimo i don't think it should have that error in there
12:36 timotimo it seems to work for all syncstreams
12:37 jnthn RT thinks not :)
12:37 timotimo urgh.
12:38 jnthn Somebody over the weekend was noting problems dealing with sync sockets over threads
12:38 timotimo yeah, sockets, sure
12:38 timotimo but stdin?
12:38 nwc10 windows doesn't implement one subtle part of the BSD socket API correctly. For socketpair()s (or any sockets between processes for that matter), implicit process exit discards data if the reader hasn't read it yet
12:39 nwc10 this is bloody annoyingh
12:39 kjs_ joined #moarvm
12:39 jnthn Ouch
12:39 nwc10 because it breaks one subtle but elegant part of the *nix API - the program doesn't need to care what STDOUT is.
12:39 nwc10 and doesn't need to take special measures to be run from inetd (or similar) as a network daemon
12:40 nwc10 it's why Perl 5 Test::Harness can't (easily) do parallel tests on Win32
12:40 nwc10 at least, not "easily" the way it works on *nix
12:40 timotimo ugh, i didn't know that at all
12:41 nwc10 I've never experienced it directly, because I rarely have cause to use Win32 machines
12:41 nwc10 but I've seen it break "portability" assumptions
12:41 jnthn It's probably why spectset on Windows has to set tests off in "batches"
12:42 nwc10 one hack is to have multiple threads. The thread has a Win32 pipe (forget the exact correct API term) from the "child"
12:42 nwc10 the pipe doesn't drop data on the floor
12:42 nwc10 but doesn't work with the select() emulation
12:42 nwc10 each thread reads from the pipe (preserving data) and writes to a socket
12:42 nwc10 and knows to flush its socket before exiting itself
12:43 nwc10 and the two ends (child, and select loop) don't know any different
12:43 nwc10 there's a Perl 5 IPC module that does this
12:43 nwc10 I suspect this all comes from sockets originally being Winsock, and not part of the C Run Time
12:44 timotimo is winsock no longer a thing?
12:44 timotimo i don't really know what winsock "used to be"
12:44 nwc10 I don't know enough to answer that correctly
12:44 nwc10 3.1 era third party library, IIRC
12:45 timotimo it'd be very much like windows to still have that
12:45 nwc10 or, was it an API, that one or more libraries provided an implementation of?
12:49 timotimo "Initially, all the participating developers resisted the shortening of the name to Winsock for a long time, since there was much confusion among users between the API and the DLL library file (winsock.dll) which only exposed the common WSA interfaces to applications above it."
12:50 timotimo apparently winsock is still a thing?
12:52 arnsholt Probably, given MS's zealous approach to backwards compat. The question is whether it's still something you're *supposed* to use, rather than something that still works, I suspect =)
12:52 timotimo no clue
13:08 tokuhirom_h joined #moarvm
13:12 dalek MoarVM: 9c53ac7 | jnthn++ | src/ (3 files):
13:12 dalek MoarVM: Implement setinputlineseps_fh op.
13:12 dalek MoarVM:
13:12 dalek MoarVM: Can now set multiple separators.
13:12 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/9c53ac7849
13:14 timotimo wow, we can have that many different input seps?
13:16 leont joined #moarvm
13:17 jnthn timotimo: I hope not :P
13:17 jnthn It's enough to prevent various overflows
13:17 jnthn lunch &
13:17 jnthn (and after lunch I'll impl the readlinechomp op, and multi-grapheme separators)
13:45 lizmat jnthn: in newio I specced something like :nl-reading and :nl-saying to seperate input from output
13:45 lizmat would it make sense to put nl-saying also in Moar, and then include an nqp::say op ?
13:46 timotimo FWIW, reading from and writing to the same file is kind of icky as soon as you have something like a variable-length encoding
13:46 lizmat it would certainly make life a lot easier on the Perl 6 side, and potentially large output effiiciency consequences
13:46 timotimo how do you mean?
13:46 timotimo oh
13:46 timotimo i missed that second line
13:47 lizmat my $out := $*OUT;
13:47 lizmat my str $str = nqp::concat(nqp::unbox_s(x),$out.nl);
13:47 lizmat $out.print($str);
13:47 lizmat says it all, really
13:47 lizmat nqp::say(nqp::unbox_s(x))
13:47 lizmat would be *much* simpler and faster I would think
13:48 timotimo ah
13:48 timotimo well, concat ought to be somewhat efficient due to our ropes implementation getting that without doing a copy, but still ...
13:51 lizmat it's a lot HLL stuff that's really low level
13:51 lizmat the only thing missing is an :nl-saying attribute on VMIO
14:01 jnthn I suspect nl-in/nl-out would be better
14:01 ilmari do any of the regular moarvm hackers use emacs? if so, what c-mode settings do you use?
14:02 ilmari and could you stick them in a .dir-locals.el?
14:02 * jnthn doesn't, but would be fine with such a file
14:03 jnthn lizmat: Will consider it, anyways
14:03 timotimo same
14:03 jnthn lizmat: For now I'm pondering API for multiple separators
14:03 lizmat that will make [Tux] very happy  :-)
14:04 jnthn What does he want, ooc?
14:04 jnthn Just to be able to set multiple seps?
14:04 lizmat being able to split on multiple different line endings :-)
14:04 lizmat yup
14:04 jnthn OK, that I already have just written a test for, though they're all single char so far
14:05 timotimo hm. so did we have vm-level line splitting so far? i think we did, but we didn't have chomping in VM, right?
14:06 jnthn Correct, and it could only split on a single splitter
14:06 timotimo right. having chomp in vm sounds worthwhile
14:06 jnthn And my \r\n -> 1 grapheme blocked on that
14:06 jnthn Which is why this issue ended up leap-frogging finishing up that.
14:20 dalek MoarVM: b2c7ecc | jnthn++ | src/ (8 files):
14:20 dalek MoarVM: Refactor readline I/O to pass down a chomp flag.
14:20 dalek MoarVM:
14:20 dalek MoarVM: Reaches the decode stream now, but it doesn't yet know what to do
14:20 dalek MoarVM: with it.
14:20 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b2c7ecc995
14:20 dalek MoarVM: bea376b | jnthn++ | src/core/interp.c:
14:20 dalek MoarVM: Implement readlinechomp_fh op.
14:20 dalek MoarVM:
14:20 dalek MoarVM: Doesn't work yet, due to missing decode stream functionality.
14:20 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/bea376b968
14:27 kjs_ joined #moarvm
14:43 dalek MoarVM: 61795ad | jnthn++ | src/strings/decode_stream.c:
14:43 dalek MoarVM: Support chomping separators when reading lines.
14:43 dalek MoarVM:
14:43 dalek MoarVM: For now, assumes that all separartors are a single grapheme, since
14:43 dalek MoarVM: that is all the separator finding logic can handle so far. Instead of
14:43 dalek MoarVM: constructing a string and then chomping it, we just produce a readily
14:43 dalek MoarVM: chomped string.
14:43 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/61795ad6b8
14:44 ilmari making moarvm's string encode functions throw exceptions on unencodeable characters breaks zero nqp tests...
14:44 nwc10 "write more tests"?
14:45 nwc10 I'm not actually sure what the desired right answer is here.
14:46 ilmari currently they unconditionally throw, the plan is to allow specifying a replacement char (or string?)
14:46 ilmari https://rt.perl.org/Public/Bug/Display.html?id=123673
14:46 jnthn Well, we tend to ultimately rely on the Perl 6 spectests to give good coverage.
14:53 TimToady joined #moarvm
14:57 dalek MoarVM: aef902f | jnthn++ | src/strings/decode_stream.c:
14:57 dalek MoarVM: Refactor to prepare for multi-graph line seps.
14:57 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/aef902fa02
15:01 ilmari what's the incantation for running spectest suite?
15:01 ilmari the README only talks about fudging
15:01 jnthn "make spectest" inside the Rakudo build directory
15:01 jnthn (It does a checkout of the spectests under t/spec)
15:01 ilmari ah
15:05 ilmari in the ticket, moritz suggests specifying the replacement as a Buf, but I thought it migth be easier to sepcify it as a Str wihch will be encoded with the same encoding (throwing an exception if it's not encodable, of course)
15:14 jnthn A Buf does indeed carry the risk that we spit out something that ain't valid in the encoding, yeah.
15:19 arnsholt Sounds like a good idea. I guess the Buf is more reasonable when you're primed to think about ASCII, since any byte sequence is valid ASCII
15:21 itz_stmuk *cough*
15:21 itz_stmuk https://github.com/MoarVM/MoarVM/pull/288
15:22 tokuhirom joined #moarvm
15:23 [Coke] itz_stmuk: "The Travis CI build is in progress" :P
15:23 [Coke] but yah, that looks pretty harmless.
15:23 itz_stmuk how long does it take? and it's not like moar has its own tests ;)
15:24 moritz arnsholt: any byte sequence with bytes <= 127 is valid ASCII :-)
15:28 timo joined #moarvm
15:29 arnsholt moritz: Durr, yeah. ASCII with high bit set is of course any one of a bajillion extended ascii charsets
15:29 arnsholt >.<
15:29 dalek MoarVM: fcb8f1a | jnthn++ | src/strings/decode_stream.c:
15:29 dalek MoarVM: Handle multi-grapheme line separators.
15:29 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/fcb8f1a16f
15:30 itz_stmu1 joined #moarvm
15:31 dalek MoarVM: e1f8e6c | (Steve Mynott)++ | build/setup.pm:
15:31 dalek MoarVM: fix moar build on DragonFly BSD
15:31 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e1f8e6cc97
15:31 dalek MoarVM: 73c0269 | jnthn++ | build/setup.pm:
15:31 dalek MoarVM: Merge pull request #288 from stmuk/master
15:31 dalek MoarVM:
15:31 dalek MoarVM: fix moar build on DragonFly BSD
15:31 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/73c026919e
15:47 hoelzro joined #moarvm
15:47 travis-ci joined #moarvm
15:47 travis-ci MoarVM build failed. jnthn 'Handle multi-grapheme line separators.'
15:47 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/88809584 https://github.com/MoarVM/MoarVM/compare/aef902fa0214...fcb8f1a16f01
15:47 travis-ci left #moarvm
15:48 jnthn orly?
15:50 jnthn hah, interesting race condition :)
15:51 jnthn (It started the Moar build, but then I merged something else and bumped MOAR_REVISION, and so when the build grabbed NQP it was a too old Moar already)
15:57 kjs_ joined #moarvm
16:52 LLamaRider joined #moarvm
17:19 diakopter jnthn: lolz
17:30 ggoebel2 joined #moarvm
18:15 kjs_ joined #moarvm
19:22 leont joined #moarvm
19:39 tokuhirom_h joined #moarvm
19:45 vendethiel joined #moarvm
19:48 Ven_ joined #moarvm
20:18 kjs_ joined #moarvm
20:37 leont joined #moarvm
21:12 Ven_ joined #moarvm
21:21 dalek MoarVM: b2c18a3 | jnthn++ | build/setup.pm:
21:21 dalek MoarVM: Disable logical-op-parentheses warning on clang.
21:21 dalek MoarVM:
21:21 dalek MoarVM: Not all that is precedence is obvious, but surely this case is.
21:21 dalek MoarVM: review: https://github.com/MoarVM/MoarVM/commit/b2c18a31e0
21:31 Peter_R joined #moarvm
21:33 Ven_ joined #moarvm
21:45 Ven_ joined #moarvm
21:50 harrow joined #moarvm
21:57 Ven_ joined #moarvm
22:16 tokuhiro_ joined #moarvm
22:18 stmuk joined #moarvm
22:20 kjs_ joined #moarvm
23:12 Ven_ joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary