Perl 6 - the future is here, just unevenly distributed

IRC log for #moarvm, 2017-02-28

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 agentzh joined #moarvm
00:03 hoelzro joined #moarvm
00:14 pyrimidine joined #moarvm
00:32 lizmat joined #moarvm
00:40 dalek joined #moarvm
01:26 agentzh joined #moarvm
01:57 timotimo for the record, i don't think adding the blocked & unblocked code for get_or_vivify_loop is wrong
02:11 pyrimidine joined #moarvm
02:49 ilbot3 joined #moarvm
02:49 Topic for #moarvm is now https://github.com/moarvm/moarvm | IRC logs at  http://irclog.perlgeek.de/moarvm/today
03:10 pyrimidine joined #moarvm
03:11 agentzh joined #moarvm
03:20 pyrimidine joined #moarvm
04:04 pyrimidine joined #moarvm
05:08 pyrimidine joined #moarvm
05:16 pyrimidine joined #moarvm
06:04 pyrimidine joined #moarvm
06:23 pyrimidine joined #moarvm
07:05 pyrimidine joined #moarvm
07:06 brrt joined #moarvm
07:06 brrt good * #moarvm
07:07 brrt .tell puddu we can have a reasoned argument about whether synchronicity 'sucks' or not. fwiw, it was just a realizaiton on my part how limiting and arbitrary synchronicity really is
07:07 brrt and it seems to me that clone() or fork() or what have you… are pretty heavy weight concurrency primitives
07:28 pyrimidine joined #moarvm
07:57 domidumont joined #moarvm
08:13 brrt joined #moarvm
08:20 domidumont joined #moarvm
08:32 zakharyas joined #moarvm
09:50 jnthn morning o/
09:57 brrt moarning
09:57 brrt so, i only have a few thorny things to resolve and then arglist compilation is finished
09:58 jnthn :)
09:58 brrt but, it feels like there's something i've forgotten about that
09:58 jnthn Hope not overly thorny
09:58 brrt well, just, messy, i guess
09:59 brrt part of the mess is;
10:00 brrt the function that does this now weighs 719 - 585 LoC
10:00 brrt 134
10:01 brrt and still needs to deal with: CALL nodes that can take register arguments
10:01 brrt (i.e. dynamic dispatch)
10:01 brrt and spilling-over-CALL
10:02 brrt i.e. all values that 'survive' the CALL node are supposed to be spilled because they are not really in memory
10:02 brrt eh, i said that all wrong
10:02 brrt the call may overwrite all of them
10:04 brrt joined #moarvm
10:05 jnthn Hopefully there'll be some way to break it into slightly smaller parts :)
10:13 dogbert11 morning, have all ENOCOFFEE errors been taken care of?
10:13 jnthn Just about :)
10:14 jnthn Fixed one thing from yesterday already; now looking at https://github.com/MoarVM/MoarVM/issues/543
10:18 dogbert11 that looks vaguely familiar :)
10:27 agentzh joined #moarvm
10:28 Sufrostico[m] joined #moarvm
10:31 brrt joined #moarvm
10:37 jnthn Think I found it
10:38 jnthn And yeah, a program could only ever hit this once in its lifetime or never again :)
10:38 jnthn So unlikely...but, as you discovered, possible
10:39 dogbert11 jnthn++ you made quick work of that one :)
10:41 dogbert11 did it have something to to with starting the threads?
10:41 jnthn Waiting for 50 runs to complete
10:41 jnthn Yeah, we just forgot to tell the GC we were blocking when starting the event loop worker thread
10:41 jnthn So it couldn't work-steal if it GC'd at exactly the wrong time
10:45 Geth ¦ MoarVM: e3e3b4cae4 | (Jonathan Worthington)++ | src/io/eventloop.c
10:45 Geth ¦ MoarVM: Missing GC block marking in event loop starting.
10:45 Geth ¦ MoarVM:
10:45 Geth ¦ MoarVM: This could cause very occasional hangs if GC happened right when we
10:45 Geth ¦ MoarVM: were starting up the event loop worker thread. Fixes #543.
10:45 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/e3e3b4cae4
10:48 jnthn Yup, 50 runs completed fine
10:53 Geth ¦ MoarVM: ec99d41f59 | (Jonathan Worthington)++ | src/6model/reprs/CArray.c
10:53 Geth ¦ MoarVM: Fix CArray marshalling of type objects.
10:53 Geth ¦ MoarVM:
10:53 Geth ¦ MoarVM: We didn't detect them and passed junk, instead of the NULL we should
10:53 Geth ¦ MoarVM: have.
10:53 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/ec99d41f59
10:54 jnthn That's one I discovered yesterday
10:54 jnthn Passed total junk instead of a NULL
10:56 jnthn CStruct seems to have a similar bug
10:56 dogbert11 you're on a roll
10:58 dogbert11 there are three RT's related to CStruct, i.e. 129798, 127237 and 127730. One of them might be related.
11:01 jnthn Sadly, it ain't one of those
11:02 dogbert11 :(
11:03 dogbert11 have you found anything else while working on the async stuff?
11:06 jnthn No, the patch I just pushed is the only Moar/Rakudo issue I ran into
11:06 jnthn Except the lack of support for function pointers in CStructs, which is a NYI rather than a bug
11:11 * dogbert11 sneaks off for lunch
11:12 zakharyas joined #moarvm
11:15 jnthn Oh, I was wrong, CStruct is correct
11:15 jnthn Ah well, at least now there's test coverage :)
11:31 travis-ci joined #moarvm
11:31 travis-ci MoarVM build failed. Jonathan Worthington 'Missing GC block marking in event loop starting.
11:31 travis-ci https://travis-ci.org/MoarVM/MoarVM/builds/206133785 https://github.com/MoarVM/MoarVM/compare/772f60da3005...e3e3b4cae4a4
11:31 travis-ci left #moarvm
11:40 ilmari[m] joined #moarvm
11:53 jnthn Failure was a race with NQP_REVISION bump, fwiw
11:56 zakharyas joined #moarvm
12:03 lizmat joined #moarvm
12:08 ilmari[m] joined #moarvm
12:09 * dogbert11 is back, downloads the latest MoarVM ...
12:11 brrt left #moarvm
12:11 brrt joined #moarvm
12:28 lizmat just as a data point, but HARNESS_TYPE is still segfaulting occasionally
12:28 lizmat HARNESS_TYPE=6  :-)
12:29 brrt get thee a core dump
12:29 nwc10 I can't replicate this with valgrind
12:29 nwc10 I can with ASAN
12:29 brrt or that
12:30 lizmat joined #moarvm
12:33 dogbert11 lizmat: it would be nice if we could get rid of that error :-)
12:34 lizmat indeed :-)
12:35 dogbert11 question is if jnthn has enough info to be able to fix it, we'll see after lunch
12:36 brrt (or anyone else for that matter :-))
12:36 dogbert11 indeed :)
12:38 dogbert11 I believe that nwc10 has ASAN output for the harness6 SEGV. Here's some gdb stuff https://gist.github.com/dogbert17/71bd542c007241aa10c45e0bf44dae61
12:39 brrt hmm, can you get that again, with --optimize=0
12:39 brrt (MoarVM compiled with that)
12:39 dogbert11 unfortunately harness6 is doubly broken, if TEST_JOBS=1 it runs out of file handles and if TEST_JOBS=1+ it often SEGV's
12:39 dogbert11 brrt: will fix ...
12:40 timotimo that's a very bogus pointer :)
12:40 timotimo (well, it's most probably an offset from a null pointer)
12:40 brrt thx dogbert11
12:41 dogbert11 with my luck it probably won't crash
12:42 lizmat ===(       0;216  50/51  0/?  95/97  834/842  134/134  0/?  112/112  1moar(49314,0x70000ce37000) malloc: *** error for object 0x7fdbbd4d6dc0: pointer being freed was not allocated
12:42 lizmat *** set a breakpoint in malloc_error_break to debug
12:43 dogbert11 brrt, lizmat: have it running now, gdb is attached
12:44 brrt bt full :-)
12:44 brrt (i wonder if this would be a good use for hack)
12:45 dogbert11 I'm running it with TEST_JOBS=4, should be enough, we'll see
12:47 pyrimidine joined #moarvm
12:52 dogbert11 brrt: https://gist.github.com/dogbert17/9dc960277399ffdc295b1bcd9eee8662
12:53 pyrimidine joined #moarvm
12:55 brrt i see
12:56 brrt thats pretty suspicious
12:56 brrt dogbert11: still have an open session?
12:56 dogbert11 yes
12:56 timotimo there's no "if(cur_bytes)" in front of that? o_O
12:56 brrt cur_bytes is not NULL
12:56 timotimo i mean, how else would you reach 0x38?
12:57 brrt well,  not using initialized memory
12:57 brrt dogbert11: can you do: up 1, print *decoder
12:58 dogbert11 (gdb) print *decoder
12:58 dogbert11 $1 = {common = {header = {sc_forward_u = {forwarder = 0x4dc06ef8, sc = {sc_idx = 28408, idx = 19904}, sci = 0x4dc06ef8, st = 0x4dc06ef8}, owner = 3, flags = 16, size = 24}, st = 0xa6a1a68}, body = {
12:58 dogbert11 ds = 0x445f45a8, sep_spec = 0x4dd5a408}}
13:00 brrt doesn't look obviously corrupt...
13:02 timotimo well, what does the ds look like?
13:05 dogbert11 how do I get to that?
13:07 dogbert11 is this correct or nonsense?
13:07 dogbert11 (gdb) p *decoder.body.ds
13:07 dogbert11 $7 = {bytes_head = 0x38, bytes_tail = 0x0, chars_head = 0x0, chars_tail = 0x0, abs_byte_pos = 0, bytes_head_pos = 0, chars_head_pos = 0, encoding = 0, norm = {form = MVM_NORMALIZE_NFD, buffer = 0x0,
13:07 dogbert11 buffer_size = 0, buffer_start = 0, buffer_end = 0, buffer_norm_end = 0, first_significant = 0, quick_check_property = 0, translate_newlines = 0}, decoder_state = 0x50}
13:08 brrt it appears kind of uninitialized to me
13:08 Geth joined #moarvm
13:15 dogbert11 question is whether the 'real' problem is visible in the gdb output
13:17 brrt it's pretty informative though :-)
13:20 dogbert11 added the result of a 'p MVM_dump_backtrace(tc)' to the end of the gist
13:47 Geth ¦ MoarVM: 5f9d6985a9 | (Jonathan Worthington)++ | 3 files
13:47 Geth ¦ MoarVM: Provide a way to put Decoder in nl-translate mode.
13:47 Geth ¦ MoarVM:
13:47 Geth ¦ MoarVM: So that we can use it in Proc::Async and have \r\n -> \n happen.
13:47 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/5f9d6985a9
13:48 nwc10 jnthn++ # back from lunch with patches
13:48 * nwc10 wonders what ilmari[m] will return with
13:48 jnthn Lunch featured mushrooms :)
13:49 nwc10 *I* like mushrooms. Some people don'
13:49 nwc10 't
13:49 ilmari nwc10: a stomach full of burrito
13:49 nwc10 I forget the list of people I can "steal" unwanted mushrooms from
13:49 nwc10 ilmari++
13:50 jnthn My wife is in the set of people I can "steal" mushrooms from :)
13:50 jnthn Meaning that I rarely cook them at home
13:50 nwc10 aha right now your comment makes more sense
13:51 jnthn But today was lunch out
13:51 jnthn And there was a mushroom-including dish to be had :)
14:05 dogbert11 do you pick mushrooms in the forests when it's season?
14:05 jnthn No; I'd have no idea which ones might kill me
14:06 Sufrostico[m] joined #moarvm
14:06 dogbert11 there's one in particular, .i.e. Amanita Virosa
14:08 lizmat jnthn: also, if you pick mushrooms in that area, be sure to also take your geiger counter, as mushrooms are known concentrators of fallout
14:09 lizmat http://www.theverge.com/2017/2/24/14733094/radioactive-pigs-boars-czech-republic-central-europe-germany-chernobyl
14:11 jnthn Going to the supermarket/restaurant feels like so much less hassle :P
14:24 dogbert11 jnthn, do you have any theories wrt lizmat's harness6 SEGV or is more debugging necessary? If so any suggestions on what I should do.
14:26 jnthn dogbert11: I need to take a look over the various bits of ASAN/GDB output and see if I can get a reproduction
14:26 jnthn Still working on the Proc::Async newline stuff at the moment though
14:28 dogbert11 cool, 'make [spectest|stresstest] HARNESS_TYPE=6 TEST_JOBS=4+' probably does the trick when you get to it
14:29 jnthn Just doing a Windows build to make sure the changes help there :)
14:38 dogbert11 hopefully all your changes will work on the first try (they never do for me though :-)
14:41 jnthn Yeah, the test file now passes on Windows
14:51 hoelzro joined #moarvm
15:08 Sufrostico[m] left #moarvm
15:14 jnthn Looking at https://gist.github.com/dogbert17/eaba7dfc536b7a65114a52b1748fbe23 I'm pondering putting a sanity check into the Decoder REPR impl to make sure we're never using it from multiple threads at the same time
15:15 jnthn To either rule that out, or perhaps to discover that is somehow bizzarely what's happening
15:16 dogbert11 I can try it (and probably lizmat as well)
15:16 * jnthn is running harness6 at the moment
15:16 lizmat dogbert11: actually am in the middle of something else atm, so please go ahead  :-)
15:16 jnthn So far up to S10 and going strong
15:17 dogbert11 it usually takes a while and as always it doesn't fail everytime
15:17 jnthn have, I got a SEGV
15:17 dogbert11 .oO
15:18 lizmat hmmm... I got a fail in t/spec/S17-supply/supplier-preserving.t  (test 1)
15:18 lizmat in HARNESS_TYPE=5
15:18 dogbert11 uh oh
15:18 brrt joined #moarvm
15:18 jnthn Aww it said core dumped but I can't actually find the core file
15:19 jnthn I thought they ended up in the cwd
15:19 dogbert11 did you start from the rakudo dir?
15:20 jnthn yeah
15:20 timotimo on modern systems with journald it might have put the core file into the journal for you
15:20 dogbert11 or did you forget 'ulimit -c unlimited'
15:20 jnthn This is just a stock ubuntu 16.04
15:20 jnthn dogbert11: oh...d'oh
15:21 jnthn I guess that doesn't stick between sessions :)
15:22 jnthn lizmat: Urgh, ran that 50 times without issue earlier today...
15:22 jnthn I assume you have the latest version of the test file?
15:22 lizmat think so
15:22 jnthn (I corrected one issue in it this morning)
15:23 jnthn Though that affected test 11
15:23 lizmat yeah, most recent
15:24 lizmat flapper  :-(   no pbs this time
15:29 [Coke] u: -
15:39 dogbert11 t/04-nativecall/06-struct.t .............. Failed 5/28 subtests   # hmm
15:47 dogbert11 interesting, the test file passes if run with ./perl6 but not if they're run with make. looks like a fudgeup :)
16:00 Geth ¦ MoarVM: 296ece0d28 | (Jonathan Worthington)++ | 2 files
16:00 Geth ¦ MoarVM: Ensure Decoder REPR never sees concurrent use.
16:00 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/296ece0d28
16:01 jnthn Never tripped so far locally
16:01 jnthn Though a good sanity check to have in there
16:02 jnthn All the SEGVs I see are from gen2 gc of a p6bigint
16:05 dogbert11 so it's still a mystery?
16:06 timotimo the bigints again?!
16:06 timotimo or still?
16:06 dogbert11 are they a BIG problem :)
16:06 timotimo ugh :)
16:12 brrt joined #moarvm
16:16 jnthn Interestingly, the latest SEGV was in add_guards_and_facts
16:18 dogbert11 does that give any clues?
16:22 timotimo oh, yikes. that's spesh, isn't it?
16:22 jnthn Yeah
16:23 jnthn Too early to say; so far it's apparently that the arg we are considering contains a Scalar container, which has a value that points to a mostly-but-not-quite zeroed bit of memory
16:25 brrt joined #moarvm
16:30 agentzh joined #moarvm
16:49 jnthn Also, sadly, I only have a core dump for that one rather than having caught it under the debugger, which makes analysis a bit harder
16:54 pyrimidine joined #moarvm
16:58 timotimo joined #moarvm
17:11 domidumont joined #moarvm
17:12 zakharyas joined #moarvm
17:15 pyrimidine joined #moarvm
17:18 domidumont joined #moarvm
17:21 pyrimidine joined #moarvm
17:21 dogbert17 jnthn, are there any checks/asserts you can smuggle in somewhere, then we can test while you rest :-)
17:26 agentzh joined #moarvm
17:33 jnthn Typical, after the latest one I've just put in it didn't crash
17:33 dogbert17 :(
17:34 * dogbert17 is running as stresstest as well
17:41 jnthn Pushed another option for debugging; can turn it on by changing MVM_ARRAY_CONC_DEBUG 0 to a 1
17:41 jnthn Geth: y u no report commit?
17:41 ilmari[m] joined #moarvm
17:42 jnthn No luck so far, anyway
17:42 dogbert17 cool
17:43 dogbert17 it's an elusive bug
17:43 jnthn Indeed
17:43 jnthn Best bet seems to be gradually ruling out things it isn't
17:44 dogbert17 yes indeed
17:44 jnthn It's interesting that in every crash I've had except the spesh one, it's been in gc_cleanup
17:45 * dogbert17 checks his gists
17:45 jnthn I've never seen the decode stream one
17:46 jnthn But the thing I put in today to prevent concurrent operations on it will help us rule out the obvious failure mode there
17:47 dogbert17 did you release the first check, i.e. did it come with the latest MoarVM bump?
17:48 jnthn No, the latest two are at MoarVM HEAD
17:48 jnthn And the final one is just a debug feature
17:48 dogbert17 good to know
17:50 jnthn Seems to crash less with the latter debug check turned on
17:50 dogbert17 all my gists have MVM_string_decodestrem_destroy in them. don't remember if nwc10's ASAN report was the same
17:50 jnthn But when it does, (a) same failure mode, (b) it didn't trip the check
17:50 dogbert17 uh oh
17:50 jnthn Yeah, I don't have that in any of mine. Bizzare
17:52 dogbert17 who knows, this one might turn out to be blogworthy
17:52 agentzh joined #moarvm
17:53 jnthn :)
17:53 jnthn Think I'll break off for now
17:53 jnthn Though will keep it running to see if I get any different failure modes
17:55 dogbert17 I will try your checks in the meantime
18:17 zakharyas joined #moarvm
18:26 [Coke] https://engineering.instagram.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172#.wcdeivlvd - "Dismissing Python Garbage Collection at Instagram"
18:28 timotimo it feels like that was already mentioned here
18:28 timotimo in the context of "look, they do the 'let the OS clean house' thing, too!"
18:29 [Coke] ah, sorry
18:31 timotimo the "reading is a writing operation" point is quite interesting, too
18:31 timotimo it's good that we don't do refcounts
18:35 Geth ¦ MoarVM: 773711e114 | (Jonathan Worthington)++ | 2 files
18:35 Geth ¦ MoarVM: Debug option to detect concurrent VMArray use.
18:35 Geth ¦ MoarVM:
18:35 Geth ¦ MoarVM: We should make this memory-safe in the future; in the meantime, this
18:35 Geth ¦ MoarVM: can be turned on to see if it's to blame for problems. Doesn't yet
18:35 Geth ¦ MoarVM: catch all cases (for example, read while write isn't yet caught).
18:35 Geth ¦ MoarVM: review: https://github.com/MoarVM/MoarVM/commit/773711e114
19:29 Ven joined #moarvm
19:44 yoleaux2 joined #moarvm
19:47 pyrimidine joined #moarvm
19:52 agentzh joined #moarvm
19:54 nwc10 jnthn++
19:55 nwc10 jnthn: http://paste.scsys.co.uk/555094
19:55 nwc10 your new thing catches it
19:55 pyrimidine joined #moarvm
19:55 nwc10 "Deocder"
19:55 nwc10 I CAN HAZ VARIANT SPEELNG
20:03 dogbert17 nwc10++ nice catch
20:04 nwc10 I think it's far more doubleplussjnthn for having a hunch about what the crazy problem might be
20:04 nwc10 I just built it and ran it in a loop
20:04 dogbert17 deocders are way above my paygrade :)
20:09 jnthn Interesting
20:09 jnthn Though the question remains why I got the SEGV *and* didn't trip warning...
20:09 jnthn Uh, error
20:09 nwc10 yours or mine?
20:09 jnthn I locally got a SEGV
20:09 nwc10 ahOK
20:09 nwc10 I have
20:09 jnthn After putting that decoder check in
20:09 nwc10 #define MVM_ARRAY_CONC_DEBUG 1
20:09 nwc10 and
20:09 nwc10 define FSA_SIZE_DEBUG 1
20:10 jnthn So it's really interesting that it hit it
20:10 nwc10 and this is ASAN
20:10 jnthn But on the other hand...might not be our SEGV
20:10 nwc10 and somehow I could never make problems with valgrind
20:10 jnthn It seems very timing-sensitive
20:11 dogbert17 have run several times with 'MVM_ARRAY_CONC_DEBUG 1', no crash
20:11 jnthn Like, that MVM_ARRAY_CONC_DEBUG makes it a lot less likely to break
20:12 jnthn And that's much less of a slow-down than valgrind
20:12 jnthn Valgrind serializes everything onto one thread
20:12 jnthn All of this points fairly strongly towards a data race
20:12 nwc10 ohhhhhh. I didn't know that
20:12 jnthn The question is what in.
20:13 jnthn Might be worth me pouring over helgrind output some
20:13 lizmat jnthn: I still suspect something to do with grammars, as that is what HARNESS_TYPE=6 is doing a lot, parsing TAP
20:13 jnthn Though I might need to work on suppressions and cleanup of various things
20:14 lizmat I've been tempted to speed up prove6 by not using grammars, but then the segfaults may go
20:15 jnthn Does prove6 need speeding up? :)
20:15 jnthn But yes, let's make it work first before worrying about that :P
20:16 * jnthn got the SEGV again
20:16 jnthn Yet again, it's in mp_clear, when doing MVM_gc_collect_free_gen2_unmarked
20:17 jnthn It's especially odd that I've not yet seen the DecodeStream failure mode in a load of runs
20:17 jnthn We can likely put together something small that stresses the relevant code paths on that one, though
20:27 zakharyas joined #moarvm
20:45 dogbert17 got a SEGV
20:45 dogbert17 0  0x40024cb0 in ?? ()
20:45 dogbert17 #1  0x404f74ba in malloc_printerr (action=<optimized out>, str=0x405e9df4 "double free or corruption (out)", ptr=0x4a542578) at malloc.c:4996
20:45 dogbert17 #2  0x404f812d in _int_free (av=0x4062f420 <main_arena>, p=<optimized out>, have_lock=0) at malloc.c:3840
20:46 dogbert17 had turned off MVM_ARRAY_CONC_DEBUG though
20:55 geekosaur joined #moarvm
21:25 nwc10 got ASAN barfage, so the bug that the debug stuff traps is not the only one
21:26 nwc10 http://paste.scsys.co.uk/555095
21:30 agentzh joined #moarvm
21:31 dogbert17 and I got this bizarre output all of sudden, https://gist.github.com/dogbert17/33da77db96e4000518e745e83b1ae4f3
21:34 dogbert17 what a rats nest
21:35 dogbert17 two or more bugs interacting, brrr
21:43 dogbert17 ok, got another SEGV, (well SIGABRT) this time with MVM_ARRAY_CONC_DEBUG = 1
21:46 jnthn nwc10: Yeah, that's the one I get nearly every time
21:48 dogbert17 could there be somthing wrong with the grammar in TAP.pm6
21:50 jnthn What will be interseting is if we get any more ASAN or GDB output that points to decode streams, or if all those are now turned into the exception throws
21:50 jnthn I'm quite confident we can golf that one down
21:51 jnthn dogbert17: It could be, but...grammars aren't a particularly notable sources of Int
21:51 jnthn *source
21:52 jnthn And the SEGVs (aside from the decode stream ones, which it seems we mighta turned into an exception now) all point to GC of an Int
21:52 dogbert17 jnthn: did you see https://gist.github.com/dogbert17/33da77db96e4000518e745e83b1ae4f3 ?
21:53 jnthn oh, output-ruler
21:54 jnthn That's where we were in spesh at the point I got a core dump earlier
21:54 jnthn That's...an interesting coincidence
22:01 jnthn Off to rest o/
22:03 dogbert17 night
22:03 jnthn 'night
22:45 pyrimidine joined #moarvm
23:08 pyrimidine joined #moarvm

| Channels | #moarvm index | Today | | Search | Google Search | Plain-Text | summary