Camelia, the Perl 6 bug

IRC log for #parrot, 2011-12-26

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:01 benabik joined #parrot
02:40 benabik Merry Christmas, #parrot
02:53 sorear hola, benabik
02:54 benabik o/ sorear
03:37 Psyche^ joined #parrot
06:59 schmooster joined #parrot
07:36 jsut joined #parrot
10:03 mj41 joined #parrot
13:06 PacoLinux joined #parrot
14:17 jsut_ joined #parrot
14:26 lateau joined #parrot
14:58 Hunger joined #parrot
15:04 bluescreen joined #parrot
15:04 lateau left #parrot
15:57 benabik_ joined #parrot
16:09 Hunger joined #parrot
18:01 whiteknight joined #parrot
18:31 whiteknight good afternoon, #parrot
18:31 benabik o/ whiteknight
18:31 whiteknight hello benabik
18:31 benabik whiteknight: How's your holidays?
18:33 whiteknight benabik: I'm very glad the worst of them are over :)
18:33 benabik ouch
18:34 mj41 joined #parrot
18:34 whiteknight It's been nonstop since we moved in: Moving, unpacking, baking, shopping, wrapping presents, other preparations, etc
18:35 whiteknight I'm looking forward to just getting into a normal routine
18:35 benabik Moving and Christmas, not a good combination?
18:40 whiteknight my wife does a lot of baking for the holiday. She does cookie exchanges with friends, does breads and other stuff for gifts, cookies for family, deserts for various traditional meals, etc
18:41 whiteknight so the kitchen, which is still not 100% unpacked from the move, is a disaster
18:42 sorear hello whiteknight
18:42 whiteknight hello sorear
18:48 benabik whiteknight: Sounds like fun.  We did canning for Christmas, our housemates cooked a couple of ducks…  Ours was a mess too.  It was such a mess, I didn't want to touch it, but it cleaned up much faster than I expected.
18:49 whiteknight ooh, what did you can? I've been wanting to do some canning this year myself
18:50 benabik Apple butter and BBQ sauce.
18:50 benabik The apples took forever to simmer down.  Oy.
18:50 benabik And we had to do a ton of the BBQ.
18:50 whiteknight your own BBQ sauce recipe?
18:51 benabik We had to be careful what recipes we used because we don't have a pressure cooker.  They have to be pretty acidic if you're going to just use a water bath.
18:51 benabik My wife found it online somewhere…  I might still have the link...
18:51 benabik http://www.theyummylife.com/blog/20​11/01/117/3+Barbecue+Sauce+Recipes-​-Sweet+%26+Tangy,+Spicy,+or+Smoky
18:52 whiteknight nice
18:53 whiteknight I always find that little homemade food gifts go over very well and can be very cheap to make in bulk
18:53 benabik We haven't had time to try it out on a full meal, but the little tastes we had were gooood.
18:53 benabik Cheap in money, but the extra time it takes can be a real pain.  ;-)
18:54 whiteknight yes, hence why our december has been so crazy
18:55 benabik I think the apple butter recipe is something like this: http://simplyrecipes.com/recipes/apple_butter/
18:55 benabik No actual butter, just apples simmered for hours.  Pretty tasty.
18:56 benabik We tend to use the handmade things to fill out the piles of aunts and uncles on the list.  It seems to have gone over pretty well and keep the gift budget from killing the poor grad student.  ;-)
18:57 whiteknight I'm a big fan of apple butter, and we usually have extra apples because we like to go out picking in the fall
18:57 whiteknight my kid loves it, and I bake a mean pie with the results
18:57 whiteknight applebutter would be a nice thing to do also
19:01 benabik The canning itself is pretty simple, although you do need to be careful with sanitation.
19:12 whiteknight oh yeah, I've done that before
19:13 benabik Wife is home from boxing day.  Good luck with the cleanup, whiteknight
19:14 whiteknight thanks!
19:38 whiteknight msg alvis are you on a 32-bit or 64-bit machine? I suspect the random number generator behaves differently on each
19:38 aloha OK. I'll deliver the message.
19:45 ruoso whiteknight: if you have some time, there are some things regarding threads/concurrency I'd like to discudss
19:45 whiteknight ruoso: for you, for that topic, I always have time :)
19:47 whiteknight ruoso: I saw your email about autothreading pipes. I do think we are definitely moving in that direction
19:49 whiteknight Your idea about blocked buffer streams sounds interesting, but I don't think I understand all the details
19:52 ruoso Ok, basically this comes from one thing in Perl
19:52 ruoso *Perl 6
19:52 ruoso which is the implicit threading
19:53 ruoso basically, even if the language ends up providing threading primitives
19:53 ruoso most of the concurrency in the language will be provided by implicit threading operations
19:53 ruoso i.e. every lazy list operation is potentially concurren
19:53 ruoso *concurrent
19:54 whiteknight okay, in what sense? Like the values can be pre-computed?
19:54 ruoso yes
19:54 ruoso like... my @a = grep { $_ % 2 }, 1..inf;
19:55 ruoso this will not fail... it will become a lazy list generator
19:55 whiteknight right
19:55 ruoso but itś
19:55 ruoso but it's not completely lazy
19:55 ruoso it's  "mostly lazy"
19:55 ruoso which means the interpreter is allowed to start producing values as pleased...
19:56 ruoso and the interpreter is also allowed to do that concurrently with the main program
19:56 ruoso so the main program will continue immediately, and a new task (i use task in the parrot sense here)
19:56 whiteknight Okay, here's the problem I have. If we move processing over to a separate OS thread, there will be additional overhead associated with moving the data, synchronizing, etc
19:57 whiteknight if we do it in a Task on the current thread, we basically cut processing bandwidth for the "master" task
19:57 whiteknight so if we start using the value immediately, we don't want to have synchronized and gone through lock headaches on a separate thread
19:58 ruoso but... in the simple case above, why would you need locks?
19:58 whiteknight not just locks. Maybe no locks at all. Still, if we're operating on data on a separate thread we need some kind of synchronization
19:58 ruoso why?
19:59 ruoso I mean...
19:59 whiteknight you need to be able to guarantee thread-safe reads on complex aggregate data
19:59 ruoso you have the stream that will work as sync agent
19:59 whiteknight ok
20:00 ruoso the user is not supposed to change data that was already sent on a stream
20:00 ruoso if he does that, he is the one to blame
20:00 whiteknight "not supposed to" or "not allowed to"?
20:00 ruoso we don't need to protect against that
20:00 whiteknight ok
20:00 ruoso we could just say 'doing that results in undefined behavior'
20:01 whiteknight My only real concern is that any threading is going to add some amount of overhead. If we cut consistency measures down to the bare minimum, we still have some overhead
20:01 whiteknight so some common use-cases could end up operating slower if we get threads involved
20:02 whiteknight if the compiler is smart enough to weed those cases out, that's cool too
20:02 ruoso Yes...
20:02 ruoso a simple @a = grep { $_ % 2 } 1..10; is clearly not worth for lazyness
20:02 ruoso this is something the compiler is responsible to detect
20:03 ruoso the compiler or the runtime...
20:03 whiteknight Okay, that concern is covered. So how do we implement these buffered streams like you're talking about?
20:03 ruoso circular queue
20:04 ruoso just like unix pipes
20:04 whiteknight you're not worried about losing data?
20:04 whiteknight or do you put in write-stops to halt the stream source when the queue is full?
20:04 ruoso I am... that's why it's blocking
20:04 whiteknight okay, gotcha
20:05 ruoso and if it's done in the VM level, it can simply change the status of the task to Blocked and the scheduler will know what to do
20:06 whiteknight so basically like a mailbox, in reverse. Reading from the Task instead of writing to it
20:06 whiteknight Yeah, I think we can do that
20:06 whiteknight nine: ping
20:06 ruoso it's a pipe, really
20:07 ruoso in the my @a = grep { $_ % 2 } 1..inf; example
20:07 ruoso it would initiate two tasks
20:07 ruoso 1 producing the range
20:07 ruoso 2nd computing the grep closure
20:07 ruoso the first would use a stream to send data to the second
20:08 ruoso the second would use a stream that would be used to populate @a (whenever someone actually reads a value)
20:08 ruoso so the second stream would fill very quickly, blocking the second task, then filling the first stream and blocking the first task
20:09 ruoso at the moment someone did @a.shift, it would create some room in the second buffer, which would unblock the second task, which would consume from the first stream, unblocking the first task...
20:15 whiteknight okay, I see what you are saying
20:16 whiteknight so this is something we would have to integrate into the concurrency system pretty closely, if it wants to have reasonable control over the Task
20:17 ruoso Yes... it would be a primitive preemptiveness control
20:17 ruoso (just like unix pipes...)
20:18 whiteknight I'm not entirely familiar with how unix pipes work internally.
20:18 ruoso they are blocking streams
20:19 ruoso at the kernel level
20:19 ruoso simple as that
20:19 whiteknight So I'm trying to work out the implementation in my head. Do we want a stream type, or add stream logic to all tasks?
20:19 ruoso a stream type, I would thikn
20:19 whiteknight I assume the former, but I might not know what the most common use-case for Tasks is going to be
20:20 ruoso in the unix world, a pipe will work by two file descriptors
20:20 ruoso from/to you read/write to/from
20:21 ruoso so a pipe is really a buffer
20:21 ruoso and then you have one read handle and one write handle to that buffer
20:21 whiteknight so the Stream would hold a Task reference. When the stream is read it blocks the reader, activates the source until it has a few values?
20:21 whiteknight then it blocks the source when the stream is full?
20:22 ruoso Yes.
20:22 whiteknight until it's empty again, then it activates the source again, repeat
20:22 whiteknight okay, I like that
20:22 ruoso not empty...
20:22 ruoso it's concurrent
20:22 whiteknight what do you mean?
20:22 ruoso the reader unblocks as soon as there is anything in the buffer
20:22 ruoso the writer unblocks as soon as there is any room in the buffer
20:23 ruoso basically the buffer has "file descriptors" associated with it. The file descriptor belongs to a process
20:24 ruoso the read process is awaken when there are contents in the buffer
20:24 ruoso the write process is awaken when there is enough room for a write in the buffer
20:24 whiteknight so the buffer always wants to be full? That seems a little bit eager
20:25 ruoso Yes, the buffer always wants to be full...
20:25 whiteknight In terms of performance, I feel like we would want to calculate data on demand instead of over-calculating
20:25 whiteknight but that's a semantic issue that we can change trivially
20:25 ruoso and that can even be set at runtime
20:25 whiteknight I *suspect* a "fill when empty" rule would have better performance overall
20:26 ruoso if in the end it's IO bound, the "always full" will probably be better
20:27 ruoso but my point is that it should probably be left to runtime decision
20:27 ruoso (even if that means a pragma, or a command line argument)
20:31 ruoso I don't know how Parrot handles IO, but my suggestion would be to emulate the way file descriptors are handled and use the same APIs
20:31 ruoso with the exception that it wouldn't be a system call, but rather a VM-call
20:32 ruoso I assume you have to handle that with the green threads and system "read" calls
20:33 whiteknight In the new thread system, we're going to be moving towards VM calls for IO instead of direct system calls
20:33 ruoso that's perfect
20:34 ruoso so this would be an emulation of unix pipes
20:34 whiteknight we'll probably have a dedicated worker thread at some point for it (for systems which support it). AIO will be the default mode of operation and blocking calls will be AIO+Task Sleeps
20:35 whiteknight of course, dedicated read sources might be their own thread or something
20:37 whiteknight We don't have threads yet, so we can really do anything at this point
20:37 ruoso well... you do have the green threads already, don't you?
20:38 ruoso I don't think you need real threads to implement that primitives
20:41 whiteknight no, you're right. I'm just saying that there is lots of behavior decisions which are still up in the air
20:41 whiteknight (if you have any other recommendations as they pertain to perl6, that would be great)
20:42 ruoso I think it's a good thing we leave as much of that as possible to runtime
20:44 ruoso the only settled down threading operative in Perl 6 is "async" so far...
20:44 ruoso which is not even spec'ed to be a OS thread
20:44 ruoso it's just "async"
20:46 mj41 joined #parrot
20:47 ruoso So... if we were to think about "types", I would think: "Feed", "Feed Handle". A Feed may have several handles associated with it. A "Feed Handle" may be a read or a write handle and belongs to one task. The "Feed Handle" would also keep a status indicating if it is trying to read/write.
20:48 ruoso whenever availability changed in a "Feed", the associated handles would be searched for to see if anyone was blocked on it... changing the status of the associated task in that case.
20:48 whiteknight I'm not sure I see the value in having two types. A Feed would be it's own handle, no?
20:48 ruoso A feed is global, a feed handle belongs to a task
20:49 ruoso the handles can be closed asynchronously...
20:50 ruoso if the read side closes the handle... it needs to raise an exception in the write side
20:50 ruoso a.k.a. SIGPIPE
20:52 ruoso and also, when all handles are closed, the feed can be destroyed
20:53 ruoso Also, you may have more then one read handle per feed
20:53 ruoso if different tasks cooperatively consume one input feed
20:53 whiteknight we don't really need that. Parrot has GC, a reference to the Feed is enough to keep the feed alive
20:53 whiteknight and when there are no references, GC makes it dead
20:54 ruoso sure... that's what I meant
20:54 whiteknight if the read-side closes, do we throw an exception on the write side, or just tell the writer to block?
20:54 whiteknight I mean, an exception seems heavy-weight to me
20:56 ruoso well... if there is no one else to read
20:56 ruoso the write side needs to be interrupterd
20:57 ruoso *interrupted
20:57 ruoso note that the feed itself is only addressable by its handles
20:57 ruoso so once you close all read handles, there is no way to get a new read handle
20:59 ruoso (addressable by the tasks, that is... the vm is the only thing holding a reference to the actual buffer
21:04 particle joined #parrot
21:19 particle joined #parrot
22:54 mj41 joined #parrot
22:59 jsut joined #parrot
23:59 contingencyplan joined #parrot
23:59 schmooster joined #parrot

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced