Camelia, the Perl 6 bug

IRC log for #parrot, 2011-10-27

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:01 dalek winxed: 5e66514 | NotFound++ | winxedst1.winxed:
00:01 dalek winxed: simplify warning emision
00:01 dalek winxed: review: https://github.com/NotFoun​d/winxed/commit/5e66514ce7
00:08 soh_cah_toa NotFound: ooc, why doesn't winxed support inline pir blocks? is it just something you haven't gotten around to or is it forbidden for a reason?
00:10 NotFound soh_cah_toa: because it will be too difficult to check for unreasonable uses and because people may use it without reporting the problem or even without knowing there is a better way.
00:11 NotFound The nqp guys can tell you that this things happened a lot.
00:11 soh_cah_toa ok, i can understand that
00:12 soh_cah_toa NotFound: also, i've always wanted to know why you chose to write winxed in c++ and generate pir yourself instead of using the pct toolkit
00:14 NotFound soh_cah_toa: several reasons: I like hand-written recursive descent parsers, I failed to made even toy languages with pct, and I'm much more fluent with C-family syntax than with perl-alikes.
00:15 soh_cah_toa ok, cool
00:18 NotFound Last but not least, Stroustrup says in "The Design and Evolution of C++" that people convinced him to implement it using yacc and family but in restrocpetive he thinks it will be better his first idea of a recursive descent parser.
00:18 NotFound And I'm a Stroustrup fan.
00:19 NotFound And a last point: is easier to generate yourself the pir you want than figuring how to tell pct to generate it.
00:19 soh_cah_toa eh...college has destroyed any appreciation of c++ i could have hade
00:20 NotFound soh_cah_toa: try to rewrite winxed stage 0 in C ;)
00:20 soh_cah_toa god no ;)
00:21 whiteknight C is not the only other possible alternative
00:21 NotFound whiteknight: try Pascal, then ;)
00:21 soh_cah_toa ew
00:21 whiteknight if I were doing it today, I would do it in C#
00:23 NotFound Is not a bad language, but I don't use windows and I don't like mono.
00:23 * soh_cah_toa concurs
00:26 NotFound_b joined #parrot
00:28 NotFound Anyway, I managed to go from zero to a useful self-compiling stage 1 in a short time, so the decisions were not too bad ;)
00:30 Themeruta joined #parrot
00:43 Themeruta joined #parrot
01:14 jsut_ joined #parrot
01:40 cotto This is the same thing I'm thinking about for Mole.
01:56 benabik joined #parrot
02:19 cotto ~~
02:43 alester joined #parrot
03:02 alester seen dukeleto
03:02 aloha dukeleto was last seen in #perl6 9 hours 17 mins ago saying "[~~]".
04:48 jsut joined #parrot
04:57 dukeleto alester: pong
05:04 alester Let's discuss TWM
05:04 alester I don't know the state of it.
05:05 alester I haven't pulled any changes of yours.
05:05 cotto TWM?
05:05 dukeleto Test::WWW::Mechanize
05:06 dukeleto alester: i thought i saw you merge my custom_linter branch?
05:07 alester oh, long ago
05:07 dukeleto alester: do you want me to add some more tests before a release?
05:07 alester hold on, checking
05:07 dukeleto alester: i only wrote a very basic test
05:08 sorear I feel old, I guessed /usr/bin/X11/twm at first.
05:09 sorear opbots trust alester
05:09 slavorg Ok
05:09 alester OK, so I need a Changes update
05:10 alester writing that
05:17 alester where do MYMETA.json come frmo?
05:19 alester dukeleto: any guess where https://github.com/petdance/​test-www-mechanize/issues/4 is?
05:24 alester nm
05:24 alester I found the MYMETA.*
05:37 dukeleto MYMETA.json is generated by newer Module::Build's
05:37 dukeleto alester: i add it to .gitignore , since it is a generated file. But you could commit it. Doesn't really matter.
05:39 dukeleto slavorg, where does your code live?
05:40 alester I just uploaded TWM 1.36
05:42 dukeleto alester++
05:43 sorear good question, nobody really knows.
05:43 sorear maybe asking #london.pm would be a good idea; it seems one of them runs slavorg
05:48 alester ok, g'night
06:20 schmooster joined #parrot
06:26 mj41 joined #parrot
06:26 mj41_nb joined #parrot
06:48 mj41_nbx joined #parrot
06:51 SHODAN joined #parrot
07:00 contingencyplan joined #parrot
08:34 alvis joined #parrot
10:27 Hunger joined #parrot
10:29 lucian joined #parrot
10:29 lucian_ joined #parrot
10:45 ambs joined #parrot
11:39 Psyche^ joined #parrot
11:50 whiteknight joined #parrot
11:51 benabik joined #parrot
11:51 whiteknight good morning, #parrot
11:53 nine good morning, whiteknight
11:53 benabik good morning!
11:53 whiteknight hello nine, benabik. how are you two doing this morning?
11:54 nine fine, fine
11:54 benabik Pretty good, other than forgetting to eat breakfast on my way out the door.
11:54 whiteknight benabik: it's the most important meal of the day!
11:54 benabik whiteknight: Yup.  I made lunch and forgot to grab a breakfast.  Bought a bagel at school though.
11:55 * benabik digs into his cream cheese.
11:55 nine whiteknight: is it? I find it's the meal I can skip most easily
11:55 benabik nine: Skipping breakfast has been shown to slow your metabolism for the rest of the day.  Less energy, easier to gain weight, etc etc.
11:56 * benabik apparently still has flashbacks to working at a vitamin store.
11:58 nine benabik: I think the verdict is still open on that one... http://www.fitnesscontrarian.com/conventional​-wisdom-vs-fitness-contrarian-skip-breakfast/
12:00 benabik nine: Large breakfasts aren't really a great idea either.  The best advice I've seen (and was most effective for me) wasn't to skip meals but to eat more.  Four or five small meals tended to be easier for me.
12:02 nine benabik: yep. I think that's a proven truth by now. But also quite difficult to follow
12:02 * benabik has lots of stuff in his desk to eat.
12:03 benabik I will admit I was on that schedule when I was working from home.
12:06 nine Oh at home it's easy :) I usually work from home on Fridays. There I usually have two small breakfasts and a snack for lunch before heading out to the university.
12:11 whiteknight nine: I was playing with your branch last night, but I don't think I pushed the newest version to parrot/parrot/green_threads. I'm going to do that now and then I'm going to open a pull request
12:12 nine whiteknight: as nice as the DVCS stuff is, it can make things quite complicated as well :)
12:13 whiteknight yeah, pros and cons
12:13 whiteknight at least it's *possible* in git. svn we would be shit out of luck
12:13 whiteknight not that you could have easily cloned the repo with svn anyway
12:15 bluescreen joined #parrot
12:17 dalek parrot/green_threads: b5ee6aa | (Stefan Seifert)++ | src/scheduler.c:
12:17 dalek parrot/green_threads: Clean a nested comment start
12:17 dalek parrot/green_threads: review: https://github.com/parrot/parrot/commit/b5ee6aac6a
12:17 dalek parrot/green_threads: f1329e5 | (Stefan Seifert)++ | src/scheduler.c:
12:17 dalek parrot/green_threads: Reworked the zero overhead patch to reduce overhead
12:17 dalek parrot/green_threads:
12:17 dalek parrot/green_threads: moritz++ reported a 6 % performance loss with green_threads. Suspect it
12:17 dalek parrot/green_threads: to be the enabled scheduler timer tick as soon as some alarm is active.
12:17 dalek parrot/green_threads: review: https://github.com/parrot/parrot/commit/f1329e5cfc
12:22 whiteknight nine: after the merge here, what's the next thing you want to work on?
12:32 nine There's still the open issue of passing in a nested runloop. Personally I would like bailing out with a usefull error message most, since it would not affect the API
12:33 tadzik can we make a thread, which will block on IO, so scheduler will not run it until IO happens, so we get async IO?
12:34 whiteknight tadzik: I'm planning AIO in my head right now
12:34 moritz tadzik: I think that requires OS level threads so far
12:35 whiteknight I'm that's the next project for me, unless nine wants it :)
12:35 whiteknight moritz: it might require an OS-level worker thread, but doesn't require interpreter-level concurrency
12:35 tadzik moritz: ah, I'm reading too much Tanenbaum recently :)
12:36 whiteknight moritz: and most modern OSes have asynch IO APIs available to use. All we would need to do is upgrade the scheduler to handle callbacks and schedule tasks with the results
12:36 tadzik but that's right, the os scheduler will block the whole process in this case
12:36 moritz whiteknight: I'm aware of that part, yes
12:39 nine whiteknight: on my personal schedule, I need to have hybrid threads up till end of the year latest to be able to finish my paper in time...
12:39 nine on the other hand async I/O would make tasks much more useful quickly
12:40 whiteknight nine: end of calendar year?
12:40 nine whiteknight: yep
12:42 whiteknight nine: Not a moment to spare! Okay, we can focus our attention on that. AIO is going to require a little bit a cleanup work first anyway, which I can do on the side
12:44 benabik At some point, I'm going to have to implement a mini-SML in Parrot.
12:46 Coke SML?
12:47 moritz surface modeling language?
12:47 benabik Standard ML, one of the more theory-based functional languages.
12:47 * moritz likes the term "theory-based"
12:48 benabik One of the first languages to have rigorous semantics.  It's something akin to a less pure haskell.
12:48 benabik (And less lazy)
12:50 jsut_ joined #parrot
12:51 benabik Mostly I want a strong functional language in Parrot and since my thesis involves working on a SML compiler, I'm going to be very familiar with it soon.  :-D
12:52 moritz just note that we don't have tools for static analysis, type inference etc.
12:52 moritz oh well, you know what our compiler tools are geared towards :-)
12:52 benabik :-D
12:53 benabik One of my goals with PACT is widening the scope of the compiler tools.  :-D
12:53 benabik Really it's pattern matching that I want in Parrot.  Tends to be very useful when writing compilers.
12:55 whiteknight nine: so what do you think we need next? I think we're going to need an API for a simple low-level threading compatibility layer to gloss over the differences between Posix and Win32
12:55 Coke benabik: nqp* aren't good enough?
12:56 nine whiteknight: exactly. Since we already had that at some point this shouldn't be too difficult.
12:57 whiteknight nine: no, not at all
12:57 whiteknight nine: the hardest part is really answering the basic architectural questions about how we want to handle data integrity, how we want to handle multi-threading with GC, etc
12:57 nine whiteknight: I think one of the most important next steps is you explaining to me how you imagine the whole "represent variables owned by other threads as proxies" thing.
12:57 benabik Coke: Not really.  NQP is designed to have P6 semantics rather than Parrot semantics, so there's a distinct mismatch when writing other things.
12:58 benabik Coke: Winxed is a nice system level language, exposing the capabilities more directly.
12:58 whiteknight nine: We have two PMCs: One is the original PMC, the other is a proxy. We overload the vtables on the proxy to schedule requests to the original, instead of performing them directly
12:58 benabik Coke: Also I mean pattern matching of data structures, not strings.
12:58 whiteknight nine: we combine that maybe with a simple mailbox-like thread-safe queue and I think we can ignore the rest of the dirty details
12:59 nine whiteknight: that's the part I understand easily. What I have not found out yet is where exactly the proxy PMC gets created. Simply create a proxy for every known PMC when moving a task to a different thread?
12:59 whiteknight The way I see it, assuming GC can be magically thread-safe, we can put a lock on the scheduler task queues, and we can put a lock on the mailbox type, and the rest of the system can be essentially lock-free so long as we follow the approved interfaces
13:00 whiteknight nine: No, that would be the task of the mailbox. We would have a mailbox between two threads. MailBox.send_object(my_object) would create a proxy on the target thread for the my_object variable
13:01 whiteknight so when you did "var foo = MailBox.receive_object()" or whatever, you would be receiving a proxy
13:01 whiteknight ...unless you were receiving from the same thread, then I suppose you could be getting a reference to the original
13:01 whiteknight so a Mailbox type is really the crux of the idea
13:01 nine whiteknight: so it's more a task for the HLL compiler to generate the code to share variables between threads? Like Perl 5's my $foo : shared?
13:02 whiteknight nine: I think so, yes. So long as you pass data around in a consistent way, you can avoid data corruption
13:02 whiteknight that's what I really want most: a system which makes it almost impossible to corrupt data internally
13:04 pbaker joined #parrot
13:05 whiteknight if cross-thread data updates are basically scheduled tasks, then we can implement critical sections easily by disabling green thread task preemption on a given thread
13:05 whiteknight then when we exit the critical section, the scheduler can run through and play out all the update tasks
13:07 nine aaah....and so the picture comes together. Now I can easily explain tomorrow, why green threads are actually useful for a full threads implementation
13:08 benabik green_threads needs a reconfigure, doesn't it?
13:09 whiteknight that's the trade-off I'm making with this design. We don't really need locks at the PIR/HLL level, because cross-thread data writes are disabled by default. Use message-passing to schedule tasks across threads instead, and make modifications in-band
13:09 whiteknight that means writing multi-threaded code is exactly the same as writing green_threaded code, so long as you make proper use of the scheduler and mailboxes
13:10 nine regarding GC: since every thread has it's own interpreter, I think it should also have it's own GC. MailBox could contain a list of referenced objects so the GC would not kill an object that's still used in other threads.
13:10 whiteknight yes, that's what I think will be the easiest to do first
13:10 whiteknight Eventually, I have a paper at home about a concurrent GC which operates on a separate thread and requires no global stops
13:10 nine that would be a fascinating read
13:11 whiteknight but I don't think we need that for the first draft
13:11 nine and one that FlightGear people would probably like to read as well... they have big problems with GC stopping the sim for very noticable amounts of time
13:13 whiteknight here's the link, but you need ACM membership to get the paper: http://dl.acm.org/citation.cfm?id=286878
13:14 whiteknight you can probably get a copy through university library or whatever
13:14 benabik whiteknight: Just being on campus does it for me.  :-D
13:14 benabik Probably free link: http://doc.cat-v.org/inferno/concurrent_gc/
13:15 whiteknight ah, yes
13:15 nine benabik: thanks :)
13:15 whiteknight benabik++. I don't think I've had a digital copy of that link for a year or more. I printed it out a long time ago and keep it by the side of my bed
13:16 whiteknight (sadly, that's true)
13:16 benabik Heh.  I've found that more and more researchers are posting their papers themselves.
13:16 nine Open Access++
13:17 mtk joined #parrot
13:18 benabik For example, the research I'm basing my thesis on: http://cseweb.ucsd.edu/~rtate/publications/eqsat/
13:20 benabik I guess I have to rebuild NQP after installing green_threads?  "Failed allocation of 7166182913849190824 bytes"
13:20 whiteknight benabik: yeah, probably
13:20 moritz nine: just re-benchmarked rakudo on the latest green_threads branch: Patched version is faster than the original by 0.02%, signficance 0.1σ
13:21 moritz so no measurable difference
13:21 whiteknight moritz: so it's equivalent to master?
13:21 nine moritz: wow...so I guess the overhead is witin the error margin?
13:21 moritz whiteknight, nine: yes to both
13:21 whiteknight okay, let's ship that sucker
13:21 moritz (on this benchmark; not sure how others perform :-)
13:21 nine moritz: my benchmark yesterday was conducted with an older Rakudo version. Noticed that I forgot to pull when I ran spectests
13:21 moritz and all spectests pass
13:22 whiteknight moritz: besides scheduling overhead, the simple single-threaded case should be identical
13:22 benabik I'm running a rebuild to compare setting time on my machine.
13:22 moritz nine: depending on how old it was, that might have made a difference
13:22 benabik whiteknight: Isn't the idea that there's little to no scheduling overhead in the single-tasked case?
13:22 whiteknight benabik: and that's what moritz++ just demonstrated
13:22 nine benabik: there is _no_ scheduling overhead when only a single task is running.
13:23 * benabik ponders using Scala's model of extractor objects for pattern matching in Parrot. http://www.scala-lang.org/node/112
13:23 nine the scheduler alarm is only set if there are other tasks in the queue. The previous version just had a bug where the scheduler alarm would get set if any alarms were active.
13:23 * moritz thinks he should should turn his small stats script into a module
13:24 nine moritz: if you're thinking that, you really should :)
13:28 PerlJam benabik: that equality saturation stuff looks interesting.
13:29 benabik PerlJam: I thought so too, otherwise I wouldn't be so eager to spend the next 6-8 months working on it.  :-D
13:30 PerlJam benabik: seems to me that alot of the work got moved from individual optimizations to the global probability heuristic.
13:31 benabik PerlJam: The conversion to/from PEGs also does a scary amount of work.
13:33 nine whiteknight: tomorrow I'll report about my progress to my professor. This will include some demonstration. Any idea for that other than printing As and Bs? :)
13:34 whiteknight nine: unfortunately, not really. Rosella has an Event library that I need to update to use tasks (once we merge to master), and I've been kicking around the idea of writing an IRC client that uses task-based eventing, but I don't have it yet
13:34 moritz nine: randomly walking turtles?
13:34 benabik PerlJam: The global profitability heuristic both does and doesn't do a lot of work.  To some extent it's a simple idea: assign costs to nodes and pick the lowest.  But solving that problem is in NP.  So I really doubt EqSat is going to be the _fastest_ optimizer.
13:35 whiteknight nine: ah yes, we do have a conio wrapper library somewhere, you could do something graphical like that
13:36 PerlJam benabik: indeed.
13:37 PerlJam benabik: one of the interesting things to me is that you can move the "hardware tunability" to one place (the GPH)
13:37 PerlJam heh ... I just noticed that I said "global *probability* heuristic" earlier  :-)
13:39 benabik PerlJam: The further work they're doing on learning optimizations is also _very_ interesting.
13:39 whiteknight nine: do we have an operator to kill a task, or kill the current task?
13:40 benabik PerlJam: Basically run it with the simple axioms over a program and then it can learn the kind of meta-axioms that your program ends up with.
13:40 nine whiteknight: t/pmc/task.t tests it, yes
13:40 whiteknight nice
13:41 whiteknight okay, we really aren't far away from basic AIO. We could teach the Select PMC how to use callbacks instead of returning descriptors very easily
13:41 whiteknight that actually might make a great demonstration, if you want to play with it :)
13:41 benabik Select PMC bleh.
13:41 nine whiteknight: the current task is killed simply by ending
13:41 whiteknight nine: okay, but is there an explicit kill_me operation?
13:41 whiteknight or is that just a return?
13:41 benabik whiteknight: return?
13:42 benabik jinx
13:42 nine whiteknight: just return
13:42 whiteknight okay. We might want to add an explicit kill operation too, for cases where we are in a task but nested down a call chain
13:42 JimmyZ joined #parrot
13:42 whiteknight but that's a detail
13:44 nine whiteknight: seems like the end op does exactly that. Description says it halts the interpreter, but actually it ends just the current task.
13:47 mtk joined #parrot
13:53 whiteknight okay, that's all we need then
13:54 benabik That description should be updated then.
13:54 whiteknight especially for AIO, we're going to want the ability to launch a request and terminate the current task, so we can resume in the callback task
13:54 benabik Callback = current continuation?
13:54 whiteknight benabik: basically, yes
13:54 benabik callback/cc
13:54 whiteknight or a task constructed by it
13:55 benabik Probably more along the lines of "the return continuation given to the supposedly synchronous IO function".  But eh.
14:00 nine whiteknight: for AIO a simple call to Parrot_cx_preempt_task should suffice. That and some runnable flag in the task pmc that gets set by the AIO callback.
14:01 nine a flag or a separate queue for inactive tasks. But that's an implementation detail that should be hidden by the API anyway. Like the enable_preemption flag that will be moved to interp
14:07 SHODAN joined #parrot
14:11 whiteknight The Select PMC should be easy to update with tasks. For asynchronous operations on FileHandle, for instance, it's going to be a little bit more involved. We're going to need to add some kind of "scheduleable" role for FileHandles and the like that the scheduler can test for readiness and extract a callback from when it is ready
14:11 whiteknight at least, that seems like the best way to me
14:11 whiteknight so we can do filehandle.dispatch_operation(), scheduler.wait_for(filehandle, callback)
14:11 whiteknight or something like that
14:11 whiteknight filehandle.dispatch_operation(callback), scheduler.wait_for(filehandle), end
14:12 dalek nqp: 1f3891e | moritz++ | t/p6regex/01-regex.t:
14:12 dalek nqp: work around hang of p6regex tests at end of file
14:12 dalek nqp: review: https://github.com/perl6/nqp/commit/1f3891e627
14:13 whiteknight msg arvis the Scala docs just got upgraded to a new look: http://docs.scala-lang.org/
14:13 aloha OK. I'll deliver the message.
14:14 benabik whiteknight: Snazzy
14:15 whiteknight benabik: I haven't looked too deep, but it is pretty
14:19 dalek parrot.github.com: da44683 | Whiteknight++ | _layouts/site.html:
14:19 dalek parrot.github.com: try to fix a few links
14:19 dalek parrot.github.com: review: https://github.com/parrot/parro​t.github.com/commit/da44683e79
14:22 dalek parrot.github.com: 3fd3de5 | Whiteknight++ | _layouts/site.html:
14:22 dalek parrot.github.com: re-fix links
14:22 dalek parrot.github.com: review: https://github.com/parrot/parro​t.github.com/commit/3fd3de5cb7
14:26 Coke benabik: so you want something more like XPath ?
14:26 nine Should this make me nervous? src/gc/gc_gms.c:2223: failed assertion 'Dead object found!'
14:26 nine Running http://paste.scsys.co.uk/154238
14:27 moritz in PIR, if obj and foo are PMCs, what does  obj.foo() do?
14:27 benabik Coke: Not familiar with XPath.  Thinking of case statements from Haskell or ML.
14:27 moritz assume that foo is a code object, and invoke it with obj as invocant?
14:27 benabik moritz: Probably the same as foo(obj), but not sure.
14:27 Coke benabik: are you familiar with xml?
14:28 benabik Coke: Vaugely.
14:34 NotFound moritz: I think it sets obj as current object an vtable-invoke foo
14:35 NotFound s/an/and
14:35 moritz NotFound: thanks
14:36 dalek nqp: fa9f991 | moritz++ | src/QRegex/Cursor.nqp:
14:36 dalek nqp: try to implement <before>
14:36 dalek nqp:
14:36 dalek nqp: It seems a bit wrong since it never captures, but it does look-ahead
14:36 dalek nqp: review: https://github.com/perl6/nqp/commit/fa9f991ced
14:36 NotFound And invoke usually pass the curent object as invocant to the overriden method, if any.
14:38 contingencyplan joined #parrot
14:39 dalek nqp: 9ce492d | benabik++ | t/p6regex/01-regex.t.old:
14:39 dalek nqp: Remove t/p6regex/01-regex.t.old
14:39 dalek nqp:
14:39 dalek nqp: It's not being used and if someone wants to refer to it, it's still
14:39 dalek nqp: available in the git history.
14:39 dalek nqp: review: https://github.com/perl6/nqp/commit/9ce492d9b0
14:39 dalek nqp: d24711b | benabik++ | t/ (2 files):
14:39 dalek nqp: Fix TAP parsing error with plan at end of tests
14:39 dalek nqp:
14:39 dalek nqp: prove doesn't like a plan at the end.  Instead just emit the 1..N done
14:39 dalek nqp: testing marker.
14:39 dalek nqp: review: https://github.com/perl6/nqp/commit/d24711bc7b
14:39 dalek nqp: c73056a | benabik++ | t/ (5 files):
14:39 dalek nqp: TODO regex test regressions from -rx
14:39 dalek nqp:
14:39 dalek nqp: We know these tests are going to fail, so let's just mark that to be
14:39 dalek nqp: the case.  They're marked regression so that we know which ones to
14:39 NotFound I'm thinking about a PMC that implements all vtables by dumping its arguments and aborting, or throwing a fatal exception.
14:39 dalek nqp: focus on.
14:39 dalek nqp:
14:39 dalek nqp: One test description got altered since using # for the issue number
14:39 dalek nqp: prevented prove from seeing the # TODO later in the line.
14:39 dalek nqp: review: https://github.com/perl6/nqp/commit/c73056a31a
14:39 dalek nqp: f858b3a | benabik++ | t/ (7 files):
14:39 dalek nqp: UnTODO passing regex tests
14:39 dalek nqp:
14:39 dalek nqp: It appears that NQP has several more features than its test suite
14:39 dalek nqp: thinks it does.
14:39 dalek nqp: review: https://github.com/perl6/nqp/commit/f858b3adf5
14:40 NotFound It can be a nice debugging tool.
14:41 NotFound (all but init, of course)
14:42 whiteknight NotFound: null already does something similar
14:43 NotFound whiteknight: it doesn't show its arguments, nor pass them to the exeception thrown.
14:43 whiteknight okay, I see what you are saying
14:43 whiteknight yes, that would be an interesting debugging tool. Maybe make a dynpmc library for debugging tools like that?
14:44 NotFound That's the idea, yes, dynamica loadable so we don't pay for it when not used.
14:45 nine Whiteknight: would you mind having a look at the assertion failure running http://paste.scsys.co.uk/154238
14:54 whiteknight nine: sure, I can take a look in a few minutes
14:57 nine whiteknight: thanks
15:00 dmalcolm joined #parrot
15:05 dalek parrot.github.com: 4ac9d37 | Whiteknight++ | / (8 files):
15:05 dalek parrot.github.com: Port over a few of the recent release announcements to the new format for an example. Update the index page to list them (needs CSS help badly). Add a .gitignore file to ignore jekyll generated files
15:05 dalek parrot.github.com: review: https://github.com/parrot/parro​t.github.com/commit/4ac9d370c1
15:07 whiteknight nine: I'm building green_threads now. I'll test it when it's ready
15:14 whiteknight nine: what error are you seeing?
15:14 nine whiteknight: src/gc/gc_gms.c:2223: failed assertion 'Dead object found!'
15:14 whiteknight okay, that's what I see
15:14 nine whiteknight: you may have to run it a couple of times
15:14 whiteknight lousy
15:15 nine Any idea what kind of problem I'm looking for?
15:15 benabik Probably something forgot to mark() a PMC its using.
15:15 whiteknight yeah, that's what I'm thinking
15:17 whiteknight Task->code is being prematurely collected somewhere
15:18 whiteknight where is the current task stored while it's executing?
15:21 nine whiteknight: interp->cur_task = task;
15:22 whiteknight yeah, I see that
15:22 whiteknight hold on, I'm testing a fix now
15:23 whiteknight ...nope, that didn't work
15:24 whiteknight bleh, I suspect we're missing a GC write barrier somewhere
15:28 nine whiteknight++ # answering my questions in blog posts before I even have them
15:35 whiteknight ...okay, I think I fixed it
15:36 whiteknight I'll push as soon as this program exits
15:36 whiteknight man, it really does start to slow down around 2000 tasks for me
15:36 benabik O.o
15:36 nine yeah! You just saved me a couple of hours of head scratching
15:37 whiteknight of course, I'm on a 1-core virtual machine here, so everything is a little slow
15:37 nine multiple cores would probably not help you much yet ;)
15:38 dalek parrot/green_threads: e212575 | Whiteknight++ | src/ (2 files):
15:38 dalek parrot/green_threads: Make sure to GC mark interp->cur_task. Add GC write barriers in scheduler.c where the internals of Tasks et al. are modified directly instead of going through safe vtable wrappers.
15:38 dalek parrot/green_threads: review: https://github.com/parrot/parrot/commit/e2125756b8
15:38 benabik nine: It would help let things other than parrot run while testing.
15:38 whiteknight nine: they help when I'm not running all background OS utilities, the console, the text editor, etc on the same core while debugging
15:42 whiteknight plus, the host machine is running VisualStudio and a few other things, so I'm generally pretty bogged down
15:47 whiteknight nine: for future reference, whenever we modify the contents of a PMC we need to set a write barrier flag so the GC picks up the changes. most of the setter VTABLEs automatically generate code to do that, so if you use vtables you're usually safe
15:47 whiteknight nine: but in scheduler.c where we modify the struct contents directly, we need to manually set the write barriers. it's tricky
15:49 iana joined #parrot
15:51 whiteknight nine: if you see similar gc-related errors, like a !Dead object found assertion, or some other issue where it looks like a PMC of one type is being accessed as if it were another type, that's probably the cause
16:10 cotto ~~
16:27 whiteknight msg dukeleto I made a few changes/additions to parrot.github.com. Take a look and see if it's heading in the direction you envision
16:27 aloha OK. I'll deliver the message.
16:35 fperrad joined #parrot
17:08 nine whiteknight: many thanks. Your patch is very instructive
17:13 whiteknight nine: good! Glad to help
17:14 dukeleto ~~
17:24 SHODAN joined #parrot
17:28 nine Seems like parrot needs 41KiB of RAM per task
17:30 whiteknight that seems like a lot
17:31 nine for supposedly ultra light green threads
17:32 whiteknight that doesn't make a lot of sense to me. The size needed should be the size of a single PMC
17:33 whiteknight plus bookkeeping
17:33 nine and the continuation
17:33 nine no idea about those
17:33 whiteknight okay, two pmcs. Continuations don't cost much
17:34 whiteknight where are you getting that number from?
17:35 whiteknight I'll need to look, but it didn't look like the system was doing anything stupid like trying to clone bytecode or anything
17:35 nine letting moretasks.pir create 6000 tasks and looking at top
17:35 nine after startup parrot took 6MiB. With 6000 tasks it's 249MiB
17:36 whiteknight That seems off
17:36 whiteknight let me give it a try here
17:36 whiteknight it might be doing something stupid internally
17:39 whiteknight which column in top is the amount of mem used? VIRT?
17:41 nine RES
17:41 nine as long as it's not swapping. Then it's nearly impossible to tell since VIRT contains memory mapped files as well
17:44 nine Interesting: task.pmc has a completely unused Parrot_jump_buff abort_jump. There's a setjmp call but no corresponding longjmp. But removing abort_jump does not reduce memory consumption.
17:46 whiteknight okay, each Task represents 4 PMCs. If you take out the schedule op, it creates 4 pmcs each loop
17:47 whiteknight https://gist.github.com/1320270
17:47 whiteknight with the schedule op, it's a lot more than that
17:48 whiteknight 4930 pmcs total
17:48 whiteknight no, wrong
17:48 whiteknight it goes up each time
17:49 whiteknight each time we schedule a task, it allocates more PMCs than the time before
17:49 nine that doesn't sound good
17:49 whiteknight aloha: 96063 - 91133
17:49 aloha whiteknight: 4930
17:50 whiteknight aloha: 91133 - 85285
17:50 aloha whiteknight: 5848
17:50 whiteknight okay, maybe not
17:50 whiteknight aloha: 85285 - 81213
17:50 aloha whiteknight: 4072
17:50 whiteknight okay, so it's a weirdly random number
17:51 nine wtf?
17:52 nine Parrot_cx_schedule_task looks pretty innocent to me. If you give it a task, all it does is call VTABLE_push_pmc
17:55 bluescreen joined #parrot
17:59 whiteknight nine: you have valgrind installed?
18:00 nine yes
18:01 nine those pmc numbers seem totally random at times. Especially if I raise the task count
18:02 whiteknight I hope the GC is keeping accurate tally
18:02 tadzik oh wtf
18:02 tadzik I just measured rakudo spectest on green_threads
18:02 tadzik make spectest  1428.83s user 117.25s system 94% cpu 27:20.81 total
18:02 tadzik now guess the yesterday's result
18:02 whiteknight ?
18:03 tadzik 2011-10-19 22:55:32     tadzik  make spectest  2210.87s user 148.94s system 95% cpu 41:04.40 total
18:03 nine wtf?
18:03 tadzik I don't believe time anymore
18:03 tadzik (yesterday's results = parrot master)
18:04 tadzik must be my cpu acting silly or something
18:04 whiteknight nine: I think that GC number is accurate. If you backscroll you can clearly see places where GC sweep is picking them up
18:04 whiteknight or, if not accurate, then close enough
18:04 moritz tadzik: do you have two different shell environments, one where TEST_JOBS is set and one where it is not?
18:05 tadzik moritz: no, I never messed with TEST_JOBS
18:06 whiteknight bleh, I have to install kcachegrind here
18:06 nine whiteknight: looks very random to me: scheduling 351 pmcs: 567557, scheduling 352 pmcs: 9592, scheduling 353 pmcs: 15040
18:06 whiteknight nine: that small number is where a GC collect run takes place
18:08 tadzik moritz: that may be my CPU going into powersave state (1.2 GHz instead of 2.1) when it's not in the mood
18:08 tadzik I'll pay double attention to that from now on
18:08 tadzik that's my only sensemaking guess
18:09 whiteknight tadzik: the only thing that should change in green_threads for current rakudo is a little bit of startup overhead
18:09 tadzik anyway, spectest passes and it doesn't seem to be slower :P
18:09 whiteknight so any numbers that look wildly different are probably erroneous
18:09 tadzik yeah
18:09 tadzik so, the only thing I can report is no failures on this machine
18:10 nine which is good news :)
18:10 tadzik I'll try nqp and parrot too, Justin Case
18:10 whiteknight nine: Parrot_cx_schedule_sleep is creating 29,013 PMCs
18:11 whiteknight I lowered test.pir to schedule only 200 tasks
18:11 whiteknight Task.invoke is creating 49,066 PMCs
18:11 whiteknight Parrot_cx_stop_task is creating 48662 PMCs
18:11 whiteknight so, there's the problem
18:11 whiteknight if sleep is creating a PMC each time, and each task is in a tight loop around sleep, that's a lot of PMCs
18:12 whiteknight 10 per task per second
18:12 whiteknight I assume that amount is being folded into the value of Parrot_Task_invoke
18:12 Coke without looking at the implementation, do we need those, what, 4 PMCs per task?
18:13 whiteknight Coke: I don't know, but I doubt that's the biggest part of the current problem
18:13 nine whiteknight: a lot of PMCs that should get garbage collected soon after
18:14 Coke wonder if chromatic is busy ;)
18:14 tadzik (:
18:14 whiteknight Coke: The 4 PMC attributes in a Task are the code (which is not created for the Task), the mailbox which we don't always need to use, the list of waiters which we don't usually need, and the user-supplied data
18:14 whiteknight so we can create the mailbox and the waiters list lazily
18:15 nine so maybe it's no problem at all but just normal behaviour for a program generating lots and lots of short lived objects in a garbage collected runtime?
18:15 tadzik PASS on parrot, nqp, rakudo. Green threads, green light
18:16 whiteknight nine: maybe, but it seems weird to me that sleep would have to create a PMC. Maybe the interp can keep around a pre-allocated sleep/delay PMC that we reuse
18:18 moritz tadzik: you can log   while true; do cat /proc/cpuinfo |grep 'cpu MHz'; sleep 5; done   while running your spectests
18:18 dalek parrot/green_threads: d42dc2a | Whiteknight++ | src/pmc/task.pmc:
18:18 dalek parrot/green_threads: Don't create Task.waiters or Task.mailbox unless we need them. This saves us two PMCs for simple tasks
18:18 dalek parrot/green_threads: review: https://github.com/parrot/parrot/commit/d42dc2a75f
18:19 nine whiteknight: the pmc it creates is the alarm. I can't see how it could work without that. You'd need one of those per task which kind of defeates the purpose.
18:20 whiteknight nine: so what if we store the alarm in the Task, and cache it so we aren't creating one each time we sleep?
18:20 whiteknight or, is that a solution which is only beneficial in the case of this synthetic benchmark?
18:21 nine whiteknight: that's what I think: we're talking about optimizing a very special case. And anyway "optimizing sleep performance" sounds kinda funny to me :)
18:21 NotFound To optimize sleep, buy a better bed.
18:22 whiteknight Parrot_cx_stop_task should take a third parameter, a flag which says whether we need to take a continuation or not
18:23 whiteknight We don't need to create a continuation, and we don't need to keep the task if we don't expect it to ever be used again
18:23 tadzik moritz: well, it shouldn't go powersave if the power cable's in, and I never do anything Rakudo-y on battery :) but I'll probably try it next time, thanks
18:23 NotFound whiteknight: maybe better another function.
18:25 NotFound winxed examples/fly runs fine in the green_threads branch.
18:25 nine whiteknight: when would we expect the task to be not used again?
18:34 whiteknight nine: when the task completes
18:34 Coke oooh, I can finally sign up for google+. Too bad about the silly name policy, though.
18:35 nine whiteknight: when the task completex Parrot_cx_stop_task is not called. Parrot_cx_next_task simply returns to Parrot_cx_outer_runloop and the task is not put again into the queue
18:38 tadzik Coke: are you on Diaspora yet? :)
18:44 dukeleto we really need Google Code-In tasks
18:45 tadzik yep
18:45 dukeleto can somebody help me with that? I have absolutely no time for it
18:45 dukeleto and we will not be accepted in our current state
18:45 tadzik what are we aiming for? Tests? Does moving TT to GH count?
18:45 tadzik documentation maybe? There's been talking about that
18:45 whiteknight dukeleto: when's the deadline?
18:47 tadzik Alvis is the new docuchief, maybe he can look through it and figure out some LHF for GCI students?
18:52 Coke tadzik: nope.
18:52 Coke documentation /generation/ is tough. /reorganization/ would probably be a good task.
18:53 Coke I wonder, will we have tasks that can be handled by students who have never done any coding before?
18:53 dukeleto whiteknight: yesterday
18:53 Coke (I know the tasks themselves don't have to be coding)
18:53 dukeleto https://github.com/parrot/parrot​/wiki/Google-Code-In-Task-Ideas
18:53 whiteknight dukeleto: Well, that stinks
18:53 dukeleto everything is explained there
18:54 dukeleto whiteknight: i don't know, but i shouldn't have applied us with 0 tasks listed, but I did
18:54 whiteknight yeah, I'm working on that page now
18:54 dukeleto whiteknight: they will look at them soon, and if they look at them when there are no tasks => no bueno
18:54 dukeleto whiteknight: i applied us when i had the motivation and time to do it
18:59 dukeleto whiteknight: what is your melange id?
18:59 whiteknight wknight8111, I think
18:59 dukeleto msg particle i put you as backup admin for Google Code-In, just as a temporary measure. No worries.
18:59 aloha OK. I'll deliver the message.
19:00 dukeleto whiteknight: i can never remember that
19:01 dukeleto whiteknight: just made you backup admin, you cool with that?
19:01 whiteknight NO I DEMAND PRIMACY!
19:01 whiteknight actually, that's cool
19:11 whiteknight okay, we have 9 tasks
19:13 dukeleto whiteknight++
19:15 whiteknight dukeleto: I don't see how we are supposed to have 5 tasks for each category. We don't have much in the way of user interfaces or training, and we are a little sour on translations
19:15 whiteknight aloha: coverage
19:15 whiteknight aloha: coverage?
19:15 aloha whiteknight: coverage is http://cv.perl6.cz or http://tapir2.ro.vutbr.cz/cover/cover-results/
19:16 Coke I didn't realize the reason we had so many translation tasks before was that google was pushing them.
19:17 dukeleto the rules changed
19:17 cotto I don't think that was the case last year.
19:17 dukeleto there was no "must have 5 tasks in each category" rule last year
19:18 NotFound User interfaces tasks? Easy: write a module for: gtk+, qt, gnome, kde, motif... ;)
19:18 atrodo Ahh, motif...
19:26 tadzik do we really want Translation tasks? We got a few last year, but it doesn't seem like they were found really useful
19:26 tadzik maybe "promote Perl somewhere in your local language" could be assigned to that
19:28 cotto tadzik: we might do better focusing on one language
19:29 whiteknight 16 tasks. Anybody with any ideas would be welcome
19:31 benabik Comparing our own documentation with the Parrot VM wikibook?  Might be multiple tasks
19:32 whiteknight the wikibook is hella old
19:33 benabik but it was one of the first things I found for info back in the day
19:34 benabik Find places in the documentation that say things like "documentation needed", TODO, or are just blank?
19:37 bluescreen joined #parrot
19:45 soh_cah_toa joined #parrot
19:51 whiteknight okay, I've put in all the tasks I can do right now. I can put in more tonight
20:05 ambs joined #parrot
20:15 tadzik "Outreach" section, how about "promote parrot $somewhere"?
20:15 whiteknight unfortunately, our code coverage is generally pretty high for most important files, and we don't have tools to measure coverage of PIR/Rakudo/HLL files yet
20:18 dukeleto tadzik++ # make promo posters, stickers, give talks about parrot at a local tech user group
20:18 NotFound Funny fact: I discovered an unused variable in winxed stage 1 in a C++ warning while testing winxedxx
20:18 dukeleto NotFound: why funny?
20:19 dukeleto NotFound: did the variable tell a good joke? ;)
20:19 NotFound dukeleto: I've not expected such usefulness
20:20 NotFound Using a C++ compiler to find pitfalls in winxed sources sounds funny to me.
20:20 NotFound Try to do that with python ;)
20:22 tadzik dukeleto: should I add that?
20:22 tadzik nicely worded
20:23 tadzik ok, forgiveness > permission
20:24 dukeleto tadzik: don't ask permission! only forgiveness :)
20:25 NotFound "Promote Parrot in google, in order to stimulate them to give us generous donations"
20:25 dukeleto tadzik++
20:25 dukeleto NotFound: they already do. It's called GSoC and GCI
20:26 NotFound dukeleto: politicallty correct fix: "Promote Parrot in google, in order to stimulate them to give us even more generous donations"
20:28 NotFound User interface: "Write one"
20:32 tadzik dukeleto: did you file me as a mentor?
20:32 dukeleto tadzik: mentors can volunteer if/when we get accepted
20:32 tadzik ok
20:33 dukeleto the UI task can be create.parrot.org, which I already started hacking on
20:33 dukeleto to generate some new graphics and pretty CSS for it
20:39 cotto also, a clone of dddash.ep.io
20:43 alvis joined #parrot
20:51 alvis Hello, #parrot
20:51 tadzik hello alvis
20:51 alvis Just now got time to catch up. Fwiw, I agree with benabik:
20:52 benabik alvis: Whatnow?
20:52 alvis Wikibooks srsly burned away hours of my time on useless, out-of-date stuff.
20:52 benabik Ahhh...
20:57 alvis Btw, does anyone want to further respond to my post on parrot-dev before I reply?
20:58 alvis I wanna give folks sufficient time.
21:00 alvis NotFound: Game 6 was postponed, so I got pine up and running instead. :)
21:01 NotFound Game 6?
21:01 alvis NotFound: Sorry, the World Series.
21:02 NotFound alvis: I don't follow any USA sport
21:02 NotFound Almost any sport, in general.
21:04 alvis NotFound: I understand. Lotta folks don't.
21:09 dukeleto alvis: i don't think you need to wait for any more replies
21:09 dukeleto alvis: people may add more suggestions, but run with what you have now
21:10 alvis dukeleto: agreed & ok
21:10 dukeleto can somebody add this as a task? https://github.com/letolabs/app-parrot-create
21:10 dukeleto 1 task is to finish the dancer app that runs it (coding) another is a UI task to create a cool logo and css for the website
21:10 dukeleto tadzik: ^^^
21:10 tadzik sure
21:11 dukeleto tadzik: also, add a task (coding) that is adding tests + implementing functions in https://github.com/letolabs/libgit2
21:12 dukeleto tadzik: just write a sentence like "since this is a large amount of potential work, it will be broken up into 10-20 tasks that are smaller chunks of work"
21:13 dukeleto tadzik: blarg! wrong link i gave you
21:13 dukeleto tadzik: https://github.com/letolabs/parrot-libgit2
21:15 tadzik dukeleto: done
21:34 dukeleto tadzik: we need to emulate structure here: http://wiki.wesnoth.org/GCI
21:35 dukeleto tadzik: description, requirements, expected time, difficulty
21:35 dukeleto tadzik: and deliverable
21:35 dukeleto tadzik: like the task template. if you have time, could you please expand some of the task ideas?
21:51 NotFound Possible task: write a task editor for next year GCI ;)
21:52 benabik HEh.
22:12 jsut joined #parrot
22:15 dukeleto https://github.com/leto/gci
22:15 dukeleto i made a repo of all the tasks i wrote for last years gci
22:15 dukeleto feel free to use it
22:15 rfw joined #parrot
22:15 dukeleto rfw: oh noes!
22:17 rfw hi dukeleto
22:18 rfw i've been lurking around here for the last few months :)
22:22 tadzik dukeleto: sorry, ran short of tuits, will have some tomorrow
22:43 cotto https://github.com/andreasgal/pdf.js/pull/603
22:43 cotto I want one of those.
22:54 whiteknight joined #parrot
22:55 whiteknight good evening, #parrot
23:19 jsut_ joined #parrot

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced