Camelia, the Perl 6 bug

IRC log for #parrot, 2011-09-07

Parrot | source cross referenced

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:50 preflex joined #parrot
01:14 kid51 joined #parrot
01:30 woosley joined #parrot
01:43 dalek Heuristic branch merge: pushed 26 commits to parrot/whiteknight/kill_threads by Whiteknight
01:52 plobsing_ NotFound: why not allow tailcalls from try blocks?
02:21 benabik joined #parrot
02:57 benabik joined #parrot
03:30 particle1 joined #parrot
04:06 nbrown joined #parrot
05:25 cotto ~~
05:45 dalek parrot: 37f7be7 | dukeleto++ | t/README.pod:
05:45 dalek parrot: Mention the allhlltest target in our test readme
05:45 dalek parrot: review: https://github.com/parrot/parrot/commit/37f7be70c5
06:30 moritz msg whitenight FYI rakudo tests fine on the kill_threads branch
06:30 aloha OK. I'll deliver the message.
06:31 cotto moritz, that's what I was pinging you about in #perl6
06:31 cotto heh
06:32 moritz (testing rakudo on a branch is no magic. One can just say  perl Configure --gen-parrot=$branchname; make spectest  in rakudo and wait an hour)
06:32 cotto doing so now
06:33 cotto just comparing that against master
06:46 JimmyZ joined #parrot
06:51 rfw joined #parrot
06:55 rfw joined #parrot
07:00 mj41 joined #parrot
07:27 woosley joined #parrot
08:02 dalek parrot: dbfea5d | jimmy++ | src/gc/fixed_allocator. (2 files):
08:02 dalek parrot: revert s/Pool_Allocator_Free_List/Pool_Allocator_Arena/, since not_gerd said it's different, or rename them to Pool_Allocater_List, I'm not sure it's ok or not
08:02 dalek parrot: review: https://github.com/parrot/parrot/commit/dbfea5d7ea
08:02 dalek parrot: b9d7d56 | jimmy++ | / (11 files):
08:02 dalek parrot: Merge branch 'master' of github.com:parrot/parrot
08:02 dalek parrot: review: https://github.com/parrot/parrot/commit/b9d7d56acf
08:06 woosley1 joined #parrot
08:06 woosley1 left #parrot
08:09 dodathome joined #parrot
08:21 JimmyZ_ joined #parrot
08:30 mls morning!
08:35 tadzik good morning #parrot
08:39 JimmyZ moring! mls
08:39 JimmyZ morning! mls
09:11 jsut_ joined #parrot
11:12 nbrown_ joined #parrot
11:34 mls anybody online with imcc knowledge?
11:34 mls the debug segemtn data seems to be wrong
11:35 mls suggested patch: https://gist.github.com/1200337
11:39 Psyche^ joined #parrot
11:43 JimmyZ mls: plobsing_ or whiteknight
11:43 mls thanks!
11:45 JimmyZ mls: pull a requrest is better , and  your patch can't be easily lost since they are not here
11:48 benabik joined #parrot
11:48 benabik o/ #parrot
11:52 benabik_ joined #parrot
12:03 whiteknight joined #parrot
12:10 bluescreen joined #parrot
12:11 benabik joined #parrot
12:13 whiteknight good morning, #parrot
12:13 benabik_ joined #parrot
12:13 mls Hi whiteknight!
12:14 mls the data in the debug section seems to be wrong
12:14 atrodo joined #parrot
12:14 mls patch: https://gist.github.com/1200337
12:15 mls (feel free to change the "if" statement if you don't like the empty block)
12:19 whiteknight mls: debug section of the packfile?
12:19 mls yes
12:20 whiteknight do you have an example of how they are wrong?
12:20 mls they contain spurious zeros
12:21 mls and the sub data starts at the wrong offset if the file contains more than one sub
12:21 mls (thus the zeros between the subs)
12:22 mls the data in the debug section maps op number -> line
12:23 mls imcc counts annotations as op, thus the misplaced entries
12:23 jnthn This could explain a lot of rather off line numbers...
12:23 kid51 joined #parrot
12:23 jnthn mls++
12:23 mls jnthn: who/what uses the debug data?
12:24 jnthn mls: When there's a runtime error and we need to display a line number in the PIR backtrace, it uses it.
12:24 mls ah yes, could be.
12:24 jnthn mls: And they don't always do too well on accuracy.
12:25 jnthn The HLL annotations tend to do much better.
12:25 mls note that files without annotations have correct debug data
12:26 jnthn oh!
12:26 jnthn :)
12:26 kid51 mls: The parrot_debugger has long been considered dysfunctional.  We recently had a GSOC project to replace it, but even that ran into problems dealing with IMCC.
12:28 kid51 afk
12:30 whiteknight mls: Can you put together anything like a test case that shows how the system was broke and how that patch fixes it?
12:30 whiteknight I have no problems merging it, I just want to see how it makes things better and by how much
12:30 mls sure
12:35 mls run this pir program: https://gist.github.com/1200447
12:36 mls without patch: current instr.: 'bar' pc 5 (/suse/mls/foo.pir:0) (foo.pir:3)
12:36 whiteknight that second .annotate is on line 4, but says line 3
12:36 whiteknight (not sure if that matters)
12:36 mls with patch: current instr.: 'bar' pc 5 (/suse/mls/foo.pir:9) (foo.pir:3)
12:36 whiteknight I have to fire up my VM. give me a few minutes
12:37 whiteknight that's pretty helpful
12:38 mls if ins->opsize is 0 for annotations, you might want to change the patch to
12:38 mls if (ins->opname && *ins->opname && !STREQ(ins->opname, ".annotate"))
12:39 mls that's probably cleaner than the empty block
12:41 whiteknight I'm not accustomed to using words like "cleaner" when dealing with IMCC
12:41 mls lol
12:44 mls success! HLL profiling works!
12:44 benabik ...
12:44 benabik Really?
12:44 benabik mls++
12:44 benabik mls++
12:45 mls I'm just looking at the profile of the rakudo setting compilation
12:46 whiteknight I'm going to do a little bit more cleanup here, while I'm staring at it
12:47 mls IMCC needs cleanup? How come...
12:49 mls next tasks for the profiler: 1) implement op profiling  2) split into three runloops  3) cleanup the code
12:50 jnthn mls: Would love to see the profile of the setting compilation :)
12:50 mls uploading...
12:54 mls http://www4.informatik.uni-erlange​n.de/~mlschroe/setting_prof.out.gz
12:58 mls (could be that the tick percentages are a bit off, as I did some work while the setting compiled)
12:59 jnthn thanks, I look when I get home from $dayjob
13:00 dalek parrot: 00fe23d | Whiteknight++ | compilers/imcc/pbc.c:
13:00 dalek parrot: Don't increment line numbers on .annotate directives. This fixes some line number disparities and places where the line number is reported as 0. mls++
13:00 dalek parrot: review: https://github.com/parrot/parrot/commit/00fe23d2b6
13:11 JimmyZ joined #parrot
13:13 dalek parrot: dd8108e | Whiteknight++ | compilers/imcc/pbc.c:
13:13 dalek parrot: Some cleanups in the get_code_size function. De-nest, don't test the same things twice, and mark some values as constant in the loop body
13:13 dalek parrot: review: https://github.com/parrot/parrot/commit/dd8108ef15
13:18 dalek parrot: 3f0c008 | NotFound++ | src/pmc/scheduler.pmc:
13:18 dalek parrot: use a hash keyed by int in Schecduler task_list to avoid string-int conversions
13:18 dalek parrot: review: https://github.com/parrot/parrot/commit/3f0c0084e8
13:26 mtk joined #parrot
13:30 mtk joined #parrot
13:30 tadzik atrodo: could isparrotfastyet possibly measure Rakudo performance on the nom branch?
13:35 dalek parrot: 5a7d8a5 | NotFound++ | src/pmc/timer.pmc:
13:35 dalek parrot: delegate most of Timer init to its super and auto_attrs
13:35 dalek parrot: review: https://github.com/parrot/parrot/commit/5a7d8a5302
13:48 whiteknight msg pmichaud I updated https://gist.github.com/1198521. For each item I added a section for "upgrade effort for HLLs" and "Benefits for HLLs". These estimates are base-line. For most items we can talk about trading more disruption for more features (or working on a phase-in schedule)
13:48 aloha OK. I'll deliver the message.
13:48 whiteknight msg kid51: https://gist.github.com/1198521
13:48 aloha OK. I'll deliver the message.
13:55 dalek parrot: d588fc6 | NotFound++ | / (3 files):
13:55 dalek parrot: delete unused Scheduler cache_valid private PObj flag
13:55 dalek parrot: review: https://github.com/parrot/parrot/commit/d588fc6fdd
14:02 dalek parrot: 4f1d005 | NotFound++ | src/pmc/scheduler.pmc:
14:02 dalek parrot: delete unused Scheduler 'pending' attribute
14:02 dalek parrot: review: https://github.com/parrot/parrot/commit/4f1d005e5c
14:06 tadzik whiteknight: if I can say, I really like the gist (:
14:06 whiteknight thanks
14:08 NotFound I'm not sure I like the idea of lookup exception handler.
14:11 NotFound For uses more compilcated that "handle it or let it pass to next level" continuations should be more appropiate.
14:13 whiteknight right, but you have to find something to handle it
14:13 whiteknight and the user should be able to do the finding
14:13 PerlJam whiteknight: there seem to be some unmentioned mild dependency relationships or at least "X should happen before Y".
14:14 whiteknight PerlJam: the whole thing is in quasi-chronological order. Things towards the bottom might depend on things towards the top
14:14 NotFound whiteknight: Who is "the user"? The thrower? The handler? The exception?
14:14 whiteknight NotFound: the HLL dev
14:14 PerlJam whiteknight: gotcha
14:14 PerlJam whiteknight: oh, you even say that in the text.
14:14 * PerlJam obviously can't read
14:15 NotFound whiteknight: ugh... that may lead to yet another problem for language interoperability.
14:15 whiteknight it's okay. There is a lot of information there
14:15 whiteknight NotFound: how? HLLs can subclass CallContext. CallContext does lookup on exception handlers
14:16 whiteknight When I call CallContext.get_exception_handler(ex), it looks up the chain until a handler is found
14:16 whiteknight and if no handler is found, we exit with an unhandled exception
14:16 NotFound whiteknight: so if I call a sub in some HLL my unconditional exception handler can be bypassed?
14:17 whiteknight NotFound: if you call a sub in some HLL and it throws an exception, the HLL can handle it before returning to you
14:17 whiteknight or, if can call up the chain into your context
14:17 whiteknight it
14:19 NotFound whiteknight: if it does its own lookup and dispatch, it can dispatch to a previous level in the chain.
14:19 whiteknight NotFound: maybe. I think that's unsafe but you can do that
14:20 whiteknight if you can subclass CallContext, you can do the lookup anywhere you want. Or if you modify the call chain, you can look up anywhere you want. You can do that now
14:22 NotFound Yes, I can write the general assumptions in a lot of ways, but that is no reason to make them easier to break, even inadvertently.
14:22 NotFound s/write/break
14:23 NotFound My main question is: there are real use cases for that feature?
14:23 whiteknight maybe not. I think so. It's just an idea right now
14:25 whiteknight I don't like Parrot doing magic dispatch. The HLL knows the rules about which handler is "best"
14:25 NotFound I think some way of automatic dispatch between handlers in the same context can be helpful, but doing it between different levels can be error prone and hard to debug.
14:26 NotFound And a serious barrier to interoperability.
14:29 whiteknight I don't think it will be so bad
14:29 NotFound if (interp->scheduler) { ... } else if (interp->scheduler) .... WTF?
14:30 whiteknight Another thing it could do is inline exception dispatch, so the JIT can optimize it
14:30 NotFound src/scheduler.c:411
14:30 whiteknight awesome
14:30 * tadzik chuckles
14:31 whiteknight when I talk about parrot having a lot of technical debt to pay down, and lots of ugly code to clean, I'm not lying
14:34 * JimmyZ agrees
14:34 benabik Random thought: We may want something akin to scala's PartialFunction…  a MultiSub that we can ask "can you dispatch with these arguments?"  That could be useful for exception handlers if dispatch becomes based on multis…  for(eh in handlers) { if(eh.can(ex)) { eh(ex); break; } }
14:39 whiteknight benabik: yes, I like that idea
14:42 benabik Although adding something like .can to MultiSub in general isn't a bad plan…  It would be nice to not have to use exceptions to discover a call won't dispatch.  Our current ExceptionHandler could be a subclass of MultiSub that adds logic for checking severity and type to .can()
14:42 * benabik is handwaving...
14:44 plobsing_ not all exception dispatch is type-based. Ωη wants identity-based exception dispatch. Perl 6 exceptions can't dispatch based on type until they are in the handler.
14:46 plobsing_ so please do not make that case any slower
14:47 dalek rakudo/nom: fc57d1c | moritz++ | src/Perl6/Metamodel/ParrotInterop.pm:
14:47 dalek rakudo/nom: fix line endings in ParrotInterop.pm; no functional changes
14:47 dalek rakudo/nom: review: https://github.com/rakudo/rakudo/commit/fc57d1c4d4
14:48 atrodo tadzik> In theory, yes
14:48 NotFound Creating multis and adding elements to them for each try level will not be cheap.
14:49 NotFound Also, removing them while when descending levels will not be trivial.
14:49 benabik The `eh(ex) if eh.can(ex)` bit would allow the handler system to be more general than just multis.  I was commenting based on the "Expose ExceptionHandler lookup and dispatch" bit.
14:49 whiteknight plobsing_: that's the thing. Different projects need different types of exception dispatch
14:49 whiteknight the "Parrot has the single magic solution" way of thinking doesn't work
14:50 benabik I don't want to say that current MMD is a good idea in any-way-shape-or-form for exceptions.  :-/
14:50 plobsing_ oh, I agree. I just don't want us falling into the assumption that all exception dispatch is type-based
14:50 whiteknight NotFound: Creating multis for each will not be cheap, no. But if we treat the context as the multi, we already have it. The context can dispatch based on signature
14:51 whiteknight plobsing_: if we have a way to manually insert handlers and manually invoke them, they can be anything we want: continuations, Subs, MultiSubs, etc
14:52 NotFound I think that the beauty of the exception model is its simplicity: you get an exception, look at it and decide if you handle it or let it pass to the next level.
14:52 plobsing_ NotFound: there's a catch to that - you end up with a huge stack trace indicating all the exception handlers that decided they *weren't* relevant to the exception
14:53 NotFound plobsing_: there is no need to do that.
14:53 plobsing_ although, in general, I like the rethrow model
14:54 NotFound And you don't need to rethrow to do that, you can use the 'can_handle' method.
14:54 plobsing_ why make two calls when we can make one?
14:55 NotFound Because we don't have a rethrow variant that doesn't add things to the backtraces.
14:55 plobsing_ can_handle jumps into eh code. why not just have that code as a preamble on the exception
14:56 plobsing_ NotFound: don't have one *yet*
14:56 NotFound +1 to add it.
14:56 plobsing_ s/preamble on the exception/preamble on the exception handler body?/
14:57 whiteknight if you don't want added backtraces from rethrow, don't look at them. They're separated out in an array
14:57 whiteknight if you only care about the first level, only look at that level
14:57 plobsing_ whiteknight: except that *users* shouldn't need to know about them
14:57 whiteknight having the information separate but available is not a problem
14:58 whiteknight plobsing_: and does it hurt them to know if it they explicitly go looking?
14:58 whiteknight it if*
14:58 whiteknight users don't need to do anything. They don't need to look at backtraces[1]
14:58 plobsing_ we're printing it, implying that they should look at it
14:59 whiteknight we don't need to print it, if that's the problem
14:59 NotFound whiteknight: the problem is that it forces a difference between use of can_handle and us of catch/rethrow to decide if to pass the exception.
15:00 NotFound whiteknight: the level that should make that decision has no acess to the information needed to know what's the case.
15:00 whiteknight I don't think I understand that
15:01 NotFound whiteknight: I can set up a catch all handler, in it look at the code, severity, type... whatever, and rethrow if I decide I shouldn't handle it.
15:02 NotFound And there is no reason to put more info in the backtrace in this case than if I were using the can_handle method.
15:03 whiteknight okay, I see what you are saying. you want to keep the backtrace from before the rethrow, not the one after it?
15:03 NotFound Yes, the ability to decide what to do.
15:04 whiteknight All the more reason not to use magic throw/rethrow ops.
15:04 whiteknight create an exception. Ask the CallContext for a suitable handler, and call handler(ex)
15:04 whiteknight that way you can create and manipulate the exception as you see fit without magic
15:05 mls (for rakudo, we need to catch all non-CONTROL exceptions and run perl6 code to decide if we need to rethrow or not)
15:06 NotFound whiteknight: Crate a new exception in the handler?
15:06 whiteknight NotFound: You could. Or, you could pass on the original exception without updating the backtrace
15:06 whiteknight once you create it and it has a backtrace, you don't need to modify it anymore
15:07 whiteknight or, you can modify it if you want
15:08 benabik When do backtraces get added to exceptions?
15:08 benabik Creation?  Throw op?
15:08 whiteknight in the throw op, currently
15:08 whiteknight I think
15:08 whiteknight the backtrace gets added when we associate the exception with a continuation, I think. It's the backtrace of the resume continuation
15:09 mls no
15:09 whiteknight I may be mis-remembering it
15:09 mls continuation is extra, U think
15:09 mls U -> I
15:09 mls die_ii used to have no continuation
15:10 benabik mls: It does now, identical to die_s.  But those are exceptions created in C.
15:10 NotFound whiteknight: I don't mind if is an opcode, a method in the context or whatever, but having opcodes for the most common usages will be good for code size.
15:10 benabik Keep the simple simple, make the complex possible.
15:11 whiteknight https://github.com/parrot/parro​t/blob/master/src/debug.c#L3575
15:11 whiteknight it's the exception context, not the exception continuation
15:11 mls ah, the backtrace method uses wither the continuation or the "thrower", i.e. the context at throw time
15:13 whiteknight right
15:13 aloha joined #parrot
15:15 dalek parrot: fb7055f | NotFound++ | src/scheduler.c:
15:15 dalek parrot: delete nonsensical wrong check in cx_delete_task
15:15 dalek parrot: review: https://github.com/parrot/parrot/commit/fb7055f2e5
15:20 plobsing_ if the exception handler is called unconditionally (and rethrows if not applicable), we wouldn't need to manage arrays of hanlders. the handler list would be implicit in the linked-list of contexts
15:21 whiteknight that represents a huge change at the PIR level
15:21 whiteknight not that we can't do it, but we have to weigh the effort against the benefits
15:22 NotFound The handler list is in the list of context, but its elements are lists.
15:22 plobsing_ not necessarily. the old ExceptionHandler could have a special invoke vtable. or dispatch to some common built-in opcodes
15:22 bacek_at_work joined #parrot
15:22 whiteknight at the moment, exception handlers are just continuations pointing to labels. They don't have a defined "end" point
15:23 plobsing_ why would they need that?
15:24 NotFound A sub can have several try blocks, separated, nested, or both.
15:25 whiteknight plobsing_: how else do you rethrow if needed, if the PIR code doesn't change?
15:26 whiteknight I'm not making it past the hand waving here
15:26 plobsing_ whiteknight: the only time an ExceptionHandler (as we currently know it) would need to rethrow implicitly under the new model is to handle the type-check functionality
15:26 plobsing_ that happens up-front, not at the end
15:27 whiteknight right, but if we pass the exception to every handler, and expect the handlers to rethrow if it doesn't pan out, that means the code of every user handler needs to be changed
15:27 plobsing_ NotFound: admitedly, that is a problem. but the HLL compiler knows about this (it is static information), and can deal with it appropriately
15:28 benabik whiteknight: If ExceptionHandler does the check/rethrow prior to handing off to the user code, I don't think we'd need to change user code.
15:28 plobsing_ whiteknight: no it doesn't. what I am suggesting is that ExceptionHandler be kept around for compat. it would jump to its own code first to do typechecks.
15:28 NotFound plobsing_: yes, but a way that does not complicate too much code generation will be nice.
15:29 plobsing_ NotFound: it shouldn't be too complicated
15:32 dalek parrot: 2541b9c | NotFound++ | t/op/exceptions.t:
15:32 dalek parrot: declare external class to avoid a warning
15:32 dalek parrot: review: https://github.com/parrot/parrot/commit/2541b9c9f0
15:33 plobsing_ NotFound: nested exception-handlers aren't that common, are they?
15:34 whiteknight exception handlers are typically a spaghetti mess of labels. Who knows how they are commonly arranged?
15:34 plobsing_ whiteknight: I'm talking at the HLL-level, which is typically structured
15:34 NotFound plobsing_: maybe not, but with the current way I don't need to care about that in winxed, the code generation is the same.
15:37 plobsing_ NotFound: this is true. but I'll posit it is worthwhile. not managing a list of eh PMCs means not doing vtable calls on an RPA of eh PMCs anytime we want to push, pop, or call exception handlers
15:38 darbelo joined #parrot
15:38 plobsing_ pointer follows are much cheaper than vtable calls
15:39 NotFound A multiple ExceptionHandler that manages a list of start and end of try blocks, and maybe its corresponding handling labels, can be doable.
15:39 NotFound But that probably doesn't decrement too much the number of gcables.
15:40 plobsing_ NotFound: I'd say it might. if the block gets called a lot, 1 gcable vs 3 is much better.
15:40 NotFound Maybe we can create it at compile time and make it constant.
15:40 plobsing_ NotFound: that doesn't handle recursion
15:41 plobsing_ hmmm... maybe it can. interesting idea.
15:41 NotFound plobsing_: the try/handling date can be constant, and added to the EH continuation at runtime.
15:41 NotFound s/date/data
15:43 NotFound Or it can be associated to the sub, and the EH look for it from the context.
15:44 plobsing_ that could work too
15:44 plobsing_ so this is a solvable issue. are there any other issues with this approach?
15:47 NotFound Also, the pushing and handling of that kind of handler can be treated diferently of the current EH, avoiding the list iteration, and both ways can coexist.
15:47 NotFound (provided that using both in the same Sub will not be supported)
15:51 NotFound Someone is depending on the precision of Timer for something?
15:53 NotFound I'm going to change it to avoid cloning on repeating, but the change will have some impact.
15:57 plobsing_ NotFound: not using both in the same sub doesn't fall under dep policy (since one of the things is new)
15:57 NotFound plobsing_: yes, as long as we don't change the old way we are safe in that regard.
15:57 plobsing_ or the replacement (since nothing can have been using new features
15:58 NotFound We can schedule for deprecation the old way later.
16:00 whiteknight I would be very surprised if people were using Timer much. There are some obvious bugs in that system that I found that nobody else had ever noticed
16:00 whiteknight besides the few tests, I can't imagine Timers are used for much
16:01 NotFound The cloning is one of the bugs. You can't stop a Timer after its first iteration because the PMC you have is no more the active one.
16:02 whiteknight right
16:02 whiteknight better question would be to ask if people use timers. If not, maybe delete them
16:03 whiteknight we'll be able to do it better when we have threads and green threads
16:03 NotFound Maybe, but will be easier to clean up the scheduler if it becomes a bit less labyrinth.
16:04 whiteknight I don't care either way. Timers aren't doing any harm, they're just ugly and unusable
16:05 whiteknight minimally usable
16:07 darbelo_ joined #parrot
16:07 dalek parrot: be307fc | NotFound++ | src/scheduler.c:
16:07 dalek parrot: change handling of timers in the scheduler to avoid cloning on repeating.
16:07 dalek parrot: That may affect its precision, but the docs already says that there are
16:07 dalek parrot: no guarantees about it.
16:07 dalek parrot: review: https://github.com/parrot/parrot/commit/be307fcf7b
16:08 NotFound Done. Feel free to revert if somenone gets upset.
16:17 davidfetter joined #parrot
16:34 mls subprof now supports subprof_sub, subprof_hll, subprof_ops
16:35 * JimmyZ couldn't anything more, except mls++
16:36 mls (spoke too soon, it segfaults in some cases. debugging...)
16:47 darbelo joined #parrot
16:48 dalek parrot: 60b854a | NotFound++ | t/pmc/timer.t:
16:48 dalek parrot: test stop a repeating Timer
16:48 dalek parrot: review: https://github.com/parrot/parrot/commit/60b854aed2
16:58 mls hmm, seems like Parrot_sub_get_line_from_pc can go into an endless loop
16:59 mls or it's too slow for my purposes
16:59 mls testing...
17:00 mls probably too slow... rewriting...
17:23 plobsing_ mls: linear scan is too slow. plus ADD_OP_VAR_PART is stupid slow
17:23 mls and I called it for every op...
17:24 plobsing_ mls: rewritting your code, or rewriting the scan?
17:24 mls my code
17:26 plobsing_ no crazy good patches for parrot? boo!
17:27 plobsing_ although I'm not sure the code as it stands is ammenable to improvement
17:27 plobsing_ variadic ops make anything beyond linear scan a pipe dream
17:28 mls I don't think it's such a critical function. I just misused it
17:28 mls anyway, seems like all three profiling types work
17:29 mls enough hacking for today
17:29 plobsing_ that's the way the debug segment is intended to be used. if the intended usage is inefficient, we need to change it.
17:30 plobsing_ perhaps we could move that info to the annotations segment and drop the debug segment
17:30 plobsing_ annotation lookup is O(log n)
17:30 plobsing_ not O(n)
17:30 mls I needed the line info for every op in the sub, that's why it was so slow
17:31 mls it's not meant to be called that often
17:31 mls (or that way)
17:31 plobsing_ mls: there's only one other place in the code that uses that function - the profiling runcore
17:31 plobsing_ it had the same perf problem, so it hacked around it by cacheing the pc
17:31 mls Isn't it also used for backtrace generation?
17:32 plobsing_ apparently not
17:32 plobsing_ I acked the entire source
17:32 mls I think there's a "get context info" function which has a copy of the code
17:32 mls searching...
17:33 plobsing_ yeah, I'm seeing that too. we should probably unify those.
17:34 plobsing_ the offending function is src/sub.c:Parrot_sub_context_get_info
17:34 mls ah yes. similar but different ;)
17:36 plobsing_ mls: we can make this function immediately faster by only scanning from the beginning of the sub rather than the beginning of the seg. that should give you a nice perf boost. possibly enough to avoid the rewrite
17:36 plobsing_ if you are profiling rakudo, the segment is rather large, but the subs are usually fairly small
17:37 mls yes, that should help a bit. Thanks.
17:38 mls do we know the "debug offset" of the beginning of a sub?
17:38 plobsing_ sub->start_offs
17:39 mls that's the opcode offset, not the "debug offset"
17:39 plobsing_ oh shoot, no. you're right. we don't know how many ops come before.
17:39 plobsing_ wow this datastructure sucks
17:39 mls ;)
17:40 mls It's optimized for size, not speed ;)
17:40 plobsing_ mls: if you've ever looked at the code managing packfiles/pbc, you'd know size was not a top priority.
17:40 plobsing_ at least we dropped the 'fixup' segment.
17:41 mls oh well.
17:41 NotFound I'm going to trademark "Optimized for hysterical laughs"
17:41 mls anyway, I gotta go home. talk to you tomorrow.
17:41 plobsing_ hysterical? more like manic.
17:47 contingencyplan joined #parrot
17:53 cotto_work ~~
18:07 preflex joined #parrot
18:09 benabik o/
18:16 cotto_work hi benabik
18:20 bacek_at_work joined #parrot
18:22 pmichaud_ whiteknight: I've updated your gist with notes about benefit/impact to Rakudo: https://gist.github.com/1201282
18:24 cotto_work I notice a pattern.
18:24 whiteknight pmichaud_: the changes to Rakudo for the proposed PCC refactors is nil. I've already considered your custom binding code
18:25 pmichaud_ whiteknight: the nom code or the one in master?
18:25 whiteknight pmichaud_: either
18:25 pmichaud_ either way, I'll accept it as Nil and let jnthn say if it's otherwise
18:25 whiteknight okay
18:26 whiteknight I'll work with him to make sure it stays nil
18:26 aloha joined #parrot
18:26 pmichaud_ I don't care if the impact is there, though.
18:26 pmichaud_ improving invoke performance far outweighs the need for compatibility
18:26 pmichaud_ I can't express how much "speed" concerns outweigh any other concern we might have.
18:26 whiteknight okay
18:27 pmichaud_ updated impact cost to be "nil" for that one.
18:27 whiteknight almost all the changes I've proposed in that gist are internal-only. C level architecture and APIs. Not that we can't or won't change PIR (it certainly has plenty of problems), but the items in this gist aren't targeted towards it
18:28 whiteknight C-level compatibility problems have been by far the most hindering
18:29 dmalcolm joined #parrot
18:29 whiteknight pmichaud_: It's funny that you say sandboxing has zero priority. I got started thinking about it because I was specifically asked to by rakudo devs
18:29 whiteknight it is on request from rakudo devs that it even appears on the list
18:30 pmichaud_ let me check
18:32 whiteknight I think it was the eval bot, and the ability to sandbox executing scripts to not be able to monkey with certain directories
18:32 whiteknight or, that was the specific example mentioned
18:32 pmichaud_ yeah, but I'm not sure that makes it a high priority for us.  I'm polling #perl6 now.
18:33 pmichaud_ actually, I'm certain that it's *not* a high priority for us.
18:33 plobsing_ for specific platforms like that, can't you just set up something with OS sandboxing?
18:33 jnthn "Far better interoperability between 6model and Class/Object for systems that want to use both." - urgh...the idea was to kill Class/Object? But also, if you try and do anything to 6model to make it look more like those, I'd rather you didn't put it in core Parrot...
18:34 whiteknight jnthn: "kill" is a relative word. They will die from the inside and be replaced with shiney 6model-based guts
18:34 whiteknight Object and SixModelObject are going to become one
18:34 jnthn Well, if you're going to do 6model right, PMCs need to be subsumed by it too.
18:35 whiteknight Class may hang around for backwards compatibility, be repurposed, or be deleted eventually
18:35 pmichaud_ updated.
18:35 whiteknight jnthn: right. There is a lot of work to make it right
18:35 jnthn And if you're not going to do that, I don't think there's so much win to be had by bringing it core.
18:35 whiteknight jnthn: the win for Rakudo is slightly different from the win for Parrot
18:36 jnthn True.
18:36 whiteknight Parrot gets to stop having a pile of garbage as its object model
18:36 jsut joined #parrot
18:36 whiteknight If we can make win for Rakudo in the process, all the better
18:36 jnthn AIO - yes, plese. :)
18:36 jnthn *please
18:37 whiteknight Ultimately, the changes we're going to need (vtables especially) are going to be pretty big and disruptive. Most of what Parrot has is going to need to be replaced
18:37 jnthn "We need to stop referring to Class objects by string name"
18:37 jnthn +1.
18:37 whiteknight so the more Rakudo and NQP use 6model features and the less it uses other stuff, the better
18:37 jnthn 6model core doesn't actually know how to do string isa.
18:37 whiteknight jnthn: and the rest of the system won't for much longer either
18:38 jnthn Yeah, though it's quite endemic in Parrot at the moment.
18:38 NotFound Changing the class model entirely and keeping backwards compatibility high looks a bit contradictory to me.
18:38 jnthn Yeah, that ain't gonna happen.
18:38 whiteknight NotFound: We're not keeping a lot of backwards compatbility
18:38 whiteknight at least not for the object model, Object/Class, and related stuff
18:38 whiteknight the class registry
18:39 whiteknight So long as Rakudo isn't using any of that, we're much more inclined to throw the baby out with the bathwater
18:39 whiteknight if Rakudo were using it, we would have to walk on eggshells
18:39 NotFound whiteknight: I doubt that we can change the Class model in one shoot without breaking lots of things, even for rakudo.
18:40 jnthn Well, once NQP and the stack it's atop of is fully 6model based, it'll be less painful.
18:40 NotFound Once...
18:40 jnthn For now, yes, PAST and so forth are using Parrot classes since.
18:40 jnthn NotFound: That's very likely within the next couple of months.
18:40 jnthn NotFound: Because not doing that blocks other things that matter to us on the Rakudo side.
18:41 jnthn (e.g. inliing multi-dispatch candidates...)
18:43 pmichaud_ I need to grab some lunch -- will read backscroll a bit later
18:43 darbelo joined #parrot
18:43 chromatic joined #parrot
18:43 NotFound Anyway, I think that coexistence for a while will be far easier than One Big Change.
18:44 chromatic Would you be happier if Parrot merely provided a garbage collector and perhaps invoke?
18:45 NotFound Me? No.
18:46 chromatic I meant jnthn and pmichaud_.
18:46 chromatic Or if you're not a fan of synecdoche, Rakudo.
18:46 jnthn chromatic: Well, and something like the 6model core, and some kind of JIT, and threads, and some kind of concurrency primitives (like CAS, which you can build other stuff from)...and had some vague notion of a type system. I suspect Parrot's approach and what I fear a VM should look like these days are too far apart to really be reconciled.
18:46 jnthn gah, fail
18:46 benabik cotto_work: Sorry, got pulled off by work very very quickly after sitting down.  Hi!
18:46 jnthn that fear ended up in the wrong place :)
18:47 jnthn s/suspect/fear/
18:47 chromatic I see.
18:47 NotFound At least Camelia is always happy.
18:47 jnthn chromatic: If anything looks hopeful to me, it's m0.
18:48 chromatic I'm glad to hear that.
18:49 whiteknight the parrot approach in the past has not been the right one. Unfortunately, you can't turn this ship on a dime
18:49 chromatic Especially if Rakudo keeps saying "Don't do that!" to everything that will help turn the Parrot ship.
18:49 chromatic Sorry, I overspeak.
18:49 chromatic ALMOST everything.
18:50 whiteknight We're more nimble without the deprecation policy, but we still can't suddenly make things right. Making things right means breaking all the old code that relies on it being wrong
18:50 whiteknight and that's a misfortunately large amount of code
18:51 whiteknight And if we said "Stick with 3.6 for a few months. Parrot is going to disappear into a hole for a while and come back with something new and working correctly", the upgrade shock would be huge
18:52 cotto_work I'm much less apprehensive about the "keep Rakudo and other HLLs happy" than "don't break anything without proper notice"
18:53 NotFound Tell them to not be happy without a proper notice.
18:54 NotFound (if they see some aceptable break, I mean)
18:57 darbelo_ joined #parrot
18:59 alester joined #parrot
18:59 dukeleto Parrot has no focus. If we do a huge set of refactors without a new focus, we will get what we have always gotten.
19:00 chromatic Parrot has no focus because Rakudo won't allow it to have a focus.
19:00 dukeleto chromatic: i agree.
19:00 * dukeleto writes up some painful truths to parrot-dev
19:01 PerlJam chromatic: Are you speaking of now, or of the past?  or both?
19:01 chromatic Now.
19:01 chromatic Maybe the past, but I'm not interested in arguing about things past.
19:03 dukeleto The past is gone, and the more time Parrot and Rakudo spend bitterly complaining about it, the less relevant both projects become.
19:03 chromatic Yep.
19:04 PerlJam dukeleto++
19:04 plobsing_ PerlJam: it seems as though any time parrot takes the initiative, rakudo says "stop, we don't want this", or "we don't want this yet, so don't do it"
19:05 chromatic Yep.
19:05 chromatic And if you note the big speed improvements in Rakudo over the past week, they've all been from Rakudo generating better code because Rakudo knows better what it needs at the appropriate layer.
19:06 cotto_work dukeleto++
19:06 chromatic Modulo a GC tweak, which is still probably the wrong approach, but at least it's less wrong.
19:07 chromatic Parrot can't make Rakudo go fast when Rakudo generates inefficient code.
19:07 whiteknight we're going to try to move to a precise GC. I think we can do that relatively quickly
19:07 chromatic I have qualms about that, but it's worth testing.
19:07 PerlJam plobsing_: this is why I suggested on #ps yesterday that parrot devs write down all the things they want to do with parrot (for whatever reasons), and rakudo devs write down the things they want to do with rakudo and then get together to look for (or work out) some sort of alignment.
19:07 whiteknight which qualms?
19:08 PerlJam plobsing_: at least that way the impedance mismatch should be lessened.
19:08 chromatic As long as external projects have C code, they'll have to hew very carefully to the precision strategy to avoid leaving things where the GC can't find them (because they're anchored on a stack or present in a register).
19:09 plobsing_ chromatic: we already have that problem (or a very similar one) with generational gc
19:09 chromatic Saying "You need these barriers" is different from "Don't write code where an optimizing compiler will do the wrong thing."
19:09 plobsing_ fair enough
19:10 chromatic It's a matter of degree, but at least one you can analyze statically. Mostl.
19:10 whiteknight we've never made any promises that writing extensions in C won't be painful or difficult to maintain
19:11 chromatic Ha, what about the debate over whether the deprecation policy extended to struct member ordering?
19:11 plobsing_ I'd like to leverage some static analysis tool to be able to detect all the places in core that need to be fixed up. I see no reason why that tool cannot also be used by extenders.
19:11 whiteknight chromatic: historical and misguided
19:12 plobsing_ the deprecation policy lead to a lot of dysfunction
19:12 chromatic Yep, Parrot needs the right to be wrong.
19:12 whiteknight plobsing_: and it preserved a lot of preexisting dysfunction. I've never liked that policy
19:13 plobsing_ I'm talking cultural dysfunction, not just technical dysfunction
19:13 whiteknight ah yes, that too
19:13 plobsing_ that struct debate is one that should have never happened
19:14 dukeleto plobsing_: what is the "struct debate" ?
19:14 plobsing_ struct ordering under dep policy or not
19:14 whiteknight it's a weird dichotomy to be doing things like agile-ish weekly standup meetings, but then going around the room and nobody can work on anything important until the next supported release
19:14 atrodo chromatic++
19:14 whiteknight or people choose not to work on important tasks because of the deprecation headaches involved
19:15 whiteknight or people work on important things, and the branch has to sit for 5 months, and then it's almost too bitrotten to merge
19:15 whiteknight cutting it down to a 3 month wait was hardly the solution
19:16 chromatic Yes, but removing things without a viable replacement is not the solution either. The middle ground is important.
19:16 dalek winxed: 2db4057 | NotFound++ | winxedst1.winxed:
19:16 dalek winxed: fix class operator for mixed HLL case:
19:16 dalek winxed: it was giving the Namespace, not the Class
19:16 dalek winxed: review: https://github.com/NotFoun​d/winxed/commit/2db40572bd
19:16 benabik There's a reason Lua forces all the access to lua tables to be through the API is so that it can do memory management correctly.  Returning raw object pointers to external C code is fraught with danger.
19:17 benabik For ops, we can use opsc to munge PMC * variables to register them correctly…  But for extend/embed we need to be very careful.
19:17 benabik (re: precise GC)
19:17 whiteknight chromatic: We couldn't make viable replacements if they required changes to supported interfaces. And nobody looks at changes anyway until the day after a supported release when branches come crashing in
19:18 NotFound Returning raw pointers doesn't need to be bad, as long as they are really used as raw.
19:18 NotFound That is, just passing them to the appropiate functions.
19:19 atrodo NotFound> You mean treating them as opaque pointers?
19:19 NotFound atrodo: yes
19:19 benabik NotFound: opaque pointers that can't be trusted to continue to exist.
19:20 NotFound benabik: bad != not easy
19:21 benabik NotFound: If any C user stores them across a GC run without registering it, pop goes the segfault.
19:21 benabik NotFound: I'd call that bad.
19:21 NotFound benabik: then register it
19:23 NotFound Or register a Hash or a PMC Array and store all your objects here.
19:23 benabik NotFound: As I said, careful.  We can ensure it by registering any PMC we return through the API.  C users can get memory leaks if they don't release it, but that's better than segfault.
19:23 benabik NotFound: I'm not saying impossible, just that we have to be careful.
19:23 pmichaud_ back from lunch, briefly
19:25 NotFound benabik: people embeding ot extending in C should know what they are doing. No one can prevent every possible misuse.
19:26 pmichaud_ 19:07 <chromatic> Parrot can't make Rakudo go fast when Rakudo generates inefficient code.
19:26 NotFound We can provide API functions that doesn't take nor return GCables for a lot of cases, though.
19:26 pmichaud_ Rakudo generates more efficient code now *because it can*
19:26 pmichaud_ previously it couldn't do that because the underlying object model and other core components wouldn't allow it
19:28 pmichaud_ 19:10 <whiteknight> we've never made any promises that writing extensions in C won't be painful or difficult to maintain
19:29 benabik NotFound: Is GC stopped when we're in C?  If not, then every time you get a PMC* back from the API, there's a chance GC will eat it before you can register it.
19:29 pmichaud_ what's interesting is that Parrot's position has often seemed to be an expectation that HLLs would write their own custom PMCs, binders, etc. to provide the functionality Parrot didn't provide.  This is what Rakudo has done.
19:29 pmichaud_ So I don't think we can entirely blame Rakudo for having done what Parrot's design said it was intended to do.
19:30 whiteknight pmichaud_: that position predates me. I've never been one to claim that HLL devs need to write more stuff in C
19:30 pmichaud_ whiteknight: I guarantee that was Parrot's original design expectation, and it was carried through to the implementation of PCC
19:30 chromatic I'm fully comfortable blaming Rakudo for writing piles of awful code that pokes into Parrot's guts in bad ways and blaming Parrot for the mess.
19:31 pmichaud_ chromatic: Please identify at least one area where we've blamed Parrot for breakages when we've poked into Parrot guts.  You still haven't identified such an instance, and I wish you would stop bring up that strawman.
19:31 whiteknight pmichaud_: the PCC refactors I'm proposing now go in the other direction. Do less crap in C code, expose more capabilities to PIR code
19:31 NotFound benabik: unless we get a threaded GC, you can't get a GC run until you pass control to parrot.
19:32 chromatic Half a dozen memory leaks and segfaults I fixed in Rakudo. git log --author chromatic in Rakudo for a list.
19:32 pmichaud_ chromatic: but did we blame parrot for those?
19:32 chromatic Yep.
19:33 pmichaud_ in the sense of "this breaks the deprecation policy"?
19:33 * dukeleto wonders if he will be tarred and feathered for his next parrot-dev post
19:33 chromatic In the sense of "Rakudo's crap doesn't stink"?
19:33 pmichaud_ in the sense of "parrot shouldn't have made those changes"?
19:33 * benabik starts boiling tar.
19:33 jnthn left #parrot
19:33 pmichaud_ you're claiming that we're wanting parrot to not change its internal.  I'd like to see proof of that.
19:33 atrodo dukeleto> On the bright side, you'll have enough materials for a new roof and a nice pillow
19:34 chromatic I'm claiming that Rakudo has free reign to break everything and Parrot gets whipped every time there's a hint of change.
19:34 pmichaud_ I disagree with that claim.
19:34 chromatic I'm not surprised.
19:35 pmichaud_ I'm concerned about continued changes that don't really help Rakudo. (more)
19:35 dukeleto pmichaud_: what is the current status on Rakudo's research for alternate VM backends?
19:35 pmichaud_ the point of my comments to whiteknight's gist isn't to say "don't do these things", but rather to point out how Parrot's priorities (and perception of HLL needs) are so different from Rakudo's perceptions of its needs
19:37 NotFound The search for alternate VMs for rakudo is not a parrot concern.
19:37 pmichaud_ dukeleto: we have a prototype implementation of 6model on CLR, and a fledgling prototype (not really complete) on the jvm.
19:37 dukeleto NotFound: yes it is.
19:37 NotFound Nor for me.
19:37 chromatic If you want speed, please look at my year old wiki page about speed problems and proposed solutions for Rakudo and Parrot.
19:38 NotFound s/Nor/Not
19:38 pmichaud_ so, the design of nqp has been based on the knowledge gained from the CLR 6model implementation
19:39 benabik whiteknight: I'd like to add "fix lexicals" to a list of things Parrot should do.  Searching for lexicals by string is just as bad as class names.  And separating lexpads from subs should allow for something better than creating piles of subs.  If inner scopes have the same context, they can easily refer to lexicals directly by register #
19:39 whiteknight benabik: that's a feature addition, not really a major fix
19:39 whiteknight but yes, it would be beneficial
19:39 pmichaud_ we've introduced the nqp:: pseudo-opcode space so that we can identify those opcodes that are core to nqp and rakudo
19:39 pmichaud_ s/opcodes/functions
19:40 pmichaud_ and that tells us what we need our target vms to provide
19:40 pmichaud_ brb, kid pickup
19:40 benabik whiteknight: The nested scopes are a feature, but lexicals not by string is IMHO a fix.
19:41 chromatic Recursive lexical lookup by string name is a bug. benabik is completely right.
19:42 bubaflub chromatic: i'd like to work on that list that you have on the trac wiki but would need some pointers
19:43 chromatic The biggest gain is probably vtable overrides.
19:43 NotFound Accesing lexicals by number will be probably doable with some pir sugar and a bit of changes.
19:44 chromatic I'm not sure PIR needs to change for lexical index access.
19:45 NotFound No, but it may make easier the transition.
19:45 chromatic It's really a compiler optimization.
19:45 benabik find_lex_p_sc that retrieve/stores based on string constant # instead of value…
19:45 benabik It would still have to be under the value to for non-constant lookup.
19:45 chromatic If lexical scopes are statically decidable (and they are), the compiler can emit the appropriate find_lex to look up the appropriate slot in the appropriate scope.
19:46 NotFound The lexical is stored in a register in some context, we just need a quick way to access that register.
19:46 chromatic Lexicals are stored in contexts? That doesn't sound right.
19:46 NotFound Its values.
19:46 benabik Lexicals are stored in registers.
19:47 whiteknight lexical name->register mappings are stored in the LexPad
19:47 ambs joined #parrot
19:47 whiteknight or LexInfo
19:47 whiteknight one of the two
19:47 benabik Or both.  :-/
19:47 chromatic What we really need is some notion of scope which is a compile-time constant entity.
19:47 * dukeleto throws his hat into the ring of fire (also known as parrot-dev)
19:51 chromatic "Parrot needs to assume that Rakudo will port themselves to another VM or write their own."
19:51 chromatic Yep.
19:51 chromatic That's why I stopped committing.
19:52 moritz to rakudo? or to parrot?
19:52 chromatic Both.
19:54 bluescreen joined #parrot
19:54 pmichaud_ I agree with the statement also.
19:56 chromatic That answers the "What does Rakudo want?" question.
19:57 pmichaud_ ...with what answer?
19:57 chromatic "Not Parrot."
19:57 pmichaud_ False.
19:57 Coke ho, chromatic
19:57 pmichaud_ We want whatever VM can give us the best Rakudo.
19:57 chromatic Coke.
19:58 pmichaud_ although I will admit that Parrot as it sits today doesn't fully meet our needs.
19:58 pmichaud_ but then, neither do any of the other VMs at present
19:58 chromatic Everyone on this channel admits that.
19:58 chromatic As I see it, you have three choices.
19:58 NotFound But why "parrot needs to assume"? Rakudo can target whatever want without any assumption from parrot land.
19:58 chromatic 1) Write your own VM. We'll see you in five years.
19:59 chromatic 2) Port to another VM. Good luck getting code in them.
19:59 chromatic 3) Help Parrot get better. Most everyone on this channel wants that.
19:59 pmichaud_ I'm here because of 3.  What I've been trying to say is that what Parrot thinks we need isn't what we think we need.
19:59 pmichaud_ Apparently I'm not doing that well.
19:59 chromatic You've made that abundantly clear.
20:00 chromatic What we're all trying to get you to tell us is, in specific, what Rakudo needs.
20:00 pmichaud_ Faster PCC.
20:00 chromatic That means getting rid of the binder.
20:00 pmichaud_ Fine.
20:00 pmichaud_ No problem.  Break the binder all you want, we'll adapt.
20:00 chromatic No, DELETE the binder.
20:00 pmichaud_ Delete the binder.  We'll adapt.
20:01 pmichaud_ We've always agreed to adapt in places where performance will improve, and even sometimes where it won't.
20:01 chromatic We are not asking you to adapt.
20:01 pmichaud_ When the new GC came in, we didn't complain about having to add write barriers outside the deprecation policy, we gladly added them.
20:01 pmichaud_ If you're complaining about Rakudo "poking into Parrot guts", that sounds to me like "asking to adapt".
20:01 pmichaud_ Or, you're asking us to stop, when we don't see an alternative at present.
20:01 chromatic Then please allow me to attempt to rephrase to be more precise.
20:01 pmichaud_ absolutely.
20:02 chromatic I think the best possible approach is for Parrot to provide code which makes Rakudo's binder all or mostly irrelevant, and which removes the need for any and all C code.
20:02 chromatic at least on the part of Rakudo
20:02 * Coke wonders if he's the only one imaging this as a lot of yelling.
20:02 pmichaud_ Coke:  no, you're not.
20:03 Coke well, you two are doing a great job still talking, then. I would have walked by now. ;)
20:03 Coke chromatic++ pmichaud++
20:03 chromatic To make that binder irrelevant or at least smaller, Parrot needs to know what Rakudo's binder does, what it doesn't do, and where the mismatches are between it and Parrot.
20:03 benabik Coke: I have memories of a discussion like this…  It wasn't _quite_ yelling...
20:03 pmichaud_ At the moment, I suspect the Rakudo's binder is intimately tied to 6model.
20:03 pmichaud_ Let me check with jnthn.
20:04 chromatic Parrot also needs to provide extension possibilities for the binder -- whether delegation, data-driven custom initialization, or something else -- which do not require the presence of C code in Rakudo.
20:04 NotFound Acces to lexicals by number can just need a bunch of ..._keyed_int vtable functions to LexPad
20:04 pmichaud_ Coke: For my part, I continue to talk with chromatic++ because his past actions have convinced me he only has good intentions for what he says.
20:05 moritz NotFound: aren't vtables too slow to be used excessively in the binder?
20:05 pmichaud_ and that he has his reasons, which are valid more often than not.
20:05 chromatic And I likewise assume good faith for pmichaud_.
20:05 pmichaud_ vtables are slow only if they call into PIR
20:05 pmichaud_ vtables into C are fastest
20:05 moritz ok
20:05 NotFound moritz: You mean slower than using string keyed access *and* LexPad vtables?
20:06 pmichaud_ jnthn appears to be unavailable at the moment.  However, I will get an answer to the question.  But if Rakudo's binder is indeed tied to 6model, then we will either need to rethink it for Parrot or bring 6model in as Parrot's core type
20:07 pmichaud_ I suspect it's tied to 6model because the binder has to know a fair bit about types.
20:07 chromatic I've been assuming that 6model is essential to Parrot's core types.
20:07 pmichaud_ That's a reasonable assumption, but see earlier discussion about keeping Class/Object compatibility.
20:08 chromatic I don't know anyone who will mourn the loss of Class/Object for long.
20:08 NotFound Not for long, but a transition period.
20:08 chromatic vtable overrides in PIR are slow because their lookup requires recursive string lookup and because they require nested runloops.
20:08 pmichaud_ and the only way that a HLL can affect vtables is by using PIR or with C code
20:09 pmichaud_ so, a HLL is left with the choice of doing it in C, or doing it in (3000x slower) PIR
20:09 chromatic Unless vtable behavior were parametric with regard to (constant) metadata.
20:09 pmichaud_ yes, a change in Parrot's implementation would help there.  :)
20:09 chromatic Certainly moving PMCs to M0 instead of C will be a big help, but that's further away.
20:10 pmichaud_ I'm merely speaking of the choices we have to day, not what "might be".  Changing vtable semantics has been largely considered a "too hard to deal with now" issue in Parrot, from my perspective.
20:10 pmichaud_ i.e., nobody's seriously proposed it
20:10 pmichaud_ (and it's not been my position to propose it)
20:10 chromatic I'm not debating whether the binder is necessary or appropriate right now in Rakudo, because it clearly is. I'm saying I want it to go away because it's not necessary.
20:11 pmichaud_ I understand.
20:11 chromatic WHEN it's not necessary, that is.
20:12 chromatic I've also pondered a change to the Parrot runloop such that dispatching to a PIR vtable would destructure into a regular method call and resume appropriately, but that's not easy.
20:12 chromatic Maybe if I could clone plobsing_ and NotFound.
20:12 pmichaud_ I know this is going to come across wrongly, but...
20:12 NotFound Changing vtable overrides is hard because they are C functions and are called from C. No way to set up a return frame and no get_results to retrieve it.
20:13 pmichaud_ actually, never mind (more)
20:13 chromatic If we're already in PIR and don't have to go C->PIR->C->PIR, we can do much better.
20:13 chromatic I'm talking about munging VTABLE_whatever inside an op body.
20:13 pmichaud_ I was going to say that moving the binder from Rakudo into Parrot indeed improves things a lot, but it doesn't come across as a win from what Rakudo does now.  But if it leads to faster PCC, it's definitely a win.
20:14 chromatic Putting dispatch into one place (and not crossing lots of language boundaries) to do it means lots more optimization possibilities and lots less fragile code in Rakudo.
20:14 pmichaud_ from a Rakudo perspective, we feel like changes have been put "on hold" a bit because "they'll be address in M0"
20:14 chromatic Where "fragile" means "highly susceptible to Parrot changes".
20:15 pmichaud_ if the deprecation policy is gone, I'm much less worried about fragile
20:15 chromatic Munging VTABLE_whatever inside an op body probably means writing an inside out Duff's device with at least two separate parts in two separate places.
20:16 pmichaud_ when we have to wait 3+ months for things to roll in/out of Parrot, that becomes intolerable from a development timeline perspective.  If it's O(month), it's really no problem.
20:16 chromatic Provided that notification is far more careful and reimplementations are in place before removal, I think you'll find the results far more tolerable.
20:17 chromatic Only recently has Parrot begun to manage feature branch development effectively.
20:17 whiteknight I really like the 6model way of doing things. Scrap most of the vtables and have 6model routines to return invokables that can be invoked directly from the runloop
20:17 chromatic Vtables are more like trampoliney things?
20:17 whiteknight vtable overrides are only expensive when we need to support C vtables and (much more common) HLL vtables
20:18 whiteknight if they are all running on top of Parrot, we don't need the expensive nested runloops
20:18 pmichaud_ From a Perl 6 perspective, Parrot vtables are outside-in
20:18 pmichaud_ i.e., they expose the interface backwards from what we really need
20:18 whiteknight pmichaud_: from our perspective as well
20:18 chromatic You still have to deal with the case of calling a vtable from within code called from a vtable.
20:18 chromatic ... which is silly in a system supposedly built on CPS.
20:18 whiteknight chromatic: If it's just an invoke, return mechanisms and call chains bring us back to where we need to be
20:19 chromatic Depends how far you've chained since the initial invoke.
20:19 whiteknight are you talking about recursion limits?
20:19 pmichaud_ anyway, I have to take a break for a while.  To reiterate my earlier point -- my comments to whiteknight's gist are not intended to say "don't make Parrot changes at all".  They simply say "these changes help or hurt us to this extent".
20:19 whiteknight pmichaud_: Thanks for that. I appreciate the priority info
20:19 pmichaud_ s/us/Rakudo/
20:19 whiteknight it helps inform what to work on next
20:20 chromatic pmichaud_, can you suggest a concrete improvement that Parrot could work on next? Something more specific than "PCC should be faster" would be useful.
20:20 whiteknight chromatic; I think I have PCC figured out anyway, so we can work on that too
20:20 chromatic Not recursion limits, but think of what VTABLE_get_pmc_keyed_str eventually has to do and how many C calls it makes.
20:21 pmichaud_ I think I answered this question recently; let me see if I can dig up the post.
20:23 whiteknight chromatic: I don't think it's going to get as hairy as you suggest
20:24 chromatic I hope you're right.
20:24 pmichaud_ I suspect the other area of improvement would be in better profiling tools.
20:25 pmichaud_ with mls++'s sub-profiler branch we were able to finally profile CORE.setting this past week
20:25 pmichaud_ and that yielded a lot of useful information.  also, the GC speed improvement (20%+ overall improvement) came from doing profiling
20:26 cotto_work I'm excited to understand that code well enough to either maintain it or steal the relevant bits into the profiling runcore as the new default behavior.
20:26 pmichaud_ the faster that can happen, the better
20:26 pmichaud_ as it is now, the branch is well behind other important changes to master
20:26 chromatic Did I see a bugfix in packfiles to improve line numbers in debugging?
20:26 cotto_work the mls/sub-profiling branch?  I can sync with master if needed.
20:27 pmichaud_ sync with master +1 .   merge to master soon +2  :)
20:27 cotto_work have you seen the code?
20:27 pmichaud_ I'm sure it's not up to standard, yes.
20:27 pmichaud_ I'm just saying that the sooner it gets into master, the more likely it is to be used/usable and maintained :)
20:27 cotto_work I'm thinking vice versa
20:27 cotto_work but I want both as soon as we can manage
20:28 chromatic Merge now, clean up later?
20:28 pmichaud_ right now if I want to profile, I have to make sure that the branch is relatively up-to-date with master, build a special parrot instance from the branch, then rebuild nqp+rakudo from that branch
20:28 whiteknight +1 to merge. Get more eyes on it. Also, give mls a commit bit or three
20:29 cotto_work look at the code and then tell me to merge
20:29 Coke +1 on getting mls a commit bit.
20:29 cotto_work https://github.com/mlschroe/parrot/blob​/mls/sub-profiler/src/runcore/subprof.c
20:29 chromatic How hard would it be to add GC stats to the code?
20:29 * whiteknight has to to home now. I'll stare at it tonight
20:29 Coke mls, have you interest in getting a commit bit? if so, submit a CLA, kthanks.
20:29 pmichaud_ I'm planning to add gc stats into --stagestats of HLL::Compiler
20:30 pmichaud_ it's very coarse grained
20:30 chromatic Sub-level will be more useful.
20:30 pmichaud_ but might be better than nothing
20:30 cotto_work I would like to get mls a commit bit
20:30 pmichaud_ yes, Sub-level would be hugely useful.
20:30 cotto_work As gnarly as the code is, he's obviously a competent coder.
20:30 benabik +1 to mls commit bit.  He made IMCC line numbers better.  Anyone who does that gets my approval.  :-D
20:30 pmichaud_ anyway, it seems like GC stats would not be hard to add to the mls profiler
20:31 chromatic pmichaud_, one thing I did note: that optimization you did for reusing registers in initialization code (I think it was imports) is something you can't rely on a register allocator to do.
20:31 Coke by that logic, you should give chromatic a commit bit!
20:31 pmichaud_ simply count the number of runs at each sub invoke
20:31 pmichaud_ chromatic: agreed
20:31 chromatic Certainly an O(n^2) allocator is awful, but even an O(n) allocator gets bad when your basic blocks have n > 1000 registers to allocate.
20:32 chromatic If you know of other places where generated code has large basic blocks and lots of registers used, those are obvious candidates for a similar optimization.
20:32 pmichaud_ it wasn't a known pain point, because we never had such large subs before.
20:32 pmichaud_ that optimization is now in all of our code
20:32 chromatic Okay good. I thought I'd seen that pattern before in generated code and wanted to bring it up.
20:32 pmichaud_ register reuse occurs at the statement level
20:32 pmichaud_ so, the end of a statement marks "release all temporaries" (except the one used to return a value) here.
20:33 chromatic That's the kind of optimization I mention that has to take place on the language semantics level, because there's just not enough information at the opcode level to do it right.
20:33 pmichaud_ no argument there.
20:33 pmichaud_ but it wasn't until we had profiling on the compilation time that it showed itself to be an issue
20:33 chromatic So if I'm stubborn about Rakudo needing to do some of these optimizations, I'm not *trying* to be a stickler for anything other than efficacy.
20:34 pmichaud_ I'm just not in favor of premature optimizations :)
20:34 pmichaud_ or, I don't want to spend a lot of my tuits optimizing things that we don't know will have significant impact
20:34 benabik In theory you can assign registers to the tree and reuse them based on tree depth.  But POST gets very flat very quickly.
20:34 pmichaud_ benabik: right, so we're doing the optimization at the PAST level.  It works out fine.
20:35 chromatic P*ST is still a better level to do this than at the register allocation level. Basic block size limits come into play.
20:35 pmichaud_ it also helps to clarify what a PAST::Stmt node is responsible for doing.
20:35 chromatic There's a reason aggressive inliners reject inlining code blocks above a certain size, and that's algorithmic complexity.
20:36 pmichaud_ I totally agree with moving optimizations and code to higher levels when it's appropriate.
20:36 pmichaud_ I'm less convinced when it looks like waterbed pushing, though.
20:36 chromatic My bias is to getting code *out* of Rakudo, for what that's worth.
20:37 chromatic About GC profiling, how about a count of GCables allocated within the bounds of the Sub?
20:37 pmichaud_ yes
20:37 chromatic Inclusive okay to start?
20:38 pmichaud_ I'd think it should be handle-able similar to the way ticks and instruction counts are handled now
20:38 pmichaud_ it's just another count, yes?
20:38 chromatic Should be, but the complexity comes in where a run happens within a Sub.
20:38 pmichaud_ however ticks are being measured, I'd think GCables should be able to use the same approach.
20:39 NotFound I must mention that winxed has been using such temprary reuse from its beginning.
20:39 pmichaud_ NotFound++
20:40 pmichaud_ anyway, Inclusive is definitely okay
20:41 NotFound And needs it probably more than rakudo because its blocks are not separate .sub
20:43 pmichaud_ oh!  one problem we have now is that .const isn't really .const
20:43 pmichaud_ .const 'Sub' $P999 = 'foo'   actually allocates a register
20:43 schmooster joined #parrot
20:43 NotFound Yes, I noticed that some weeks ago.
20:43 pmichaud_ and I'm not sure it's legal for me to re-use   $P999 again later for a different purpose
20:44 NotFound I've tested, is reusable... but very confusing.
20:44 pmichaud_ well, I mean "per spec" as well as "per implementation"
20:44 NotFound Did we have a spec? ;)
20:45 pmichaud_ pdd19 would be the closest to that, I suspect.
20:45 NotFound Fixinf it need a few 'c' opcode variants.
20:46 pmichaud_ anyway, some clarity on that would enable us to eliminate a bunch more unique register allocations (that are also a pain point at the moment)
20:47 NotFound The opcode part sholdn't be hard, but the pir...
20:52 chromatic Yeah, reuse is going to be hard.
20:52 NotFound Looking at the lexical handling code, I think an intermediate solution can be a lexical cache in the Sub PMC. Not sure about its possible speed impact, though.
20:52 pmichaud_ lexical cache was what we were thinking would be a good first approximation
20:52 pmichaud_ bonus: not all lexicals can be determined statically.  see "eval"
20:52 pmichaud_ (clearly most lexicals can be determined statically, but not all.)
20:53 NotFound And given that cache, int keyed access to it is a posibility.
20:53 pmichaud_ a cache would be able to speed up the dynamic lookups
20:54 pmichaud_ yes, one could do int keyed access into the cache
20:54 pmichaud_ I think jnthn++ has plans for adding this capability into nqp (and rakudo).  Parrot could look at Rakudo's lexpad PMC implementation, perhaps, and adopt whatever it's doing.
20:55 pmichaud_ and then we could eliminate Rakudo's lexpad pmc
20:55 chromatic Still seems like it depends on fixing Sub.
20:55 chromatic Or is at least easier.
20:55 pmichaud_ well, it wants some changes to Sub, yes.
20:55 NotFound In the context, maybe.
20:56 pmichaud_ whether that feature requires a wholesale Sub refactor, I don't know. (more)
20:58 pmichaud_ I recognize that Sub likely needs a refactor, and I don't have a problem with that.  NQP and Rakudo conceptually treat Sub as black-boxish anyway, and wrap Parrot subs into objects that we can manipulate.
20:58 pmichaud_ Ultimately Parrot needs a fundamental rethink of code and lexical scopes, definitely.
20:58 chromatic Completely agree.
20:59 chromatic If Parrot provided some sort of scope object (and that could be compile-time in most cases), attaching compile-time exception handlers would be a lot cheaper.
20:59 pmichaud_ refactoring Sub simply to make its internals look less messy doesn't help (Rakudo), though, unless it leads directly to the other performance-enhancing improvements
20:59 chromatic A lot of Rakudo sub preludes could possibly go away.
21:00 pmichaud_ note that we've already eliminated the exception handler for return.
21:00 pmichaud_ we may have to add some back at some point, but we no longer create an exception handler on every HLL sub entrance
21:00 chromatic The refactoring is primarily for separating compile-time and run-time behavior.
21:02 bluescreen joined #parrot
21:03 NotFound method_name and vtable_index, for example?
21:03 chromatic That too.
21:04 pmichaud_ We do create a continuation on HLL sub entrance, though.
21:04 pmichaud_ anyway, reducing the sub prelude would be a plus.
21:05 chromatic Maybe there could be a prelude attached to an invokable PMC such that invoking the PMC always runs the prelude code, and that prelude can be shared between multiple subs.
21:06 pmichaud_ does that actually result in a performance win, as opposed to inlining the code directly into the sub ?
21:06 pmichaud_ *the prelude code
21:06 chromatic Don't know. It's less code to generate and reconstitute.
21:07 pmichaud_ I think it might be improving compile-time at the expense of runtime
21:07 chromatic We're at the level of speculation which involves caches and memory.
21:07 NotFound Eventually a JIT can re-inline it.
21:09 NotFound BTW, TT #2181, HashIteratorKey .value, is really a bug?
21:11 plobsing_ I'd speculate that reducing the code size might increase the cache-friendlyness
21:12 plobsing_ guesses like that have worked out in the past
21:12 chromatic If reductions are on the order of 15-20%, certainly.
21:52 dalek parrot: 40afde8 | NotFound++ | src/pmc/lexpad.pmc:
21:52 dalek parrot: put repeated code in helper functions, delete unused parameters, and fix interp/INTERP in LexPad
21:52 dalek parrot: review: https://github.com/parrot/parrot/commit/40afde80bb
21:58 NotFound Could not upload report to Smolder at http://smolder.parrot.org
21:58 NotFound HTTP CODE: 500 (Internal Server Error)
22:01 benabik smolder--
22:05 Coke momentito.
22:07 Coke NotFound: try smolder again.
22:07 Coke I really need to set up that nightly cron job to hup it.
22:08 Coke we're at 90% inode usage again. also need to clean that up.
22:08 NotFound There is some make target to retry, or should I use the command?
22:08 Coke Iunno, sorry
22:08 Coke common enough we should probably add a target to resend last smoke results.\
22:09 NotFound perldoc lib/Parrot/Harness/Smoke.pm suggest using: perl -Ilib -MParrot::Harness::Smoke -e 'Parrot::Harness::Smoke::send_archiv​e_to_smolder(Parrot::Harness::Smoke:​:collect_test_environment_data())'
22:09 NotFound Not particularly easy to remember ;)
22:11 NotFound Test report successfully sent to Smolder at
22:11 NotFound http://smolder.parrot.org/app/​projects/report_details/22653
22:15 dukeleto NotFound: the same can be acomplished with parrot ./examples/io/post.pir
22:16 NotFound A bit easier, yes.
22:35 panda joined #parrot
22:35 panda hello
22:37 cotto_work hi panda
22:38 panda hi
22:38 panda I am trying to find out if parrot is right for me
22:39 panda I am not sure if this is right place to ask
22:39 panda but if it is, I have a few questions
22:41 cotto_work it is.  Ask away.
22:44 panda well I am not perl guy. What annoys me with Python is that is damn slow language. No optional static typing to speed things up so you have to switch to C++ to do anything. Java is fast enough but it is just too bloated and bureaucratic
22:44 panda what about parrot and new perl 6? Does it have optional static typing to speed things up?
22:45 panda basically I am looking for replacement for Python that is faster
22:45 panda but obviously I know next to nothing about parrot VM
22:45 dukeleto panda: just curious, what kind of computations are you doing that you are finding Python slow?
22:45 plobsing_ panda: you probably want #perl6 over at freenode
22:45 dukeleto panda: also, welcome
22:45 cotto_work panda: Rakudo Perl 6 and Parrot are separate (but related) projects.  Rakudo is working on gradual typing, though questions to them should go in #perl6 in freenode.
22:46 plobsing_ but to answer your questions, rakudo is currently going through a refactor that has the potential make the optional static types into performance improvements (right now, however they are simply runtime checks, which slow things down)
22:47 cotto_work Rakudo is focusing aggressively on speed, but they're not in the same range as P* (Perl5, Python, PHP, Java) yet.
22:47 dukeleto panda: though, to warn you, if python is too slow for you now, you will find rakudo perl 6 to be many times slower than that, in the short term
22:48 plobsing_ panda: if all you care about is speed and similarity to python, have you looked at node and/or lua?
22:48 cotto_work panda: you might take a look at winxed
22:48 panda have not looked at lua. But since lua is dynamically typed I imagine it can't be very fast
22:50 panda I need language with optional static typing (that can actually translate to performance, not as runtime checks)
22:50 chromatic Lua with LuaJIT is very fast.
22:50 plobsing_ panda: that is a common misconception of dynamically typed languages.
22:51 plobsing_ it is possible to get them to be very fast, it is just *harder* to do.
22:51 cotto_work LuaJIT has some very impressive optimizations.  Don't knock it 'til you've tried it.
22:52 panda thank you, I'll try it
22:53 plobsing_ what the hell are we doing guys? we just lost a customer to another VM! ;)
22:53 cotto_work panda: winxed lives on top of Parrot but does a pretty good job of being low-overhead.
22:54 cotto_work panda: what kind of code are you writing?
22:58 panda well, I am hobby game developer. Mostly actionscript 3.0 (flash) games so far. AS 3.0 is actually pretty fast, with optional static typing. I have half finished 2d real-time strategy written in it. The problem is that it is flash and it doesn't really communicate with anyting outside itself. I am basically trying to see if there is viable platform outside C++ or Java. I have tried pygame, but
22:58 panda python is just inadequate
22:59 panda Nothing wrong with C++ or Java in terms of speed but developing anything big and complex on those platforms is just slow and painful
23:00 panda I don't need some ground breaking performance
23:00 panda but I need something decent
23:00 panda Thanks for answers so far
23:02 panda also I am looking for alternate platform for general programming
23:05 panda so I guess this answers your question, cotto
23:14 plobsing_ panda: naughty dog used lisp (game oriented assembly lisp) and had spectacular results.
23:14 panda hm lisp
23:14 panda I tried lisp couple of times
23:14 plobsing_ yeah, it isn't for everyone
23:14 panda I actually liked it... liked idea
23:15 panda what is this assembly lisp you are talking about?
23:15 plobsing_ it is a custom lisp engine developped by the naughty dog game studio.
23:16 panda let me guess: it is either unavailable or expensive ?
23:16 plobsing_ they had very tight requirements. for most uses, standard common lisp compilers do pretty well.
23:16 arnsholt_ You can get pretty good performance with Common Lisp
23:16 arnsholt_ Yeah, what plobsing_ said =)
23:16 panda I was thinking of clojure for VM... but thats functional only language so not best suited...
23:17 arnsholt_ (CL has the option to add type declarations too)
23:17 panda I know
23:17 panda I know it does.... but thats practically dead language
23:17 panda lisp it is cool and I like it, but who maintains that stuff any more
23:17 arnsholt_ More people than you think, I suspect
23:18 plobsing_ there is a small but devoted community
23:18 arnsholt_ But it's not a language that shows up in the frontend a lot
23:18 rfw joined #parrot
23:19 panda I gues I'll have to google it
23:21 panda there is a lot to like about lisp, actually
23:22 arnsholt_ Common Lisp is a great language. Just too bad the syntax sucks =)
23:23 panda parentheses? Well there is prupose to it. Makes code generation easy
23:23 panda enables macroes
23:24 plobsing_ racket is also doing pretty interesting things
23:24 panda someone needs to ressurect that language from the grave
23:24 panda never gonna happen
23:24 plobsing_ racket is alive and well
23:25 panda racket?
23:25 panda what is that?
23:25 panda or who?
23:26 plobsing_ formerly mzscheme
23:27 plobsing_ they have cool things like a module system, optional static types with type inference
23:27 plobsing_ module system is only really cool from a scheme perspective
23:28 panda is it free?
23:28 plobsing_ yes
23:28 panda how fast?
23:29 plobsing_ http://shootout.alioth.debian.org/u32/benchmark​.php?test=all&amp;lang=racket&amp;lang2=python3
23:30 panda holly molly, thank you plobsing
23:30 panda please tell me they have multicore support
23:30 panda multithreading
23:30 panda I am downloading racket now
23:32 plobsing_ I don't know that much about racket (I don't use it). I just look at it trying to steal ideas for parrot.
23:33 panda well, I am looking at wikipedia and downloading it right now and it looks balls to the wall awesome
23:33 panda how come I never heard of it?
23:34 panda there must be a catch somewhere
23:34 plobsing_ you've never heard of mzscheme, pltscheme, or drscheme?
23:34 plobsing_ they recently changed their name (because they are diverging with the scheme standard)
23:34 panda nope. I am not lisp zealot. I liked the concept of a language but assumed it is dead and never really looked at ut
23:34 panda it
23:35 panda but now it looks awesome
23:38 plobsing joined #parrot
23:39 panda anyway I am going
23:41 whiteknight joined #parrot
23:41 whiteknight good evening, #parrot
23:43 cotto_work good evening, whiteknight
23:47 nopaste "chromatic" at 192.168.1.3 pasted "Batch Frees in GC Sweep" (234 lines) at http://nopaste.snit.ch/79126
23:47 chromatic That patch has some promise.
23:47 preflex_ joined #parrot
23:47 chromatic One obvious tuning is to pre-size the new PAs.

| Channels | #parrot index | Today | | Search | Google Search | Plain-Text | summary

Parrot | source cross referenced