Perl 6 - the future is here, just unevenly distributed

IRC log for #pdl, 2013-06-12

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:57 Meiermann joined #pdl
04:20 _vicash_ joined #pdl
04:28 _vicash_ run4flat_: I use PDL with Moose in my financial plotting application… not sure why PDL needs to incorporate Moose internally as that is just needless overhead… The data I have and use needs PDL to work fast and with low memory footprints.. even Microsoft Excel cannot handle this data without being slow as molasses or dying. PDL handles it faster than any other interpreted language or plotter
04:28 _vicash_ run4flat: I use PDL with Moose in my financial plotting application… not sure why PDL needs to incorporate Moose internally as that is just needless overhead… The data I have and use needs PDL to work fast and with low memory footprints.. even Microsoft Excel cannot handle this data without being slow as molasses or dying. PDL handles it faster than any other interpreted language or plotter
04:30 _vicash_ the more abstractions you add to PDL the slower it will get and will stop solving the problem it is intended for — being the fastest Perl library to manipulate large timeseries and matrices
12:01 _vicash_ joined #pdl
13:03 jberger _vicash_, I dont't think we are intending to make PDL a moose based object, but we want to increase interoperablility
13:55 _vicash_ joined #pdl
14:02 _vicash_ jberger: i am curious what you mean by interoperability ? i use Moose features like Type Coercion, Constraints and overriding classes to load large datasets and then invoke the PDL objects internally in my class. So my class is "Moose"-ified with the guts using PDL and other possible classes. In your opinion, what is lacking in this kind of a design that PDL would need Moose as an internal ? How would the changes in PDL affect my current design ?
14:03 _vicash_ just curious as i will then have to budget time to make my code compliant to the newer releases accordingly
14:07 run4flat _vicash_, different people have different ideas for Moo[se] compliance; we're still converging
14:07 run4flat I think the basic idea is to have PDL be a fast base class
14:08 run4flat but make it Moose-overridable if so desired
14:08 run4flat At the moment, there are no plans for changing PDL in such a way that it would break old code
14:08 run4flat that is, PDL3 is supposed to be backwards compatible
14:08 run4flat future major versions of PDL may not be backwards compatible
14:09 run4flat but I, for my part, hope that we make a break in the name itself, perhaps to "PDL4"
14:09 run4flat so you would say "use PDL4"
14:09 run4flat that way, both PDL and PDL4 could live in the same Perl install
14:14 _vicash_ Sounds great
14:16 _vicash_ but I think we can always do "use PDL 2.0" or "use PDL 4.0" or similar version selections already in Perl, so having PDL and PDL4 is redundant
14:16 run4flat not quite
14:16 run4flat if I say "use PDL 4.0"
14:16 run4flat then it will require that I have v 4.0
14:17 run4flat if I say "use PDL 2.0", it will require that I have v 2.0
14:17 run4flat this works great if 4.0 is backwards compatible with 2.0
14:17 run4flat because it actually checks that I have v2.0 or later
14:17 run4flat but if 4.0 is not backwards compatible, then all old scripts die
14:18 run4flat if non-backwards-compatible stuff goes under PDL4, then submodules live in completely seperate directoryes
14:18 run4flat *directories
14:19 _vicash_ right..
14:40 judd joined #pdl
14:42 judd I dunno about moose. Using a new namespace may not fix things either. It seems inevitable that project developers are going to go with the new version and abandon the old "fast" PDL. I almost think that if a new namespace is going to be defined, it may need to be a new project entirely.
14:43 run4flat judd, I don't think PDL would take a speed hit with the thoughts about Moose
14:43 judd As a new project, I think it's exciting and will support it. However, I think the performance questions are too large at this point to know if this is the permanent direction in which PDL should be going.
14:43 run4flat we're just talking about ways to use Moose to subclass PDL, or impose type constraints
14:44 run4flat Everybody who's voiced an opinion has said that execution speed is our top priority
14:44 run4flat so we're not going to do anything that will slow things down
14:45 zowie joined #pdl
14:45 judd I was thinking this was part of the plan for the new datatype support... as in moose objects on the data type level.
14:45 run4flat yes, that idea has been suggested
14:45 run4flat but there's no clear way to implement that
14:46 run4flat and I highly doubt we would be able to get it working by August
14:46 run4flat zowie, good idea about making PDL_SV higher than doubles
14:46 judd In addition to that, I've got some issues with PDL as a whole requiring moose as a dependency. It's a monstorous amount of packages that would now need to be added to get Moose running on a given OS... otherwise I would be using it in my own code so far. But as of yet, I can't make the coding benefits outweigh the costs of deployment for Moose.
14:47 run4flat judd, think Moose-compatible
14:47 run4flat not Moose-dependent
14:47 run4flat Chris has suggested rewriting PDL :: PP using Moo
14:47 run4flat but Moo has a much smaller dependency chain
14:48 judd That's good. But if this has nothing to do with requiring moose, which has it been suggested to make this a new namespace?
14:48 run4flat and everything Moo depends on is pure-Perl
14:48 run4flat the new namespace is for backwards compatibility
14:48 run4flat if we completely rewrite PDL's internals and change things up, we should use a new namespace
14:48 run4flat not like that'll happen any time soon...
14:48 run4flat :-)
14:49 judd I kinda like the way PDL::PP works now, unfortunately :( I think the code gen is the genius of PDL in that you can prototype code as quickly as in IDL, except it will be fast and type safe already. I need to look into Moo.
14:49 run4flat Yes, I agree, PDL::PP is the gem of PDL
14:50 run4flat but it's damn near impossible to comprehend
14:50 judd How do you get backwards compatibility with a new namespace without changing all of the code to use PDL2 instead of use PDL?
14:50 run4flat for example, if I want to implement sparce matrices, how do I do it?
14:50 run4flat you wouldn't change the old code
14:50 run4flat that's the whole point
14:50 run4flat PDL would live on
14:50 run4flat your old scripts would continue to use the old modules
14:50 run4flat and PDL4 would make a break
14:51 run4flat In fact, this would mean that new scripts could use *both* PDL4 and PDL
14:51 judd Have you guys looked at things like Boost C++ as a generated code instead of C in PDL::PP?  Boost hasn't been as fast as PDL in my experience, but in some instances it could handle things like sparse matricies.
14:52 run4flat I'm aware of Boost; I don't like having C++ as a dependency for PDL
14:52 run4flat not sure what others think
14:52 judd You could algorithmically solve the sparse matrix problem... use a full one of just coordinates.
14:52 run4flat but you could suggest it on the list
14:52 judd If PDL::PP switches to using Moo... can we just call it Poo?
14:52 judd ;)
14:52 run4flat haha!
14:53 run4flat PDL::Poo?
14:53 run4flat like Poo bear... right...
14:53 judd Well, PDL::PP is already "doing the dirty work", right?
14:54 * run4flat shakes head
14:55 judd Just curious, what part of PDL::PP is so hard to comprehend? It seems like a simple code gen to me. For really complicated stuff, it's just not the right tool for the job. For that I'd probably use PDL::CallExt
14:57 judd Also, can you give me an example where a subclass of PDL is needed where a container relationship wouldn't work as things exist now?
14:57 run4flat no, I cannot
14:57 run4flat I always use a has-a relationship
14:57 run4flat However, if we decide to allow a new datatype that is an SV*, then there will be many ways that people will want to subclass it
14:58 run4flat As far as the comprehensibility of PDL :: PP, well...
14:58 run4flat PDL :: PP creates code that runs quickly under most use cases
14:59 run4flat but tweaking it to be cutting-edge for multicore stuff is next to impossible
14:59 judd Also, about moose (while I'm rambling) what kills me is that there aren't any good IDE design tools out there for it. I mean, what's the point of being so compatible with code gen/reverse engineering if you can't get diagrams? I'm using argouml now, and it's a major PITA to work up a design, then code it... change the code and then your diagrams are pretty much useless now.
14:59 run4flat basically, the code that implements the threading engine is a black box, even to me, and I've spent hours studying the code
15:00 * run4flat has never worked with class diagrams
15:01 run4flat judd, do you just want something that'll build a picture with the "isa" and "does" relationships?
15:01 run4flat i.e. inheritance and role consumption?
15:01 run4flat I'd be a little surprised if that's not available on CPAN somewhere, the metadata is certainly available...
15:01 judd I work alone, so it's not a major part of my work (it's primarily usefull for telling people how to do stuff). But some of the code I write is more complicated than I can comprehend, and the diagrams make getting it to work and then maintaining it possible. A fix that would take days of debugging otherwise can take just minutes with good documentation like class and sequence diagrams.
15:02 run4flat hmmm
15:02 judd There's nothing that works out there. The primary driver for it is the IDE/modelling environment. Most of those are geared towards god awful java.
15:02 run4flat uh, does this help? http://code.google.com/p/umlwiki/wiki/ExampleMoose
15:04 run4flat It's also discussed on Stack Overflow, with a script link in the last post: http://stackoverflow.com/question​s/4945771/uml-tool-for-moose-perl
15:06 judd Yeah, I've read that a dozen times. The tool won't build on any of my distros, and I don't have time to fix it.
15:06 judd Is there a way to post a pic here?
15:06 run4flat jberger, ^^ ?
15:07 run4flat You'd have to find a website to host the pic
15:08 judd sequence diagram: http://c5fes.orbitalsystems.com/H​RPTInputClientSequenceDiagram.png
15:09 * run4flat checks
15:09 run4flat hmm, that's really dark on my machine
15:09 run4flat but I can see the links between the pillars
15:10 judd It's a transparent background, and firefox seems to be thinking that black would be a great color for transparent graphics. Maybe save it and open it with something that doesn't suck.
15:10 run4flat hahaha
15:10 run4flat sure
15:10 run4flat oh firefox
15:12 zowie joined #pdl
15:12 judd Anyways, this is the kind of thing that's required because all of these parts won't fit in my head. This is for a daemon that does thin client streaming of satellite imagery in real time. The diagrams is tracking the flow of the control/data notifications through all of the various parts of the system. Since it's a heavily OO design, it's confiusing as heck otherwise.
15:13 run4flat yeah, I can only imagine
15:13 run4flat I have the great fortune of only working with comparatively simple systems
15:13 judd I can't. That's the problem! :)
15:14 run4flat heh
15:15 judd Still, over the years I've been moving more and more towards OO designs where this type of modeling is a good idea. The code itself ends up being much better as far as maintenance and extension.
15:15 run4flat is there any Perl object system that supports UML?
15:15 judd I'd like to use Moose as a basis for this stuff, but not having modeling tools is like like doing math when you can only add and negate.
15:15 run4flat The great thing about Moose is that it has the proper introspection to achieve this
15:16 run4flat even if nobody has done it yet
15:16 judd Not really. Perl can do a lot more than UML thinks is possible... but you can shoe horn it in a lot of diagramming tools.
15:17 judd Perl OO seems to be able to do a lot more than Moose thinks is possible, too. Like magically changing from one class to another.
15:18 judd I mean that without a type cast. However, Multiple Inheritance just doesn't work in Perl. It's something you always learn about, but whenever you implement it, you realize it's a terrible idea, just like replacing all subs with GOTO calls.
15:18 judd There are times when that would be nice if it worked. I'm not sure if it works in Moose.
15:19 run4flat Moose supports multiple inheritance (I believe), but highly recommends roles for almost anything where multiple inheritance is traditionally used
15:20 judd Perl OO also doesn't do interfaces in any sane manner. I'm guessing Moose takes care of that.
15:20 * run4flat isn't familiar with interface
15:20 run4flat is an interface an abstract base class?
15:22 judd You define an interface as an abstract class that has to implement a certain set of attrs and operations. Any class that implements those calls/attrs is an implementation of the interface. This doesn't exist in perl OO, and is hacky to implement on your own (like BEGIN blocks check that things are defined).
15:23 run4flat I'm not sure
15:23 run4flat this sounds vaguely familiar
15:23 judd Hard to describe. Do you have any Java experience? Have you danced with the devil?
15:23 run4flat no
15:23 run4flat I am familiar with C++, though
15:24 judd They are used heavily in Java, and it's one that that works out pretty well. I think interfaces are in C++... but it's been so long I can't remember.
15:25 run4flat These are good questions, but I'm not the person to answer them
15:25 judd So how it's useful is when a class is defined as
15:25 judd class foo(junk) implements DataReceiver;
15:25 run4flat if you hop on to #moose, I'll bet they can answer your questions
15:26 run4flat (say /join #moose if you're not sure how to do that)
15:26 judd Then you have some function that's looking for DataReceivers as an arguement. DataReceiver is abstract, so it doesn't exists, but if DataReceiver is an implemented interface, the function doesn't have to care what data type was actually sent.
15:26 run4flat yeah, I think you get this in Moose with roles
15:26 judd Yeah, this is getting off topic.
15:27 run4flat and actually, it sounds like you would benefit a lot if we ensured that PDL was compatible with Moo[se]
15:27 run4flat since a UML tool can be written for Moose
15:27 run4flat again, we don't want PDL to depend on Moose
15:27 run4flat just be compatible with it
15:28 judd Well, that's where the subclass/containership arguement comes in. My objects typically just contain a PDL variable in a specific format, then do their operations on the PDL var, and spit the results out. I find this is much easier than trying to subclass.
15:28 judd In that sense, it's already compatible with everything that can hold a PDL variable.
15:29 run4flat yeah, me too
15:29 run4flat but the couple of emails that have just gone out about new Datatypes have me thinking about lots of subclasses
15:30 judd That's why I'd like to know why subclassing is worth jumping through a bunch of hoops.
15:30 run4flat well, I don't think it's a bunch of hoops, actually
15:30 run4flat it'd probably only require proper documentation. :-)
15:31 judd Even for datatypes, wouldn't it be better if PDL::Datatype was an interface as described earlier, where anything that implements it can be threaded automagically?
15:31 run4flat and maybe tightening how PDL interacts with inhereted types
15:31 run4flat which it already supports
15:31 run4flat "can be threaded automagically" is really, really hard
15:31 run4flat and that's where we come back to PDL::PP
15:32 run4flat and the need to revise it
15:33 judd Well, from my experience, this is stupid easy to make work. HOWEVER, it'll be slow as molasses. The only way out is to use a code gen on the level of PDL::PP to make the realization of these type classes in pure C OO (that's an oxymoron, but there are techniques out there to do this).
15:33 run4flat I am quite familiar with Pure C OO (not C++ OO, C OO)
15:34 run4flat and the problem with PDL::PP is that it essentially provides a highly constrained and non-introspectable C OO
15:34 run4flat there is no way to change anything at runtime
15:34 judd Moose, or anything perl level won't make this fast enough... heck even C++ may not be fast enough. If you're generating code, you can make it fast, but it becomes cryptic just like the PDL::PP output. But, it'll work.
15:34 judd What do you want to do at runtime?
15:35 run4flat For example, true sparse data handling should access data using methods
15:35 run4flat but currently PDL::PP always accesses data through a pointer dereference
15:35 run4flat this means that sparse data must be copied to a temporary buffer
15:36 judd Not following
15:37 run4flat in your PDL::PP code, when you say "$piddle(n => 4) = 100", that gets translated into some C code that is something like "_piddle[n_stride * 4] = 100"
15:37 run4flat roughly speaking
15:37 run4flat that means that if I want to perform a Ufunc (i.e. a reduction) on sparse data,
15:38 judd So you mean the sparse buffer will have to be expanded into real memory (non sparse) to dereference?
15:38 run4flat PDL needs to create a temporary buffer where it can copy that data
15:38 run4flat yes
15:38 run4flat exactly
15:38 judd I see that as ass backwards.
15:38 run4flat that's how non-affine slices work
15:38 run4flat i.e. slices created using "where" or "index"
15:40 judd Sparse stuff is by nature backwards. You iterate over what you've got, then calc any neighbor dependents for your calculation and then carry it out. Rather than go row or col wise. I don't see this being compatible with what PDL::PP does at all. Working efficently with sparse data is  just not compatible with algorithms that work with non sparse data.
15:40 judd Or at least, that's how I see it.
15:41 run4flat Sure
15:41 run4flat but if I wanted to make PDL::PP more general, to make it not impossible to support sparse data, for example
15:41 run4flat well, I can't do that
15:41 run4flat PDL::PP is just too dense for me
15:41 judd The additional costs of calculating the dependent locations is made up for by the gain from not having to deal with empty data locations. Shoehorning iterative operations (a la PDL::PP) into that kind of algorithm is not efficient, nor is shoe horning sparse calcs into the iterative algorithms...
15:41 run4flat and it doesn't provide any hooks that I could modify
15:42 run4flat I would have to modify PDL::PP itself
15:42 run4flat alas, I gotta get to work
15:43 run4flat I'll be around for further discussion, but I must turn to other things
15:43 judd I'm not so sure. Most of my uses of PDL::PP are actually type specific and un=PDL like. However, there's no reason you couldn't use it to code a sparse algorithm and just use your own looping mechanism to do what you want.
15:43 judd Then it would still be type generic, so it's still a gain to use PDL::PP for that purpose.
15:50 run4flat Bender1, trust jberger
15:50 Bender1 OK, run4flat
15:50 run4flat Bender1, trust pdurbin
15:50 Bender1 OK, run4flat
15:50 run4flat Bender2 trust jberger
15:50 Bender2 run4flat, I already trust jberger
15:50 run4flat really?
15:50 run4flat Bender2 trust pdurbin
15:50 Bender2 run4flat, I already trust pdurbin
15:50 * run4flat shakes fist at bots not restoring opers
15:51 run4flat zowie, judd, I would be happy to give you guys ops, too
15:51 run4flat just let me know. :-)
15:51 * run4flat shuffles papers and returns to work
15:58 jberger admittedly I skimmed the converstation above, but I think there are two levels, PDL::PP the "language" is more or less ok
15:59 jberger is PDL::PP the code-generator/engine that is too difficult to comprehend/extend
15:59 jberger that is one of our targets for a future PDLNext namespace IMHO
15:59 jberger among other things
16:00 sivoais it may be worth looking at <http://wiki.scipy.org/Cookbook/Recarray>. apparently numpy has two ways to access a complex datatype. I don't know the implementation details.
16:00 jberger also, and off topic, the Go language is built around interfaces (exclusively if I understand correctly)
16:08 sivoais I am just looking around and I suppose this could lead to more complete support for formats like FITS?
16:09 run4flat sivoais, I don't follow
16:09 run4flat oh, bother, never mind
16:09 * run4flat is too easily distracted
17:28 run4flat was kicked by pdurbin: run4flat
17:29 run4flat joined #pdl
17:30 run4flat pdurbin?
17:30 pdurbin run4flat: don't trust pdurbin
17:30 run4flat haha
17:30 pdurbin :)
17:31 run4flat ok, how do I unop someone?
17:31 run4flat Bender1, don't trust pdurbin
17:31 run4flat hmmm
17:32 run4flat well, at any rate, pdurbin, you are an operator so that if the rest of us get kicked off or drop off, and you're still here, we can retrieve ops without bugging perigrin
17:33 run4flat you seem to hang around
17:33 run4flat :-)
17:33 pdurbin I like it here. oh, netbeans as a IDE for perl. saw a tweet about that the other day. crazy!
17:35 pdurbin here it is: http://netbeans.dzone.com/arti​cles/perl-netbeans-ide-beta-2
17:43 pdurbin I've never tried Padre. I guess maybe I should. I'm more of a vim guy
17:44 perigrin Bender2: distrust pdurbin
17:44 Bender2 OK, perigrin
17:44 perigrin Bender2: trust pdurbin
17:44 Bender2 OK, perigrin
17:44 perigrin run4flat: ^^^ like that
18:11 run4flat perigrin, thanks!
18:12 perigrin np
18:12 run4flat Bender1, distrust pdurbin
18:12 Bender1 OK, run4flat
18:12 run4flat Bender2,  distrust pdurbin
18:12 Bender2 OK, run4flat
18:12 run4flat there you go, pdurbin :-)
18:28 _vicash_ joined #pdl
18:39 pdurbin no need to trust anyone who uses netbeans
18:40 run4flat heh
19:05 dystan joined #pdl
19:08 _vicash_ joined #pdl
19:25 _vicash_ run4flat,judd: Interfaces in C++ are done using pure virtual functions in class definitions. this forces the user of the class to inherit the class first, define the required methods. There is no "interface" keyword like in Java but pretty much the same thing.
19:47 run4flat _vicash_, thanks. I figured that was how it worked in C++.
19:47 run4flat So it's not a formal part of the language, but this is the practice to get the same effect
20:23 lungching joined #pdl
21:11 * perigrin is spoilt by roles ... finds interfaces lacking horribly.
21:13 run4flat perigrin, I got the impression that interfaces are really useful for compile-time checking
21:13 run4flat for a language like Perl, where we are less concerned about compile-time stuff (except strictures), roles are a better fit
21:13 run4flat maybe?
21:16 perigrin kinda, interfaces are afaict a subset of Roles
21:17 perigrin but yes they're mainly useful for contract / api enforcement which is *one* of their uses in Moose.
21:29 sivoais I love roles as well. Composing behaviours seems to scale nicely while still fitting in my head. :-)
21:34 _vicash_ joined #pdl
21:35 _vicash_ it might be formalized in C++0x11 or whatever the new standard is. i am not aware.. pure virtual functions are definitely a formal part of the language. the concept of interface is not. that is just a design idiom
21:36 run4flat yes, pure virtual functions have been around in C++ for a long time
21:36 * run4flat likes the phrase "design idiom"
22:05 judd In Java they are formalized, however. class foo implements bar. Interface bar...
22:06 judd However, the key advantage to them as they relate to perl is that they are a sort of metadatatype... in that a stricture can require an interface, which can be one of many a class implements, and there is no type checking against the data type itself.
22:07 judd It allows useful type checking, without having to check for super/sub class types
22:07 perigrin judd: that's exactly how Roles (can) behave in Moose.
22:07 judd This means you can use sub baz(PDL var), and pass it PDLSub, which would otherwise break.
22:08 perigrin except we don't have type signatures in a useable way yet
22:08 judd That's... disappointing.
22:08 run4flat there is type checking
22:08 judd That was the main thing I was hoping was supported.
22:08 perigrin well there are several competing implmentations on CPAN
22:09 perigrin and at least two of them will let you use roles exactly like that
22:09 run4flat we should differentiate between "optimal syntax" and "existence of capability"
22:09 perigrin sub baz(RoleFoo var) { ... }
22:09 * run4flat smacks head
22:10 run4flat I was focusing on a different thing
22:10 run4flat judd: https://metacpan.org/module/Method::Signatures
22:11 run4flat note the type constraints near the bottom of the synopsis
22:12 run4flat hmm... Method::Signatures requires Mouse?
22:12 run4flat really?
22:13 run4flat anyway, Method::Signatures solves this particular problem really, really well
22:14 run4flat and with Peter Martini's work on adding function signatures to Perl's core, I think that the dependency chain for this sort of thing will get much shorter over time
22:15 run4flat Martini's patch, as I recall, actually allows for pragmatic modules to indicate their signature *parser*
22:15 run4flat so the basic signatures won't have support for type constraints
22:15 run4flat but more advanced ones, eventually, will
22:15 run4flat But then you'll have to require Perl 5.20
22:15 run4flat :-)
22:22 run4flat o/
22:24 _vicash_ You can use Moose basic "isa" to do simple type checking
22:25 _vicash_ depending on your class requirements you can use Roles.. i would advise using Roles only when simple type checking fails. if you have an interface that is your base class and if you inherit from that class, Moose will understand it and match it
22:25 _vicash_ if it doesn't then you will need to write a Role for it and then it will match it
22:26 _vicash_ if your class is super-complex and has unimaginable properties and methods that will need to be type checked then use the slightly lower level MOP layer in Moose
22:26 _vicash_ Moose is built on top of MOP and you can leverage the meta stuff to do type checking
22:28 perigrin Moose generally exposes sugar for 90% of what you'd need to do ... in 6 years I can't remember when I needed to reach for the MOP to do type validation.
22:29 _vicash_ which is why i said "unimaginable properties"
22:29 _vicash_ i myself could not find a use for using MOP
22:58 judd hunting through an isa tree is exactly what I hope to avoid.
23:47 _vicash_ joined #pdl
23:49 jberger I have talked to barefoot and Method::Signatures is investigating a move to Moo
23:50 jberger see also Function::Parameters
23:50 jberger finally: Peter's code does two things, provides a signatures api and a reference implementation
23:51 jberger the reference implelemtation doesn't do types, but I expect other implementation to pop up on CPAN immediately
23:52 jberger possibly including some of the existing projects

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary