Perl 6 - the future is here, just unevenly distributed

IRC log for #pdl, 2013-06-11

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:16 _vicash_ joined #pdl
04:40 Meiermann joined #pdl
13:21 _vicash_ joined #pdl
13:52 chm joined #pdl
13:54 chm run4flat: Have you had time to put your PDL3 thoughts together?
13:54 chm jberger: Ditto...
13:55 jberger run4flat and I have discussed at length
13:55 jberger but its usually more like PDL5 :-)
13:55 chm Great, anything ready to come out of the black hole event horizon?
13:56 jberger or NewPDL
13:56 jberger the problem is attaching the C level data
13:56 chm Hmmm.  That was sort of what I got from David's brief "not Moo[se] post" but there were no details.
13:56 jberger while exposing a useful/modern object
13:56 jberger he is working on extracting the Prima object system from the GUI code
13:57 jberger and from what I've seen, you should like the end result :-)
13:58 chm Sounds intriguing.  My main aim for PDL3 is that it works, allows for migration forward, and is done by end of August
13:59 chm The Moo[se] stuff was just a hand wave at supporting more modern object infrastructure
13:59 jberger right, which is why I don't think anything we are doing is likely coming for PDL3
13:59 chm and have some interoperability with other Perl modules.
13:59 jberger I would love a Moo[se] like object, the problem is how to do it
14:00 chm I think I am talking about something orthogonal to what you and David are here.
14:00 chm The idea was to support Moo[se] from PDL objects but not be Moo[se] objects
14:01 jberger chm, how would you do that?
14:01 chm since there really is no nice C-level object for Moo[se], just hashes.
14:01 jberger right, thats the problem
14:02 chm I'm around today and can irc for a change so I'll try to write something up and come back
14:02 jberger ok sounds good
14:02 chm later.  Have to step out for a while.  Later o/
14:03 jberger o/
14:03 jberger I'm very curious to see what your come up with, we keep arriving at the need for a C level object system
14:11 run4flat_ joined #pdl
14:11 run4flat_ o/
14:11 run4flat_ hola, chm
14:11 run4flat_ jberger: thanks for alerting me. :-)
14:15 jberger np
14:16 jberger "chm> I'm around today and can irc for a change so I'll try to write something up and come back"
14:16 jberger is the log up
14:16 jberger ?
14:16 jberger it looks like it
14:16 * run4flat_ checks
14:18 * run4flat_ is caught up
14:18 run4flat_ well, since chm is on, this'll get output to is irc client, so I'll just let it rip
14:19 run4flat_ First of all, Moo[se] compatible PDL would be pretty cool and I think it would be fun to get that to work
14:19 _vicash_ joined #pdl
14:19 run4flat_ for example, PDL as a supported type for Moose would be great
14:20 run4flat_ The ideas I've been kicking around are much to grand to get done by August
14:20 run4flat_ unless I take off work from now until August. :-)
14:21 run4flat_ oh, re Moose/PDL integration, I think it would be pretty easy to get Moose to use a PDL back-end
14:21 run4flat_ just like you can have inside out objects in Moose, I bet we could have PDL-based objects in Moose
14:21 run4flat_ they would just (ab)use the PDL header
14:38 run4flat_ joined #pdl
14:38 run4flat_ argh, internet connection was being dumb
14:39 * run4flat_ checks log to see what was actually posted
14:39 run4flat_ Good, not too much
14:39 run4flat_ on a related topic, jberger and I have discussed reworking PDL :: PP using Moo[se]
14:40 run4flat_ I was initially *quite* skeptical it would work
14:40 run4flat_ PDL :: PP essentially creates a C-based object system for PDL data structures, except there is absolutely no way to tweak most of it at runtime
14:40 run4flat_ let alone introspect it
14:40 run4flat_ The current pipeline architecture is about as good as it gets, I think
14:40 run4flat_ but I thought some more on it
14:40 run4flat_ and I realized that we might be able to take a different approach, in which PDL :: PP2 provides a ghost class heierarchy of sorts
14:40 run4flat_ that way, if you want to create a slice method that is only slightly different from, say, range, you would simply inherit your slice method class from range and apply your modifications
14:40 run4flat_ Then you would instantiate an object of your PDL :: PP2 slice class and call the "generate_code" method, or some such
14:40 run4flat_ Of course, this means that PDL :: PP2 would have to also create modules that other PDL :: PP2 scripts could consume at their Build time
14:41 run4flat_ 've played around with this a little bit
14:41 run4flat_ see https://metacpan.org/module/PDL::Fit::ExpRate
14:41 run4flat_ which provides a PDL method for fitting an exponential decay
14:41 run4flat_ it could be more robust...
14:42 run4flat_ but the interesting part for this discussion is this: https://metacpan.org/module/P​DL::PP::Include::Fit::ExpRate
14:44 run4flat_ Alas, that's not superbly documented
14:45 run4flat_ the idea is that in your .pm.PL file, you "use" this module, and this module makes a few PDL :: PP calls to set up function pointers that you can use in your own PP code
14:46 run4flat_ It adds some code to your boot section that ensures that PDL::Fit::ExpRate has been loaded and sets up the function pointers
14:47 run4flat_ alright, that's enough of my thoughts for now. I'll wait until everybody has caught up. :-)
15:16 chm run4flat, jberger: splatting some thoughts in response to the PP discussion...
15:17 chm Re: PDL::PP using Moo[se]
15:17 chm I think it would make the code more comprehensible and that the possibility of MOP expansion (from Moo to Moose) means that some of the PDL::PP2 ideas might be applicable as a role or some such.
15:17 chm MOP and roles seems to correspond roughly to the idea of a "ghost class hierarchy"
15:17 chm As to moving PDL to use Moo[se], my thought was to start with the existing PDL OO support for inheritance via has-a.  We would replace
15:17 chm the idea of a generic hash with a PDL key by a Moo[se] clase with a PDL attribute.  This automatically gives us a quick boost to modern
15:17 chm OO programming, allows for cleaner implementation of the PDL hdr hash stuff---see mailing list why this is a problem---and the work
15:17 chm is pretty much done since the support is already there for the hash case.  Specifying to Moo[se] should be a minor refactoring job.
15:18 chm The *very cool* thing is that PDL already has support for the case that the PDL key/attribute is a code reference---did you know that?
15:18 chm I remember thinking of how to implement piddle data flow callbacks in a list discussion and now realize if I had used an "inside out"
15:18 chm object, it was already there.
15:18 chm The one impact I can see for PDL3 vs PDL2 performance by going to the Moo[se] objects is for object creation and things like that.
15:18 chm But with MOP, we have the possibility of "lifting" the hash part out to replace it by p2 or whatever coolness David and Joel are
15:18 _vicash_ joined #pdl
15:18 chm brewing in the bat cave....
15:18 chm Looking at David's PDL::PP::Include::Fit::ExpRate, I think using Moo[se] and MOP could make the implementation more general still.
15:19 * run4flat_ was away; reads
15:20 chm Sorry about the cut-n-paste format.  I need to see if cygwin has a real irc chat program I can use...
15:20 chm I'm using Mibbit at the moment with not much control
15:20 run4flat_ yeah, me too
15:21 run4flat_ ok, I'm caught up
15:22 run4flat_ Q: "PDL already has support for the case that the PDL key/attribute is a code reference" I have *never* seen this!
15:22 chm This should be low risk, doable by end of Summer and will give a way to migrate to PDL5 more seamlessly
15:22 run4flat_ where is this documented and/or implemented (if not documented)?
15:22 run4flat_ I knew that you could have a derived class be hash-based, and PDL would automagically pull out the piddle under the PDL key
15:23 run4flat_ but a code reference... neat!
15:23 chm Look in PDL...sorry try pdldoc -a object, my system is hanging for some reason....
15:24 * run4flat_ checks
15:24 chm PDL::Objects    Manual: Object-Orientation, what is it and how to exploit it
15:24 chm It's in there.  I read it a long time ago but never needed it.
15:24 run4flat_ whereis PDL::Obje ts
15:24 liddle_piddle_bot run4flat_: Could not find PDL::Obje ts.
15:25 run4flat_ whereis PDL::Objets
15:25 liddle_piddle_bot run4flat_: Could not find PDL::Objets.
15:25 run4flat_ liddle_piddle_bot: c'mon
15:25 jberger whereis PDL::Objects
15:25 liddle_piddle_bot jberger: Found PDL::Objects in PDL::Objects.pod (http://pdl.sourceforge.net/PDLdocs/Objects.html)
15:25 run4flat_ thanks, jberger
15:25 chm That's it.
15:26 run4flat_ Here's the link on CPAN, which looks prettier (IMO): http://search.cpan.org/~chm/PD​L-2.006/Basic/Pod/Objects.pod
15:26 chm I didn't do much with it since the management of inheritance was iffy.  Look at the issues with PDL::Complex stuff.
15:26 run4flat_ hahaha: "NEED STUFF ABOUT CODE REFs!!"
15:26 run4flat_ oops
15:26 chm However, Moo[se] and roles would be a perfect combination.
15:27 run4flat_ I've been meaning to read that thread for some time. :-)
15:27 chm The code refs, I don't know, maybe it was when I was reading the PDL/Basic/Core stuff for the 64bit migration
15:27 chm Read the source, Luke...
15:27 run4flat_ well, yeah, I just figured the coderef stuff could be almost anywhere, so I asked. :-)
15:28 chm Yes, I know it should be doc'd but you know how it goes...
15:28 chm The code ref handling is the same place in the core that handles the check for a hash and then PDL key
15:29 run4flat_ ok, that helps. I don't know where that's located exactly, but it'll give me more to ack for
15:29 run4flat_ Oh, btw, I had an idea for lexically scoped PDL methods
15:29 run4flat_ yet another idea
15:29 run4flat_ :-)
15:29 run4flat_ but maybe we should hold off on that one for the moment
15:29 chm Well, here is an idea you and Joel won't/don't appear to like but I think would be huge for PDL3....
15:30 chm Get the Alien::XXX stuff working for dependencies so that can be moved out of the PDL core.
15:30 chm We could then safely spin chunks of the library based code out to separate CPAN modules
15:30 chm Making the kernel more lean, mean and portable....
15:30 run4flat_ chm, I think that is a *FANTASTIC* idea!
15:31 run4flat_ :-)
15:31 run4flat_ I've been a bit skeptical as to whether we can get arbitrary autoconf-based libraries to build with Alien::Base on Windows machines
15:31 run4flat_ but that has nothing to do with whether we should spin them off.
15:31 run4flat_ :-)
15:31 chm We don't need arbitrary libraries to build, just the ones for PDL
15:31 jberger agreed, however I wonder one thing, why is spinning those out dependent on Alien modules? I would think that their continued presence in PDL core is dependent on Alien modules
15:32 jberger the reason to spin them off is that they are hard to install right?
15:32 chm Because the probing for parameters and stuff is all intertwined with the Makefile.PL and build time config stuff
15:33 chm If we move to Alien::PROJ4, for example, that mess becomes 'use Alien::Proj4;' and then everything is simple.
15:33 jberger dont get me wrong, I'm all for 1) spinning them off and 2) Alien modules
15:33 jberger well I'm happy to be a guide
15:33 run4flat_ I think we all agree on this ^^
15:34 chm Another motivation is that that would put PDL at a good place in that improvements
15:34 chm going forward would not require so much spin up time and expert knowledge.  PDL now builds
15:35 chm "everywhere" but the dependency stuff makes it pretty fragile without care.
15:35 chm I'm also at the point that I need to spend some time using PDL rather than pushing it forward so
15:35 chm this could be my last push for quite a while.  If PDL is not cleaned up and ready to move forward, it may
15:36 chm languish...
15:36 run4flat_ chm, understood
15:36 chm Not saying I won't be contributing but won't have time to push for releases and stuff...
15:37 chm Fortunately, I actually have some time coming this summer that I can use for PDL work
15:37 run4flat_ I think it's about time I finally caught up on all those PDL email threads I've been holding off reading
15:37 chm Top of my personal list is to finally update OpenGL and refactor TriD to be what it should be
15:38 run4flat_ what are the most relevant threads for this?
15:38 run4flat_ yeah, new OpenGL and TriD would be great
15:38 chm Unfortunately, they are spread out over a number of months/years
15:38 run4flat_ I just meant the most recent ones, from mid-march or so
15:38 run4flat_ I think I have most of the other conversations in my head. :-)
15:39 chm I'm trying to set up a web2project for PDL3 on our sf.net that we can use to coordinate the effort
15:39 run4flat_ I mean, does the discussion about complex numbers have something I should read *right* *now* to be conversant?
15:39 run4flat_ :-)
15:39 chm The discussion about complex numbers is the general problem of better type support
15:40 chm which is why I think your idea to refactor the PDL::PP to use Moo[se] might be a big
15:40 chm help to simplify any changes going forward.
15:40 liddle_piddle_bot chm: http://pdl.perl.org/?docs=to simplify any changes going forward.&title=PDL::to simplify any changes going forward.
15:40 run4flat_ haha
15:40 chm (not sure what happened there---any docs for liddle....?)
15:41 run4flat_ yeah, send a /msg to liddle_piddle_bot with some text
15:41 run4flat_ it'll give you a link
15:41 chm I don't know what a /msg is, I was just typing and the response happened
15:42 chm Looks like it was in response to the word 'help'
15:42 run4flat_ sorry, I meant, that would cause liddle_piddle_bot to send you the link to its docs
15:42 run4flat_ yes, it was
15:42 run4flat_ https://github.com/PDLPorters/pdl/wiki/PDL-IRC
15:43 chm Thanks
15:43 chm FYI, here is the link to web2project: http://web2project.net/
15:44 chm The other items on my plate: finish the new improved NiceSlice support since I know what to do, just need to do it
15:45 chm I'm behind schedule but should have web2project up by next week.
15:45 chm I think it would be valuable to try a few skype/web meetings of PDL developers for PDL3 work planning...
15:46 run4flat_ yes, I agree
15:46 chm If we could get together for a PDL3 hackathon---even if virtual, that could be useful...
15:46 run4flat_ indeed!
15:47 chm BTW, I'm trying web2project because it looked simple.  Some of the functionality overlaps with github.com
15:47 run4flat_ btw, I don't know if I told you, but I'm moving to Carlisle, PA in mid-July
15:47 run4flat_ so, we might be able to get together some time over the summer to hack on things. :-)
15:47 chm but there was a lot of embedded dependencies and a lack of flexibility.  On sf.net, we can control the web site ourselves.
15:47 chm That would be great.
15:47 run4flat_ yep
15:48 run4flat_ is there a good getting-started-for-users document for web2project?
15:49 run4flat_ the wiki is... a bit scant
15:49 chm It is actually pretty simple, you set up the database, put the files in a location on your site, and it has scripts that handle the install.
15:49 chm There is a link to a PDF on getting started under docs I think let me look...
15:50 chm It is under downloads, web2Project Guides
15:50 run4flat_ great, thanks
15:50 chm http://sourceforge.net/projects/web​2project/files/web2Project%20Guides​/web2Project%20User%20Guide%20v1/
15:51 * run4flat_ reads
15:51 run4flat_ haha, love the stock photos
15:52 chm The reason it is not set up already is that I need to learn enough about web stuff now
15:52 chm to ensure I don't mess anything up.  Someone with more web knowledge could probably set things
15:52 chm up in an hour or less.
15:54 run4flat_ ooh, Gantt charts. That's not on Github. :-)
15:54 chm I know
15:55 run4flat_ (bear in mind, I am *terrible* at estimating the time it'll take me to complete something, but then again, maybe it'll improve with practice)
15:55 chm The cool thing is that I think it would be a way to make a more asynchronous project team for PDL work better...
15:56 chm I could see people working on tasks, and documenting steps needed if they step away and someone could chip in from there...
15:56 * run4flat_ nods
15:56 chm If we know where things stand, we can add help or modify schedules as required...
15:57 chm Back to the type support: I was thinking if we could get some sort of run time PDL::PP support
15:58 chm working then we might simplify things by having a default type->double->compute->type
15:58 chm and then generate specific code for cases as needed.  In many cases, I think the generic loop with casting
15:58 chm could work good enough which would make things simpler as far as types go...
15:59 run4flat_ eh, I fear that run time PDL :: PP support is a big project
15:59 run4flat_ unless we also just JIT
16:00 run4flat_ and if that's in the cards, we should look closely at TCC
16:00 run4flat_ see http://bellard.org/tcc/ and http://en.wikipedia.org/wiki/Tiny_C_Compiler
16:00 chm Actually, with Moo[se] and MOP, I think we could apply that at runtime and cache the result
16:00 chm rather than only at config/build time.
16:01 chm I like the p2 stuff better since tcc is platform restricted isn't it?
16:01 run4flat_ TCC compiles code for x86, x64, and ARM
16:01 run4flat_ for jit-compiling, it works on Linux, Windows, and Mac
16:01 run4flat_ it cannot compile down to an executable on Mac (I don't think)
16:02 run4flat_ because Macs use Mach-O binary formats instead of ELF
16:02 run4flat_ but it can create executable code, and it compiles very quickly
16:03 chm Neat.  Of course the trick is not to get sucked into a huge, low-level development effort at first.  Just set things up for it to work with PDL3 going forward
16:03 chm Otherwise we'll never get there from here.  :-)
16:03 run4flat_ I agree. Just something to keep in mind
16:03 run4flat_ but, I don't see how we would setup runtime support for PDL :: PP
16:03 chm That is exactly the type of thing I was thinking about with MOP and JIT
16:04 chm Add in the OpenCL and CUDA and it gets amazing!
16:04 run4flat_ yep. :-)
16:04 run4flat_ A lot of the PDL :: PP code modifications modify the source code that gets compiled
16:04 run4flat_ If we want runtime modifiable stuff, we would need those modifications to change an entry in a vtable
16:05 run4flat_ but most of PDL :: PP's functionality doesn't use a vtable
16:05 chm Where is that.... all the PDL::PP stuff I know just produces C code to be compiled
16:05 run4flat_ yes, exactly
16:05 chm it generates, not modifies.  More like table based code generation.
16:06 run4flat_ which means that if we want to have runtime changes, we have to compile new code
16:06 chm Any vtable runtime changes would probably be in the core somewhere and not PP
16:06 run4flat_ at runtime
16:06 chm Right, we already have Inline::PP
16:06 chm I'm not trying for JIT, just adding another type support after install as needed.
16:06 run4flat_ yes, and that could be our starting-off point
16:06 run4flat_ ooooh
16:07 run4flat_ runtime addition for new types
16:07 run4flat_ thank you
16:07 chm That would mitigate the need for the huge thread loop unless they were needed.
16:07 run4flat_ sorry for my confusion
16:07 chm A lot of the PDL slow compiles is the huge set of types we build for.
16:07 run4flat_ yeah. Arghhh
16:07 chm Start with one default (or a few) and add the ability to supplement, things could go much faster.
16:08 run4flat_ I usually explicitly specify only the types I want my code to support
16:08 run4flat_ makes compiles a lot faster
16:08 chm If that is a useful feature, it needs to be supported.  One shouldn't need to be a PDL guru or a Perl developer to make PDL sing...
16:09 chm What I've proposed seems reasonable to achieve by Summer's end.
16:09 * run4flat_ is thinking
16:09 chm Especially with coordination and help from our current cadre of PDL developers
16:09 run4flat_ the major problem I see with reducing the size of the typeloop is that we will loose in our benchmarks
16:10 run4flat_ If we provide type coersion, that could potentially involve a lot of copying of data
16:10 chm I don't see why the current build-all wouldn't be available if needed.
16:10 run4flat_ oh, ok
16:11 run4flat_ you have a build-all-and-include-generic-type idea?
16:11 chm Also, with tcc/jit stuff, we may be able to do binary loadable modules by type or some such
16:11 * run4flat_ steps away for a second
16:12 chm The idea is that the type coercion would happed in the CPU.  That is processor intensive but with modern architectures, it might not be so bad.
16:13 chm 4chm steps away too
16:15 run4flat_ chm, I'm not quite sure what you mean by "type coercion would happen in the CPU"
16:15 run4flat_ do you mean we would have lazy type coercion?
16:16 run4flat_ even if we do that, the threading engine will have to convert everything requested by the signature
16:16 run4flat_ so that if you want to run a Fourier transform on a gigabyte long time sample, it'll have to convert all your data to a double array first
16:16 run4flat_ Obviously, for Fourier transform, that's how it presently works anyway
16:17 run4flat_ sorry, let me clarify that last statement
16:17 run4flat_ suppose I have a piddle with 10 x 1,000,000,000 ints
16:17 run4flat_ 10 billion ints
16:18 run4flat_ obviously memory mapped
16:18 run4flat_ now if I perform a Fourier transform, PDL will create a single 1-billion element double array and copy those ints into the double array
16:18 run4flat_ it doesn't convert all 10 billion ints at once; it only works with 1 billion ints
16:19 run4flat_ but it's still converting 1 billion ints to 1 billion doubles before performing the operation
16:20 jberger ok, so I'm WAAAYYY behind here, but I cannot even construct a PDL subclass correctly
16:21 run4flat_ really?
16:21 * run4flat_ has never actually tried
16:21 run4flat_ did you look at the test suite?
16:21 run4flat_ t/subclass4.t
16:21 jberger not yet, no
16:22 run4flat_ dear god, hand-written tap
16:22 * run4flat_ shakes fist
16:23 run4flat_ whoah, subclass.t exercises the coderef thingy
16:24 jberger ok, well my problem is the constructor functions, like xvals
16:25 run4flat_ I think you need both an initialize and a copy method
16:25 run4flat_ for your subclass
16:25 jberger and subclass4.t also has its own new
16:26 run4flat_ yeah, all the subclass methods seem to have their own new
16:26 run4flat_ s/methods/tests/
16:27 run4flat_ I'm itching to just rewrite these tests using Test::More
16:27 run4flat_ I know, it doesn't matter, but still...
16:28 jberger but if they use their own new, then the exported PDL constructor functions have no chance of working correctly
16:28 * jberger is thinking of a Moose class that delegates correctly
16:28 run4flat_ ??
16:28 jberger it shouldn't be PDL's job to notice that its an attribute
16:28 run4flat_ $a = xvals(20) <-- this is a normal, double PDL
16:29 run4flat_ $b = PDL::Derived->new(20)  <-- this is a derived PDL
16:29 run4flat_ ugh
16:29 run4flat_ $b = PDL :: Derived->new(20)
16:29 run4flat_ $c = $b->xvals  <-- this should be derived
16:29 jberger ok
16:30 run4flat_ method calls on already created piddles copy the type
16:30 run4flat_ otherwise, you get a double PDL. :-)
16:30 run4flat_ or, you have to specify the type as the first argument
16:30 * run4flat_ just realized a hole in his logic
16:31 run4flat_ what does $a = xvals('PDL :: Derived', 20) give you?
16:31 run4flat_ and, what should it give you?
16:31 run4flat_ :-)
16:33 jberger joined #pdl
16:34 jberger AAAAAAAAAAANNNND that test crashed my computer
16:34 chm 4chm is back
16:34 chm 4chm catching up on a lot
16:34 run4flat_ hahaha!
16:34 run4flat_ jberger: did that test actually cause you to disconnect from irc?
16:34 run4flat_ HAHAHA
16:35 jberger caused me to need to hard reboot!
16:35 jberger liddle_piddle_bot, paste
16:35 jberger paste
16:35 liddle_piddle_bot jberger: http://scsys.co.uk:8001/
16:36 shadowpaste "jberger" at 217.168.150.38 pasted "subclass?" (31 lines) at http://paste.scsys.co.uk/254974
16:38 run4flat_ this forced you to hard reboot?
16:38 run4flat_ wow
16:38 chm I get an out of memory error on cygwin/win7
16:38 run4flat_ but the docs say: "All PDL constructors will call initialize() to make sure that your extensions are added by all PDL constructors automatically"
16:39 chm I think some of these issues have been around for a while which is why not many examples of subclassing PDL
16:39 run4flat_ nobody has a need to subclass
16:40 run4flat_ It is a rare thing to need something to have an "isa" relationship with a piddle, and yet not actually be a piddle
16:40 jberger I think its worth rethinking that whole mechanism tbh
16:40 chm I was meaning via the has-a implementation.  Not many fully working examples of that
16:41 run4flat_ but...
16:41 chm I was thinking that maybe somehow PDL-ism could be a role
16:41 run4flat_ an object that has-a piddle works just fine
16:41 chm Of course that was why going to Moo[se] had promise for the future refactoring and architecture changes
16:41 run4flat_ it's when you want said class to behave like a piddle
16:42 chm but shouldn't it?
16:42 jberger if you just has-a you don't get all the overloading
16:42 run4flat_ I guess I'm just trying to argue that code is better reused by writing a PDL method
16:42 jberger it must be isa, but have delegation
16:42 run4flat_ so everybody writes PDL methods
16:43 run4flat_ instead of creating inherited data types
16:43 chm right
16:43 jberger monkey-patching forevermore
16:43 run4flat_ right
16:43 run4flat_ which brings me back to my idea of lexically-scoped methods
16:43 run4flat_ :-)
16:43 chm a complex piddle should be a piddle too, right?
16:43 jberger but that whole concept is not very moosey
16:44 run4flat_ jberger: lexically scoped methods are not moosey, no
16:44 run4flat_ chm, that is one of only two cases I can think of, yes
16:44 run4flat_ the other being a matrix class
16:44 run4flat_ which, essentially, inverts the first two dims
16:44 run4flat_ though I've never had a need for that in practice
16:44 chm because it doesn't really work, I think
16:45 run4flat_ I've never needed a matrix subclass, I think, because I conceive of my code as working on arrays, not matrices
16:45 run4flat_ more of a programmer's approach than a mathemtatician's
16:46 run4flat_ chm, here's an idea for custom types
16:46 run4flat_ allow for arbitrary objects to be passed into the threadloop
16:46 run4flat_ er, up to the threadloop
16:46 run4flat_ then, figure out what type is going to be executed in the threadloop
16:47 run4flat_ and call the custom type's 'get_dataref_for_type' method
16:47 chm That was also in the list of improved type support.  It would allow fully generic types (at some performance cost, I expect)
16:47 chm The trick is how to drop into optimal type-specific code if/when needed
16:47 run4flat_ but that's just the point
16:47 chm But I agree
16:48 run4flat_ it would only be an extra check
16:48 run4flat_ methods would only be called when the type is custom
16:48 chm As for lexical methods, couldn't something like that be implementable via method modifiers?
16:48 run4flat_ maybe?
16:48 run4flat_ seems unlikely, though
16:49 chm what were you thinking of?
16:49 run4flat_ My idea was to implement an AUTOLOAD that would ask the hints hash for a mapping from a bareword method name to a fully-qualified package name
16:49 run4flat_ let me cook up an example
16:50 chm That could be done via a Moo[se] role (class or object)
16:52 shadowpaste "run4flat_" at 217.168.150.38 pasted "example usage, lexical methods" (17 lines) at http://paste.scsys.co.uk/254981
16:52 run4flat_ no, I mean lexical from the user's standpoint
16:52 chm This discussion reminds me that we should probably add some basic performance benchmarks to PDL3 build process tests (so we can see if things are being slowed down or sped up).
16:52 run4flat_ yes, we should. :-)
16:53 run4flat_ The idea about lexical methods is that it would provide a very contolled means for polluting the PDL namespace, and only within your current lexical scope
16:53 run4flat_ As far as I know, this hasn't been done in the Perl world
16:53 run4flat_ because if you want to have derived functionality, you create a derived type
16:53 run4flat_ but PDL is different. We have a type that has all kinds of uses
16:54 run4flat_ so we add methods to our symbol table
16:54 * run4flat_ apologizes for his rant
16:55 shadowpaste Someone at 217.168.150.38 pasted "subclass" (19 lines) at http://paste.scsys.co.uk/254982
16:56 chm Maybe I don't understand something but the method calls don't pollute.  It is the fact
16:56 chm that use PDL dumps a bunch of things into main: for convenience that causes the problems.
16:56 run4flat_ actually, I'm not worried about polluting main.
16:56 chm I think we should enable use PDL qw(); so everything would be methods
16:57 run4flat_ I'm worried about having two different modules that define the fft method, for example
16:57 jberger chm isn't that what use PDL::Lite is
16:57 run4flat_ though, polluting main is something we should address
16:57 chm Roles would give an error
16:57 chm You could specify delegation if needed.
16:57 run4flat_ but roles apply at class creation time
16:57 run4flat_ or, you modify the whole class
16:58 run4flat_ btw, who posted the latest paste? Does it work?
16:58 chm Roles can be applied to objects as well (they create an anonymous class with the features to which the object is blessed)
16:58 run4flat_ oh
16:58 run4flat_ cool
16:59 chm I agree about use PDL::Lite, but maybe we should support use PDL qw() to do the same thing---more of a standard usage thing
17:00 chm The use of roles and composition is one of the reasons I think going to Moo[se] a a basis would enable
17:00 jberger sorry, it was me, and yes it does
17:00 run4flat_ yes, I agree
17:00 run4flat_ PDL's import neds to be standardized
17:00 run4flat_ jberger: cool
17:00 chm major steps going forwards (implement in perl first, then other performance as neede)
17:00 jberger for Moose proper you need `use Moose; use MooseX::NonMoose;`
17:00 run4flat_ chm, I think we're addressing two different things
17:01 run4flat_ you're talking about making it super-easy to subclass
17:01 run4flat_ I'm talking about controlling the PDL:: namespace
17:02 chm Right but the dynamic application of roles to objects is an already extant method to base a first implementation upon.
17:02 chm We get it working first, then revise, refactor, JIT, ...whatever makes sense.
17:02 run4flat_ yes, sure
17:03 chm The key is we can start playing around without diving into the low level details immediately.
17:03 chm And we can layer on functionality to try things out before jit/tcc or p2 or whatever.
17:04 run4flat_ right, that is why subclassing PDL is of fundamental importance
17:04 run4flat_ My notion of lexical methods was a different beast
17:04 run4flat_ tangential to that conversation
17:04 chm I understand but maybe that discussion is more orthogonal than tangential?  :-)
17:04 run4flat_ haha, sure
17:05 chm Another reason for Moo[se], we can start trying things out with PDL3 directly.
17:05 chm Frankly, I would like to avoid significant C-code changes this go-around since the testing and verification would be a huge issue.
17:06 chm Enough changes with the jump to 64bit support.
17:06 chm BTW, have either of you had a chance to try it out?  Any performance impacts?
17:06 run4flat_ I am pretty sure I have the latest on my machine
17:06 run4flat_ I haven't noticed anything
17:06 run4flat_ so, good to go here. :-)
17:07 chm There are still issues with 32bit platforms that we need to work out
17:07 chm I'm also interested in 32bit vs 64bit benchmarks to make sure we don't do someting
17:07 run4flat_ yet another email thread 've put off reading. Sorry
17:07 chm stupid like break the cache
17:08 run4flat_ I can't imagine that switching to 64 bit would break the cache
17:08 run4flat_ operating on huge datasets enabled by 64 bit support -- now *that* would break the cache
17:08 chm The size of the pdl struct is different, register pressure will be doubles for index values,...
17:08 run4flat_ but that is to be expected
17:08 run4flat_ sure, sure
17:09 run4flat_ but piddles are relatively few in number, so the increased struct size isn't that big a deal right?
17:09 chm yes, but there a a number of ways to mitigate any problems---if we know they are there.
17:10 chm This may all be moot since modern processors are bigger in many ways that the cpus PDL was first developed for.
17:10 run4flat_ yeah. I think it's unavoidable
17:10 run4flat_ and it'll become less of a problem as CPU caches grow
17:10 chm For example, if the pdl struct is now more than one cacheline size, then it will take
17:11 run4flat_ oh, yeah, that's true
17:11 chm more IO to load it, and it will have 2X the chance of being purged during operations
17:11 chm That kind of thing can *kill* performance.
17:11 run4flat_ wrestling the PDL struct size is not a project for PDL3, though
17:11 run4flat_ there are a lot of unused pointers in the PDL struct
17:12 run4flat_ or, I thought there were
17:12 chm There are a number of preallocated entities in pdl that could be adjusted.
17:12 run4flat_ yep, that too
17:12 chm Not saying this is a problem, but I think we should check.  The more agressive mods to the architecture should definitely monitor performance
17:13 chm Since one goal is to get things closer to the bare metal in performance.
17:13 run4flat_ yes, found what I was looking for: http://perldoc.perl.org/perlpragma.html
17:13 run4flat_ this is *impossible* to find
17:13 run4flat_ anyway, I would hook into this to create lexically scoped methods
17:14 run4flat_ sorry for the orthogonal conversation again
17:14 run4flat_ :-)
17:14 chm That would be nice but I think it could be done independent of PDL specific things
17:14 run4flat_ sure
17:14 chm Other perl modules might use such a feature
17:14 run4flat_ haha
17:15 run4flat_ there are many things of general use pioneered in PDL
17:15 run4flat_ that stayed in PDL
17:15 run4flat_ I'll play with it and see what I can come up with
17:15 run4flat_ :-)
17:15 chm Speaking of which, I think we should move to using PDL::Test or whatever for our tests in the Core.
17:16 run4flat_ chm++
17:16 chm What do you think?  A lot of cruft could go away in t/
17:16 run4flat_ but, didn't edward vote against that?
17:16 run4flat_ I thought he wanted to keep it as its own module
17:17 chm We would just add it as a dependency.  We could distribute a copy in our inc/ folder for the build to avoid that even...
17:17 run4flat_ ah, perfect
17:17 chm I think more of PDL should be on CPAN with a PDL bundle for the whole kitchen sink version.
17:17 run4flat_ honestly, I wish he would let us pull it in
17:17 run4flat_ yep
17:17 chm That is another argument for getting the Alien support in there.
17:18 run4flat_ because then a kitchen sink install could be dead-simple
17:18 chm Well, I've enjoyed this conversation and have a more positive hope for where we're going.
17:18 run4flat_ just install, no conditional dependencies, etc
17:18 run4flat_ yep, me too
17:19 run4flat_ I'll start digging into some ideas you've got me thinking about
17:19 chm Got to get back to other thing herel  I'll post a follow up to my Moo-dest proposal and let you
17:19 chm know when the web2project is up and available.  o/
17:21 run4flat_ sounds good
17:21 run4flat_ o/
17:21 run4flat_ question: is this "scratching my head"?   o'\
17:21 run4flat_ just wondering.
17:21 run4flat_ :-)
17:21 run4flat_ or would it just be o\
21:01 MeierM joined #pdl
21:07 run4flat_ jberger: Alright, I got it to work: lexically scoping a set of methods for a specific class: https://gist.github.com/run4flat/5760681

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary