Perl 6 - the future is here, just unevenly distributed

IRC log for #pdl, 2013-12-10

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:51 jberger_ joined #pdl
01:52 jberger joined #pdl
05:05 jberger_ joined #pdl
05:16 jberger joined #pdl
05:53 sivoais this poster looks like they should be introduced to PDL :-) <http://www.reddit.com/r/perl/comments/1sara3/n00b_here_would_love_some_support_automate_the/>
13:24 jberger_ joined #pdl
13:39 vicash joined #pdl
14:25 run4flat joined #pdl
14:49 chm joined #pdl
16:14 chm sivoais : the poster also demonstrates why we need simple, clear, beginner tutorials
16:17 chm webart : regarding http://pointclouds.org/ and http://opencv.org/ , 64bit index support and more general types should make these bindings cleaner and more elegant
16:21 chm run4flat: I keep seeing these bender statements and stuff like topic setting, how does that work and can I help?
16:23 run4flat chm, the Bender1 and Bender2 folks are actually bots
16:23 run4flat their job is to ensure that we have channel operators
16:23 run4flat and the only real job of a channel operator is to occasionally update the topic
16:24 chm ok
16:24 run4flat It's really just an irc managerial thing
16:24 run4flat do you want ops?
16:24 run4flat :-)
16:24 run4flat speaking of which ...
16:24 chm Sure but without a manual, it would be a sorcerer's apprentice situation
16:24 run4flat Bender1, trust jberger
16:24 Bender1 OK, run4flat
16:25 run4flat Bender1, trust chm
16:25 Bender1 OK, run4flat
16:25 chm Uh-oh... :-)
16:25 run4flat duh-duh-duuuuuuh
16:25 chm I'm snowed in today so planning to clean up some organizational PDL2 and PDL3 release stuff
16:26 run4flat nice
16:26 chm todo lists, features planned, status at the moment,...
16:26 chm I took a look at NumPy and see that they have 26 developers at the numpy.org site
16:26 chm I don't think we have that many core, 5-ish for PDL
16:26 run4flat yeah...
16:27 chm And 5 is only when we all happen to work at the same time  :-(
16:27 run4flat yeah, a veritable army
16:27 run4flat and yet PDL's threading engine is still more sophisticated...
16:27 chm I think things are getting better but it is difficult to tell.
16:28 chm I'm pretty excited about the thread engine and core clean up work.
16:28 run4flat I've seen some new folks on the mailing list, and that is encouraging
16:28 run4flat I've not yet looked at the thread engine and core cleanup myself
16:28 chm The new error messages Craig put it really make it much easier to generate totally threaded code
16:28 run4flat oh, nice
16:29 chm The final 64bit index fix (which I might get to today---snowed in) is based on the core-cleanup branch
16:29 run4flat cool.
16:29 run4flat I must admit I haven't put too many spare cycles towards PDL stuff lately
16:29 run4flat I've been focused on tinycc bindings
16:29 chm I'm excited about a few things that have come up recently: attributes for dimensions are at the top of the easy list
16:30 run4flat yes, I think those would be nice.
16:30 run4flat After looking over ggplot2 (R's big plotting library), I started thinking about categorical data and PDL
16:30 chm tinycc, great!  I've been working on the 64bit index and getting modern opengl done that I haven't had time to follow up at all.
16:30 run4flat I think I can hack something like categorical data with a subclass of PDL
16:30 run4flat but it'll have to wait for a little while
16:30 chm What is catagorical data?
16:31 run4flat categorical data would be a mapping from an integer to a word
16:31 run4flat like 1 -> gorilla, 2 -> orangutan
16:31 chm Ok, that would easily fall under the planned improved type support.
16:31 run4flat not even
16:31 chm Really?  How not?
16:31 run4flat it should be simple enough to simply subclass PDL
16:32 chm The new type support is a superset of PDL-2.x, so clearly it would be easier  QED
16:32 run4flat heheh
16:32 run4flat sure
16:32 run4flat but my idea is to just use one of the integer PDL data types
16:33 run4flat and include a mapping from integer to name, as well as from name to integer
16:33 chm Sure, like a bi-directional list/pointer
16:33 run4flat sorta...
16:33 sivoais or an enumeration
16:34 chm Is your thought to use tinycc or such to generate threadloop code?
16:34 chm I would imagine it might be a bit trickier than just for looping?
16:34 run4flat no, no
16:34 run4flat it would be almost exclusively for lookups
16:34 run4flat slicing
16:35 chm ok, you're right, easy.
16:35 run4flat for example, $gorilla_weights = $weights->where($category eq 'gorilla')
16:35 run4flat $category would be a categorical piddle
16:35 run4flat it would overload the 'eq' operator
16:35 run4flat which would look up the word "gorilla" and find its value
16:36 chm While this could be glued onto PDL2, I think using the proposed Moo basis, it could be much cleaner.  Maybe a role could work, per-pdl even
16:36 run4flat it would return the equivalent of $weights->where($internal_category_integers == $lookup_integer_for_gorilla)
16:36 run4flat Yes, I think Moo could make it easier to deal with
16:37 run4flat but actually, I can't ever imagine a situation in which one would want to, say, *add* two categorical data sets
16:37 run4flat so, actually, it may not even be necessary to have this class inherit from PDL itself
16:37 chm What about something like a direct product via generator code rather than in-memory, on-demand?
16:38 run4flat I'm afraid I don't quite follow
16:39 chm Sorry, that didn't make sense on reading.  I thought you were going in another direction
16:39 run4flat heheh, I try to keep folks on their toes. :-)
16:39 chm I should probably not opine on categorical data until I have more background experience.
16:40 webart joined #pdl
16:40 run4flat As far as I'm aware, it's most common in biology and other soft sciences
16:40 run4flat astronomers are more likely to describe their data quantitatively
16:40 chm Re attributes in PDL3, the idea would be that we could tag dimensions and dimension sets with information
16:40 run4flat yeah
16:40 run4flat that's a different beas
16:40 run4flat t
16:40 run4flat very helpful, but in a different way
16:40 chm Loops and threading could be applied on the $coords, or $colors,...
16:41 run4flat yeah.
16:41 run4flat $reshaped = $original->mv ('dim-name', -1)
16:41 chm Then it would also allow things like $H->on(mom)->integrate to integrate over momentum or some such
16:41 run4flat hmm
16:42 run4flat so on($dim_name) would basically be like mv($dim_name, 0)?
16:42 run4flat nice
16:42 sivoais are data frames on the PDL TODO list?
16:42 run4flat sivoais, not by that name
16:42 chm Similarly, a piddle could be tagged with ->wrap(pos) which would do the move to dim(0), operate, move back transparantly
16:43 run4flat data frames can be thought of as a hash of piddles
16:43 run4flat so that the top-level "slice" is by category rather than by index or offset
16:43 chm yes, ->on(pos) could be all coords or ->on(x,y) might be the x and y axes
16:43 chm chm looks up data frames
16:44 run4flat chm, data frames are R's standard way of storing data
16:44 chm seems like an R term, what is the difference with piddles?
16:44 run4flat if you imagine that you take household surveys, you might ask how many people live in a house, if they have a TV, how many hours a week they watch TV
16:45 run4flat each question gets its own column in your data frame
16:45 run4flat each household gets a row
16:45 sivoais it's pretty much like a spreadsheet table. You can have multiple types of data along the columns
16:45 chm It seems like the attribute names and categorical data would map very simply to that
16:45 run4flat chm, yes
16:46 chm Maybe maggiexyz will have a nice present next year?  :-)
16:46 run4flat ggplot2 makes very clever use of data frames. When you plot something your plot works on a single data frame; you pick the x-data by name, and the y-data by name
16:47 chm Anyway, with all the power of PDL2 and PDL3 coming, it is critical that some/many simple tutorials and intros get made
16:47 chm perl monks' most recent bit of PDL info is from 6 years ago
16:47 run4flat I saw your email; I decided to swing by Perlmonks
16:47 run4flat haven't done that in a long time
16:47 chm Even PDL core developers aren't aware of what the current PDL can do
16:47 run4flat :-)
16:48 chm I've missed things that you've pointed out, and vice versa
16:48 sivoais the way data frames are integrated into R is rather nice. For example, if you have R, run
16:48 sivoais summary(iris)
16:48 chm The improved docs (intelligent searching anyone) would *really* help
16:49 run4flat sivoais, that's exactly why data frames are so powerful: because they're ubiquitous (in R)
16:49 chm sivoais, is summary(iris) sort of like a data frame version of help $pdl in the shell?
16:49 run4flat It would be easy to create a Perl-alike structure using a hash of arrays, or a hash of piddles (or a hash of both)
16:50 run4flat and because it's so easy, nobody ever formalized it
16:50 chm sivoais, run4flat : this is another case of something that could be simply implemented in modern PDL, if we had the developers/users
16:50 sivoais chm: no, it does summary stats on each column, see <https://gist.github.com/zmughal/b4124b36ccf16797bc8b>
16:50 chm oh, that would be $pdl->statsover
16:50 run4flat but prettier
16:50 run4flat :-)
16:51 chm Of course, PDL doesn't have the column headers natively, which could easily be implemented with the planned work
16:51 sivoais but note that the last column isn't numerical. It's categorical and gives you counts of each category. It's not hard to do, just requires careful thought.
16:51 chm The key is PDL can do all of these things if we can lower the barrier to entry (he hypothesizes)
16:51 sivoais But...! We have R, so we can copy what they do
16:52 chm We need some who knows R and PDL to do it
16:52 run4flat yeah, it's nice to have a working, known interface that people like
16:52 chm I'm tapped out with index support and trying to get OpenGL/TriD back on track
16:53 chm There are similar things between matlab and PDL (dots anyone?)
16:53 run4flat I'm somewhat interested in this, because I'd like to make an interface to PDL::Graphics::Prima that looks like ggplot2
16:53 chm PDL and NumPy,...
16:53 run4flat but it'll be hit or miss for me for a little while
16:53 chm run4flat, my biggest motivation for OpenGL support is that I think the PDL graphics needs to use OpenGL, not glyphs to render
16:54 chm Of course, I haven't had time to get the new opengl stuff working and you've been able to do ok with the native Prima types
16:54 sivoais I'm actually looking at the MATLAB.pod doc right now and trying to see where there can be improvements to the docs
16:54 run4flat sivoais, that would be awesome!
16:54 run4flat that hasn't been cleaned up for a couple of years
16:55 run4flat chm, yeah, I've been pretty happy with Prima support
16:55 sivoais And I might do the same for a possible R.pod
16:55 chm Nonetheless, using a 3D texture, it should be possible to have animations, embedded images, PDF/PS output, printer support, data exploration
16:55 run4flat sivoais, that would be even awesomer!
16:55 run4flat chm, 3D textures can output to pdf?
16:55 chm sivoais, I put together a list of common matlab/octave routine stub.pdl files
16:56 chm the plan was to allow user contributions to provide a more octave/matlab friendly set of methods
16:56 sivoais chm: where are they?
16:56 run4flat chm, animations and embedded images seem obvious, but I didn't think that 3D textures knew how to output to postscript or printers
16:57 chm run4flat, a 3D texture could be output as a view on a volume render, maybe as an animation (clearly there might be other formats for videos..)
16:58 chm sivoais, I never stood them up, no interest and I needed to review them for license issues----been on the back burner for a while
16:59 chm It could easily be reproduced, let me see if I have an example I can show you...
17:01 sivoais btw, you should check out Hadley Wickam's testthat package <http://cran.r-project.org/web/packages/testthat/index.html> for testing (like Test::More)
17:01 sivoais except when things aren't equal, it shows the  relative mean difference <http://adv-r.had.co.nz/Testing.html> ;-)
17:02 sivoais I thought that was a nice touch for a stats tool
17:02 chm http://paste.scsys.co.uk/284315?hl=on&amp;store=on&amp;submit=Format+it!
17:04 sivoais I see. I had that idea, but I was going to approach it another way... I downloaded the entire MATLAB File Central repository and was going to index it for commonly used functions.
17:04 chm sivoais : such stub routines would also serve as tutorial code for octave and matlab users
17:04 sivoais but then I got sidetracked with writing a MATLAB grammar with Marpa :-P
17:04 chm That's where I got my list of stubs, from usage of some matlab using colleagues
17:05 chm checkerboard.pdl just implemented something I needed and a proof of concept
17:05 chm for octave/matlab, starting with size() or all the basics would be better
17:07 sivoais one problem that I foresee with implementing lots of MATLAB functions is that some of them actually change behaviour between MATLAB versions :-O But the basics don't for the most part.
17:07 chm the idea would be training wheels, not racing car
17:08 chm that said, one thing I would like for matlab specifically, is a mex binding to allow bidirectional calls matlab session to pdl and pdl matlab session
17:09 chm That way I could send matlab using colleagues pdl routines and similarly, I could make use of their routines if needed
17:09 sivoais if you have a list of functions you want, I would volunteer to write some
17:09 sivoais Ah, yes. I've done some MEX in the past
17:11 chm sivoais, I have the list I generated from two matlab users, probably more indicative than representative, I can send you the list if you like
17:11 sivoais sure, that'd be great
17:13 chm sivoais, here you go http://paste.scsys.co.uk/284316
17:14 sivoais thanks!
17:14 chm stepping out, nice chatting, maybe more later, o/
17:17 run4flat chm++, sivoais++
17:17 run4flat sorry I had to step away for the moment
17:17 run4flat and now, off to review an exam
17:17 run4flat :-)
19:10 preaction joined #pdl
19:40 vicash jberger: is there a definitive document on how to use Alien::Base ? perldoc says nothing except that it is a base class. For example, I want to create a PDL::Finance module that depends on a C library and want to use Alien::Base to pull that dependency in instead of using wget/LWP to download a tarball and build it
19:42 jberger vicash: you must not have looked at that tiny document very closely
19:43 chm joined #pdl
19:43 chm jberger, you forgot the smiley...
20:04 vicash jberger: you're right. my mistake.
20:05 jberger :-)
22:02 drrho joined #pdl
22:29 run4flat o/
23:50 chm joined #pdl

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary