Perl 6 - the future is here, just unevenly distributed

IRC log for #pdl, 2013-12-01

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:58 jberger__ joined #pdl
02:50 Mithaldu how do i get one-dimensional slice from two-dimensional piddle
03:12 sivoais Mithaldu: $p = sequence(10,10); $p(2,:)->squeeze; # is that what you mean
03:13 Mithaldu sivoais: are you going to be around for a few minutes so i can dump shit at you and figure this crap out?
03:13 sivoais Mithaldu: $p(:,3); $p(,(3)); $p(,3;-) # TIMTOWDI
03:13 sivoais Mithaldu: sure :-)
03:13 Mithaldu good
03:14 Mithaldu so i got this thing: https://gist.github.com/wchr​istian/d84b4b31fbf4a3132707
03:14 Mithaldu and i run this:
03:14 Mithaldu $pVertices->slice( "3,0:2" ).''
03:14 Mithaldu and the result i see is this:
03:14 Mithaldu [
03:14 Mithaldu [0 1 0]
03:14 Mithaldu ]
03:15 Mithaldu how do i get, instead of that, this:
03:15 Mithaldu [0 1 0]
03:16 Mithaldu also
03:16 Mithaldu i refuse to use niceslice
03:16 Mithaldu i'm already dealing with enough magical bullshit here
03:16 Mithaldu i don't need source filters on top of that
03:17 pdurbin :)
03:17 sivoais yeah, just run squeeze() on it. It gets rid of all dimensions of length 1 (singleton dimensions)
03:17 Mithaldu and that slice will still be properly connected to the values in $pVertices so i can add to it and assign to it?
03:18 sivoais yes, it doesn't sever() the piddles
03:19 Mithaldu ok, so i have this:
03:19 Mithaldu $pVertices->slice( "3,0:2" )->squeeze.''
03:19 Mithaldu which is for some reason this?
03:19 Mithaldu [
03:19 Mithaldu [0]
03:19 Mithaldu [1]
03:19 Mithaldu [0]
03:19 Mithaldu ]
03:19 Mithaldu it managed to get *worse*
03:21 Mithaldu or maybe the print of that is just dumb?
03:22 sivoais yeah, what does $pVertices->slice( "3,0:2" )->squeeze->dims
03:22 sivoais say
03:22 Mithaldu how do i debug pdl properly without spewing to stdout
03:22 Mithaldu 3
03:22 sivoais :-O
03:24 Mithaldu at least the cross product of that is correct
03:24 Mithaldu and it normalizes fine
03:24 Mithaldu it just has the wrong dimensional layout
03:25 sivoais what about
03:25 sivoais $pVertices->slice( "3,0:2" )->squeeze->ndims
03:25 Mithaldu 1
03:25 sivoais ok, so that means it is a one-dimensional piddle. It worked fine. I think I had this issue of confusion with the output before
03:26 sivoais it doesn't seem to behave like the MATLAB I'm used to
03:27 sivoais now I use <https://metacpan.org/relea​se/Data-Printer-Filter-PDL> (a packaged version of jberger__'s script)
03:27 sivoais much easier to read, imo
03:28 Mithaldu i'd be happy if print() reflected reality :x
03:28 Mithaldu otoh
03:28 Mithaldu it seems to work slightly better
03:28 jberger__ sivois++
03:28 jberger__ +a
03:29 Mithaldu output is still fucked, but oh well
03:30 jberger__ I meant to put that in for a core pdl enhancement,  but forgot
03:31 Mithaldu sivoais, jberger, if either of you feel like looking over tutorial18.pl in branch perl of this repo to see if i fucked up PDL stuff, that would be much appreciated
03:31 Mithaldu https://github.com/wchristian​/ogldev/tree/perl/tutorial18
03:33 Mithaldu hurf, i'm an idiot
03:33 Mithaldu helps to actually read the size of vertexes when passing byte offsets
03:38 Mithaldu hell yeah, it works
03:38 Mithaldu diffuse lighting is implemented and pushed
03:38 Mithaldu thanks sivoais :)
03:40 sivoais ooh nice! OpenGL is one area I haven't delved into. I've gotten away with not learning it by using VTK since it has a bunch of volume rendering algorithms built in.
03:41 sivoais that's <http://www.vtk.org/>
03:41 Mithaldu honestly, right now it's a pain in the arse to deal with anyhow
03:42 Mithaldu especially since pdl has a different matrix majority than glsl
03:42 sivoais ah, yeah, that can always be a pain when interfacing numerical stuff
05:13 jberger__ joined #pdl
14:04 chm joined #pdl
14:06 chm Mithaldu : maybe you are seeing a bug, I cannot reproduce the problem on my PDL-2.007_01 install
14:06 chm pdl> p $vv  [  [0 1 0] ]    pdl> p $vv->info PDL: Double D [3,1]   pdl> p $vv->squeeze [0 1 0]   pdl> p $vv->mv(1,0)  [  [0]  [1]  [0] ]    pdl> p $vv->mv(1,0)->info PDL: Double D [1,3]   pdl> p $vv->mv(1,0)->squeeze [0 1 0]
14:07 chm Well, that wasn't helpful---Mibbit doesn't do multi-line pasting apparently.
14:07 vicash joined #pdl
16:39 drrho joined #pdl
16:58 Mithaldu chm, sivoais, jberger: if you'd like to try a minimized version: https://gist.github.com/wchr​istian/258ed61440dffa5bb952
17:57 chm joined #pdl
17:59 chm Mithaldu : I'm not sure what is up but running the script worked but line-by-line in pdl2 generated a hang and then segfault.
17:59 Mithaldu chm: note the printed output
17:59 chm It is also confusing about the mpdl stuff.  I appears that the slice doesn't do what you think for an mpdl
17:59 Mithaldu it's giving me 3 single element piddles inside one piddle
18:00 Mithaldu i'm glad it still works, but it might be an actual bug
18:00 chm It might be clearer to use the normal pdl declaration and the reorder to pass to OpenGL.
18:00 Mithaldu that makes it silly hard to have an array of vertices
18:01 Mithaldu since pdl does not allow me to declare logical structures
18:02 Mithaldu i'll have to break down and make an array of vertices object i think
18:02 chm I'm confused, what do you mean?  The example just had an mpdl...
18:02 Mithaldu please clarify which part you're confused about, i said a bunch of things
18:03 chm yes, that, but starting with "that makes it silly hard to have an array of vertices"
18:03 Mithaldu not using mpdl and instead using pdl makes that hard
18:03 chm How can you not have an array of vertices.  Isn't that just a multidim array?
18:03 Mithaldu clearer?
18:04 chm Do you have an example where is is hard, so I can see what you mean?
18:04 Mithaldu how about this, you try and rewrite the code i pasted to how you think, and i tell you if i don't like it
18:04 chm I thought it was mainly about dimension ordering.
18:08 chm If mpdl works for you, then feel free to use it.  If you have some code
18:09 chm showing the problem, please feel free to post to the mailing list as I'm trying to collect
18:09 Mithaldu /o\
18:09 Mithaldu i thought you saw the gist?
18:09 chm ideas and requirements for PDL3 work.  The discussions on #pdl never end up in a useful
18:09 Mithaldu look man
18:10 Mithaldu the script above
18:10 Mithaldu in the gist
18:10 chm form for further development.  Especially when the answers are quick but sometimes partial or incorrect.
18:10 Mithaldu you told me, paraphrased
18:10 Mithaldu "use pdl, not mpdl"
18:10 chm I said I thought that there might be a bug in the mpdl stuff.  I haven't used
18:10 Mithaldu now given that a pdl is simply a one-dimensional array, that makes things really arse, as i can't just ask it "how many rows do you have"
18:11 Mithaldu and since you know more, i asked you to do the replacement in that code to show me how you think it should look like
18:11 Mithaldu if you don't want to do that, that's cool, i'll just keep on rolling and hope this stuff doesn't bite me in the ass later
18:11 chm Yes, but again, I don't even know what the problem is.  As in, I can't see what would
18:12 chm be different except for switching dims in the slice args and and using pdl instead of mpdl.
18:12 chm The only difference on my system is that the mpdl hangs in pdl2 (a bad sign) which
18:13 chm leads me to suspect that something funny is going on.
18:14 chm The standard 2D pdl is something like long( [ 1, 2, 3 ], [ 4, 5, 6 ] ) which is a shape [3,2] piddle
18:14 Mithaldu i think it might be the transpose that's missing
18:15 chm If you use mpdl, then the first two dims are swapped and then you have shape [2,3] for the piddle but the dims are swapped in operations
18:15 chm Basically a transpose, operate, transpose back is taking place under the covers.  If there is a bug somewhere,
18:15 chm then things can break.
18:16 Mithaldu ok, i think what i want to do is stop using mpdl and stop using transpose and things might work
18:17 chm When I stepped through the gist, it seemed that the slice was being applied to the dims of the pdl and not the mpdl (backwards)
18:17 chm If you don't get a crash, you could definitely get weird results.
18:18 Mithaldu https://gist.github.com/wchr​istian/d6c28393f31ea2b9a163
18:18 Mithaldu those two give me the same result
18:18 Mithaldu and i think i got the idea i need to use mpdl because the docs being quite confusing
18:19 chm I think I see part of the problem, mpdl silently switches the order of the dimensions
18:20 chm in the dimension array, but *doesn't* change the underlying data.  It seems that get_dataref just
18:20 chm drills down to the original pdl data as-is, skipping the dimension tricks of mpdl, which is why a transpose is needed.
18:20 chm Does that help?
18:21 Mithaldu possibly
18:21 Mithaldu i'll refactor all the tutorials to drop mpdl and see what breaks
18:21 Mithaldu main issue is i can't remember why i started using it exactly
18:21 Mithaldu and had wrong ideas about how it works
18:21 chm I don't use mpdl myself because it can get *really* confusing to track multiple dimensions (>2) when a couple are backwards.
18:22 chm I also think you might have had better luck sorting this out via the mailing list.
18:23 chm Mainly because you have more eyes on the problem, and problems with PDL itself can be discovered and fixed
18:23 Mithaldu i don't do well with email
18:23 chm if needed.  Not the case with #pdl.  Even I don't remember what I've talked about from day to day.  The list
18:23 Mithaldu doesn't fit my mental model
18:23 chm is at least a log of discussion and record.  I do understand that you don't work well with the list but
18:24 chm the catch is you have a pretty good chance that the answer is not complete or correct and that bugs discovered are not reported or acted upon.
18:25 chm I'd like #pdl to work better but none of the proponents of #pdl have come up with a way to make that happen.
18:25 chm In fact, if I didn't read the back logs periodically, there wouldn't be the information there is.
18:25 chm Good luck.  o/
18:26 sivoais Mithaldu: I've got Perl 5.16 and I get "Useless assignment to a temporary" on line 40. Instead of = that should be .=, right chm?
18:26 chm Yes
18:27 Mithaldu well, it works for the rest of the perl community here on irc.perl.org because they understand that IRC is something where /quit is a gross mistake, and where the primary mode of communication is realtime exchange of small messages
18:27 Mithaldu sivoais: haha, wow, 5.12 it works :(
18:27 Mithaldu thanks for the hint though, i'll have to look up that bit of magic
18:29 chm Difference between elementwise assignment (into a slice for example) or ordinary assignment (replaces the reference which usually isn't what you want for slices on LHS)
18:30 Mithaldu oh, right i know why it works
18:30 Mithaldu the algorithm above works out in such a way that the vectors i'm poking are already normalized
18:30 Mithaldu so the assignment is a no-op
18:31 Mithaldu (and 5.12 doesn't know it should be complaining)
20:13 Mithaldu does PDL have a function to convert between degrees and radians?
20:15 jberger_ 180/pi
20:15 jberger_ :-P
20:15 * jberger_ shuts up
20:15 Mithaldu :(
20:15 * jberger_ runs away
20:16 Mithaldu also, so far it seems excising mpdl works fine
20:16 jberger_ Mithaldu++
20:18 Mithaldu can i have pdls with more dimensions than 2?
20:18 jberger_ yeah
20:18 Mithaldu huh
20:18 Mithaldu i really wonder what my earlier hangup was then
20:18 jberger_ in fact that is where pdl shines
20:18 Mithaldu i'm just trying to figure out why i used mpdl at all
20:19 jberger_ :-)
20:19 Mithaldu also, upside, with less transpose calls i get higher fps
20:20 * jberger_ wonders if xchg is faster than transpose
20:27 Mithaldu hm
20:27 Mithaldu Slice cannot start or end above limit at Basic/Core/Core.pm.PL (i.e. PDL::Core.pm) line 805.
20:29 Mithaldu i guess i need to do this?
20:29 Mithaldu $pVertices->slice( ":,$Index0" )->slice("5:7")
20:29 Mithaldu weird
20:30 jberger_ ?
20:31 jberger_ my slicing edge is dulled of late unfortunately
20:31 Mithaldu with mpdl i was doing
20:32 jberger_ I haven't used PDL in anger in a while
20:32 Mithaldu $pVertices->slice( "($Index0),5:7")
20:32 Mithaldu and now i have to do the above
20:32 jberger_ seems like you should be able to do the transposed operation
20:33 Mithaldu which would be?
20:33 Mithaldu just switch the parameters around the ,?
20:33 jberger_ "5:7,($Index0)"
20:33 Mithaldu hurf durf, of course
20:33 Mithaldu yes, it works :)
20:34 jberger_ good, I'm not losing it then ;-)
20:36 Mithaldu Error in slice:Cannot obliterate dimension after end
20:36 Mithaldu that is new
20:37 jberger_ I think that is too many commas
20:37 jberger_ have you already gone down a dimension?
20:37 Mithaldu it doesn't like
20:37 Mithaldu my $norm = $pVertices->slice( "5:7,($i)" )->norm;
20:38 Mithaldu i think i'm going too far
20:38 jberger_ what does norm do again? that word is so overloaded in mathematics
20:38 Mithaldu normalize a vector
20:38 Mithaldu i'm sure it's just that my loop loops too far
20:38 jberger_ yeah, I think so too
20:39 Mithaldu yeah, vertexcount 8 is wrong
20:39 Mithaldu i think that's why i was using mpdl
20:39 Mithaldu because, now dim(0) is wrong to get the vertex count
20:39 jberger_ why are you looping at all? usually an explicit loop in pdl is a code smell
20:39 Mithaldu sec
20:39 jberger_ dim(1)
20:40 Mithaldu yeah, dim 1 is right
20:40 Mithaldu it's just semantically bullshit
20:40 Mithaldu shit, the model is wrong :x
20:40 jberger_ ->xchg(0,1)->dim(0) :-P
20:41 Mithaldu ah, i was reading the length of a vertex with dim(1)
20:41 Mithaldu which now needs to be dim(0)
20:42 sivoais btw, transpose seems somewhat slower than xchg with this silly little microbenchmark. I'm curious why
20:42 jberger_ I'm pretty sure that all mpdl does is (badly) overload the methods with an extra ->xchg(0,1) in all over the place
20:42 sivoais $T = 500000; $N = 10; use PDL; use Benchmark q(:all); cmpthese($T, { trans => sub { sequence($N,$N)->transpose }, xchg => sub { sequence($N,$N)->xchg(0,1) }  }); 1;
20:43 jberger_ I think xchg doesn't physically transpose, just reoders the indexes
20:43 Mithaldu jberger: https://gist.github.com/wchr​istian/d6c28393f31ea2b9a163
20:43 Mithaldu feel free to play with that
20:43 Mithaldu oh wait, needs the indices
20:44 Mithaldu jberger: F5 the page, i added some stuff
20:44 Mithaldu if you remove the gl stuff it should run as is
20:45 Mithaldu and the loop i was talking about is the loop at the bottom of the normalize thing
20:46 Mithaldu ok, cool, everything works, mpdl/transpose excised
20:46 Mithaldu only downside is that the semantics are kind of ugh now
20:46 jberger_ I suspect that you could probably remove all those loops
20:47 jberger_ but my pdl-foo is weak at the moment
20:47 Mithaldu but i didn't need to use the semantics before tutorial 18, so it'll be fine
20:47 Mithaldu jberger: well, i'm intending to make a third branch where i rewrite all of these tutorials into pretty perl
20:47 Mithaldu so if you feel like playing with it at some point i'd appreciate that :)
20:48 jberger_ when doing image processing, an image is a 2D piddle, so I make some processing function for that
20:48 Mithaldu i mean, there's a ton of stuff in there that's just pointless repetition because that's how the C tutorial i'm translating looks like
20:48 jberger_ then when I want to do multiple images, you just make a 3D piddle by stacking 2D piddles, and PDL just does it, returning a vector of the results
20:49 jberger_ PDL autoloops over higher dimensions
20:49 jberger_ and that is its major feature over things like numpy
20:49 Mithaldu well
20:49 Mithaldu that's way above my level of experience with pdl
20:49 Mithaldu right now i flail at it to see what sticks :D
20:50 jberger_ agreed, that is approaching guru level
20:50 jberger_ and as I have said, I haven't used PDL in anger in over a year, so I'm having a hard time seeing how you would do it too
20:50 jberger_ so flail for now, it works
20:50 jberger_ but if you gave this to run4flat, he could probably do it in no time
20:51 Mithaldu i have another 25 tutorials before i get to prettying ;)
20:51 jberger_ hehe, good luck
20:51 Mithaldu right now though i need to fix up the commits
20:52 Mithaldu and then let my fingers rest
20:52 jberger_ still this looks useful, even if not 100% PDL guru-level
20:52 Mithaldu too much typing this weekend
20:52 Mithaldu it's useful inasmuch it actually does what it's meant to do
20:52 jberger_ in fact, a direct port might be better if you are porting tutorials
20:52 jberger_ then it doesn't assume your reader is a PDL guru either
20:52 Mithaldu that's what i'm thinking
20:53 jberger_ still you might get a speed bump, that loop is then done in C
20:53 Mithaldu haha
20:53 Mithaldu i'm gonna get speed bumps all over the place
20:53 jberger_ PDL is really fast
20:53 Mithaldu some of the matrix math i'm doing makes zero sense to do in perl every frame
20:54 jberger_ then you rewrite your logic in PDLpp and you get another boost
20:54 Mithaldu yeah
20:54 jberger_ again, see run4flat
20:54 jberger_ I've never actually done that myself
20:55 jberger_ anything that compiles to XS is above my pay grade!
20:55 Mithaldu i did, it's in my repo
20:55 Mithaldu and i showed it in here :P
20:55 jberger_ I haven't kept up completely in #pdl
20:55 Mithaldu https://github.com/wchristian/ogld​ev/blob/perl/tutorial18/math_3d.pm
20:56 Mithaldu rotate a vector by an angle about a vector
20:56 jberger_ nice!
20:56 Mithaldu i'm still not sure that couldn't be golfed
20:56 jberger_ doesn't matter, its C code
20:57 jberger_ also PI == 4*atan2(1,1)
20:57 jberger_ if you don't want to have to type it in by hand
20:59 Mithaldu ah, helpful :)
21:04 sivoais the roughly equivalent concept to autothreading in NumPy is called broadcasting <http://docs.scipy.org/doc/numpy​/user/basics.broadcasting.html>, but afaict, PDL has it more built in. And let's not talk about MATLAB's bsxfun...
21:05 jberger_ I hate the name threading as used in PDL
21:05 jberger_ the word means something else now
21:05 jberger_ so I always call it autolooping, since that is what it really is
21:06 Mithaldu s/// the docs :)
21:06 sivoais hehe, yeah...
21:06 jberger_ we really should
21:07 Mithaldu at least in PDL3 there's nothing stopping you
21:07 jberger_ Christmans
21:07 jberger_ Christmas
21:16 Mithaldu whee, all commits remade and pushed
21:39 chm Mithaldu : hi, I was taking a look at the loop code to see how it could be vectorized with pdl "threading"
21:39 chm Could you explain what the elements of the vertices buffer are?
21:39 Mithaldu i have no idea what the first sentence means, but sure
21:39 Mithaldu the bugger contains a list of vertex data elements
21:39 chm I'm new to the shader stuff but I think it is actually packed data for a number of different things?
21:40 Mithaldu the first three numbers in each element are the 3d coordinates with the position of the vertex
21:40 Mithaldu the next two numbers are the coordinates uv in a texture which that vertex corresponds to
21:40 chm vectorize <=> used here to mean replacing loop constructs with auto-loop/threading which happens at the C,not perl level, much faster...
21:41 Mithaldu the last three are the normal vector of that vertex, i.e. the direction at which a vector is exactly 90° to the surface the vertex is on
21:42 Mithaldu ah, so vectorize means "auto-loop it" :)
21:43 Mithaldu anyhow, chm, does that answer the question?
21:44 chm As an example, the first loop in CalcNormals appears to actually be an unrolled loop over a set of coords.
21:45 Mithaldu not entirely sure if you see the same thing as i am, but possibly, yes
21:45 chm If that is so, the natural shape of the pdl data would be [3,4] where dim0 is the coord and dim1 the number
21:45 Mithaldu it takes 3 vertices which form a surface, then calculates the normal of that surface and adds it to each vector
21:46 Mithaldu you mean the shape of the index list, and yes :)
21:46 chm So if indices were shaped [3,4] then you could calculate all the vertex pair at the same time with some threading.
21:46 chm BTW, how do you do the gist?  I'll take a look at this and get back.  Right now I'm being called to dinner.
21:48 Mithaldu i don't understand the question
21:49 Mithaldu and yes, it sounds reasonable that much of that can be automated in some fashion
21:49 chm How do you do the gist pasting stuff?
21:49 Mithaldu oh, get a github account, login at gist.github.com and click the + icon
21:49 chm Ok, I'll have to dig up my account info.  Thanks, later. o/
21:49 Mithaldu cheers :)

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary