Perl 6 - the future is here, just unevenly distributed

IRC log for #pdl, 2013-12-02

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:57 vicash joined #pdl
09:32 Mithaldu figured out why i went with mpdl last time
09:33 Mithaldu when using pdl not only are the dim calls all wrong, but the slice calls are all inverted too
09:33 Mithaldu i think i'll need to write a wrapper or something :x
09:59 Mithaldu jberger: yoyoyoyo i remember you working on alien stuffs, please let me know when you're around for brain picking!
10:10 Mithaldu jberger: specifically i am wondering how to provide https://github.com/assimp/assimp or https://github.com/acgessler/assimp2json
10:11 Mithaldu many options open, but i'm not sure which are sensible
12:44 Mithaldu chm or others: is PDL 3 planned to change the majority to match Perl's?
12:44 Mithaldu *access majority
15:38 webart joined #pdl
16:47 vicash Hi, is there a quick way to check where Perl is leaking memory ? I have a script that uses an XS module which grows the memory usage to 1GB+ while executing and finishes. However the data is about 128KB only. On an embedded system with 512MB memory (Beaglebone Black) the OS kills the script for using up too much memory.
17:02 perigrin "quick" not exactly ... Devel::SizeMe might help
17:03 perigrin eventually it should be come the right answer, I just don't know how much Tim has gotten to work on it since Austin.
17:03 vicash thanks. i am researching Devel::Leak as well.. the problem is in the XS module as if I dont use the XS version the leak is not there but the time taken is 100 fold
17:04 perigrin then you'll want to reach for C debugging tools I think.
17:04 perigrin valgrind etc.
17:04 perigrin and I'm *so* not the right person to help.
17:04 vicash yea i might have to go that route...
17:05 perigrin The other choice is to benchmark the non-XS version and see if you can improve it's performance without resorting to XS
17:05 perigrin :)
17:05 perigrin at which point one of Tim's *other* popular modules: Devel::NYTProf is your friend
17:07 vicash thanks. i might stop using the XS module for now since it has a known CpanRT bug of excessive memory usage that hasn't been fixed by the author. it is not a leak because they seem to free the memory at the end of the script but if you use way too much data then memory drives up too much..
17:08 * perigrin nods
17:08 perigrin what module out of curiosity?
17:09 vicash Algorithm::Diff::XS
17:09 vicash i have 2 binary files that are of size 128KB that i need to diff byte by byte.. and find all the bytes that are different and their indexes.
17:09 vicash Algorithm::Diff works well but only for files < 2KB it finishes in under 5 seconds but otherwise can take minutes for this..
17:10 perigrin bleh
17:10 perigrin yeah
17:11 perigrin might be worth looking at the profile for Algorithm::Diff to see if you can get some improvement out of the pure perl version
17:11 vicash i might just write the stuff in C if nothing works as  a quick fix solution. there is binwalk in python that might work but then i have to make it work on my embedded hardware too
17:11 vicash ok i will try that
17:19 vicash i just solved my problem by doing a dumb diff of the data in question since the sizes are expected to be the same .. if not the same they can use the more complex algorithm::diff.. the speedup is tremendous
17:38 perigrin :)
17:59 Mithaldu vicash: maybe just shell out to diff?
18:09 vicash that is a possibility but needs hexdumping the data first to actually grab the bytes and also tracking indexes. current method works alright for now. later will fix it.
20:24 run4flat joined #pdl
21:14 run4flat Mithaldu, I'm going to have to leave in a couple of minutes, but I'd be happy to golf some of your gist
21:14 run4flat :-)
21:14 Mithaldu \o/
21:14 run4flat for example, this for loop: https://gist.github.com/wchristian/d6c28393f31ea2b9a163#file-asd-pl-L30
21:14 run4flat I *think* it could be replaced with something like this:
21:15 run4flat $pVertices->slice("5:7") .= $pVertices->slice("5:7")->xchg(0,1)->norm->xchg(0,1);
21:15 run4flat or, maybe just this: $pVertices->slice("5:7") .= $pVertices->slice("5:7")->norm
21:15 Mithaldu ... ahaha
21:15 Mithaldu the latter would be hilariously obtuse but amazing
21:16 * run4flat thinks Mithaldu meant to say "the former"
21:16 run4flat yes, somewhat obtuse, but that was the design decision
21:17 run4flat which compares to, say, numpy, where you specify the "axis" over which an operation is to be performed
21:17 run4flat with PDL, the operations like norm always operate along the first, so you use xchg or transpose to rearrange
21:18 run4flat actually, if the latter works, I wonder if you could do $pVertices->slice("5:7")->inplace->norm
21:18 run4flat or if the forerm, then perhaps $pVertices->slice("5:7")->xchg(0,1)->inplace->norm
21:19 run4flat s/forerm/former/
21:19 run4flat as for the for-loop starting line 15, I'd have to understand the data better
21:20 Mithaldu now those two look nice :D
21:20 Mithaldu run4flat: you can just run it and print $pVertices as you go
21:20 Mithaldu it should become obvious
21:20 Mithaldu run4flat: do you know 3d programming?
21:21 run4flat no
21:21 run4flat I've glanced at OpenGL stuff
21:21 run4flat but I've never done anything with it
21:21 Mithaldu hm, ok, i'll try to explain it rather simply then
21:21 Mithaldu pVertices is a list of vertex elements
21:21 Mithaldu each element contains:
21:21 Mithaldu x, y, z, texture x, texture z, normal x, normal y, normal z
21:22 Mithaldu the indices pdl is a list of triangles
21:22 Mithaldu with each triangle consisting of three vertices
21:22 Mithaldu so it generates the normals for each triangle, adds them to each vertex in turn
21:22 Mithaldu then at the end normalizes the normal vector of each vertix
21:23 Mithaldu make sense?
21:23 * run4flat just realized Mithaldu was chatting
21:24 run4flat yeah
21:24 run4flat so all of your data is the same datatype
21:24 run4flat and slicing different pieces may give you data, or textures, or some (probably useless) combination
21:24 Mithaldu it gives me either:
21:24 Mithaldu 3d coordinate of vertex
21:25 Mithaldu 2d coordinate of vertex in texture
21:25 Mithaldu 3d vector of normal of vertex
21:25 run4flat :-)
21:25 run4flat only if you slice it right
21:25 Mithaldu right :)
21:26 run4flat alas, I gotta run
21:26 run4flat we can pick this back up tomorrow
21:26 run4flat :-)
21:26 Mithaldu sure, no hurry
21:26 Mithaldu the golfing will come very late in the process :)
21:27 Mithaldu i need to get shit working first :D

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary