Perl 6 - the future is here, just unevenly distributed

IRC log for #pdl, 2015-09-20

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
02:22 travis-ci shawnlaffan/pdla#1 (master - 3f09b2d : Shawn Laffan): The build failed.
02:22 travis-ci Change view : https://github.com/shawnlaffan/pdla/​compare/1784fa410511...3f09b2d9fcc5
02:22 travis-ci Build details : https://travis-ci.org/shawn​laffan/pdla/builds/81221649
02:38 travis-ci shawnlaffan/pdla#2 (master - 21c541e : Shawn Laffan): The build failed.
02:38 travis-ci Change view : https://github.com/shawnlaffan/pdla/​compare/3f09b2d9fcc5...21c541e63449
02:38 travis-ci Build details : https://travis-ci.org/shawn​laffan/pdla/builds/81222418
02:50 travis-ci shawnlaffan/pdla#3 (master - 12b4cb1 : Shawn Laffan): The build is still failing.
02:50 travis-ci Change view : https://github.com/shawnlaffan/pdla/​compare/21c541e63449...12b4cb13ef4a
02:50 travis-ci Build details : https://travis-ci.org/shawn​laffan/pdla/builds/81223203
03:21 travis-ci shawnlaffan/pdla#4 (master - b760399 : Shawn Laffan): The build is still failing.
03:21 travis-ci Change view : https://github.com/shawnlaffan/pdla/​compare/12b4cb13ef4a...b760399b49d2
03:21 travis-ci Build details : https://travis-ci.org/shawn​laffan/pdla/builds/81225460
03:27 travis-ci shawnlaffan/pdla-core#1 (master - 2233aa1 : Shawn Laffan): The build failed.
03:27 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/ecb55f58744e...2233aa156a21
03:27 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81226004
03:35 travis-ci shawnlaffan/pdla-core#2 (master - 0972618 : Shawn Laffan): The build failed.
03:35 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/2233aa156a21...097261878967
03:35 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81226144
03:40 travis-ci shawnlaffan/pdla-core#3 (master - 093423c : Shawn Laffan): The build passed.
03:40 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/097261878967...093423c97664
03:40 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81226472
03:47 travis-ci shawnlaffan/pdla-core#4 (master - 006765d : Shawn Laffan): The build was fixed.
03:47 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/093423c97664...006765df10d7
03:47 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81226659
03:55 travis-ci shawnlaffan/pdla-core#5 (master - 6f57a29 : Shawn Laffan): The build passed.
03:55 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/006765df10d7...6f57a292f571
03:55 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81227435
04:02 opkick [pdla-core] shawnlaffan opened pull request #12: PDLA::Lite exports match documentation, and update documentation (master...master) http://git.io/vnnf2
16:34 chm joined #pdl
16:48 chm sivoais, mohawk: I noticed the lldf test fail in t/bigmem.t because of OOM problems.
16:49 chm Looking for suggestions how to cleanly conditionalize the test to avoid the "failure"
16:54 chm Is there an easy way to determine the memory limits of the build system?
18:01 mohawk chm, the most general one would be just to try allocating (using pure perl) the appropriate amount of memory at the start of the test, and skip_all if that failed
18:02 chm I did some googling and I think it might be simplest to avoid running for $AUTOMATED_TESTING set.
18:02 chm The memory usage is so high, only hand tests are likely to be working.
18:03 chm I'm putting that in at the moment to move things along.
18:04 chm Is there a way to allocate in "pure perl" such that the test harness process won't die?
18:04 chm I thought OOM issues are tricky to catch and handle?
18:07 chm mohawk: Do you know if AUTOMATED_TESTING is set in our Travis-
18:07 chm CI builds?
18:07 chm Also, your name comes up in yellow which is almost impossible to read on my white xterm.
18:07 chm Are you setting that specifically to yellow?
18:09 chm It looks like reverse video shows better.  I'll see if I can set irssi defaults to that as a work-around.
18:10 chm mohawk, sivoais:  So far, the latest lldf branch appears to be working for large pdl ops.
18:10 chm If you have a chance to test, write tests, please do so.
18:10 opkick [pdl] wchristian pushed 1 new commit to longlong-double-fix: http://git.io/vnCRi
18:10 opkick pdl/longlong-double-fix fd83457 Chris Marshall: Only run t/bigmem.t if $AUTOMATED_TESTING is false...
18:10 chm I'm working on some basic checks for the mapflex() routines at the moment.
18:11 chm For something this tricky, I would appreciate more eyes on the tests (writing and running).
18:11 chm I don't want to say 64bit support is working with out some checkout.
18:11 chm Otherwise, it could be awkward if some weird issue is not discovered until someone starts working with a 27GiB piddle.
18:12 chm Hmm, I wonder why we didn't test that?  :-)
18:19 chm AFAICT, OOM failuers are not catchable for perl so maybe the best is a diag warning on running t/bigmem.t
18:19 chm With being run interactively, at least the user can see that a failure could be from lack of memory.
18:19 chm Probably need to add outputs relating to size of memory being used.
18:20 chm Also, incrementing the size until failure with stepwise diag's would be more informative.
18:23 chm sivoais: re testing 64bit index support with bigmem, I'm working to set up SPP for a 64GiB system to run some tests.
18:23 chm If I can get the SPP ported to use Windows Powershell from using CMD and .bat files, I could try a bigger memory system.
18:24 chm Any windows PDL users tried setting the SHELL to windows powershell?  What works/what doesn
18:40 opkick [pdla-core] mohawk2 commented on issue #12: Thanks for this!... http://git.io/vnCw3
18:46 mohawk chm, i'm not setting any colour things in my irc client
18:47 mohawk a way to allocate memory in perl might be (untested):
18:47 chm mohawk: sigh, I didn't think so, just frustrating not being able to tell when you are on-irc
18:48 mohawk eval { my $bigmem = '0' x 2 << 64; }; $succeeded = !$@;
18:48 mohawk my client is nearly always connected to irc
18:48 mohawk that doesn't necessarily mean i am reading it, sorry :-)
18:48 chm I know.  I can no longer get irc or the logs during the day
18:49 chm I try to catch up when I can.
18:49 chm mohawk: re memory, I review some links from the last time I looked at this and there is no easy "in perl" fix
18:50 mohawk ok
18:50 chm Most of the perl safeguards work for allocating something big elementwise and there is a cache of overflow memory.
18:50 mohawk i think at best i have a precedence problem in the above (just testing now)
18:50 chm That breaks for PDL since our "big" allocations can be a single multi GiB one.
18:51 chm It can be very slow to allocate as well.
18:51 mohawk it seems if i try to allocate too much memory in perl, it crashes perl
18:51 chm I think a way around might be to use an auxilliary proces that does the equivalent
18:52 mohawk $ perl -e 'eval { my $x = "0" x (2<<40) }; die !!$@'
18:52 mohawk Out of memory!
18:52 mohawk panic: fold_constants JMPENV_PUSH returned 2 at -e line 1.
18:52 chm with malloc/free.  Then it would not cause perl to OOM and a decision could be made from the output.
18:52 mohawk $ perl -e 'eval { my $x = "0" x (2<<30) }; die !!$@'
18:52 mohawk (ran, but i killed it as it was taking ages)
18:52 mohawk yes, i was just thinking the external process option as well
18:52 chm mohawk: One of the reasons mapflex is so nice is that it can avoid some of these OOM possibilities.
18:53 mohawk there's still a likelihood it'll take ages though
18:53 chm This is a real problem in the PDL shells since the OOM kills the shell. :-(
18:53 mohawk indeed
18:53 mohawk is it pdl code itself that allocates the memory?
18:54 chm Did you see my email re PDL-2.014 and lldf integration?  I think we're pretty close to good to go...
18:54 mohawk i glanced over your email of 20 mins ago
18:54 chm The current PDL implements by calling the underlying perl new routine for a string (don't remember the name)
18:54 mohawk you can squash the commits together yourself, it's easy :-)
18:54 mohawk git rebase -i master
18:55 chm Yes, you say that but you may still underestimate my "git fu"
18:55 mohawk then change the first "pick" to "reword" and the other "pick" to "fixup"
18:55 mohawk it's literally that
18:55 chm I'll give it a try but be prepared to fix....  :-)  Thanks for the tips.
18:55 mohawk if you want to make a throwaway branch:
18:55 mohawk git checkout -b "try-squash"
18:55 chm good idea
18:55 chm !
18:55 mohawk then you won't lose your lldf branch!
18:55 chm Thanks
18:56 chm Sorry about not moving faster but things have been busy outside of $PDL
18:56 mohawk i do understand
18:56 mohawk i say this neutrally: if only one person can do things, everything that happens depends on the availability of that person
18:56 chm I'll keep #pdl and pdl-devel postted on my progress.
18:57 mohawk if you have a team collaboration, then you don't have a single point of holdup anymore
18:57 chm Believe me, I understand.  That is why I'm delighted to have resolved the PDL/PDLA logjam.
18:57 mohawk ha ha
18:57 mohawk well, it's still a work in progress
18:57 chm Got to go, laundry calls.  o/
18:57 mohawk i want to figure how to cache the build of PDLA::Core
18:57 mohawk have a good one!
18:57 mohawk enjoy the laundry ;-)
21:08 travis-ci shawnlaffan/pdla-core#6 (master - bec8290 : Shawn Laffan): The build was broken.
21:08 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/6f57a292f571...bec829093b84
21:08 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81294678
21:16 opkick [pdl] devel-chm commented on issue #145: Apparently, AUTOMATED_TESTING is not being set by Travis-CI http://git.io/vnWe0
21:21 travis-ci shawnlaffan/pdla-core#7 (master - 3017f72 : Shawn Laffan): The build was fixed.
21:21 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/bec829093b84...3017f72822f8
21:21 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81296205
21:47 travis-ci shawnlaffan/pdla-core#8 (master - cb10fe5 : Shawn Laffan): The build passed.
21:47 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/3017f72822f8...cb10fe5973ce
21:47 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81299629
21:54 travis-ci shawnlaffan/pdla-core#9 (master - 10b4e68 : Shawn Laffan): The build passed.
21:54 travis-ci Change view : https://github.com/shawnlaffan/pdla-co​re/compare/cb10fe5973ce...10b4e68f3f84
21:54 travis-ci Build details : https://travis-ci.org/shawnlaf​fan/pdla-core/builds/81300279
22:10 opkick [pdla-core] shawnlaffan commented on issue #12: No worries.  ... http://git.io/vnWLL

| Channels | #pdl index | Today | | Search | Google Search | Plain-Text | summary