Perl 6 - the future is here, just unevenly distributed

IRC log for #metacpan, 2014-10-16

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:57 kentnl the /distribution/ endpoint is really a weird name given what it presently contains
01:02 vroom joined #metacpan
01:10 vroom joined #metacpan
01:12 Khisanth joined #metacpan
01:13 Mso150 joined #metacpan
01:23 klapperl_ joined #metacpan
01:34 vroom joined #metacpan
01:47 Mso150 joined #metacpan
01:47 kentnl *gripes* getting scrolled search to cache is a nightmare :(
01:48 * kentnl is monkey patching URI.pm just to get close to being able
01:49 kentnl https://github.com/libwww-perl/uri/pull/18 # Helps a lot, but is yet to be merged
01:53 dipsy [ _query.pm: Sort hash keys if given a hash. by kentfredric · Pull Request #18 · libwww-perl/uri · GitHub ]
01:58 kentnl ahudfrf. And of course ->file() differs from the rest of the API, in that it doesn't let you use search queries.
02:17 oalders no search queries on file?
02:20 vroom joined #metacpan
02:20 kentnl I think its just because ->module() does that instead.
02:20 kentnl but its just weird, file() is exclusively a _get(), while module() is a _search_or_get
02:21 kentnl the other fun part of scroll search caching I haven't worked out how to fix, is if your process fails mid-way through iterating the scroll, then you need to invalidate the *whole* scroll search, because you cant resume it after it fails
02:22 kentnl and of course, add to that chi's cache being slightly random and you might lose parts of the scroll without the whole thing >_>
02:23 vroom joined #metacpan
02:25 * kentnl double checks the docs
02:25 kentnl Phew, that wont happen unless I explicitly set expires_variance.
02:26 kentnl though having the start of the cache vanish before the end of it could be awkward
02:28 kentnl Heh. ether presently has more +1's for all the dists she's  $RELEASER[-1] for, than she has modules that she was $RELEASER[-1] for
02:28 kentnl 922 +1s vs 733 modules.
02:31 ether most of those would be for Moose
02:31 kentnl Yeah.
02:32 kentnl there's a few for Task::Kensho too
02:33 vroom joined #metacpan
02:40 Khisanth joined #metacpan
02:45 oalders kentnl: i think file should be a _search_or_get() as well, since it's way more than just modules in the file type
02:46 vroom joined #metacpan
02:57 kentnl oalders: ->modules() also returns non-module files though, so its confusion all the way down
03:01 kentnl what I really need is some sort of Elasticsearch proxy database I can run which basically goes: user query -> proxy -> forward request to upstream -> upstream returns list of object ID's to proxy, proxy fetches and caches id's it doesn't already have
03:01 kentnl -> returns objects to user
03:01 kentnl because these queries are so *painfully* slow to execute.
03:06 kentnl 16 minutes to run a script once is a bit  painful :(
03:08 kentnl actually, that figure includes partial caching :/
03:14 oalders maybe just using Elasticsearch.pm would be faster
03:15 oalders although, i'd rather see this stuff fixed in MetaCPAN::Client if possible
03:15 oalders + modules() should try hard not to return non-modules
03:19 vroom joined #metacpan
03:35 vroom joined #metacpan
03:43 vroom joined #metacpan
04:03 jwang joined #metacpan
04:35 vroom joined #metacpan
05:13 kentnl nah, the problem is not that I'm using the MetaCPAN client, the problem is my network overheads.
05:13 kentnl network overheads + slow queries + multiple requests for a scroll request => Its going to be slow
05:25 kentnl Ok, I switched to using a File cache for my query run, and a single pass has me at 230M of disk usage. Nice.
05:25 kentnl and that's using Sereal
05:26 kentnl or ... or not. Seems Sereal never got invoked and its all flatfiles
05:57 gvenkat joined #metacpan
07:02 FROGGS joined #metacpan
07:32 dp_ joined #metacpan
07:49 Mso150 joined #metacpan
08:18 neilb joined #metacpan
09:33 FROGGS joined #metacpan
09:34 bvierra joined #metacpan
10:02 grantm joined #metacpan
11:17 vroom joined #metacpan
11:42 vroom joined #metacpan
12:28 vroom joined #metacpan
12:37 oalders kentnl: well, for me i think it's a bit slower because i really only need one field from every release, but it's just getting everything
13:10 metacpan joined #metacpan
13:10 metacpan [metacpan-examples] oalders pushed 1 new commit to master: http://git.io/dvJ7bQ
13:10 metacpan metacpan-examples/master 5f9c7ec Olaf Alders: Adds script which scrolls through all releases.
13:10 metacpan left #metacpan
13:12 dipsy [ Adds script which scrolls through all releases. · 5f9c7ec · CPAN-API/metacpan-examples · GitHub ]
13:13 fedelippo joined #metacpan
13:23 vroom joined #metacpan
13:57 vroom left #metacpan
14:01 vroom joined #metacpan
15:43 n0body is there a way to sort search results by date?
15:43 n0body or by ++
15:58 vanstyn joined #metacpan
15:59 vroom joined #metacpan
16:01 oalders n0body: not currently, but would be nice to have
16:01 vanstyn joined #metacpan
16:03 n0body oalders: it sure would! thanks though, at least i'm not missing a secret trick!
16:03 oalders :)
16:40 neilb joined #metacpan
16:42 Mso150 joined #metacpan
17:00 neilb joined #metacpan
17:09 vroom joined #metacpan
17:13 kentnl that's basically what I'm trying to do, expand the list of AUTHOR results to include ++ and N-RDEPS
17:18 * kentnl thinks I'll have to directly subclass Elasticsearch::Scroll to make caching viable, because waiting till it hits the web layer seems to be far too complex
17:19 kentnl my present problem is even after a full scroll run, even after caching the whole scroll, it still for some bizarre reason tries to re-send the scroll to the API, for reasons I assume to be "ES generates a slightly different query somewhere in the middle causing a cache miss"
17:22 ether wtf, clicking on 'source' no longer shows the embedded pod?
17:22 ether https://metacpan.org/source/SFINK/Acme-CPANAuthors-Not-0.01/lib/Acme/CPANAuthors/Not.pm only has code, no pod -- but the pod is in the same .pm file when I browse.
17:23 dipsy [ lib/Acme/CPANAuthors/Not.pm - metacpan.org ]
17:24 ether confirmed, with other modules. It looks like the pod is being stripped from the 'source' view.
17:24 kentnl there's a toggle on the left
17:25 ether ah phew
17:25 ether that was really confusing for a minute
17:26 kentnl Its really handy on modules like https://metacpan.org/source/ADAMK/Module-Changes-ADAMK-0.11/lib/Module/Changes/ADAMK.pm , where it makes reading the source itself easier slighty
17:26 kentnl bad example, but I CBA fidning a goood one
17:26 dipsy [ lib/Module/Changes/ADAMK.pm - metacpan.org ]
17:27 kentnl https://metacpan.org/source/PTROST/Disk-SMART-0.08/lib/Disk/SMART.pm # better example
17:27 dipsy [ lib/Disk/SMART.pm - metacpan.org ]
17:28 kentnl ether: It may be worth having that mode draw a contraction decoration where the lines are folded
17:28 haarg i have a half finished branch that does that
17:28 kentnl sort of like how vim folding mode works
17:28 kentnl haarg++
17:29 haarg but i'm also working on replacing the syntax highlighter, so i haven't finished it
17:32 ether a horizontal line showing the fold would definitely help
17:33 ether I was confused about where the pod went, so I hunted down the installed source to find the line numbers, to confirm the numbering did indeed jump over the gap
17:36 haarg i recently re-wrote that function, but there wasn't an intentional change in behavior
17:52 haarg http://i.imgur.com/V6rMQdK.png http://i.imgur.com/irz5Xz9.png
17:54 ether haarg++
18:12 neilb joined #metacpan
18:16 neilb_ joined #metacpan
18:24 haarg the current syntax highlighter is quite annoying to work with.  it overrides pretty much everything using !important.
18:41 oalders that looks really good
19:06 Mso150 joined #metacpan
20:10 kentnl haarg++
20:39 vroom joined #metacpan
22:05 fedelippo joined #metacpan
22:13 neilb_ joined #metacpan
22:31 ilbot2 joined #metacpan
22:31 Topic for #metacpan is now Have you installed your MetaCPAN VM? https://github.com/CPAN-API/metacpan-developer | Chat logs available at http://irclog.perlgeek.de/metacpan/ | Can't find your module on MetaCPAN? https://metacpan.org/about/missing_modules | Feature suggestions now belong on the wish list: https://github.com/CPAN-API/cpan-api/wiki/Wishlist
23:10 vroom joined #metacpan
23:55 ilbot2 joined #metacpan
23:55 Topic for #metacpan is now Have you installed your MetaCPAN VM? https://github.com/CPAN-API/metacpan-developer | Chat logs available at http://irclog.perlgeek.de/metacpan/ | Can't find your module on MetaCPAN? https://metacpan.org/about/missing_modules | Feature suggestions now belong on the wish list: https://github.com/CPAN-API/cpan-api/wiki/Wishlist

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary