Perl 6 - the future is here, just unevenly distributed

IRC log for #metacpan, 2016-11-28

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
05:03 metacpan joined #metacpan
05:03 metacpan [metacpan-web] tsibley force-pushed tsibley/metahack-sponsors from 3f15948 to 7d6dfb4: https://git.io/v1IlH
05:03 metacpan metacpan-web/tsibley/metahack-sponsors 37925eb Thomas Sibley: meta::hack sponsors on the front page and the full sponsors list...
05:03 metacpan metacpan-web/tsibley/metahack-sponsors 5c2a0c0 Thomas Sibley: Stop using floats poorly for the about/account sidebar...
05:03 metacpan metacpan-web/tsibley/metahack-sponsors 7d6dfb4 Thomas Sibley: meta::hack: Link to more blog posts
05:03 metacpan left #metacpan
05:27 metacpan joined #metacpan
05:27 metacpan [metacpan-web] tsibley pushed 1 new commit to tsibley/metahack-sponsors: https://git.io/v1I82
05:27 metacpan metacpan-web/tsibley/metahack-sponsors d2423ca Thomas Sibley: meta::hack platinum sponsors at top of front page
05:27 metacpan left #metacpan
05:31 metacpan joined #metacpan
05:31 metacpan [metacpan-web] tsibley force-pushed tsibley/metahack-sponsors from d2423ca to a5217ea: https://git.io/v1IlH
05:31 metacpan metacpan-web/tsibley/metahack-sponsors 47726ff Thomas Sibley: Use a better Booking logo from Mickey
05:31 metacpan metacpan-web/tsibley/metahack-sponsors 27a12af Thomas Sibley: Remove unused CSS
05:31 metacpan metacpan-web/tsibley/metahack-sponsors 8d4b560 Olaf Alders: Adds meta::hack sponsor page with some boilerplate content.
05:31 metacpan left #metacpan
05:33 metacpan joined #metacpan
05:33 metacpan [metacpan-web] tsibley pushed 1 new commit to tsibley/metahack-sponsors: https://git.io/v1I8y
05:33 metacpan metacpan-web/tsibley/metahack-sponsors 2e9e6c0 Thomas Sibley: Add meta::hack page to about navigation bar
05:33 metacpan left #metacpan
07:11 [Tux] joined #metacpan
07:32 nakiro joined #metacpan
07:39 nakiro joined #metacpan
09:01 osfabibisi joined #metacpan
09:12 Relequestual joined #metacpan
10:33 Tux joined #metacpan
11:28 neilb joined #metacpan
12:01 neilb joined #metacpan
12:14 edward joined #metacpan
13:00 metacpan joined #metacpan
13:00 metacpan [metacpan-api] mickeyn created mickey/split_author_index (+4 new commits): https://git.io/v1IjU
13:00 metacpan metacpan-api/mickey/split_author_index e72e492 Mickey Nasriachi: script/mapping: added type emptying functionality (for indices split)
13:00 metacpan metacpan-api/mickey/split_author_index 2a6f6ba Mickey Nasriachi: move script prompting to the script role (reuse in other scripts)
13:00 metacpan metacpan-api/mickey/split_author_index d8bd2df Mickey Nasriachi: script/author: prompt if not using author dedicated index
13:00 metacpan left #metacpan
13:00 neilb_ joined #metacpan
13:31 neilb joined #metacpan
14:28 Khisanth joined #metacpan
15:53 Yysachinnyy joined #metacpan
15:57 Guest69 joined #metacpan
18:32 oalders anybody who has metacpan-web set up can test out these changes by jberger https://github.com/metacpan/metacpan-web/pull/1822
19:10 metacpan joined #metacpan
19:10 metacpan [metacpan-api] ranguard pushed 1 new commit to master: https://git.io/v1L5H
19:10 metacpan metacpan-api/master e1bd089 Leo Lapworth: Merge pull request #585 from metacpan/mickey/split_author_index...
19:10 metacpan left #metacpan
19:22 mst https://metacpan.org/pod/POE::Kernel
19:22 mst ranguard: that ... looks wrong
19:22 ranguard looking thanks
19:23 mst 19:23 < treehug88> daemon mst it seems to correlate roughly in time with this:
19:23 mst https://github.com/metacpan/metacpan-api/commit/e1bd0896d37808cd31a85c2bdeefadea32ed197b
19:23 ranguard mickey: did you restart the starman service?
19:23 mickey מם
19:23 mickey no
19:24 ranguard restarting :)
19:25 ranguard mickey: tail /var/log/starman/metacpan-api/starman_error.log on lw-mc-03
19:26 ranguard [illegal_argument_exception] Alias [cpan] has more than one indices associated with it [[author_01, favorite_01,
19:26 mickey good idea :)
19:27 mickey oh, dear
19:27 ranguard mickey: https://github.com/elastic/elasticsearch/issues/7621 ?
19:28 autarch joined #metacpan
19:28 mickey k, let me revert first
19:28 autarch are you all aware of the current breakage?
19:29 mickey autarch: yes
19:29 autarch cool
19:29 mickey we broke it :)
19:29 TBSliver erm.... somethings gone weird? getting the html in a raw thing https://metacpan.org/source/MIYAGAWA/App-cpanminus-1.7042... oh hang on :P
19:29 autarch I figure ;)
19:29 Topic for #metacpan is now We are fixing what we broke
19:29 TBSliver hah okies :)
19:30 jnbek can y'all fix the stuff someone else broke too?
19:30 * jnbek runs for his life
19:31 mickey jnbek: no, we're busy fixing the stuff we broke :)
19:32 jnbek :)
19:33 mickey ranguard: can you restart again?
19:33 ranguard done
19:33 ranguard working
19:33 mickey data is back in cpan_v1_01 and aliases removed
19:34 Topic for #metacpan is now https://github.com/metacpan/metacpan-api/wiki/MetaCPAN-V1-announcement-and-migration-plan | MetaCPAN Developer VM https://github.com/metacpan/metacpan-developer | Chat logs available at http://irclog.perlgeek.de/metacpan/ | Can't find your module on MetaCPAN? https://metacpan.org/about/missing_modules
19:34 mickey should be back to normal... *maybe* we need to restart ES?
19:34 ranguard shouldn't think so
19:34 mickey k, will keep an eye on the log
19:35 ranguard ok, I have to head offline - you have my mobile for emergencies :)
19:35 mickey will have to fix the crons path
19:37 mickey will do it in a couple of minutes... shouldn't be urgent
19:40 autarch left #metacpan
19:44 metacpan joined #metacpan
19:44 metacpan [metacpan-api] mickeyn created mickey/fix_cron_scripts_index (+1 new commit): https://git.io/v1Lpe
19:44 metacpan metacpan-api/mickey/fix_cron_scripts_index 54612e3 Mickey Nasriachi: temp. revert cron scripts to use 'old' index
19:44 metacpan left #metacpan
20:03 metacpan joined #metacpan
20:03 metacpan [metacpan-api] mickeyn pushed 1 new commit to master: https://git.io/v1ten
20:03 metacpan metacpan-api/master 0664dd0 Mickey: Merge pull request #586 from metacpan/mickey/fix_cron_scripts_index...
20:03 metacpan left #metacpan
20:13 metacpan joined #metacpan
20:13 metacpan [metacpan-api] mickeyn created mickey/fix_author_script (+1 new commit): https://git.io/v1tfz
20:13 metacpan metacpan-api/mickey/fix_author_script d57ab1b Mickey Nasriachi: temp. disable index check for author script
20:13 metacpan left #metacpan
20:14 metacpan joined #metacpan
20:14 metacpan [metacpan-api] mickeyn pushed 1 new commit to master: https://git.io/v1tfg
20:14 metacpan metacpan-api/master 2c4ddd1 Mickey: Merge pull request #587 from metacpan/mickey/fix_author_script...
20:14 metacpan left #metacpan
20:20 ranguard mickey++ # breaker and fixer all rolled into one
20:21 mickey "job security"++ :)
20:41 karjala I need to search for a module on metacpan API. It's got to be authorized and latest. However I'm afraid the query I came up with could return more than one results. Let me show you my script.
20:43 karjala http://pastebin.com/7eVbQ5pH
20:43 karjala does this return at most one result? I think not.
20:43 mickey karjala: a module name is not uniqe
20:44 karjala but the latest authorized one is, isn't it?
20:44 mickey to get a unique result you need to fix release+author
20:44 karjala ok
20:45 karjala Here's my problem:
20:45 karjala My site's being given the name of a module (like Algorithm::Diff) and it's supposed to come up with the correct distribution that owns that module (Algorithm-Diff in this case)
20:46 karjala I'm not being given author + release
20:46 karjala I wonder what I can do about that. Let me think a bit.
20:46 mickey this query you pasted returns one result btw
20:46 mickey do you have an example that gave you trouble?
20:47 Grinnz the 'latest' is what would limit it to one result in this case, isn't it?
20:47 haarg unfortunately no
20:47 karjala I don't have an example yet, but I need to make sure I won't run into trouble either, mickey
20:47 haarg it will
20:47 karjala it will what? give me trouble?
20:47 haarg yes
20:48 haarg there are currently two issues with 'latest' that i am aware of
20:49 karjala One problem with the above query, is that I think anyone can make an authorized module (like XML::MyXML) that contains in the same file (XML/MyXML.pm) a package Algorithm::Diff statement
20:50 mickey a good example that will give you trouble 'warnings' (based on an issue we just had)
20:50 haarg "module" in the query refers to package statements
20:50 karjala then module.authorized => 1 will return true, for XML::MyXML, so the query will return this Algorithm::Diff (that belongs to XML::MyxML) too
20:50 haarg so authorized won't be true
20:50 karjala ok
20:52 karjala In the past, before I wanted to use MetaCPAN::Client, I did a GET query on /v0/module/Algorithm::Diff and caught the "distribution" parameter of the one result that came
20:52 karjala is that the way to go?
20:53 karjala But I can't do it this way anymore, because I need to be able to catch up to 500 modules in a single query
20:54 karjala with many "or"s
20:54 karjala a big "or", i mean
20:54 mickey this has nothing to do with the client
20:55 mickey the endpoint just gave back one result (most recent upload date if i'm not mistaken)
20:55 karjala oh
20:55 karjala good
20:56 karjala I can get multiple results for each module name, and sort them by date with perl and capture just the latest one
20:56 mickey best current hack would be querying download_url with a module name ... we're working to fix all module based queries to use its logic
20:56 karjala the latest of the latest authorized modules
20:57 karjala of what use would the download_url be to me, though?
20:57 karjala a URL?
20:58 karjala I need to capture 500 distribution names with a single query
20:58 mickey 1. we could add dist info into it
20:59 mickey 2. it gives you a file path which is the based on the release (dist+ver) name
20:59 mickey did i mention it's a *hack* ? :)
21:00 mickey wait, let me check something
21:02 karjala yes, you mentioned it!
21:03 mickey this issue was fixed
21:03 mickey so
21:03 karjala I'll do it with a complex query, with a big OR and require status=latest & authorized=1
21:03 karjala what issue?
21:03 karjala does anyone see any problem with what I plan to do?
21:03 mickey you can do $mcpan->module('Algorithm::Diff')
21:03 karjala I'll filter through the search results to find the most recent result for each module
21:03 mickey it will have latest and authorized built into the query it uses
21:04 mickey reyjrar fixed it last week :)
21:04 karjala ok, nice, but that will only return one module (I need 500 at a time)
21:05 mickey so you can reuse the same query it uses... sec.
21:05 karjala oh aha
21:05 karjala i see
21:07 mickey check out https://github.com/metacpan/metacpan-api/blob/master/lib/MetaCPAN/Document/File/Set.pm#L20-L79
21:07 mickey atm, that's what I can offer... we are in the process of improving/simplifying the API... it will take time though, we just migrated the ES version
21:08 karjala my god that's complicated
21:08 karjala ok
21:10 haarg i would also like to adjust our data so that we don't need a query that complex
21:11 mickey haarg++ :)
22:03 karjala By the way, everyone, you can check out my script again. Now it brings back TWO modules! http://pastebin.com/7eVbQ5pH
22:03 karjala The most recent one is the wrong one!
22:04 karjala the correct query is with nested, I guess
22:06 karjala mickey: Is it possible to place all that huge query you showed me, into $mcpan->all ?
22:08 mickey karjala: look at the es_filter option
22:08 karjala ok thx
22:09 karjala Is there a limit in how huge a query can be? May I request information for 500 modules?
22:09 karjala listed one after the other
22:10 mickey you get back a scroller (iterator)... read from it 500 times
22:11 karjala I mean, may the REQUEST be huge?
22:12 mickey there are some ES limitations... at some point you will hit the limit on number of values
22:13 mickey I think 500 values should be fine though (untested)
22:15 mickey going to bed, if you want - please move this to a GH issue so we can try to come up with a proper solution within the client
22:34 karjala thanks all (and mickey) - the super complex query solved my problem
22:55 karjala I don't understand what the issue should be about. There already is a way to query a module, it's just: $mcpan->module("XML::MyXML").
22:55 karjala Should I be asking for a way to query multiple modules in one go, with the Client?
23:33 mib_9wsnep joined #metacpan
23:34 mib_9wsnep so there is this thing in the examples: https://explorer.metacpan.org/#/5521729
23:34 mib_9wsnep what would a similar recipe look like to find the name of the largest tarball on CPAN?

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary