Perl 6 - the future is here, just unevenly distributed

IRC log for #metacpan, 2016-07-04

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
03:18 kentnl joined #metacpan
03:38 alh_ joined #metacpan
05:37 alh__ joined #metacpan
05:39 oiami joined #metacpan
06:07 [Tux] yes, I am using a token
07:09 neilb joined #metacpan
07:24 neilb joined #metacpan
07:50 neilb joined #metacpan
08:00 punter joined #metacpan
08:15 Relequestual joined #metacpan
08:18 neilb joined #metacpan
09:15 ilmari https://v1.metacpan.org/release/FREW/DBIx-Class-Helpers-2.033000 is 404ing
09:16 ilmari uploaded 3h ago, works on v0
09:18 mickey last entry in 'recent' is 13h old
10:57 punter joined #metacpan
12:42 punter I have Mojo-Facebook in my metacpan favorites, which doesn't seem to exist anymore (displays 404) and can't remove the favorite (I'm cpan developer karjala)
12:42 punter any nice person can take it off my list?
12:45 punter or, alternatively, what should I do?
12:46 punter likewise for Pinto::Common
13:12 kyzn joined #metacpan
13:30 ranguard punter: ideally you should submit a patch so the code handles this :)
13:30 ranguard mickey: can mc-web search backpan?
13:30 ranguard I mean directly?
13:31 ranguard because technically those modules are now in backpan
13:56 punter I'm too bored to submit a patch... I wish somebody else could do it.
13:56 punter like someone who is more familiar with the platform
14:01 mickey ranguard: in which controller?
14:01 mickey btw, i'm done updating the statuses... looks much better now
14:03 mickey it was a struggle with the ES timeouts (not sure it's ES, might be the bulk module)... but it's over - hopefully we won't need to run this script anytime soon
14:04 mickey ranguard: I need to know to which URL your question was referring, mc-web fires ES queries, most of them will filter out backpan... or at least they can
14:35 ranguard mickey: https://v1.metacpan.org/author/KARJALA - that's listing Pinto-Common
14:35 ranguard but https://v1.metacpan.org/release/Pinto-Common does a 404
14:35 ranguard punter: sure, also it's a non-trivial thing
14:37 ranguard mickey: the question is... should we hi-light that that's actually backpan.. and if so do we have a link that will get the user to it?
14:37 ranguard I don't think we should just delete a user's favs (something might go backpan and then come back!)
14:38 mickey we can (i guess)
14:38 Tux oalders, I now added three issues to Pithub. (and I have no idea how feasible these are)
14:38 ranguard but then maybe we should just not show the link by default and have a 'show backpaned favs'
14:38 mickey punter: you don't have to fix it, but please open an issue
14:38 Tux my main goal is to make the generation of dashboards 1. using the API instead of webscraping, 2. fast, 3. reliable
14:38 mickey ranguard: let's move this to an issue - it will be easier to handle
14:39 ranguard I guess the minimal fix is to have a button to let the user delete it... actually is there something in /account already?
14:39 ranguard mickey: ok
14:54 mickey ranguard: do we have any logging on the release script cron run?
14:56 ranguard mickey: I've not set up anything specific, it would all go into the metacpan.log file AFAIK
15:09 punter mickey, I opened this issue: https://github.com/metacpan/metacpan-web/issues/1742
15:10 punter as you asked
15:10 mickey punter: thanks
15:26 neilb joined #metacpan
15:32 mickey ranguard: running it manually to check for errors... we can also switch the cron to use the new --queue flag to have it all run through minion
15:34 mickey ilmari: the link you mentioned now works... it's our recent release cron that seem to not have run for some reason
15:35 ilmari mickey: cheers
15:35 mickey thanks for reporting
15:40 mickey ranguard: the manual run finished successfully. not sure why the cron didn't run. can you please set logging for it?
15:46 oalders Tux: thanks for creating those issues
15:46 oalders there's a bunch of stuff around the favourites that we can do better
15:47 oalders for instance, I'd like to see the ++ button on the favorites list that appears on my own author page
15:48 oalders so that i can easily remove stuff from that list
15:48 oalders we make it easy to add modules, but you have to go to the same module page to remove the ++
15:48 oalders which means the lists easily get outdated
15:49 Tux Most of http://tux.nl/perl.html is generated by the result of web-scraping
15:49 Tux API's are more reliable ans generaly faster
15:50 oalders Tux: how many requests/hour are you trying to get from the Github API?
15:50 Tux If I can optimized the required throughput to just fetch what is displayed/analyzed, it will be faster
15:51 Tux hah! as I have no idea how many requests Pithub makes for my top-level call I cannot answer that :)
15:52 oalders Tux: you can apply LWP::ConsoleLogger::Easy to your useragent before you pass it to Pithub https://excelmicro.freshbooks.com/showInvoice?invoiceid=6957252&_alt_domain_cookies=W10%3D
15:52 oalders heh, oops
15:52 oalders https://metacpan.org/pod/LWP::ConsoleLogger::Easy
15:52 Tux on avarage I'd say that I need about 20 "values" (or data points) per module/release
15:52 oalders that will show you how many API calls are being generated
15:52 Tux I currently have 17 modules
15:53 oalders also, that will show if if the Github token is actually being used
15:53 Tux that is a point I also try to make in the issues
15:54 oalders You're hitting API limits on just 17 modules?
15:54 Tux the list of modules for an author uses the token, but I cannot see how the IF for issues and commits can be invoked from the ->new
15:54 Tux I'm not yet hitting the limit for just *my* modules
15:54 oalders the IF?
15:54 Tux I hit it when testing
15:55 Tux IF = InterFace
15:55 Tux nd I also hit it when I *also* create dashboards for ether, rjbs, book and so on
15:55 Tux http://tux.nl/Files/dashboards.html
15:56 oalders You get 5,000 requests per hour
15:56 Tux that should suffice once I slim my code down
15:56 oalders Have you tried using WWW::Mechanize::Cached while testing?
15:57 oalders That won't eat up extra API calls when you re-run the code
15:57 Tux no, I had to spend time on family stuff
15:57 Tux :)
15:58 oalders :)
15:58 oalders Well, I'd start with that when you're testing.  It's faster as well.
17:46 punter joined #metacpan
17:58 ranguard mickey: If it goes via --queue then the logging would be what ever minion queue does
17:58 ranguard currently it would end up as an email to noc@mc.org
17:59 ranguard mickey: should I deploy: https://github.com/metacpan/metacpan-puppet/commit/59d3122faefb3fa00fccb866c94553d42d41dbd6 ?
18:00 ranguard jberger: is there a nice tool (other than pgsql) for looking at, reporting and/or logging failed jobs in the minion queue?
18:01 jberger The tooling around minion is still rather young i would say
18:02 jberger They're is the cli interface that can help you report
18:02 ranguard jberger: so that's a 'no' then :)
18:02 jberger Or there is my Minion::Notifier which gives you better access to events
18:02 ranguard ok, well we'll need to build something I guess
18:02 jberger I'd probably start there
18:03 ranguard cheers
18:03 * ranguard goes off to make dinner for now though... will leave it on our todo list
18:04 jberger I could start something if you give me a little direction
18:05 jberger Can't promise a full polished thing but i could probably hack out a poc
18:12 jberger ranguard: ^^
18:13 jberger All i really need is where do you want/need the notification/report to go
18:16 oalders jberger++
18:20 mickey ranguard: sure, let's try it and see how if works.
18:21 mickey it will give us warnings on specific releases, but i suspect it won't solve the mistery of why the cron didn't run
18:22 mickey *mystery
18:23 mickey i think it's still not running... last entry in 'recent' is 3h old (since manual run)
18:23 mickey that's not good
18:23 oalders yep, let's use the queue there
18:23 oalders less messing around with crons can't be a bad thing
18:25 jberger just to clarify, we're back on master (for metacpan/metacpan-api) right?
18:27 oiami left #metacpan
18:30 oiami joined #metacpan
18:41 oalders jberger: that is correct
18:41 oalders all repos should be back to master now
18:44 jberger nice
18:51 jberger so this is the simplest possible demonstration: http://pastie.org/10898777
18:51 jberger that is basically what Minion::Notifier does to attach to the failed (and finished) events
18:52 jberger it goes further and also attaches to a message bus so that anything in the ecosystem can get notifications
18:52 jberger from your perspective if you want to say send an email or the like you can just do it from the worker process there
19:09 jberger ranguard / oalders (and others): does that make sense?
19:10 oalders jberger: looks good
19:18 mickey jberger: that's really nice
19:25 jberger like I said, the tooling is still young but the hooks are pretty robust
19:25 jberger no one has quite set out to build a generic reporting system yet though
19:25 jberger the closest might be my Minion::Monitor (which is still unreleased) but even it doesn't actually alert it just gives you a web view into stats etc
19:26 jberger if anyone wants to hack on it they are certainly welcome
19:37 mickey ranguard: 'release' cron is set to run once a day (?) and 'latest --force' every 30m? that's odd
19:43 oalders mickey: we have to update latest every 30m because we know about packages before they appear in 02packages
19:44 oalders if we waited longer then new releases wouldn't show up as "latest"
19:44 oalders at least not in a timely manner
19:48 ranguard jberger: cool thanks
19:49 ranguard mickey: oalders knows about the cron's more than me, I'm just a sysadmin and do what I'm told :)
19:50 oalders the release cron is run daily to (i think) catch anything that might have gotten missed over a 24h period
19:55 mickey oalders: 'latest' scrolls over indexed files
19:55 mickey which cron then indexes them?
19:55 oalders mickey: the watcher should do the indexing
19:55 mickey also, 'latest' takes >30m AFAIK
19:55 oalders watcher indexes then cron sets "latest" flag
19:56 mickey is watcher supposed to run as a cron?
19:56 oalders no, it's a daemon
19:57 oalders /etc/init.d/metacpan-watcher
19:57 oalders does latest take > 30 m on v0?
19:57 oalders i know we slowed it down a bit for v`
19:57 oalders *v1
19:58 mickey we run it with --force (because some issue) so should take longer
19:58 mickey i don't see the watcher running on 03
19:58 mickey maybe that's why recent didn't get updated
19:59 mickey restarted (indeed, restart complained about some pid not running)
20:00 mickey ranguard: can you please add a cron/something to check on the watcher?
20:04 punter joined #metacpan
20:10 mickey running 'release' again to cover the missing recent 4h
20:11 oalders if the watcher wasn't running, then that would be the problem
20:12 ranguard mickey: ok, please add ticket, running puppet should have sorted that
20:12 mickey oalders: seems so :)
20:13 mickey ranguard: sure. restarting wasn't the problem... but nothing checked the pid actually exists
20:14 mickey will open the ticket now
20:15 mickey ranguard: https://github.com/metacpan/metacpan-puppet/issues/112
20:16 * mickey going to watch the dumb-box for a while
20:16 ranguard mickey: thanks
20:25 punter joined #metacpan
23:41 alh___ joined #metacpan

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary