Perl 6 - the future is here, just unevenly distributed

IRC log for #metacpan, 2016-06-22

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:53 kyzn joined #metacpan
01:08 kivanc joined #metacpan
01:09 kyzn joined #metacpan
01:18 guest9049 joined #metacpan
01:21 guest9049 left #metacpan
02:08 alh__ joined #metacpan
02:10 kivanc joined #metacpan
02:11 kivanc joined #metacpan
02:18 alh___ joined #metacpan
02:46 kyzn joined #metacpan
04:33 kyzn joined #metacpan
06:50 oiami joined #metacpan
07:14 neilb joined #metacpan
07:43 kyzn joined #metacpan
07:52 neilb joined #metacpan
08:05 fuzzix joined #metacpan
08:07 Relequestual joined #metacpan
08:13 neilb joined #metacpan
08:54 ranguard ahh, ok, so v1.mc.org hits fastly, and I set that up to point to lw-mc-03 but change the host header to be just metacpan.org
08:54 ranguard so that it was as close to being what we will put in production as possible!
08:55 ranguard but that's why $c->req->base is reporting https://metacpan.org/
08:56 haarg fastly
08:56 ranguard mickey: I can change that, or we just make it relatie instead?
08:56 haarg somehow i forgot about that when considering the layers for this issue
08:57 haarg if you just have v1 report v1 as its host header it should fix the issues
08:57 ranguard haarg: v1.mc.org isn't actually in the nginx config at the moment
08:57 haarg ah, missed that bit
08:57 ranguard but yea I can add that in and then change the host header
08:58 haarg i was confused, because the nginx config as set shouldn't have caused the issues we were seeing
08:58 haarg so i was kind of guessing
08:59 haarg getting catalyst to respond with relative URLs is going to be awful or worse
08:59 haarg i checked the code
08:59 ranguard heh, ok, I'll sort with nginx/fastly
08:59 haarg it manually blesses things into various URI packages
09:05 ranguard fixed
10:59 Tux I get "Cannot send to channel" on #cpantesters
10:59 Tux Error 503 backend read error
10:59 Tux backend read error
10:59 Tux Guru Mediation:
10:59 Tux Details: cache-fra1233-FRA 1466592914 133818886
10:59 Tux Varnish cache server
11:01 Tux right, /me adds -discuss
11:05 haarg on cpantesters?
11:05 haarg seems like #perl-qa or #cpantesters-discuss?
11:06 haarg well i guess you're crossposting
11:09 oiami joined #metacpan
11:55 oiami joined #metacpan
13:09 mickey ranguard++
13:38 metacpan joined #metacpan
13:38 metacpan [metacpan-web] mickeyn created fix/links (+1 new commit): https://git.io/voiQ9
13:38 metacpan metacpan-web/fix/links ee928fc Mickey Nasriachi: fix more links
13:38 metacpan left #metacpan
13:40 ranguard mickey: that links pull has some wierd stuff in it
13:41 metacpan joined #metacpan
13:41 metacpan [metacpan-web] mickeyn pushed 1 new commit to fix/links: https://git.io/voi7t
13:41 metacpan metacpan-web/fix/links ead9fa9 Mickey Nasriachi: doh
13:41 metacpan left #metacpan
13:41 ranguard Just to confirm https://api-v1.metacpan.org/ will be our new API domain
13:41 mickey is it just the `about`?
13:42 ranguard But once ready we will make 'https://metacpan.org/' point to the new servers and drop v1.mc.org
13:42 ranguard root/static/opensearch.xml
13:42 mickey sure, we just don't want to currently link from v1 to v0
13:44 mickey i'll update that one back
13:45 ranguard :)
13:45 * ranguard adds another comment as well, canonical link MUST be to mc.org NOT v1.mc.org
13:45 metacpan joined #metacpan
13:45 metacpan [metacpan-web] mickeyn force-pushed fix/links from ead9fa9 to 87f88ad: https://git.io/voi5Y
13:45 metacpan metacpan-web/fix/links 87f88ad Mickey Nasriachi: fix more links
13:45 metacpan left #metacpan
13:47 metacpan joined #metacpan
13:47 metacpan [metacpan-web] mickeyn force-pushed fix/links from 87f88ad to 2c52fd8: https://git.io/voi5Y
13:47 metacpan metacpan-web/fix/links 2c52fd8 Mickey Nasriachi: fix more links
13:47 metacpan left #metacpan
13:48 mickey ranguard: rolled back the ones you didn't like
13:49 oalders ranguard: are you saying we'll have https://api-v1.metacpan.org/v1 permanently once we go live?
13:51 ranguard oalders: yes, as we've said we'll support api.mc.org/v0 for a while and that can't go through fastly so has to be a seperate domain
13:52 oalders ranguard: ok.  just trying to figure out the versioning.  seems weird to have it twice in the url
13:53 ranguard you dont actually need the '/v1' in the URL AFAIK, but think MC::Client needs it to specify an index?
13:54 oalders no, you don't need it, but we've been trying to encourage people to use it.
13:55 oalders we should clean everything up not to use a v1 in the top dir then
13:55 ranguard mickey: comments added
13:56 ranguard oalders: your call on if we keep the '/v1' in or not, I don't know the remifications
13:57 mst no
13:58 mst oalders: the /v1/ permanently is a good thing
13:58 oalders mst: so versioning in the host and the path?
13:59 mst I don't care at all about 'in the host'
13:59 * ranguard goes to do school run
13:59 oalders I think it would be nice if MetaCPAN::Client would DTRT with the url based on the version arg
14:00 ranguard oalders: maybe now is the point to enforce that '/v1/ is added *shrug*
14:00 oalders mst: ok. any particular reason?
14:00 * ranguard goes to do the school run
14:00 mst but I think having it in the path would be really helpful because, for example, were I to throw up a dev rig
14:00 mst or my own local index
14:00 mst I may not be allowed to provision extra hosts
14:00 mst but I can totally provision URL path parts
14:00 mst so I think that's where the versioning should live
14:01 oalders that's a compelling argument
14:01 oalders i'm fine with that then
14:02 mst it's one of those 'obvious the instant you've seen it' things
14:02 oalders :)
14:07 kyzn joined #metacpan
14:31 zostay joined #metacpan
14:36 metacpan joined #metacpan
14:36 metacpan [metacpan-client] mickeyn deleted oalders/fix-dzil-build at 4d929ca: https://git.io/voih1
14:36 metacpan left #metacpan
14:36 metacpan joined #metacpan
14:36 metacpan [metacpan-client] mickeyn deleted oalders/travis at 7f2bba3: https://git.io/voihM
14:36 metacpan left #metacpan
14:36 metacpan joined #metacpan
14:36 metacpan [metacpan-client] mickeyn deleted oalders/domain at a28c5ec: https://git.io/voihD
14:36 metacpan left #metacpan
14:54 kivanc joined #metacpan
14:55 kyzn joined #metacpan
14:55 kyzn joined #metacpan
15:12 kivanc joined #metacpan
17:58 neilb joined #metacpan
18:10 ranguard oalders: so I'm going to make /v1 required in the URL
18:13 trs Will that mean that clients which don't explicitly talk to /v0 break?
18:15 ranguard trs: they have to upgrade their domain in anycase, so as we're doing breaking stuff seemed better to do it in one go
18:15 ranguard s/upgrade/update/
18:16 ranguard The /v0 will continue for $time after api-v1 is made master (see annountment)
18:17 ranguard so it will only be clients talking to api-v1.mc.org which will have to have /v1
18:20 ranguard hmm, mickey / oalders please change mc-web to use the version path!
18:21 * ranguard updates the breaking changes in https://github.com/CPAN-API/cpan-api/wiki/MetaCPAN-V1-announcement-and-migration-plan
18:29 trs Oh, I thought api.mc.org/v0 and api.mc.org/v1 is what was being proposed.
18:30 trs and that api.mc.org/.../ sans v0 or v1 gets v0 until it's retired, and then it gets v1.
18:34 oalders trs: that's what i thought as well
18:36 trs api-v1.mc.org feels like an implementation detail that should be papered over.
18:36 oalders ranguard: is this related to fastly?
18:37 ranguard oalders: yep
18:38 ranguard we can't have api.mc.org go via fastly, as any currently client that uses GET + Body will go boom
18:39 ranguard so we need another domain name if we are having both old and new running at the same time
18:39 oalders right
18:40 ranguard but it doesn't have to be 'api-v1.mc.org' - could be 'fastapi.mc.org/v1'
18:41 oalders good point.  i'll let you and trs battle it out on the hostname ;)
18:42 * ranguard quite likes fastapi.mc.org
18:44 trs anything that's not api.mc.org feels like an implementation detail. :)
18:45 ranguard trs: it IS, but it's an important one :)
18:48 ranguard going once for 'fastapi'...
18:49 trs Yeah, I understand the problem now.
18:52 trs In the future, it'd be nice to see <single-domain>/v{1,2,3,...,n}/
18:54 ranguard trs: totally
18:55 oalders i suppose after v0 is retired we can change this and redirect everything to that domain
18:58 ranguard yea
19:08 trs Right, after v0 is gone, api.mc.org can go through fastly and fastapi.mc.org can redirect to api.mc.org.
19:08 trs And then versions will only be in the URL not the host.
19:08 oalders That sounds like something everyone can live with?
19:14 ranguard ++
19:14 ranguard I'll set it up after dinner
19:23 trs I can live with whatever folks decide, since I'm merely pitching in my 2¢ and you all are doing the work.
19:31 mickey ++
19:33 kyzn joined #metacpan
19:36 ranguard fastapi or fast-api :) ? - seeing as I'm here!
19:38 ranguard mickey: oalders ^^ :) ?
19:38 mickey i'm fine with both, likes fastapi slightly more
19:39 ranguard first responder wins :)
19:40 mickey \o/
19:41 ranguard mickey: and you win.... researching how to get mc-web to use /v1/ in it's request path :)
19:42 mickey doh!
19:43 mickey i might have some time tomorrow, but then traveling back so out until prettry much the end of the weekend
19:44 oalders what are the most pressing issues right now?
19:44 mickey isn't it just changing the config?
19:44 ranguard mickey: might be, I can have a try if you don' tknow of the top of your head :)
19:45 mickey i'm pretty sure that's what it *should* be
19:45 mickey oalders: login?
19:46 ranguard well, login we need to test, but not actually a requirement to make live?
19:47 ranguard mickey: what was that issue about somethings not be being fully indexed/searchable?
19:47 ranguard that and the download_url end point I'd guess
19:47 mickey i think the 'latest' marking
19:47 mickey we can solve it by changing the cron to use --force
19:47 mickey it's slower but will fix the problem
19:48 mickey don't see why we should care
19:48 ranguard mickey: lets do that then :)
19:48 ranguard with a comment about why maybe?
19:49 oalders mickey: right, login is key
19:49 oalders and download_url as well
19:49 mickey sure - temp. until we figure why it's not working properly :)
19:49 ranguard oalders: https://github.com/CPAN-API/metacpan-puppet/blob/master/hieradata/common.yaml#L279 that one?
19:49 ranguard or https://github.com/CPAN-API/metacpan-puppet/blob/master/hieradata/common.yaml#L309 ?
19:49 ranguard (or both)
19:50 oalders i'm actually not sure what --force does
19:50 mickey ranguard: first one
19:50 ranguard thanks
19:50 mickey oalders: i added it not to skip the 'lastest' marking on distributions... it will just override it always
19:51 mickey i think you merged it :)
19:52 oalders heh
19:52 oalders yeah, that'll be slower then, but probably fine
19:53 mickey on $work server it took me 1-2h ... should be roughly the same
19:53 oalders sounds good
19:53 oalders i think the v1 will need to be enforced in MetaCPAN::Web::Model::API::request()
19:54 oalders we just need to clean up $path
19:54 ranguard deployed the --force
19:54 oalders ranguard++
19:54 mickey ranguard++
19:54 ranguard oh, this is in cron every 30 mins - issue?
19:55 ranguard oh, or rather ever 30 mins past the hour, so once an hour
19:55 mickey why do we need it every 30m? the release one is running with --latest
19:56 * ranguard shrugs, that's just what it is - what should it be?
19:56 ranguard nightly?
19:56 mickey i thought it was one a day or something...
19:57 mickey i say yes, let's try that and see if we have any issues with it
19:57 ranguard ok
19:57 mickey the release script should keep the marking correct
20:00 oalders we need that one to run on the hour
20:00 oalders or nothing gets marked as latest
20:01 oalders we would run it more often, but in the past it has been a real pig
20:01 ranguard oh!
20:01 oalders unless the queued items are now getting --latest applied
20:02 oalders actually, i don't think that's enough in every case
20:02 oalders given the timing of when we get the file vs when the other PAUSE files get updated
20:02 ranguard so we do want `latest --force` to be hourly, but it'll take more than an hour?
20:03 ranguard does that matter?
20:03 oalders mickey: which script did you have run for 1-2 hours?
20:03 oalders the one that runs on the hour shouldn't take that long.  it just hogs resources
20:03 oalders having said that, now that we're on a cluster we could look at running it on one of the boxes that isn't serving anything
20:03 ranguard oalders: actually it runs at 30 mins past the hour
20:04 mickey latest
20:04 oalders s/on the hour/hourly/
20:04 mickey it forced an update for the 'lastest' field for every release
20:05 mickey i don't think the release cron actually queues the tasks
20:05 mickey i can tough, i added --queue support
20:05 ranguard no, it'll just start another one at the same time
20:06 oalders moving everything to the queue could help with the resources as well
20:06 ranguard so just add that as well to `latest --force` ?
20:06 mickey we can make it - but it will slow full reindex... should we care?
20:07 oalders can we disable it for the full re-index?
20:07 mickey sure
20:07 ranguard is there a reason we actually need this.. I'm guessing it's because we have a bug somewhere else?
20:08 oalders the initial problem was that we got tarballs _before_ 02packages was updated
20:08 oalders so we indexed right away, but we could check perms until much later
20:08 mickey ranguard: obviously a bug somewhere :) it's just a quick fix until we figure it otu
20:08 mickey *out
20:09 ranguard oalders: ahh,
20:09 oalders 02packages gets updated more frequently now, but i'm not sure if it's in real time.  i thought it was every 5 minutes
20:09 oalders what we could probably do insert a job into the queue that says "reindex this dist in 8 minutes"
20:09 mickey i think we agree most of the logic in that script needs some love... isn't that on our hackathon list?
20:09 oalders :)
20:10 oalders using the queue would mean that we could just re-index the one dist rather than looking at everything from the last hour or whatever
20:10 oalders that would be cheaper
20:10 ranguard ok, adding comments and --queue to the `latest --force` command for now
20:12 oalders thanks!
20:12 ranguard https://github.com/CPAN-API/metacpan-puppet/blob/master/hieradata/common.yaml#L279 updated
20:13 metacpan joined #metacpan
20:13 metacpan [cpan-api] mickeyn created mickey/release_queue_w_latest (+1 new commit): https://git.io/voXk0
20:13 metacpan cpan-api/mickey/release_queue_w_latest 08c1a15 Mickey Nasriachi: when queueing from script/release also provide --latest
20:13 metacpan left #metacpan
20:14 mickey --queue to latest won't work ... until we patch it to
20:15 ranguard mickey: quick patch :) ?
20:15 oalders you can tell which one of you is the manager
20:16 ranguard yea mickey stop being so bossy :)
20:16 oalders :)
20:17 mickey ok
20:17 mickey i stepped down as a manager for a reason
20:18 oalders ah, i forgot you were a manager as well
20:18 ranguard heh, I stay one because I know others are better at code :)
20:19 ranguard https://fastapi.metacpan.org/v1/release/ETHER/Moose-2.1603 now works
20:20 oalders ranguard: you're just in it for the perks
20:20 oalders what was the problem with that URL before?
20:20 mickey eh, it will still be a bit slow cause it first reads the entire list of distributions
20:21 ranguard oalders: note the domain name :)
20:21 oalders ah, right
20:21 oalders should we redirect api-v1?
20:21 mickey i'll try to work something better than just a quick fix for this one
20:22 oalders mickey: i _think_ we just need to queue a job that will do this per dist at some interval after the initial indexing has run
20:22 oalders we can tell Minion to wait before running a job
20:22 ranguard oalders: once we've switched our own code over to use the new one :)
20:23 mickey the script already supports --distribution... so it will be faster to loop over the list and run it for each with --queue (to be implemented)
20:23 mickey as it is now it will just take a while _before_ starting to queue
20:24 mickey nm, i just think i will do it wrong if i keep coding during the Larry interview :)
20:29 metacpan joined #metacpan
20:29 metacpan [cpan-api] mickeyn force-pushed mickey/release_queue_w_latest from 08c1a15 to 0bcecfb: https://git.io/voXt2
20:29 metacpan cpan-api/mickey/release_queue_w_latest 0bcecfb Mickey Nasriachi: when queueing from script/release also provide --latest
20:29 metacpan left #metacpan
20:30 ranguard FYI, the api_secure is just domain, adding the path doesn't work - should be just changing API.pm
20:48 reyjrar joined #metacpan
22:01 kivanc joined #metacpan
22:02 kivanc joined #metacpan
22:04 leejo joined #metacpan
22:24 kyzn joined #metacpan
22:47 kyzn joined #metacpan
22:47 kyzn joined #metacpan

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary