Perl 6 - the future is here, just unevenly distributed

IRC log for #metacpan, 2014-08-06

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:49 metacpan joined #metacpan
00:49 metacpan [metacpan-web] rwstauner created bootstrap3 (+32 new commits): http://git.io/0PEWzQ
00:49 metacpan metacpan-web/bootstrap3 e11081f oiami: Migrate to Bootstrap 3.1.1 and introduce Font Awesome
00:49 metacpan metacpan-web/bootstrap3 9208112 oiami: Adjust styling on tablet
00:49 metacpan metacpan-web/bootstrap3 0a7c8ad oiami: Change visible-phone to visible-xs
00:49 metacpan left #metacpan
00:49 dipsy [ Comparing e11081fcbe06^...03230a49a588 · CPAN-API/metacpan-web · GitHub ]
00:50 oiami Finally, it merged !!! :D hooooray !!  :D
00:51 oiami (B)<:o)
00:52 oiami ohh noes ! sorry
00:52 oiami it's just creating branch :p
01:36 klapperl_ joined #metacpan
01:38 FROGGS__ joined #metacpan
02:15 metacpan joined #metacpan
02:15 metacpan [metacpan-web] rwstauner pushed 1 new commit to bootstrap3: http://git.io/o9fXow
02:15 metacpan metacpan-web/bootstrap3 3c87a8b Randy Stauner: Make /recent xpath test slightly more reliable...
02:15 metacpan left #metacpan
02:16 dipsy [ Make /recent xpath test slightly more reliable · 3c87a8b · CPAN-API/metacpan-web · GitHub ]
04:37 rashi joined #metacpan
05:35 talina_ joined #metacpan
06:37 neilb joined #metacpan
07:31 DerAlex joined #metacpan
07:52 rashi joined #metacpan
07:57 neilb joined #metacpan
08:05 chansen joined #metacpan
09:06 [Sno] joined #metacpan
09:06 [Sno] hi
09:06 dipsy hola, [Sno]
09:07 [Sno] is there an API to get the md5-sum for a distfile
09:07 [Sno] to verify the download?
09:16 ranguard [Sno]: not as far as I know but stay around others might know later
09:17 ranguard the source data, eg. http://www.cpan.org/authors/id/L/LL/LLAP/ doesn't have them - if you ask on #toolchain they might know more
09:17 dipsy [ Index of /authors/id/L/LL/LLAP ]
09:17 [Sno] ranguard: mifgt work as workaround so far, thanks
09:17 BinGOs you mean like the CHECKSUMS file
09:18 [Sno] BinGOs: yes, looking up by distname as key
09:18 ranguard doh, man I'm asleep!
09:18 ranguard BinGOs++ # awake
09:19 BinGOs also sha256 is betterer
09:19 [Sno] BinGOs: I take both when I get them ;)
09:19 [Sno] however, they should be exposed through metacpan, either
09:21 BinGOs raise a ticket!
09:21 [Sno] rt? github?
09:22 BinGOs its github I am sure.
09:22 [Sno] will do
09:22 BinGOs https://github.com/CPAN-API/cpan-api I guess.
09:22 dipsy [ CPAN-API/cpan-api · GitHub ]
09:24 [Sno] thank you
11:21 rashi joined #metacpan
12:01 oiami joined #metacpan
12:05 neilb It looks like the page summarisation in MetaCPAN is: $description = get_section_from_pod('DESCRIPTION'); $summary = substr($description, 0, $MAX_SUMMARY_LENGTH). Is that roughly right, or does it do more than just that?
12:13 haarg page summarisation?
12:13 neilb the 2 line summary shown in search results
12:16 haarg https://github.com/CPAN-API/metacpan-web/blob/master/root/search.html#L29
12:16 dipsy [ metacpan-web/search.html at master · CPAN-API/metacpan-web · GitHub ]
12:17 haarg https://github.com/CPAN-API/cpan-api/blob/master/lib/MetaCPAN/Document/File.pm#L141
12:17 dipsy [ cpan-api/File.pm at master · CPAN-API/cpan-api · GitHub ]
12:17 neilb Thanks. That's why my grep didn't find it.
13:55 metacpan joined #metacpan
13:55 metacpan [metacpan-web] rwstauner pushed 1 new commit to master: http://git.io/AAl06Q
13:55 metacpan metacpan-web/master 973828b Randy Stauner: Use search-bar template partial in one more view
13:55 metacpan left #metacpan
13:55 dipsy [ Use search-bar template partial in one more view · 973828b · CPAN-API/metacpan-web · GitHub ]
15:24 DerAlex joined #metacpan
15:52 talina_ joined #metacpan
16:35 neilb joined #metacpan
16:38 rwstauner upgrade to bootstrap 3 has been deployed
16:38 rwstauner feedback welcome
16:52 oalders :)
16:53 oalders i'm seeing lines under the cpan ratings stars https://www.dropbox.com/s/gmftyg9v4e7ltt0/Screenshot%202014-08-06%2012.52.07.png
16:53 dipsy [ Dropbox - Screenshot 2014-08-06 12.52.07.png ]
16:54 oalders that's all so far. looks really good
17:05 ilmari we're seeing intermittently corrupted file downloads from cpan.metacpan.org on travis
17:09 ribasushi ilmari: ranguard knows, seems a "wontfix" due to the only practical fix is to get off fastly
17:10 ribasushi I was trying to find a hybrid cpan/backpan mirror to use but there seems to be no alternative :/
17:11 ilmari how about pestering fastly to fix it?
17:12 ilmari "we" do know a fair number of people there
17:23 oalders ribasushi: do you remember exactly what the fastly issue is here?
17:23 oalders i do remeber this has come up probably more than once
17:24 ribasushi oalders: it generally seems to operate in "best effort" mode
17:24 oalders i'm all for pestering people :)
17:24 ribasushi that is - it may serve the whole stream, or just a part of it
17:24 ribasushi it may end it with 5xx or it may just give you a 200 back
17:24 ribasushi in either case - browsers recover better from this than cpan clients
17:25 oalders ribasushi, ilmari: do you have any debugging output we can use?
17:25 oalders i guess it's in the failing travis builds
17:25 ribasushi oalders: ranguard has plenty collected so far, and has correlation with fastly logs
17:25 oalders i guess at the very least we could set up a hostname for the mirror that bypasses fastly
17:26 oalders would that be an acceptable stopgap?
17:26 oalders or actually we could just disable fastly until that's fixed
17:26 oalders not sure there's a huge win there
17:26 oalders disable for the mirror, that is
17:29 oalders re-opened https://github.com/CPAN-API/metacpan-web/issues/1030
17:29 dipsy [ Intermittent 503's from cpan.metacpan.org · Issue #1030 · CPAN-API/metacpan-web · GitHub ]
17:30 oalders ranguard: thoughts?
17:30 ribasushi oalders: I asked ranguard about disabling fastly for the cpan mirror, he was pretty much against the idea
17:31 oalders ribasushi: ok. we'll revisit this, though. all options on the table :)
17:31 ribasushi cheers ;)
17:32 ribasushi it is a real problem, that keeps happening to me, I just gave up reporting occurrences (I just restart failed builds)
17:32 ribasushi ilmari++ # bringing it up
17:32 oalders we'll deal with ASAP
17:33 oalders we have some randomly failing metacpan builds (for different reasons, i think). it really can be aggravating
17:35 ilmari ribasushi: do you know if ranguard raised it with fastly (and if so with whom)?
17:35 ribasushi no idea
17:35 ilmari seen ranguard
17:35 dipsy ranguard was last seen in #london.pm 5 hours 43 mins ago saying 'Do we know the guy on the right, I want to say Giddeon for some reason?'.
17:38 ranguard last I looked at this I increased the timeouts and then no one reported any problems so assumed all was ok
17:43 oalders yeah, that's noted in the ticket
17:44 ribasushi sorry, reporter fatigue I guess
17:44 oalders heh
17:44 ribasushi from my pov things did improve a bit around May, and have steadily kept the pace since
17:45 oalders ribasushi: these are the DBIC builds that are failing?
17:45 ribasushi now DBIC::SL as well (ilmari's)
17:46 ranguard the graphs really arn't showing many 503's that I can see
17:46 ranguard oalders: please login to fastly and look at the stats for cpan.metacpan.org - let me know if you have any questions
17:46 ribasushi <ribasushi> that is - it may serve the whole stream, or just a part of it
17:46 ribasushi <ribasushi> it may end it with 5xx or it may just give you a 200 back
17:46 ribasushi when I get an example I will share it
17:47 ribasushi though I am not going to be internetering much this month, so likely Sat, instead
17:47 ilmari the logs I saw didn't have any useful info
17:47 ilmari ERROR: Failed to download https://cpan.metacpan.org/src/5.0/perl-5.8.5.tar.bz2
17:48 ranguard at the moment we don't save the fastly logs so we'd need to do that to do more debugging
17:48 ribasushi ilmari: timestamp for correlation is important
17:48 ranguard logs to correlate to is more imporant :)
17:48 ribasushi hmmm... this gives me an idea to force an ntp sync before every job
17:49 ilmari I reran the job, dunno if I can get the log for the previous failed one
17:49 ribasushi ranguard: this was ~2h ago, you ought to have that much log no?
17:49 ribasushi ilmari: you can't no
17:49 ribasushi it's a stupid quirk
17:50 oalders yeah, the graphs aren't telling me much for 500s
17:50 ilmari [21:55:29] ERROR: Failed to download https://cpan.metacpan.org/src/5.0/perl-5.8.3.tar.bz2
17:51 ilmari hm, that's an old one
17:51 ilmari about 20 hours ago, so yesterday
17:51 ilmari presumably UTC
17:52 * ilmari hometimes
17:52 ranguard 185.31.19.20 - - [05/Aug/2014:21:55:32 +0000] "GET /src/5.0/perl-5.8.3.tar.bz2 HTTP/1.1" 200 8666050 "-" "curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3" I'm guessing
17:53 ilmari is 8666050 the size?
17:53 ribasushi 9509716 actual size
17:53 ilmari Content-Length: 9509716
17:54 ribasushi oalders: ^^ so there you have it
17:54 trs (and it recorded a 200!)
17:54 oalders nice
17:54 ribasushi combined with partial tarballs extracting themselves just fine, and no verification in cpan clients - you get fun ;)
17:54 oalders well, you got *most* of it ;)
17:54 trs another reason to use checksums :)
17:55 trs it's too bad CHECKSUMS is so useless.
17:55 ranguard add! - so the request before was: 185.31.19.22 - - [05/Aug/2014:17:14:49 +0000] "GET /src/5.0/perl-5.8.3.tar.bz2 HTTP/1.1" 200 9509716 "-" "curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3"
17:55 ranguard 9509716 vs 8666050
17:55 ilmari ribasushi: more crucially, the HTTP client not failing when it doesn't get the number of bytes indicated in Content-Length
17:56 ranguard yea, sorry looking at configs as well
17:56 * ilmari has added --verify to the cpanm invocation in travis
17:56 ilmari (for schema::loader)
17:56 trs ilmari: I suspect there are enough bad Content-Length headers in the world that it's not worth hard failing when it doesn't match?
17:56 ribasushi .oO( still wondering why we can't just kill fastly for the mirror )
17:57 ilmari trs: I doubt it
17:57 ilmari it's such a fundamental part of http
17:58 ilmari ranguard: is that the fastly log or the origin server log?
17:58 trs and it's super easy for software to get wrong.
17:58 ilmari counting bytes is hard
17:58 ranguard ilmari: origin
17:58 ilmari if you don't know the length, use Transfer-Encoding: chunked or Connection: close
17:58 ilmari ranguard: so it _served_ the wrong number of bytes? boggle
17:59 * ilmari really should run now
17:59 ilmari back later
18:01 trs ilmari: I didn't say it was hard, I said it's easy to get wrong. I've seen tons of bad CLs in the wild.
18:01 ranguard ribasushi: because shortly we'll have 3 data centers with multiple servers at each, so we can build actual resilience and failover rather than relying on one box we can't even upgrade and dare not reboot
18:02 oalders up 165 days
18:03 ranguard yea, that was when we tried to upgrade and would have been lost without rafl doing sysmagic
18:03 oalders :)
18:06 ranguard I'm fairly sure this is our issue, be that the load on the box, the really old nginx we have or how we've configured fastly, but I've put the settings quite high, can increase further if needed I guess
18:11 [Sno] joined #metacpan
18:17 vanstyn_ joined #metacpan
18:18 BinGOs ribasushi: what's this about no verification in cpan clients? CPANPLUS (hence cpanp) does checksum checks on all dist tarballs it fetches.
18:18 BinGOs using those useless CHECKSUMS files.
18:20 trs BinGOs: I was looking at the one in /src/5.0/
18:20 trs since that was the url we were talking about
18:21 trs (there *are* individual checksum files)
18:21 BinGOs I get a 404 for http://www.cpan.org/src/5.0/CHECKSUMS
18:22 trs http://cpan.metacpan.org/src/5.0/CHECKSUMS
18:22 trs so perhaps it's on "backpan" :)
18:22 BinGOs it probably is.
18:23 BinGOs the tarballs in there are symlinks to the authors' path
18:23 BinGOs but have you noticed the .txt files?
18:23 trs ah
18:24 trs yeah, that's what I meant by individual checksum files
18:24 trs also, cpanm's --verify (mentioned by ilmari) does use CHECKSUMS
18:24 BinGOs I guessed it did. Its the default for CPANPLUS to do that.
18:25 trs I was under a false impression about all CHECKSUMS from the one in /src/5.0/ and previous overheard conversation in #toolchain about checksumming that I didn't pay enough attention to.
18:25 trs thanks for pointing out that cpanp also uses it.
18:26 BinGOs cpan (CPAN.pm) probably does too. But I only know for sure about CPANPLUS because I rewrote that code to use SHA256 instead of MD5
18:27 BinGOs Because Digest::MD5 dropped 5.6.x support, then 180'd on that decision.
18:28 BinGOs best see what the wife is up to.
18:28 * BinGOs &
19:05 oiami joined #metacpan
19:31 neilb joined #metacpan
21:09 neilb joined #metacpan
22:27 oalders http://oiami.blogspot.ca/2014/08/work-report-and-new-metacpan-result.html
22:27 dipsy [ Pattawan (oiami)'s Blog: Work report and sketch of new MetaCPAN result page. ]

| Channels | #metacpan index | Today | | Search | Google Search | Plain-Text | summary