Perl 6 - the future is here, just unevenly distributed

IRC log for #crimsonfu, 2016-09-15

crimsonfu - sysadmins who code

| Channels | #crimsonfu index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
01:47 ilbot3 joined #crimsonfu
01:47 Topic for #crimsonfu is now http://crimsonfu.github.com - ConfiguRatIon Management of Systems Or Network kung FU | logs at http://irclog.perlgeek.de/crimsonfu/today
12:18 dotplus isn't the Python Way to just use hashlib and let Python take care of which hashing functions to use?
13:49 bear joined #crimsonfu
15:47 pdurbin The question is "What is the user story?" We put an MD5 next to each file for a reason. Presumably *some* researchers will go to the trouble to calculate the MD5 locally after downloading the file to make sure it wasn't corrupted.
15:49 pdurbin Or a tool such as Archivematica will re-calculate the MD5 values asserted by Dataverse to make sure they match after the files are downloaded. MD5 values are mentioned at https://wiki.archivematica.org/Dataverse
15:49 pdurbin In preservation circles, people call this "file fixity": https://en.wikipedia.org/wiki/File_Fixity
15:51 pdurbin see also "data fixity" at http://www.dlib.org/dlib/january11/crosas/01crosas.html
15:52 pdurbin http://blogs.loc.gov/thesignal/2014/04/protect-your-data-file-fixity-and-data-integrity/
15:52 pdurbin via https://github.com/IQSS/dataverse/issues/2192#issuecomment-104904884
16:03 semiosis pdurbin: why not just use a stronger hash?  sha256?  or for that matter, use PKI signing/verification?
16:06 pdurbin Well, the intention is to be able to support any number of hashes. So sure, sha256 or whatever comes along in the future.
16:09 semiosis i think PKI is the only way to guard against MITM... if someone impersonated Dataverse they could manipulate the data and the hash value, regardless of algorithm.  but if you were signing with a well known key then that wouldn't work.
16:10 semiosis how often do files really get corrupted in transit over TCP?  that's extremely unlikely imo
16:13 bene speaking as an ex-network guy for an ISP, it definitely happens
16:13 bene it's not always the network either
16:14 bene there's all kinds of shitty HTTP caching appliances that mangle the data as it passes through
16:14 semiosis ah good point
16:15 pdurbin hence the need for file fixity, data fixity
16:18 semiosis related, have you heard of parchive?  it generates redundancy files that can be used to repair corrupted files or even replace missing chunks of data.  https://en.wikipedia.org/wiki/Parchive
17:23 pdurbin "Parchive was originally written to solve the problem of reliable file sharing on Usenet"
17:23 pdurbin wow, you'd think I'd have heard of this
21:59 semiosis joined #crimsonfu
21:59 mhayden joined #crimsonfu
22:05 semiosis joined #crimsonfu
22:44 semiosis joined #crimsonfu
22:44 semiosis joined #crimsonfu

| Channels | #crimsonfu index | Today | | Search | Google Search | Plain-Text | summary

crimsonfu - sysadmins who code