Perl 6 - the future is here, just unevenly distributed

IRC log for #perl6-toolchain, 2016-01-09

| Channels | #perl6-toolchain index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:27 leont joined #perl6-toolchain
01:50 japhb joined #perl6-toolchain
02:28 ugexe ah, pwned by EVAL 'use 007' ($dist<name>) instead of EVAL 'use _007::Runtime' ($dist<provides>.keys[0])
06:35 Zoffix joined #perl6-toolchain
09:10 llfourn joined #perl6-toolchain
09:15 hankache joined #perl6-toolchain
10:14 FROGGS_ joined #perl6-toolchain
10:33 masak oh -- is there something I can do to help on the 007 side, or is it strictly a zef thing?
11:39 nine panda now actually checks a dist's Perl 6 version requirement and errors out if it cannot be met. This should make it much easier to diagnore installation issues caused by the user having a too old Rakudo.
12:24 leont joined #perl6-toolchain
12:27 nine panda will now fall back to curl and then to wget if downloading meta data failed. This should fix the proxy issues.
13:02 nine I wonder why we use git:// URIs in the ecosystem. panda seems to be dealing with https:// just fine. Although I can't quite figure out why...
13:03 nine The second "why" is because panda's project-from-git really checks for ^git://
13:16 leont Why wouldn't we?
13:34 nine You mean why wouldn't we use git://? Because github recommends https:// as more reliable. Especially in restricted network environment.
13:35 JimmyZ +1 to https
14:01 leont Restricted network environments is a good reason
14:02 leont (though I still hate it for development)
14:26 hankache joined #perl6-toolchain
15:38 llfourn nine: I have PR to docs which uses precomp to replace EVAL slurp. It may be of interest to you https://github.com/perl6/doc/pull/334/files :)
16:17 autarch joined #perl6-toolchain
16:26 nine llfourn: I _like_ that!
16:26 llfourn nine: thanks! I thought you might.
16:26 hoelzro nine: not to mention that git:// is subject to MITM attacks
16:27 hoelzro leont: ↑
16:27 leont Is it?
16:28 hoelzro mhmm: http://git.661346.n2.nabble.com/git-protocol-over-SSL-TLS-td7601259.html
16:29 leont I suppose it is when the host hasn't seen the server yet, but given almost all repositories use github, that seems rather difficult to pull off
16:29 mst also, if you actually care about integrity you should be getting checksums via a separate route
16:29 hoelzro leont: true, it's more of a theoretical than practical attack
16:30 hoelzro mst: *nod*
16:30 hoelzro it just occurred to me that this channel is probably slow enough to effectively backlog \o/
16:31 hoelzro I was going to ask a question about moving away from GH as the canonical place to fetch modules from
16:32 Zoffix GH is just temporary. jdv79 et. al. are working on the P6 PAUSE/CPAN/MetaCPAN stack
16:32 mst uhhh
16:32 mst hang on
16:32 mst we have jdv79/ranguard's prototype of putting stuff onto cpan via pause
16:33 Zoffix It works in fact. You can upload Perl 6 modules to pause.perl.org *right now*
16:33 mst yes, and cause carnage and problems because nobody knew anybody was going to do that unbtil ranguard did it
16:33 mst so everything downstream is completely unprepared
16:33 autarch I really hope that whatever the end solution is that I never go to metacpan.org and see Perl 6 stuff
16:34 Zoffix No, it'll be a separate site
16:34 hoelzro *whew*
16:34 mst there will be an mc6
16:34 mst but there's lots of things that trail the PAUSE aupload feed
16:34 mst and basically every time somebody uploads a Perl6 dist half of them crash
16:34 mst and then distro packaging teams start asking the perl5 toolchain "wtf are these? should we be packaging them? if so how?"
16:35 mst and since nobody talked to the perl5 toolchain team first our answers have mostly been "they did WHAT?! what the flying fornication?!"
16:35 mst so I would prefer that we don't encourage -lots- of people to start uploading until we're a little more certain of what we're doing and have some public messaging explaining what and why
16:36 leont Yeah, it doesn't seem very useful
16:36 Topic for #perl6-toolchain is now Fire is step THREE! | https://github.com/perl6/toolchain-bikeshed
16:37 mst it may be that slotting them all together will be
16:37 mst and it may be that cpan is the best distrribution mechanism to use for tarballs
16:37 autarch I would note that the PAUSE website for authors is not that great to use, and I'd be very happy to not have to use it for p6 stuff
16:37 mst but I think everything else is "this is an open question because it's clear nobody's actually thought it through yet, or if anybody did, they're a distinct set from the people currently trying to do things"
16:38 * leont would like to see a solution based on dumb data storage, but smart data indexing
16:38 Zoffix autarch, yeah, and the argument there is there are only so few hands to code stuff, so doing-something-new() is starved for resources
16:38 autarch yeah, I get that
16:39 leont Having non-turing-complete versions (end other requirements?) would help there, since then we could ask the index "I need a Foo that satisfies these requirements" and it can actually be helpful
16:39 nine mst: FWIW I actually tried bringing the Perl 6 on CPAN topic up at 2014's Austrian Perl Workshop, at the 2015 QA Hackathon and at YAPC::EU, but Perl 5 people didn't seem interested.
16:40 autarch nine: the problem is that the people who maintain the Perl 5 CPAN stuff are all already busy, and presumably not that enthused about doing more work for something they may not ever use
16:40 nine With few exceptions like Andreas with whom I had a very interesting chat about topics like this :)
16:40 mst I think, understandably, nobody was really convinced at that point perl6 was ever going to ship
16:41 nine yep, that was the vibe I got
16:42 nine I guess this will (slowly) change now :)
16:43 mst and andreas, sadly, barely talks to the rest of the toolchain people
16:43 autarch well, don't expect every p5 person to care about p6 - some will, some won't - there are plenty of p5 people who really don't like p6
16:43 autarch (I mean don't like the language)
16:44 mst personally, ideally, I think we would do well (ab)using the 'giant distributed tarball store with user accounts' part of cpan
16:44 nine autarch: I don't expect everyone to care. I just think, it will be harder to deny that Perl 6 exists :) Though if Perl 6 matters is a different question.
16:44 mst and then trying to basically avoid the rest of it
16:44 mst but that may require more tuits than are available
16:44 mst I dunno
16:44 mst I'd just like to not run before we can walk
16:45 nine Yes, I'd love to be able to use CPAN just as a tarball store as a first step and be able to download those tarballs with panda
16:45 nine Or even just use the tarballs for distro packages
16:46 leont You might want to write an Archive::Tar and a gzip/bzip2/xz module then, those are rather missing right now
16:47 nine leont: I'm me. I'd not hesitate to use Inline::Perl5 for that ;)
16:48 leont This is toolchain, you can't assume all platforms will have p5
16:48 nine panda already does
16:48 nine and rakudo's build
16:48 nine Well panda assumes you have a "prove"
16:48 leont I know panda does, and have a plan to change it
16:48 mst though I can imagine win32 could easily have neither perl5 nor tar
16:49 leont Exactly
16:49 mst honestly though I think "needs perl5" is not actually that high on the list of problems
16:49 autarch yeah, p6 really needs to get to the point of being self-bootstrapping if we expect more adoption
16:49 leont panda's architecture doesn't lend itself for a drop in replacement for TAP::Harness, but it's not like the module isn't written or working
16:49 autarch I should say the toolchain does - having the actual build require p5 isn't as big a deal, I suspect
16:50 leont Requiring it on build is suboptimal, but not something urgent to fix
16:50 leont Requiring it at runtime is rather embarrassing
16:50 autarch yeah
16:51 autarch I do note that you can start Archive::Tar etc by just looking for CLI utilities - as an intermediate measure
16:51 leont AT isn't that complicated, but there are lots of gritty details to it as far as I understand (because of 40 years of history).
16:52 leont A straight port of p5's AT may not be a bad starting point, because it has dealt with various of those issues
16:54 ugexe zef can download/untar from metacpan
16:55 mst I wonder if the PAUSE issues would be solved simply by removing the PErl6/ dir from the upload feed and minting a separate upload feed
16:55 mst that might stop people seeing things they're confused by
16:55 ugexe or it could a few days ago. need to update a few things to use DependencySpecification  again
16:56 mst ok, this may be a stupid question but - how did we end up with panda and zef? just two people working on ideas independently? or is there some philosophical difference that caused a deliberate split?
16:57 mst (tried as hard as I could to make the question neutral, I don't have an opinion here, I just don't know the history)
16:57 nine There's also ufo
16:58 leont Does anyone actually use ufo?
16:58 ugexe panda would fail to work for days at a time due to a dependency or something else. so zef was meant to be more plugin based and without any (perl6) external dependencies so it was easier for an end user to keep working
16:59 ugexe also to do things different from panda for the sake of doing it different. i.e. panda used CU::L::File and zef used CU::L::Installation
17:00 mst at this early stage, doing things differently for the sake of it honestly sounds like a great idea (previous sentence was not sarcasm)
17:03 leont I suspect zef would be an easier target for integrating TAP::Harness than Panda does (maybe also because it doesn't do much right now in that corner)
17:04 autarch m: say False ~~ False
17:04 camelia rakudo-moar 725348: OUTPUT«False␤»
17:04 autarch is that really intentional?
17:05 autarch doh, wrong channel
17:07 ugexe it may well be. i have not yet gotten around to using your test harness to see but its been my intention for it to be the default other than running `perl6 -Ilib t/file.t` on each file
17:10 nine ugexe: I think zef doesn't handle resources correctly yet.
17:11 nine Other than that I like what I've seen so far.
17:11 leont It's fully functional except the async parts (rakudo bugs) and currently the ANSI coloring (a rakudo regression)
17:14 ugexe yeah there is still more work to do on it like parallization, a way for the phases to talk to each other during parallelization, terminal output prettification (instead of the debug messages), although those 3 were implemented in the previous version
17:14 ranguard joined #perl6-toolchain
17:15 ugexe leont: what parts of it make it difficult to use with panda? it may make it easier for me to avoid them
17:16 nine ugexe: how does zef know that a dependency is already installed?
17:17 ugexe ive seen your .resolve fork so i havnt put much effort into that. it evals the dist's identity (Distribution.Str()) and if that fails its evals the first provide's spec string
17:18 ugexe previously it would read the manifest
17:21 ugexe a manifest that isn't multiple levels deep like the previous one just for mapping some basics like where to find the installed dist meta might not suffer from the slowness of the previous manifest
17:32 mst nine: oh, when you get a minute, can you try and summarise what you got from your conversations with andreas? I doubt they're written down anywhere
17:41 nine mst: I'm not sure I can remember all that much. Writing it down would have been an excellent idea. I know it was about Perl 6 on CPAN, cross language dependencies and how rpm already solves many of the problems we struggle with.
17:41 nine ugexe: If you have any input on .resolve, now would be the time :)
17:41 mst rpm is unusual in that it can handle having multiple versions of the same package installed so long as they don't have conflicting files
17:41 mst it continues to sadden me that /deb doesn't
17:41 mst I put this in at least one YAPC talk
17:44 ugexe nine: im not sure really. the method of a manifest that only maps dist identity + provides identities to the actual dist file allows a way to only read the appropriate meta files without having to recursively read them all until you find what you need. the drawback being its more fragile, like when someone manually deletes a distribution. im guessing either way its specific to each CU::R
17:45 ugexe i think part of the problem of the previous manifest is it had to read like 6 levels deep of every single distribution to find what it needed
17:46 mst I'm pretty sure I can create a much better repo internal format
17:46 mst but I think step zero is to make repo formats versioned+pluggable
18:15 nine ugexe: I think the biggest issue with the previous manifest was that it grew with each installed distribution and rakudo's JSON parser is dog slow
18:17 nine I guess once we provide proper uninstall support, people will be less prone to poke at repository's private files manually. And uninstall is mostly a decision on language versioning implementation away
18:17 nine Right now everything is blocked on that
18:17 mst maybe we should port JSON::Tiny
18:17 ugexe but each distribution wouldnt have to create a huge structure like it did previous, it only maps to where the huge structure is located for each dist. i dont know if that works or not
18:18 ugexe there was a new on hackernews a few months back about "the fastest json parser" or some such, where it would skip parsing over levels of the data structure that weren't needed. so if you wanted a key that started with 'F' and it reached a key that started with 'A' it somehow parses only whats needed to find the next key. a specialized manifest parser (or a format that lends itself to it) could
18:18 ugexe sidestep slowdown when 100s of modules are installed
18:21 nine There's another reason why we would want to avoid a large json file like that: it sucks for packaging modules. With the current structure, installation can really mean just dropping some files and fireing off precompilation, except for the short name lookup files. But those could be replaced by directories.
18:22 mst I think each dist should have some sort of JSON meta file, plus a tree of installed files
18:22 mst then whatever extra indices and dependents like short name and precomp can be a derived product of that
18:23 mst things you have to regenerate are best kept pure functional
18:23 nine I think I'm gonna implement uninstall even before we merge .resolve. Just to make sure the API fits nicely together
18:23 mst if you see where I'm handwaving
18:23 ugexe i cant speak for packaing modules, but size wise 1000 modules with a key-pair mapping module name to the actual meta file would be smaller than the previous manifest with like 50 modules installed
18:23 mst right, a single-file global json manifest is not something we want to be loading at runtime, ever
18:23 nine With a little adjustment to the short name lookup files, we would only have to parse the JSON data of the exact dist that's going to be loaded
18:23 mst a directory of file-per-dist json should provide the same features and be lots less painful
18:24 ugexe a manifest manifest
18:24 mst nine: right, which is why I'm thinking about repo format versions
18:24 mst maybe I should write down a textual defintioin of what ::Installation currently has on-disk-format-wise
18:24 mst and then we should ask, separate from the code, what's wrong with that
18:25 nine I think someone created a github repo for documents like that...
18:25 mst :D
18:28 nine Add version, auth and api to the short/ files and you can pick the matching dist right there. Of course this begs the question of what exactly is allowed in each of those fields. Would we allow white space? Newlines?
18:29 mst since the short/ files aren't meant to be consumed by humans
18:29 mst we could use null separated values
18:31 nine Oooh...true
18:31 nine You know when you've done Perl too long when you try to think about problematic characters and the worst you come up with is \n
18:43 nine Why is it always that literally when writing the third line of a supposedly straight forward job, you find out what's actually quite hard about it?
18:44 mst hey, this is a toolchain channel. finding out before you've finished and deployed it is always a win.
18:45 nine Of course, I could make my life much simpler by lowering requirements. If the user wants to uninstall a module that's depended on by others, who am I to deny it?
18:46 nine Because finding out the answer to "is this thing still needed" is surprisingly difficult
18:46 mst write a low level one that just uninstalls
18:46 mst we can add a more intelligent one later
18:46 mst intentionally stupid often works well at this level - see things like cpanm, fatpacker, etc.
18:54 nine Also we'd want to split up these two tasks into different methods anyway.
18:54 mst right, so at least you can shore off the crazy shit to the outer method :)
18:55 mst also, consider that you can't fully find 'depended on' anyway
18:55 mst since e.g. there could be a CUR that stacks on top of this one that has something that depends on it
18:56 mst (I've seen 'breaking one local::lib stack updating things for another' happen, sadly)
18:59 mst maybe just provide a things_depend_on_this() method for the CUR and have uninstall document it exists and people can call it if they want and then die/warn as they prefer
18:59 * mst sees our job wrt CUR as primarily to provide mechanism and let external tools use that to implement policy, roughly
19:00 mst &
19:00 patrickz joined #perl6-toolchain
19:21 lizmat "  <mst>and since nobody talked to the perl5 toolchain team first our answers have mostly been "they did WHAT?! what the flying fornication?!""
19:21 lizmat sorry mst, but that is *not* true
19:21 lizmat we already had a meeting about this at the QA hackathon in Lancaster
19:21 lizmat and we talked about it at the QA Hackathon in Berlin
19:22 lizmat sorry, I don't think P6 devs can be held responsible for P5 devs not taking P6 seriously
19:22 lizmat it was and is a deeply felt sentiment by Andreas Koenig that there should be *ONE* CPAN
19:23 lizmat and personally, I'm not ready to abandon that
19:24 lizmat .tell mst please check my monologue at http://irclog.perlgeek.de/perl6-toolchain/2016-01-09#i_11859804
19:26 TimToady that's a fine sentiment, but we also designed Perl 6 to not care where its modules come from
19:26 autarch I'm not sure the benefits of one CPAN is really worth the risk
19:26 autarch it's nice to reuse architecture and all that but any breaking critical p5 stuff that currently works seems like a real risk
19:27 autarch and while I'd like to see p6 succeed, I need p5 CPAN to keep working for my $day-job in the forseeable future
19:27 Zoffix lizmat, the .tell bot isn't here
19:27 TimToady .oO("Do not put your new wine into old wineskins, or the wineskins might burst, and the wine will be spilled out and ruined.")
19:27 TimToady we can use CPAN if it won't burst :)
19:28 lizmat TimToady: I wrote part of the design where modules could live just about anywhere, so I'm aware of that
19:28 Topic for #perl6-toolchain is now Fire is step THREE! | https://github.com/perl6/toolchain-bikeshed | Channel logs: http://irclog.perlgeek.de/perl6-toolchain/today
19:29 lizmat that being said, I do think that at least PAUSE should be able to accept P6 distributions
19:29 lizmat and afaik, it already can
19:29 lizmat *without* interfering with PAUSE itself
19:29 Zoffix right
19:30 lizmat now, that it may break stuff out there: sure, this can and will happen
19:30 lizmat but afaik, the fix is easy: ignore anything that is stored in a Perl6 subdirectory from the top of a home dir of a PAUSE user
19:31 lizmat it's even easy enough to be part of an rsync filter
19:31 lizmat so, if anything, this is an example of bad communication between PAUSE / CPAN and its P5 user base
19:32 lizmat *not* of P6 devs just blatantly trying to destroy the P5 CPAN infrastructure
19:32 lizmat sorry, but I'm quite livid at mst at the moment
19:32 * lizmat goes away fro a bit to cool off
19:51 llfourn joined #perl6-toolchain
19:59 patrickz joined #perl6-toolchain
20:15 hankache joined #perl6-toolchain
20:16 patrickz joined #perl6-toolchain
20:52 llfourn joined #perl6-toolchain
21:23 mohij joined #perl6-toolchain
21:33 llfourn joined #perl6-toolchain
23:05 patrickz joined #perl6-toolchain
23:11 mohij joined #perl6-toolchain
23:16 patrickz joined #perl6-toolchain
23:30 mohij joined #perl6-toolchain

| Channels | #perl6-toolchain index | Today | | Search | Google Search | Plain-Text | summary