Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2017-05-20

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 druonysus joined #salt
00:00 raspado joined #salt
00:06 zulutango joined #salt
00:13 sh123124213 joined #salt
00:20 manji joined #salt
00:28 VR-Jack2-H joined #salt
01:01 Trauma joined #salt
01:05 edrocks joined #salt
01:18 gilide joined #salt
01:18 nejage joined #salt
01:18 br3 joined #salt
01:18 cligr joined #salt
01:37 manji joined #salt
01:39 kline left #salt
01:48 ilbot3 joined #salt
01:48 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2016.3.6, 2016.11.5 <+> Support: https://www.saltstack.com/support/ <+> SaltStack Webinar on Carbon, Nitrogen, and Enterprise 5.1 on May 18, 2017 https://goo.gl/PvsOvQ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ <+> See also: #salt-devel, #salt-offtopic <+> Due to spam, please register with NickServ
01:50 leo_ joined #salt
02:01 onlyanegg joined #salt
02:04 manji joined #salt
02:21 cateyelaser joined #salt
02:25 cateyelaser joined #salt
02:28 onlyanegg joined #salt
02:47 schemanic joined #salt
02:48 schemanic hello
02:55 manji joined #salt
02:56 schemanic I have an issue with salt-formula on amazon linux. Please see pygit2.sls here: https://github.com/saltstack-formulas/salt-formula/blob/master/salt/gitfs/pygit2.sls. On amazon linux, the pygit-deps state requires an alternate set of packages. I could simply overwrite these files in my own clone, however I would like to do this in the 'proper' way. Can someone take a glance at the repo and advise me as to the best way to conditionally a
02:56 schemanic pply the right set of packages when on amazon linux?
03:02 fracklen joined #salt
03:06 edrocks joined #salt
03:16 schemanic can anyone tell me why salt can see some pip packages while it cannot see others?
03:22 manji joined #salt
03:32 miruoy joined #salt
03:37 onlyanegg joined #salt
03:40 schemanic Is anyone up?
03:43 manji joined #salt
03:50 manji joined #salt
03:54 jeddi joined #salt
03:55 onlyanegg joined #salt
04:02 onlyanegg joined #salt
04:03 cateyelaser joined #salt
04:06 zerocoolback joined #salt
04:12 _JZ_ joined #salt
04:17 manji joined #salt
04:18 mavhq joined #salt
04:23 bfoxwell joined #salt
04:31 MTecknology what's amazon linux?
04:33 Eugene RPM-based, started as CentOS.
04:33 Eugene It's mostly fine?
04:36 MTecknology I'm planning on switching to an internal unbound server managed by salt just because I want salt to start managing DNS
04:36 MTecknology right now, I use the unbound service included in pfsense
04:38 schemanic Good evening
04:38 schemanic Eugene, yeah, I'm a bit miffed with it
04:39 schemanic AMZ works like redhat, except there are a bunch of conventions and shortcuts it uses
04:39 schemanic My company is all in on AMZ right now
04:39 MTecknology I'm sorry
04:39 schemanic MTecknology, AMZ linux is a distro that Amazon gives you to work with in their cloud
04:41 MTecknology I'm 100% that 100% of companies that have 100% of their servers running on cloud services are running in a 100% criminally inefficient capacity.
04:41 schemanic Installing pygit2 with saltstack on AMZ has a bunch of idiosyncracies I don't like. AMZ has a pygit package in thier yum repo, except that saltstack cant see it, so you have to compile it from source
04:41 MTecknology but... not relevant, just complaining
04:42 schemanic MTecknology, I would not disagree with you on that. I am a former JS developer working in customizations who stepped up because we had no SysOps, just devs solving problems with band aids and duct tape
04:42 MTecknology schemanic: I honestly want to have an answer for you, but I don't really understand what you're asking
04:42 schemanic MTecknology, are you referring to my earlier posts?
04:43 schemanic I think I've solved the issue. I'm going to make a situation-specific state and use that
04:43 MTecknology I also have no idea what situation specific means
04:45 schemanic "The situation where I have to put a salt-master on AMZ linux" I'm going to copy pygit2.sls in the salt-formula repo and add AMZ specific changes to it so it works.
04:45 MTecknology I've seen masters on AWS
04:45 schemanic I'm sorry, i've been working late and misspoke
04:45 MTecknology don't don't pretend like they have more resources than they do and it can work
04:46 schemanic I mean getting pygit2 working on AWS
04:46 schemanic the bootstrap script works just fine to get a functioning salt-master/salt-minion
04:46 MTecknology you mean you're struggling with pygit2 on the amazon os?
04:46 schemanic but pygit2 on AWS is unpleasant
04:46 schemanic yes
04:46 manji joined #salt
04:47 MTecknology I don't have jack crap of a clue there
04:47 MTecknology others might, but I don't
04:48 schemanic I have solved the issue more or less - the salt-formula assumes a list of packages that don't exist in yum/AWS's repos, it tries to use an old version of libgit2, and it fails to present libgit2 to python in such a way that saltstack knows it has pygit2 installed, even when pip has installed pygit2 successfully
04:48 schemanic AWS has a yum installable package, but python can't see it, and it's 5 versions behind
04:49 schemanic So now my issues are less "What is my problem?" and more "How to fold my solutions into the repo so they might be useful to other people"
04:49 schemanic because I am a former JS developer who is teaching himself everything
04:50 schemanic I know python now and I'm fluent enough in YAML that I can read what Jinja is doing, though I haven't written much yet
04:50 schemanic So It's more that I'm not aware of the conventions at work in most salt formulas
04:51 schemanic ergo my question was more to the effect of 'how do I put my discoveries back in the repo and not do it like a n00b'
04:56 onlyanegg joined #salt
04:57 evle joined #salt
05:08 edrocks joined #salt
05:10 manji joined #salt
05:19 druonysus joined #salt
05:24 svij1 joined #salt
05:26 cyteen joined #salt
05:27 schemanic joined #salt
05:28 gmoro_ joined #salt
05:31 MTecknology schemanic: most people try to avoid formulas because they tend to introduce a fragmented design
05:36 whytewolf a lot of us avoid them because they are mostly designed by a chimp at a typewriter
05:37 sh123124213 joined #salt
05:39 swa_work joined #salt
05:43 cyborg-one joined #salt
05:59 schemanic joined #salt
06:05 sh123124213 joined #salt
06:08 sh123124213 joined #salt
06:12 viccuad joined #salt
06:18 inad922 joined #salt
06:40 zerocoolback joined #salt
06:42 Antiarc joined #salt
06:57 onlyanegg joined #salt
07:23 xet7 joined #salt
07:26 viccuad joined #salt
07:31 preludedrew joined #salt
07:39 zerocoolback joined #salt
07:58 onlyanegg joined #salt
08:04 oida_ joined #salt
08:13 mikecmpbll joined #salt
08:26 sh123124213 joined #salt
08:33 viccuad ok, why in this example, the execution of the states go "users from salt-formula > dotfiles > workstation", when it should go "users from salt-formula > workstation > dotfiles"?
08:33 viccuad https://github.com/viccuad/salt-bug
08:33 viccuad seems like a bug
08:37 viccuad or at least "workstation -> users -> dotfiles"…
08:38 whytewolf viccuad: as the node on the documentation states for include/exclude
08:39 whytewolf NOTE: include does not simply inject the states where you place it in the SLS file. If you need to guarantee order of execution, consider using requisites.
08:46 viccuad whytewolf: I've tried that (see https://github.com/viccuad/salt-bug/commit/573d55c48c47c75ebc6ee5ad54db05770aba1286)
08:46 viccuad it keeps failing
08:47 viccuad maybe it is a bug and is falling back to lexicographic order?
08:47 whytewolf in what way does it fail
08:48 whytewolf and no, the reason includes are not just injection. they are rendered at about the same time as jinja so anything included will get filtered to the top.
08:48 viccuad it fails in the way that with the require, I suppose it should execute "user states from the formula > workstation > dotfiles"
08:49 whytewolf what version are you on?
08:49 viccuad 2016.11.2
08:49 whytewolf and no. the what you have it in that salt-bug it will be users -> dotfiles -> workstation.
08:50 whytewolf because the workstation includes users then dotfiles so first users is loaded then dotfiles then the rest of workstation
08:50 viccuad why? does the require only affect the "dotfiles_dummy_for_require" state then?
08:50 viccuad aaah
08:50 viccuad gosh
08:51 viccuad but if I tell that dotfiles requires workstation, I was expecting that it would do workstation first
08:51 viccuad but I suppose chicken and egg
08:52 whytewolf humm. Im kind of surprised you are not getting duplicate id's with that. dotfiles includes workstation and workstation includes dotfiles
08:53 viccuad I include worsktation in dotfiles so I can do the require: workstation
08:54 viccuad but then, there's no way to specify ordering of states for .sls that only have includes in them
08:54 viccuad and the includes kind of breaks wysiwyg when reading .sls?
08:56 cyteen_ joined #salt
08:58 whytewolf so, does dotfile require workstation, or does workstation require dotfiles?
09:00 viccuad I want dotfiles to require workstation (so repos are set up to install the needed packages in dotfiles)
09:01 whytewolf okay, then a. why are you calling workstation in your topfile and not dotfiles? b. why even bother with the include dotfiles in the workstation/init.sls
09:08 cyborg-one joined #salt
09:09 yomateod joined #salt
09:14 viccuad whytewolf: a: because if I don't do it like that, can call doftiles, then the pillar/top.sls doesn't match the role:workstation, and the users pillar is not filled, so nothing happens in the user state formulas, even if they are included
09:14 viccuad s/can/and
09:15 viccuad the b: is a consequence of a:
09:15 whytewolf state loading has 0 relationship with out pillars are loaded
09:16 whytewolf pillars are setup before the highstate runs
09:19 viccuad I implemented your solution in the call-dotfiles branch, and it keeps not working: https://github.com/viccuad/salt-bug/commit/fa5481aadf5b54a93445d5bd2f42ecbbb59eda97
09:20 Praematura joined #salt
09:20 whytewolf ... why did you change the role?
09:20 viccuad (basically, the users pillars from workstation are not being taken, since we are not calling workstation but dotfiles)
09:20 viccuad in the id?
09:20 whytewolf in your top file.
09:20 whytewolf role is still workstation
09:20 whytewolf your just calling dotfiles state file
09:21 whytewolf also. pillar.get doesn't work in pillar
09:21 viccuad I'm calling the dotfile state by using the id
09:21 candyman88 joined #salt
09:22 viccuad role:workstation in the top.sls is just there, I just added the role:dotfiles
09:23 viccuad pillar.get doesn't work in pillar? it works for me perfectly in the master branch
09:23 viccuad and other repos besides mine, I've seen in the wild
09:25 whytewolf pillar.get has never worked in pillar. the only time it has worked is if you have an ext_pillar and ext_pillar running first
09:25 whytewolf as the pillars don't exist before you are calling them
09:25 whytewolf unless something has changed very recently
09:29 whytewolf humm. okay. i am wrong apperently pillar.get works but only in masterless setups
09:29 viccuad ah, makes sense
09:29 ChubYann joined #salt
09:29 viccuad I just rerun the master branch, it works
09:30 whytewolf https://github.com/saltstack/salt/issues/37893
09:32 viccuad my todo.org referencing salt bugs is growing fast.. :?
09:32 whytewolf anyway. not sure why your pillar data is not loading correctl when you change to roles:dotfiles. but that has nothing to do with your order of operations.
09:32 whytewolf as pillar has 0 to do with the order of operations of states
09:33 viccuad then it baffles me more
09:33 whytewolf unless the pillar.get is still returning workstation for the pillar.get in the pillar/top which would throw off your pillar since it now thinks it is dotfiles
09:34 viccuad may be
09:34 viccuad I'm getting the vibe that it's going to be impossible to have the pillars, and states only with includes that depend on other states
09:35 whytewolf it isn't impossable
09:35 viccuad is there a pattern/example that I can build on top of, somewhere?
09:36 whytewolf probley not. as it isn't a common pattern.
09:37 whytewolf most people don't try forcing backwords logic.
09:37 viccuad too complex?
09:37 whytewolf you want to have an entry point act like it isn't the entry point
09:37 viccuad I'd call it modular :)
09:37 whytewolf it isn't module
09:37 whytewolf modular
09:37 whytewolf it actually is quiet fragile
09:38 viccuad I would have a lot of profiles, built in a pyramidal way, ala hannoy towers
09:38 viccuad and then I point to one profile, and it gets used, plus the profiles that are below it
09:39 whytewolf honestly. try fixing your pillar issue first. as that might lead to the loading dotfiles first working.
09:39 viccuad my problem is that I don't see the issue in the pillar, from the docs, I should be able to do that
09:39 whytewolf point to a doc that says pillar.get works in pillar
09:40 whytewolf i've seen issues that say it is kinda working in masterless. but not really
09:41 whytewolf and i know it doesn't work at all in normal master mode
09:58 onlyanegg joined #salt
10:06 aaditya joined #salt
10:42 LostSoul joined #salt
10:52 Rumbles joined #salt
10:58 sh123124213 joined #salt
11:01 Trauma joined #salt
11:17 mrueg joined #salt
11:26 Praematura joined #salt
11:59 onlyanegg joined #salt
12:06 zerocoolback joined #salt
12:28 preludedrew joined #salt
12:29 nebuchadnezzar joined #salt
12:34 evle1 joined #salt
12:40 _KaszpiR_ joined #salt
12:49 aldevar joined #salt
12:49 Terminus joined #salt
12:53 o1e9 joined #salt
13:08 nebuchadnezzar joined #salt
13:28 Praematura joined #salt
13:31 relidy joined #salt
13:41 zerocoolback joined #salt
13:53 Inveracity joined #salt
13:54 zerocoolback joined #salt
14:00 onlyanegg joined #salt
14:23 viccuad joined #salt
14:28 o1e9 joined #salt
14:28 johnkeates joined #salt
14:30 dspec12 joined #salt
14:34 dspector joined #salt
14:36 dspector joined #salt
14:40 jalaziz joined #salt
14:43 Xenophon1 joined #salt
14:48 dspec12 joined #salt
14:57 davromaniak joined #salt
15:10 Felgar joined #salt
15:16 edrocks joined #salt
15:21 jalaziz joined #salt
15:24 o1e9 joined #salt
15:39 cyborg-one joined #salt
15:43 swills joined #salt
15:53 o1e9 joined #salt
16:06 heyimawesome joined #salt
16:11 o1e9 joined #salt
16:31 Praematura joined #salt
16:32 jalaziz joined #salt
16:38 iggy even times where it does work, it's fragile and not guaranteed... so don't
16:51 fracklen joined #salt
16:54 fracklen joined #salt
16:57 sh123124213 joined #salt
17:00 _KaszpiR_ joined #salt
17:04 fracklen joined #salt
17:18 edrocks joined #salt
17:18 ajv left #salt
17:29 viccuad well then, instead of using pillars, I then need to use grains. And then the minion can change its own grain and apply its own role
17:30 viccuad apart that, is it possible to read the `id` grain and use that to apply the wanted states? because I keep failing on that
17:30 viccuad plus, with init.sls only using includes, I see no way to make a require work for more that I try
17:30 iggy the normal top file match is a glob match of the minion id
17:32 viccuad yes, but I want to be able to have whatever hostname (and therefore id) on the minion, and apply whatever states from the hostname
17:32 viccuad I don't want names as db*, app* etc
17:32 iggy I guess I'm not understanding what you're doing
17:32 viccuad I have achieved it here: https://github.com/viccuad/salt-configs/tree/master/pillar/id
17:33 iggy if you're saying you want the id to be different than the hostname, you're asking for trouble
17:33 viccuad no, I want to read the id, and from that id derive a list of states to be applied
17:33 viccuad from a file that I can change
17:33 viccuad not hardcoding the states depending on the id
17:34 viccuad I have achieved that: https://github.com/viccuad/salt-configs/blob/master/pillar/id/aworkstation.sls
17:34 Trauma joined #salt
17:34 iggy I think most people go from file based using IDs to ext_pillar connected to cmdb/etc
17:35 iggy not trying to make file based fit for something huge that it's not suited for
17:36 viccuad fine. I can scratch it and put the grain roles in etc/salt/minion, per minion
17:37 viccuad but then the next stop is, I'm using states with init.sls, and that init.sls only contain "include: rest of substates"
17:38 viccuad and then one can not add a "require: " anywhere, so if you put a machine with role B, it would apply its dependency, role A, giving you A+B
17:38 fracklen joined #salt
17:40 iggy you can use require, but realistically, you're better off trying to use natural ordering rather than using requisites for ordering
17:40 iggy I try to just use require (et.al) to define dependencies rather than set ordering
17:40 viccuad by natural order you mean the salt/top.sls?
17:41 iggy the ordering changes that requisites induce are not at all natural
17:41 armyriad joined #salt
17:41 iggy I mean top-down
17:41 viccuad top down, as written in the top.sls?
17:41 iggy as written in any file
17:42 viccuad but then I need to specify the list of all roles that I want
17:43 viccuad eg, in https://i.imgur.com/mliR4vz.png, I only want to specify B, and that would apply B+D+E
17:43 viccuad or I specify role G and it would apply G+H+I+J
17:45 iggy then B should include D,E
17:45 viccuad it's not enough since includes don't guarantee the order
17:46 viccuad and it fails
17:46 iggy they will operate top-down (unless you've done other things to change the order)
17:46 viccuad really, it doesn't work. eg: https://github.com/viccuad/salt-bug/blob/master/salt/profile/dotfiles/init.sls
17:47 viccuad that doesn't work
17:47 iggy yes, because your require completely messes up the ordering
17:48 ssplatt joined #salt
17:49 iggy your include in https://github.com/viccuad/salt-bug/blob/master/salt/profile/workstation/init.sls won't work like you think it will
17:50 fracklen joined #salt
17:50 viccuad I'm aware that it goes to the top of the file
17:50 viccuad let me clean it up
17:51 Praematura joined #salt
17:51 iggy p.s. pkg.installed will run apt-get update unless you tell it not to, so running it manually, isn't required
17:53 iggy and you should use pkgs instead of names as names will cause it to invoke apt-get for each package (vs one call with multiple packages names)
17:53 viccuad iggy: where am I doing that? IIRC I'm only doing apt-get update after setting the new file with the repos
17:54 iggy yeah, you setup sources.list, run apt-get update, then run pkg.installed (that will run apt-get update itself)
17:56 viccuad I suppose I don't trust myself into remembering it, and taking that pkg.installed out and then not having any apt-get update run
17:56 viccuad autocontained states
18:00 iggy fair enough... but when you get to the point where you're tired of waiting for 80 apt-get updates to run, you can take that out
18:02 iggy you're still doing pillar.get in your pillar top file
18:07 viccuad ok, not in this branch: https://github.com/viccuad/salt-bug/tree/with-grains-and-includes
18:10 viccuad that branch has all grains to set up the roles, and dotfiles has an include for workstation, but workstation gets run yet the include for the users salt formula doesn't, so dotfiles has nothing to work on..
18:10 viccuad why? because the users salt formula is missing the info of the users pillars, which only kicks in with the workstation role
18:12 viccuad so there's no way to make a role depend on another role, and I need to know the role dependencies by heart and specify it on /etc/salt/minion
18:15 iggy yes, you will have to assign pillar data accordingly... it's really not a good idea to have pillar data dynamically assignable by other pillar data as the potential for leakage is high
18:15 viccuad the leakage is already full, I'm using grains to set up the roles
18:15 viccuad because I cannot use pillar.get
18:16 lompik joined #salt
18:16 iggy this is why most people ditch pillar_roots based pillars pretty early on in complex deployments
18:17 viccuad what should I use then? do you have any place where I can look at? :)
18:17 iggy well, I'd start by looking at the ext_pillar modules that exist already
18:18 iggy where I work now, we wrote our own ext_pillar to talk to our cmdb
18:18 iggy but you can do a lot with what's already there
18:19 iggy in particular, things like the stack and varstack pillars were written by users to scratch the itch between simple pillar_roots and going all out on database backed pillars
18:20 lompik joined #salt
18:24 viccuad I was hoping that calling the 2 pillars that I needed, at pillar/top.sls, would suffice
18:24 viccuad even if it isn't DRY and you have info about the dependency in salt/top.sls and pillar/top.sls
18:26 manji joined #salt
18:28 ssplatt joined #salt
18:28 iggy well, you can't use pillar.get in the pillar top file, so I'd start looking for other ways
18:30 viccuad pillarstack it is, then
18:30 viccuad thanks :)
18:33 fracklen joined #salt
18:46 viccuad mmh, since I'm using it masterless, I wonder if there's ext_pillar that work through gitfs
18:53 iggy if you are using it masterless, why are you worried about people being able to change the grains to access the pillar data?
18:58 manji joined #salt
19:01 viccuad I'm not now, but that doesn't solve the problem of pillar.get on top.sls
19:02 viccuad ah, why an ext_pillar for gitfs?
19:02 viccuad because that way I can push to github the confs but the pillar resides somewhere
19:03 lompik joined #salt
19:18 fracklen joined #salt
19:22 iggy the entire pillar will still get pulled down to each masterless minion... so anyone with rights on a box can see all your pillar data
19:22 iggy I mean it sounds like you understand that, just being clear
19:25 LostSoul joined #salt
19:34 viccuad I'm aware of that
19:34 viccuad can one do pillar.get in salt/top.sls?
19:41 aldevar joined #salt
19:47 zerocoolback joined #salt
19:50 viq As in can you target based on pillar contents? Yes, you can.
19:51 viq http://seedickcode.com/devops/saltstack/saltstack-a-better-salt-top-sls-part-2/  has some examples
19:52 viq Or https://pbot.rmdir.de/ldtt6P2sisvAKg6_tJNL8g
20:09 viccuad well, that did the trick. I added a pillar dictionary that contains all the states, and I include the needed pillars so all states have their pillar info to be able to run
20:09 viccuad and with that I can express the state dependencies that I wanted, for now
20:10 viccuad I suppose that if I get into pillar complexities and I need to merge them I would need to look at ext_pillar
20:10 viccuad iggy, viq, whytewolf: many thanks! :) \o/
20:14 XenophonF joined #salt
20:19 puzzlingWeirdo joined #salt
20:21 edrocks joined #salt
20:21 Yamazaki-kun joined #salt
20:22 Yamazaki-kun joined #salt
20:30 manji joined #salt
20:30 XenophonF joined #salt
20:32 snoopinator joined #salt
20:42 mavhq joined #salt
20:45 XenophonF joined #salt
20:51 rwaweber joined #salt
20:56 manji joined #salt
21:02 aldevar left #salt
21:05 swills joined #salt
21:12 mavhq joined #salt
21:21 fracklen joined #salt
21:23 Potty_ joined #salt
21:23 Potty_ left #salt
21:33 fracklen joined #salt
21:33 mavhq joined #salt
21:47 cyborg-one joined #salt
21:50 manji joined #salt
21:56 johnkeates joined #salt
22:01 swills joined #salt
22:09 jalaziz joined #salt
22:11 manji joined #salt
22:17 AndroUser2 joined #salt
22:23 edrocks joined #salt
22:37 manji joined #salt
22:52 druonysus joined #salt
23:00 manji joined #salt
23:10 manji joined #salt
23:21 XenophonF joined #salt
23:22 t0m0 joined #salt
23:34 AndroUser2 joined #salt
23:42 rav_ joined #salt
23:55 AndroUser2 joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary