Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2018-04-23

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 onslack joined #salt
00:04 orichards joined #salt
00:25 MTecknology jfindlay: I'd also pass that through the json filter to avoid rendering headaches
00:39 ws2k3 joined #salt
00:39 ws2k3 joined #salt
00:40 ws2k3 joined #salt
01:08 JacobsLadd3r joined #salt
01:08 Hybrid joined #salt
01:57 ilbot3 joined #salt
01:57 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2017.7.5, 2018.3.0 <+> Support: https://www.saltstack.com/support/ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ <+> See also: #salt-devel, #salt-offtopic, and https://saltstackcommunity.herokuapp.com (for slack) <+> We are volunteers and may not have immediate answers
02:01 toddejohnson joined #salt
02:06 shiranaihito joined #salt
02:06 masuberu joined #salt
02:10 orichards joined #salt
02:15 DammitJim joined #salt
02:18 orichards joined #salt
02:30 nixjdm joined #salt
02:43 orichards joined #salt
02:49 JPT joined #salt
03:08 masuberu joined #salt
03:13 orichards joined #salt
03:15 masber joined #salt
03:48 dxiri joined #salt
04:04 justanotheruser joined #salt
04:14 sauvin_ joined #salt
05:13 dxiri joined #salt
05:48 wongster80 joined #salt
06:08 darioleidi joined #salt
06:16 Hybrid joined #salt
06:44 awerner joined #salt
06:47 senfgurke joined #salt
06:50 orichards joined #salt
06:51 evle1 joined #salt
06:55 Bochi_ joined #salt
06:57 Elsmorian joined #salt
07:02 Yamakaja joined #salt
07:04 Pjusur joined #salt
07:05 Tucky joined #salt
07:12 lompik joined #salt
07:12 oyvindmo joined #salt
07:27 orichards joined #salt
07:37 briner joined #salt
07:38 jrenner joined #salt
07:43 Ricardo1000 joined #salt
07:43 toanju joined #salt
07:44 Elsmorian joined #salt
07:48 Elsmorian joined #salt
07:51 orichards joined #salt
07:54 DanyC joined #salt
07:57 stooj joined #salt
07:58 Naresh joined #salt
07:59 Elsmorian joined #salt
08:04 Yamakaja joined #salt
08:06 Hybrid joined #salt
08:09 cbosdonnat joined #salt
08:09 cbosdonnat joined #salt
08:12 rollniak joined #salt
08:18 bdrung_work joined #salt
08:30 Mattch joined #salt
08:51 cbosdonnat is it me or config.get doesn't read the config.DEFAULTS values?
08:53 Elsmorian joined #salt
08:56 briner joined #salt
09:14 Barbarossa left #salt
09:24 eMBee which version of salt can i use that will support debian 7, 8 and 9? repo.saltstack.com has 2016.3.6 and 2016.11.5 as the newest, but debian 9 only goes down to 2016.3.7 and 2016.11.6. can i mix those versions?
09:27 AlexLau joined #salt
09:27 gmoro joined #salt
09:32 AvengerMoJo joined #salt
09:35 xet7 joined #salt
09:44 onslack <florian.benscheidt> eMBee: <http://repo.saltstack.com/apt/debian/9/amd64/>
09:53 IdoKaplan joined #salt
09:54 eMBee oh, why was i looking in archive???
09:54 IdoKaplan left #salt
09:57 eMBee it doesn't change the problem. archive contains all the versions and there is no single version that is available accross all 3 debian releases
09:57 aanriot joined #salt
10:06 eMBee however, when i tried to install stable 2016.3.7 with bootstrap.sh it picked 2016.11.2 from the debian distribution repo instead of 2016.3.7 from the saltstack repo. i guess that will work :-)
10:20 babilen eMBee: You don't have to run the same minion version on all minions
10:20 babilen I mean you can easily run a newer version on 8 and 9, while using the latest available version for 7
10:21 babilen (but then .. you *really* should upgrade those wheezy boxes)
10:27 Elsmorian joined #salt
10:41 Elsmorian joined #salt
10:42 eMBee babilen: yes, well, i ran into the problem that the latest salt-minion does not work with our 2015.5.3 master. i know it's time to upgrade, but one thing at a time :-) need to get these machines running first, so i started looking for the salt version that will most likely work on all machines without issues. 2016.11.2 appears to be it. upgrading from debian 7 will happen soon, but then i won't need to worry about salt any more as i'll be a
10:44 pcgod joined #salt
10:44 babilen Yeah, your master has to be newer. Is your master on wheezy?
10:45 babilen And your message was cut off at "as I'll be ab"
10:47 babilen eMBee: ^
11:00 babilen Upgrading your master should be about 30 minutes of work tops, which allows you to run 2017.7 or 2018.3, while you can run all but your wheezy minions on a matching version
11:01 babilen That *far* outweighs the problems/bugs/.../lack of support you get with 2016.11 on your master, IMHO
11:01 chowmein__ joined #salt
11:01 babilen But then .. who knows what "as I'll be ab" means ;)
11:04 Kelsar joined #salt
11:05 briner joined #salt
11:06 Waples_ joined #salt
11:11 Elsmorian joined #salt
11:13 sjohnsen joined #salt
11:16 hoonetorg joined #salt
11:34 Elsmorian joined #salt
11:38 hoonetorg joined #salt
11:40 Kelsar joined #salt
11:41 eMBee babilen: ooops, the last line of my message is: i'll be able to keep the version the same throughout the debian upgrades (after i upgrade salt everywhere to 2016.11.2)
11:44 eMBee babilen my master is on debian 7, yes. i have minions on 2014... i'll see about the order to upgrade. maybe bring up the older ones to 2015 first, then upgrade the master to 2016, and finally all clients. the machines on debian 9 with 2016.11.2 now seem to work ok for now.
11:45 eMBee babilen: so you are saying that even very old minions will work fine with ther latest master (2018)?
11:45 Elsmorian joined #salt
11:55 babilen eMBee: I'd forego 2016.* on the master ..
11:55 babilen eMBee: Define "very old"
11:59 babilen I have one setup with a master on 2018.3.0 and most minions on either 2018.3 or 2017.7, but two squeeze minions on 2014.7 and they work "fine"
11:59 babilen Obviously things like "state.apply" don't work in lieu of "state.highstate" on those squeeze minions, but the rest is fine
12:00 rollniak joined #salt
12:00 Elsmorian joined #salt
12:03 babilen eMBee: Upgrading your master should be quite easy (I mean in the end it boils down to a couple of sed s/{wheezy,jessie}/{jessie,stretch} and apt{,-get} update ; apt{,-get} upgrade ; apt{,-get} full-upgrade calls
12:05 babilen The only thing you should be aware of is that SaltStack kept messing up the KillMode= setting in the minion's unit file, which caused the minion to kill running dpkg processes when it upgraded itself. I deployed http://paste.debian.net/1021617/ on all minions and generally follow https://docs.saltstack.com/en/latest/faq.html#what-is-the-best-way-to-restart-a-salt-minion-daemon-using-salt-after-upgrade for
12:05 babilen upgrades
12:05 Elsmorian joined #salt
12:12 Nahual joined #salt
12:12 Elsmorian joined #salt
12:15 CrummyGummy joined #salt
12:21 stooj joined #salt
12:25 Elsmorian joined #salt
12:28 trudelu joined #salt
12:28 mavhq joined #salt
12:42 justinl_ joined #salt
12:46 zer0def babilen: i'm curious, have you tried a virtualenv with newer salt installed on those old systems?
12:52 XenophonF I wish one could tell Jinja to use something other than the Python pretty printer for `{{ blah }}` expressions.
12:52 XenophonF e.g., swap in a YAML serializer
12:56 zer0def doesn't salt provide a `json` and `yaml` filter?
12:57 XenophonF it does but a surprising number of formula writers don't know about or use them
12:57 AngryJohnnie joined #salt
12:58 XenophonF e.g., https://github.com/saltstack-formulas/tomcat-formula/blob/master/tomcat/config.sls#L16
12:58 XenophonF I wonder if Salt 2018.3 changed something to do with Unicode strings, because now all of a sudden I'm getting this error.
12:58 XenophonF https://github.com/saltstack-formulas/tomcat-formula/issues/89
12:59 babilen zer0def: I have not and use it as a "pain point" to get people to finally get rid of those ancient systems
12:59 XenophonF That formula worked with the previous version of Salt because, luckily, the Python pretty printer output and the YAML serializer extension output were the same.
12:59 crux-capacitor joined #salt
13:00 zer0def babilen: it's a good ache point to move them away from, i'm just curious whether that'd do the job
13:00 babilen I'm sure 2018.3 changed quite a bit in terms of unicode handling for py3 support
13:00 XenophonF that's what I'm thinking too
13:00 XenophonF am scanning the changelogs now
13:00 XenophonF u'...' is Unicode string syntax
13:00 zer0def yeah, those break clearly because of pythonic marking on py2 unicode strings
13:01 sh123124213 joined #salt
13:02 babilen XenophonF: yaml/yaml_encode should indeed be what you're looking for. Lazily written formula ..
13:02 zer0def oh i'm pretty sure i've been as lazy on a whole series of opportunities
13:05 XenophonF I'm working on patches for the parts of that formula that break for me.
13:05 babilen Happy to merge
13:06 DammitJim joined #salt
13:06 XenophonF I'd patch the whole thing but can't test the stuff I don't (know how to) use.
13:07 XenophonF yeah, lots of Unicode changes in the 2018.3.0 release notes, so that makes sense
13:08 crux-capacitor joined #salt
13:11 babilen It was essentially the jinja_filter, slots, get that fancy docker module from ansible, auto discovert .. and .. first and foremost .. get py3 support ready by addressing all those unicode/byte and whatnot strings everywhere
13:11 babilen *Finally* custom jinja filters :)
13:25 DammitJim joined #salt
13:27 gh34 joined #salt
13:28 evle1 joined #salt
13:29 jerematic joined #salt
13:29 XenophonF oh I missed that!
13:43 Waples_ I too am working out if I can hop over to 2018.3.* with Py3. Gonna be fun rewriting our patches
13:51 dxiri joined #salt
13:52 aanriot joined #salt
13:54 babilen Same here ..
13:56 cgiroua joined #salt
14:00 Hybrid joined #salt
14:00 racooper joined #salt
14:06 dendazen joined #salt
14:10 dendazen joined #salt
14:10 lompik joined #salt
14:11 cgiroua joined #salt
14:15 XenophonF PR for tomcat-formula submitted
14:22 jerematic joined #salt
14:26 AngryJohnnie joined #salt
14:27 VR-Jack2-H joined #salt
14:33 cwright joined #salt
14:34 beardedeagle joined #salt
14:34 aanriot joined #salt
14:40 beardedeagle joined #salt
14:49 VR-Jack-H joined #salt
14:50 babilen XenophonF: Looks fine
14:55 Elsmorian joined #salt
14:57 XenophonF thanks
14:59 toddejohnson joined #salt
15:02 DammitJim joined #salt
15:05 noobiedubie joined #salt
15:12 briner joined #salt
15:20 Bochi_ joined #salt
15:38 delya joined #salt
16:00 Elsmorian joined #salt
16:04 jerematic joined #salt
16:19 DanyC joined #salt
16:32 sauvin joined #salt
16:34 DanyC joined #salt
16:35 DanyC_ joined #salt
16:36 raceboyer_ joined #salt
16:39 nixjdm joined #salt
16:41 Hybrid joined #salt
16:42 jab416171 joined #salt
16:45 tiwula joined #salt
16:54 crux-capacitor when setting minion config via pillar, is there a place on the minion to see that it was applied?
16:55 ddg joined #salt
16:58 zer0def you can make sure it's there by syncing pillars via `saltutil.refresh_pillar` (i'm not sure whether `pillar.items` lists what minion is aware of or what master wants it to be aware of)
17:02 Elsmorian joined #salt
17:04 DanyC joined #salt
17:04 crux-capacitor so i did that, and did a pillar.get on the setting, and it returns the proper value. i just wanted to see if there's a place to see on the minion that this setting was properly applied as a config setting, and not just pillar data
17:12 whytewolf crux-capacitor: depend on the setting. a lot of settings are exactly the same. but really there is no list of commands that are able to be set by pillar. if a config can be set through pillar it will use config.get to fetch the setting.
17:15 crux-capacitor ok cool, that works
17:16 whytewolf config.get will show pretty much everything that is out there.
17:16 whytewolf it won't tell you if the setting is coming from __opts__ or config.get
17:21 JacobsLadd3r joined #salt
17:30 DanyC joined #salt
17:35 gmoro_ joined #salt
17:36 Elsmorian joined #salt
17:37 AngryJohnnie joined #salt
17:39 Hybrid joined #salt
17:45 onlyanegg joined #salt
17:49 ymasson joined #salt
17:52 ponyofdeath joined #salt
17:59 sjorge joined #salt
18:05 DammitJim1 joined #salt
18:10 Elsmorian Is there a way to blacklist the init grain?
18:11 DammitJim joined #salt
18:11 Elsmorian I'm running salt-master without an init system, just raw from a docker container, and my logs are full of repeated  Could not determine init system from command line: (/bin/bash /usr/local/bin/docker_entrypoint.sh run_saltmaster)
18:19 noobiedubie getting a UnicodeDecodeError: 'utf8' codec can't decode byte 0xae in position 10: invalid start byte when trying to file.manage a .jar file in s3 seems to be since upgrading to 2018.3.0
18:23 noobiedubie https://paste.debian.net/hidden/82bd49b0/
18:23 noobiedubie the debug log from the file state that is failing
18:24 noobiedubie any help would be greatly appreciated
18:32 cliluw joined #salt
18:34 wongster80 joined #salt
18:37 toanju joined #salt
18:40 noraatepernos joined #salt
18:41 noraatepernos git.latest with force_checkout and force_reset still fails when I have a file modified.  “commit your changes or stash them before you can merge” — this is a deployment.  I need to force.  Any trick to doing this?
18:42 noobiedubie thought it could have been a file error or corruption but has been ongoing with a few different compiled .jar file versions. jar file runs fine and doing a unzip works perfectly os file checks all come back ok.
18:42 noobiedubie noraatepernos: your looking for force_fetch
18:44 noobiedubie really you need force_clone, force_reset, and force_fetch to avoid similar other errors
18:45 brokensyntax joined #salt
18:45 noobiedubie if you want your git.latest to always pull in the latest version of w.e branch and ref you want
18:45 noobiedubie without worrying about these edge cases
18:45 AngryJohnnie joined #salt
18:49 noraatepernos noobiedubie: Is that in 2017.7.0? force_fetch didn’t fix it.
18:50 noraatepernos Hmm I think I need hard_reset
18:52 briner joined #salt
18:54 noobiedubie noraatepernos: as I said you really need all three
18:54 noobiedubie force_fetch, force_clone, and force_reset
18:55 DanyC joined #salt
18:57 noraatepernos_ joined #salt
18:58 noraatepernos Yeah literally no combination of the force_ options in git.latest will actually force anything.
18:58 noraatepernos This is so frustrating.  Should I not be using salt to deploy/
18:58 noraatepernos Pretty basic stuff here.
18:59 Elsmorian joined #salt
19:01 noraatepernos “Set to True if the repository is to be a bare clone of the remote repository” also what does “bare” mean?
19:02 lordcirth_work noraatepernos, https://www.git-scm.com/docs/git-clone#git-clone---bare
19:03 noraatepernos Hmm.  I need to ignore local changes to a file.
19:03 noraatepernos None of the force_ options work.  Is there another solution?
19:04 noraatepernos This is a deployment using salt.
19:04 noobiedubie noraatepernos: if your really stuck and for some reason nothing else works you can file.absent the directory first then your git.latest state
19:04 noraatepernos When npm installs, it generates a package-lock.json file.  I want this commited to the repo for developers.
19:04 noobiedubie noraatepernos: you can also add that file to your .gitignore
19:05 noraatepernos No I cannot.  I want it committed.
19:05 noraatepernos It is not for deployment.  I want to force a merg.e
19:05 noobiedubie ?
19:05 noobiedubie I think this is coming down to how your using git
19:06 schemanic joined #salt
19:07 noraatepernos Yeah so basically this thing called node package manager, when you “install” packages/deps it will generate this package-lock.json file.
19:07 noobiedubie your asking salt to deploy the lastest version of a repo but want it to take into account your changes to certain files but not others potentially, seems like a workflow issue more than a salt one
19:07 noraatepernos This is fine for developers.  It’s to be committed as a reference.  Unfortunately, npm.install is *also* part of the salt state.
19:07 schemanic Hi, I'm looking for advice from folks who've used Packer before. I'm interested in packing up AMIs with the salt-minion pre-installed on them, but as I'm researching it, It becomes unclear to me how to make sure that the minion id for pre-salted minions to get set.
19:08 noraatepernos Nope.  I’m an adult and I want it to overwrite any local changes on the salt minion.
19:08 noobiedubie but you just said you wanted the changes to that file merged?
19:08 noraatepernos About to go here and just ignore it finally.  https://github.com/npm/npm/issues/18007#issuecomment-340304283
19:09 noraatepernos Nope.  Tracked in the repo.  Ignored in deployment.  Simple.
19:09 noobiedubie clearly... so simple
19:09 noraatepernos When I fetch/merge in salt state’s git.latest I want an exact replica of the repo.
19:09 schemanic I still want to call my minions into existence from salt-cloud, which is how I understand a minion ID is created when using salt-bootstrap. If I have a Packer AMI, how am I telling the salt-minion on it what it's ID is at creation time?
19:09 noraatepernos It’s so simple.
19:10 noobiedubie noraatepernos: I mean your that one saying it's so simple yet apparently not so much when you try to do in the real world but sure
19:11 noobiedubie i mean if you want simple just pull from a branch that is curated to your liking
19:11 noobiedubie in relation to that file
19:11 marcinkuzminski joined #salt
19:12 noobiedubie track it in one branch and don't have it in your deployment branch
19:12 noobiedubie pull from deployment branch solved?
19:14 noraatepernos joined #salt
19:15 tin_ joined #salt
19:18 noobiedubie schemanic: that gets really tricky any reason why using the bootstrap script and cloud won't work? You could have salt-cloud push out the minion config in your cloud.profile therefore setting things like minion-id, master, grains...etc.
19:19 schemanic noobiedubie, I am already using salt cloud and the bootstrap script, however there's a bug with salt-bootstrap that causes the arguments for installing a specific minion version not work
19:19 schemanic Edgan says I should be baking an AMI with my minion in it already
19:21 noobiedubie seems like terrible advice but i guess
19:21 noobiedubie https://paste.debian.net/hidden/2f74a8d3/
19:22 noobiedubie is how we pass minion setting during cloud deploy
19:22 schemanic I'm already doing that
19:22 schemanic I'd be happy if the bootstrap script was correct for AWS linux
19:22 noobiedubie so were is the problem setting your minion-id?
19:23 noobiedubie seems to work for us but we always install latest so haven't tried the version choosing
19:24 schemanic I'm not currently having a problem setting the minion id
19:25 schemanic I'm saying, if I were to make a packer image and install salt minion
19:25 schemanic then commission that image from salt-cloud into an instance
19:25 schemanic I'd have a salt minion with a preset ID in it that was baked in
19:26 schemanic i'd need to tell each machine 'no, you're not johnny1, you're johnny 26'
19:26 noobiedubie but that could be overwritten using the syntax above in your cloud.profile for that instance
19:27 onlyanegg joined #salt
19:27 schemanic I'm not seeing where you set a minion id in that paste
19:27 noobiedubie something as simple as setting your id to the hostname of the machine in a jinja variable and passing that in as the id: {{ hostname }} in the cloud.profile minion section
19:27 schemanic I have a state that does it the other way around
19:27 schemanic my minions derive hostname from the minion id
19:28 noobiedubie i don't there but only because we have another way of naming our instances but you can pass any minion config options under the minion: directive in the config
19:28 schemanic the minion id must be specified to the minion
19:28 schemanic Yes but the minion id is not specified in the cloud profile
19:28 noobiedubie well that seems like an issue since you'll always get a hostname first
19:28 noobiedubie and since your hardcoding in your id before hand
19:28 schemanic its specified when you say `salt-cloud --profile myProfile MINIONID`
19:29 schemanic No I'm not hardcoding it
19:29 schemanic I'm stating the minion id when I run salt-cloud
19:30 noobiedubie ok then pass that in that name in the profile file as well?
19:30 noobiedubie and cloud will set that setting in your minion config
19:30 schemanic noobiedubie, you're then telling me to have N profiles where N= the number of minions I want. this is inefficient
19:30 schemanic I'm not having a johnny1, johnny2, johnny3 set of profiles
19:31 noobiedubie not necessarily again its how you set it up
19:31 schemanic What do you mean when you say that?
19:31 noobiedubie you can have one profile and has dynamics settings for things like name and such based on other factors if you wish
19:31 schemanic right, but I still need to pass it data upon creation
19:32 morgana2313 joined #salt
19:32 noobiedubie aren't you already doing that?
19:32 schemanic No
19:32 schemanic When I need a new server, I calculate what it's minion id will be using our naming convention
19:32 noobiedubie and that can't be automated or scripted?
19:33 schemanic then I say `salt-cloud --profile TestApp OTTSAPVM0005`
19:33 schemanic then the minion knows it's OTTSAPVM0005
19:33 noobiedubie right all this can be taken care of with a git hook or a script or a bunch of other fancy ways
19:34 schemanic I want to do it with something thats built in to salt if possible
19:34 schemanic It looks like the bootstrap script can do things that aren't just installing salt
19:35 noobiedubie sure plenty of options from runners, scheduled jobs, even the bootstrap script has an option to run a script pre and post launch
19:35 schemanic it might be as simple as using the bootstraps 'dont install salt' and 'set the minion id' flags
19:35 schemanic It just depends on those working
19:36 noobiedubie i mean you could also debug whats going wrong with the bootstrap versioning system then all this is mute right?
19:37 schemanic I don't have time at the moment.
19:38 DammitJim joined #salt
19:38 noobiedubie you could also try a salt event
19:39 noobiedubie there are cloud events that the master looks for that would run a state
19:41 rollniak joined #salt
19:43 Hybrid joined #salt
19:44 wongster80 joined #salt
19:45 crux-capacitor if I was to set up a syndic to act as the master for a particular group of systems, they would have to resolve 'salt' to the IP of the syndic, correct? or could minions be connected to a master and a syndic?
19:47 whytewolf crux-capacitor: you can tell a minion what master to connect to with the master option. salt is just the default.
19:48 crux-capacitor ah, right. so I could just set that via pillar based on whatever criteria I need?
19:48 whytewolf ... this is one that you can't set via pillar.
19:48 zer0def regarding noraatepernos' problem - `force_reset: true` worked just fine
19:49 zer0def sucks to be impulsive, though
19:49 noobiedubie lol
19:50 onlyanegg joined #salt
19:53 noraatepernos joined #salt
19:54 zer0def noobiedubie: regarding your problem, have you tried with `show_changes: false`?
19:55 zer0def that said, it's still a bug worth reporting, but if it mitigates your case, that at least would be progress
20:00 Hybrid joined #salt
20:02 hiroshi joined #salt
20:06 schemanic @noobiedubie, I'm going to work with trying to get the bootstrap script fixed via an issue on the github page
20:08 Edgan schemanic: noobiedubie: The problem is all it takes is an apt repo metadata issue, can't connect to the apt server that second, etc, and you instance is going to fail to install salt, and never come up in an automated way. There are just too many things that can go wrong in my experience. Where as if it is pre-installed, even if an older version, you can come up, and run salt to do whatever
20:09 Edgan schemanic: noobiedubie: Even installing it via packer can be unreliable, but at least you get that one to work once, and then it works for actual instance creation, especially with things like auto-scaling
20:09 schemanic Edgan, I agree with you that the method is sound. I'm just trying to figure out how to make it work properly with salt-cloud and be able to spin up a new host with it's own minion ID
20:10 schemanic Like, I get the idea of just taking a bare AMI, installing salt on it straight up without actually connecting it to the master, and running packer on it
20:10 Edgan schemanic: Are you using auto_accept: True, or trying to be secure?
20:11 schemanic My minions show up right away on the master when I call them into existence
20:11 Edgan schemanic: ok, so not secure
20:12 schemanic What are your suggestions
20:12 Edgan schemanic: So if the salt-minion is pre-installed and the /etc/salt/minion is pre-configured, then you shouldn't have to do anything with salt-cloud other than tell it the right pre-made AMI and disable bootstrapping
20:13 schemanic Edgan, but then doesn't that mean that new servers come into existence with the same value in /etc/salt/minion, and then you have 18 'johnny1's?
20:13 Edgan schemanic: The key to /etc/salt/minion is master: <hostname>, which defaults to master: salt. So if the instances can resolve salt, you don't even have to change /etc/salt/minion, but I don't like that method, personally.\
20:13 whytewolf schemanic: basicly aws when it creates and instances creates the host as the name of the instance. when the minion spins up with the unconfigured minion_id it will take the hostname as the minion_id
20:14 Edgan schemanic: Your instances by default have ip-10-2-2-5.aws.internal as their hostname
20:14 schemanic Yep I see that
20:14 Edgan schemanic: So unless you have some process setting then all to johnny1, you don' actually have a problem
20:14 DammitJim joined #salt
20:14 schemanic But I want to set their minion IDs at creation time
20:14 whytewolf schemanic: create a custom boot script
20:14 Edgan schemanic: ok, here is how I do that
20:14 Edgan whytewolf: there is a better way
20:15 schemanic you're saying that I can use salt-cloud to call the packer-ed AMI which has no minion ID, and specify the minion id via the bootstrap params?
20:15 schemanic cause that's kindof what I had hoped
20:15 Edgan schemanic: You will need salt-cloud to write some userdata on instance creation. then you run cloud-init, and it sets the hostname
20:16 Edgan schemanic: I don't like how cloud-init does it, so I wrote my own bash script that sets the hostname. But then cloud-init still calls it for me.
20:16 Edgan schemanic: Let me give you an example
20:16 schemanic I would appreciate that
20:17 Edgan schemanic:  https://pastebin.com/nRwFBbiZ
20:18 Edgan schemanic: The first part of the cloud-init, hostname and fqdn are the cloud-init way of doing hostname setting.
20:18 Edgan schemanic: It has a module to do it
20:18 Edgan schemanic: The set-hostname in the runcmd section is my way of setting the hostname
20:19 Edgan schemanic: I also disable the auto start of the salt-minion in the AMI, and then enable it again with cloud-init to avoid race conditions
20:19 schemanic you say 'cloud-init' as if it is a known term. Please explain
20:19 schemanic /define
20:19 Edgan schemanic: https://pastebin.com/UhKvSZRe
20:20 Edgan schemanic: Very common open source piece of software that is in the default ubuntu ami that is meant to do stuff like this
20:20 schemanic We use Amazon Linux
20:20 Edgan schemanic: probably there too
20:20 whytewolf it is. it is in pretty much every cloud image
20:21 Edgan https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
20:21 whytewolf and yes if you are working in the cloud you should know what cloud-init is
20:21 Edgan nod
20:22 Edgan schemanic: Then if you want to get fancy you can have a process call the salt-api, have it generate a salt key, feed it to the salt-minion via cloud-init
20:23 Edgan schemanic: Then you can set auto_accept: false, and your process will actually be secure
20:23 Edgan schemanic: Where as anything in your VPC that can talk to the salt-master can fake it's hostname and pull any secrets it wants from the master pillars
20:24 Edgan schemanic: Which defeats the whole point of not just doing a '*' - everything in the pillar top.sls
20:25 Edgan schemanic: Then if you want to get even fancier, you can do jinja matching in the pillar top.sls include of hostname matching, and the pillar top.sls becomes 10x easier to manage. Where as you could do grain matching, but that wouldn't be secure.
20:25 schemanic In these pastes, am I to understand that these are commands to be run verbatim, or are you saying in things like cloud-init-config that those are files and should read as such
20:26 schemanic Edgan I am doin that already
20:26 schemanic doing*
20:26 Edgan schemanic: The commands would be run verbatim, but you would search replace the fqdn and hostname
20:26 Edgan schemanic: but with auto_accept: True, it is all for not really
20:26 schemanic So, you're saying I have to manually ssh to each server and vim these into files and run commands and somesuch?
20:26 schemanic I'm lost
20:26 Edgan schemanic: No
20:26 Edgan schemanic: The chain goes like this
20:27 Edgan schemanic: 1. Run packer with default distro AMI 2. Have it run script to install salt-minion 3. It will already have cloud-init installed 4. run salt-cloud to start instance with packer made AMI
20:28 schemanic k
20:28 Edgan schemanic: 5. have salt-cloud also include custom userdata from a template
20:28 Edgan schemanic: step 2.1. the script also includes the scripts that will be run by cloud-init
20:29 Edgan schemanic: As the instance starts, cloud-init automatically runs
20:29 schemanic Okay. At what point do I specify my desired minion id?
20:29 Edgan schemanic: It automatically reads a private to the instance api that pulls in the userdata
20:29 Edgan schemanic: in the custom userdata
20:29 Edgan schemanic: and then it processes the userdata and executes what you told it
20:29 schemanic so I write that custom userdata on the command line when I invoke salt-cloud? You pasted like 25 lines
20:29 Edgan schemanic: which in my case is set-hostname and enable-salt-minion, scripts already in the AMI
20:30 Edgan schemanic: I don't know salt-cloud well enough to tell you if it even supports userdata. I have previously done this with a custom boto script, and currently do this with terraform
20:30 Edgan whytewolf: salt-cloud and userdata?
20:31 schemanic but you said in 5 that I should have it use custom userdata, ergo you assumed it could do that before you wrote the line
20:31 Edgan schemanic: I am just trying to wrap your brain around what I do, and what you need
20:31 whytewolf https://docs.saltstack.com/en/latest/topics/cloud/aws.html#optional-settings
20:31 whytewolf and, userdata can be templated
20:31 Edgan nice
20:32 Edgan schemanic: So it sounds like you can connect the dots
20:32 schemanic yeah I think I can, but it seems quite complicated, and it still makes me write down the minion Id I want in a file first, if I'm following you
20:33 Edgan schemanic: no, he said template
20:33 schemanic I am doing the jinja-in-pillar thing though and yes that's making things much easier
20:33 schemanic template=jinja file yes?
20:33 Edgan schemanic: Which implies, and assumes that you can pull in the fqdn into the template
20:33 Edgan userdata_template: jinja
20:33 Edgan so, yes
20:33 whytewolf I'm just not sure of default values in the template. I don't use aws
20:34 Edgan schemanic: it is complex, but this stuff has been around a while, and is fairly common, even outside of salt
20:34 schemanic hmm
20:34 Edgan schemanic: userdata and cloud-init is how people user cloudformation to do poor mans configuration management. They just say run these 10 commands to setup the instance on boot
20:35 DammitJim joined #salt
20:35 whytewolf yeah. cloud-init userdata is how i used to setup systems before i got into salt. using openstacks heat for other parrts of it
20:35 Edgan schemanic: You could also combined it with salt masterless, aka salt-call and not even need a salt master
20:35 schemanic Sure. I think I'm just struggling with where some of this stuff executes.
20:36 Edgan schemanic: ask
20:36 schemanic I don't understand what you mean when you say that a template isn't necessarily a file
20:36 schemanic you're saying that I pass a file to the ec2 instance which can be templated
20:37 schemanic ergo I have to have a file with the value of my minion sitting somewhere
20:37 schemanic right now I use a salt state to set the hostname AFTER the minion id
20:37 schemanic ergo my hosts need minion ids before they get the hostname I want from them
20:37 schemanic like, it makes the fqd == minion_id
20:37 schemanic fqdn*
20:38 whytewolf so, put the value for hostname into /etc/salt/minion_id
20:38 whytewolf userdata is just a bash script.
20:38 whytewolf you can do pretty much anything you want with it
20:38 whytewolf [up to the char limit which you most likely won't hit]
20:39 schemanic It just seems to me like it'd be much simpler to just have salt-bootstrap fixed, commission a new instance with it, and then packer THAT
20:39 schemanic because salt-bootstrap either can set the minion id without bootstrapping the host, or the documentation needs clarifying
20:39 whytewolf you still would need to setup the minion after words. packer only creates images.
20:40 schemanic yeah but if salt-bootstrap works properly, then it will clobber the old minion id with the new one upon commissioning a minion am I right?
20:43 whytewolf yes, the bootstrap will overwrite the minion_id file.
20:43 schemanic right, well then bob's your uncle
20:44 whytewolf not really. it also has the added benifit of trying to change your install, and will bomb out if something goes wrong leaving your instance in a broken state
20:45 exarkun joined #salt
20:45 Edgan schemanic: minion_id = fqdn, so if you set the fqdn of the instance properly, no need to mess with minion_id
20:45 Edgan schemanic: Also it is very nice to have the minion_id, fqdn, and name tag to all match
20:46 morgana2313 Hello. I use -require: [some_service] a lot. And some_service depends on the service being installed etc. Is there a way to skip all these dependencies if the service is already running?
20:46 schemanic Edgan, I agree, That's what I want
20:46 Edgan schemanic: I have a mess of minion_ids as foo_uuid, foo.node.consul, and non-unique foo.domain
20:47 Edgan schemanic: and the fqdn can be a different one of those three
20:47 Edgan schemanic: and the name tag will be either the minion_id or the fqdn
20:47 Edgan schemanic: I inherented it
20:47 Edgan schemanic: I am cleaning up the mess
20:48 whytewolf morgana2313: require and the requisites are state based not service based. it is paying attention to the states. you could use onchange instead of require. onchange will only run the state if the one listed is changed
20:48 Edgan schemanic: I prefer to install the minion from packages, not bootstrap. It likes to pip install
20:49 Edgan schemanic: I also make my own custom debs and rpms with backported patches
20:49 whytewolf bootstrap shouldn't pip install if you don't pass -P
20:49 Edgan I also want it pulling it from my apt/yum repo, not the public ones
20:49 schemanic I'll review our conversation today and work on seeing if it's easy to operationalize. I am the only person doing DevOps at our company and I also handle all the audits and IT stuff they have here, so I only have time to sneak things in as we go
20:49 Edgan schemanic: nod
20:50 Edgan schemanic: I am mostly a one man show, and I have been a few times before this
20:50 schemanic Thanks for your suggestions though. I'm trying to build good patterns and your help has been invaluable toware that already
20:50 Edgan schemanic: You are welcome
20:50 schemanic toward*
20:50 schemanic I can't wait for my new keyboard
20:50 whytewolf schemanic: cloud-init when working with anything cloud based is a good skill to have.
20:50 Edgan schemanic: What did you get?
20:51 schemanic Massdrop CTRL.. Basically an Input Club K-Type with QMK instead of KLL
20:51 Edgan schemanic: I use https://elitekeyboards.com/products.php?sub=keyed_up_labs,tenkeyless&amp;pid=es87u_cl_bbb_al  with cherry blacks at home and work
20:51 schemanic Not getting it till august. https://www.massdrop.com/buy/massdrop-ctrl-mechanical-keyboard. Plenty of time to order switches and a nice keycap set.
20:52 schemanic Gotta run. Thanks again!
20:52 Edgan I had to replace the original keycap set on one of them from ware.
20:59 onlyanegg joined #salt
21:00 AngryJohnnie joined #salt
21:22 DanyC joined #salt
21:27 briner joined #salt
21:30 onlyanegg joined #salt
21:48 noobiedubie zer0def: your a gentleman and a scholar that does seems to be the issue though tracking down why is going to take longer
21:49 noobiedubie zer0def: yeah need to file this as a bug for sure thanks for the workaround though!
21:52 noobiedubie anyone else run into this with any other file types?
21:53 noobiedubie encoding error when salt tries to show_changes in a state? this would be on 2018.3.0
21:54 MTecknology I'm a fan of the microsoft ergonomic 4000 keyboard. It doesn't have the fancy mechanical clicky stuff, but it sure got rid of my nerve problems.
21:57 rglauser joined #salt
22:07 JacobsLa_ joined #salt
22:14 ipsecguy_ joined #salt
22:16 brokensyntax joined #salt
22:26 StolenToast joined #salt
22:26 noobiedubie joined #salt
22:30 evle joined #salt
22:47 heaje joined #salt
23:05 wongster80 joined #salt
23:07 cliluw joined #salt
23:17 Hybrid joined #salt
23:26 dendazen joined #salt
23:36 dezertol joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary