Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2017-06-19

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:25 fxhp joined #salt
00:28 onlyanegg joined #salt
00:55 keldwud joined #salt
00:56 gnomethrower joined #salt
01:08 gmoro joined #salt
01:27 cyteen joined #salt
01:28 MTecknology shanth_h: $client uses jinja templating *EVERYWHERE*... and it's needed almost nowhere.
01:28 Bock joined #salt
01:29 MTecknology I actually write my states with 1-2 bits of templating and after I get it working I wedge it into the environment and add 10-20,000 more bits.
01:31 shanth_h what do you mean 20,000 bits?
01:33 shanth_h im playing around with jinja in my test lab MTecknology, hard to tell when to use it in prod
01:34 shanth_h if i write my states in such a way that they are resusable, then using a jinja map makes sense. i think i might be getting the idea. cause i can use my 'install pkg, config file, start service' state many times
01:39 Trauma joined #salt
01:46 MTecknology shanth_h: the answer to when it's time to use jinja... is when you need it
01:46 shanth_h :D
01:48 ilbot3 joined #salt
01:48 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2016.3.6, 2016.11.5 <+> Support: https://www.saltstack.com/support/ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ <+> See also: #salt-devel, #salt-offtopic <+> We are volunteers and may not have immediate answers <+> The call for speakers for SaltConf17 is now open: http://tinyurl.com/SaltConf17
01:51 MTecknology shanth_h: example- https://gist.github.com/MTecknology/f05ff2d1ebd8819c80b13201257c02cf
01:59 whytewolf agreed, when needed is the best answer. such as my mysql sections. to install mysql not a drop of jinja is used. even in the config, although i do tell it to use jinja as a render engine. just in case. but when i get to users and database setups. i have a lot more jinja for looping through lists of lists.
01:59 MTecknology no jinja to deploy?
02:00 whytewolf https://github.com/whytewolf/salt-phase0-states/blob/master/sys/install/mysql.sls
02:01 whytewolf i only deploy on 1 os so i don't need to worry about maps and names
02:04 whytewolf if this was a non-home lab i would be more worried about versions and what not. but in my lab i expect it to break
02:05 whytewolf although still wouldn't need jinja to move to versioning. just would need to be more careful
02:08 whytewolf there is jinja to change root password though. no avoiding that. but that happens in a seperate piece of code
02:10 whytewolf https://github.com/whytewolf/salt-phase0-states/blob/master/sys/other/mysql/rh_root.sls
02:11 MTecknology whytewolf: I used jinja to populate debconf states that were used for templated deployments, as well as cluster members.
02:11 whytewolf I used to do that when i used ubuntu for my os
02:11 whytewolf but my hardware needed me to move to centos
02:12 MTecknology I'd prefer never use either
02:13 whytewolf I'm not a fan of debians it is opensource or nothing additude.
02:13 MTecknology non-free?
02:14 whytewolf unofficial?
02:14 MTecknology just means it has a whole different set of rules and policies
02:15 MTecknology I do free-only across everything except one of my laptops that requires nvidia drivers.
02:33 berto- joined #salt
02:39 fracklen joined #salt
02:45 edrocks joined #salt
02:46 JPT_ joined #salt
03:00 emid joined #salt
03:12 onlyanegg joined #salt
03:29 donmichelangelo joined #salt
03:53 jk04 joined #salt
03:59 jk04 joined #salt
04:05 onlyanegg joined #salt
04:10 evle1 joined #salt
04:15 snakegums joined #salt
04:20 hemebond What's the trick to getting custom runners working?
04:20 hemebond I've tried extmods, runner_dirs, sync_all, sync_runners..
04:21 hemebond nvm
04:21 hemebond Should have used -l debug first.
04:21 whytewolf okay, neverminded
04:22 whytewolf i think i blinked from the time you started asking till you got to the -l debug
04:22 hemebond :-D
04:22 hemebond I thought of it as I was typing the question and pressed ENTER
04:46 mosen joined #salt
04:47 edrocks joined #salt
04:58 fracklen joined #salt
05:00 qwertyco joined #salt
05:03 sh123124213 joined #salt
05:42 jk04 joined #salt
05:48 buhm joined #salt
06:05 do3meli joined #salt
06:05 do3meli left #salt
06:05 zerocool_ joined #salt
06:11 jk04 joined #salt
06:11 keltim joined #salt
06:13 preludedrew joined #salt
06:15 jk04 left #salt
06:16 jk04 joined #salt
06:16 colttt joined #salt
06:17 onlyanegg joined #salt
06:19 felskrone joined #salt
06:22 jk04 joined #salt
06:25 samodid joined #salt
06:25 darioleidi joined #salt
06:26 xet7 joined #salt
06:37 seffyroff joined #salt
06:45 jk04 joined #salt
06:48 jk04 joined #salt
06:52 jk04au joined #salt
07:00 jk04au joined #salt
07:02 cyborg-one joined #salt
07:07 jk04 joined #salt
07:08 jk04 joined #salt
07:11 jk04 joined #salt
07:13 jk04 test
07:13 hemebond test
07:13 whytewolf pong
07:14 jk04 How do I include an env in another one? Example base in production
07:15 MTecknology that sounds like an excellent way to create a confusing mess of infinite bugs
07:17 hemebond jk04: You can have multiple directories in the file root.
07:17 hemebond But not include an environment.
07:21 MTecknology You can include different sls files from different environments applied to one minion, but that's already getting pretty screwy
07:22 hemebond MTecknology: My current top.sls files would probably unnerve you :-)
07:22 MTecknology probably
07:23 iggy if it uses environments, it definitely would
07:23 fracklen joined #salt
07:25 MTecknology Someday I'd like to start using environments at home, but not until I decide to set up a real testing/QA process w/ gitea.
07:26 MTecknology I got to play with them more at $previous_employer where there was nothing before salt, but it wasn't my group that played with them. I was the only linux admin so my environment was the ProTest env.
07:26 hemebond Figuring out a process for testing to production is my current task. Not easy.
07:26 aldevar joined #salt
07:28 MTecknology I have a plan for someday, but I have a pretty big checklist first.
07:33 onlyanegg joined #salt
07:33 Hybrid joined #salt
07:35 wych joined #salt
07:48 MTecknology hemebond: I want to push to a test branch where test has unit testing (unit testing => deploy system, test stuff, destroy), done nightly. If things pass, I can do a PR to a qa branch for testing and then schedule those to migrate to prod during a change control window... or something like that
07:48 losh joined #salt
07:49 MTecknology I've never seen change control done effectively so I'm not really all that sure what it looks like.
07:52 yuhl joined #salt
07:53 coredumb MTecknology: if you find a correct way to do that please share :)
07:59 om2 joined #salt
08:00 Rumbles joined #salt
08:01 hemebond MTecknology:  Seems fairly straight-forward.
08:01 fracklen joined #salt
08:01 hemebond I think someone else in my team is looking at that method.
08:01 coredumb hemebond: is it?
08:02 hemebond The only reason I'm not using that same method is because historically my environments haven't been identical.
08:02 coredumb I feel a bit overwhelmed by all the jinja's validation
08:02 hemebond I'm weary of trying to have identical sources.
08:03 coredumb Well I guess that's doesn to one's convention
08:03 hemebond I'm also curious what a "component"-based setup would be like.
08:04 hemebond e.g., applying different things from different saltenv's to build servers.
08:05 MTecknology what does component-based mean?
08:07 hemebond Building a highstate from states across many environments.
08:07 hemebond e.g., a management saltenv that sorts out management-type stuff.
08:08 hemebond A saltenv for a particular application suite or stack.
08:08 * MTecknology shudders
08:08 hemebond
08:09 MTecknology bed time!
08:09 hemebond ????
08:09 MTecknology ... I hope, wish me luck
08:11 coredumb good night :)
08:15 absolutejam Anyone know how to remove Minion XYZ did not respond. No job will be sent. from my job output?
08:17 pbandark joined #salt
08:22 Eugene Lazy answer; pipe it through grep
08:22 justanotheruser joined #salt
08:24 qwertyco joined #salt
08:29 Mattch joined #salt
08:42 absolutejam Well I'm currently doing salt ... --static --out json > somefile
08:42 cablekev1n joined #salt
08:42 absolutejam and piping to python script, so I'll process it there
08:57 dstensnes absolutejam: maybe "show_timeout: True" in master config is related?
08:58 fracklen joined #salt
09:00 ujjain joined #salt
09:00 ujjain joined #salt
09:00 fracklen_ joined #salt
09:06 Kelsar joined #salt
09:11 ujjain joined #salt
09:11 ujjain joined #salt
09:13 Edur joined #salt
09:14 rofl____ joined #salt
09:14 lstor joined #salt
09:17 dstensnes absolutejam: did that help?
09:20 evle joined #salt
09:21 absolutejam Sorry buddy, only just seen your message
09:22 absolutejam Hm, I still want it to show for most of the time
09:22 absolutejam I'll just parse it in my script; it's no bother
09:22 absolutejam Hopefully i can move this over to HubbleStack soon anyway
09:24 dstensnes ah, okay :)
09:26 tvinson joined #salt
09:27 eseyman joined #salt
09:27 yidhra joined #salt
09:27 saltsa joined #salt
09:27 nku joined #salt
09:27 k_sze[work] joined #salt
09:33 onlyanegg joined #salt
09:36 citaret joined #salt
09:37 citaret joined #salt
09:40 citaret joined #salt
09:43 JohnnyRun joined #salt
09:55 zerocoolback joined #salt
09:55 zerocoolback joined #salt
09:56 fritz09 joined #salt
10:00 sh123124213 joined #salt
10:05 sh123124213 joined #salt
10:08 Praematura joined #salt
10:16 CrummyGummy joined #salt
10:18 zerocool_ joined #salt
10:23 zerocoolback joined #salt
10:26 om2 joined #salt
10:28 mauli joined #salt
10:30 mschiff joined #salt
10:30 mschiff joined #salt
10:32 Ouzo_12 joined #salt
10:34 jor joined #salt
10:38 cyborg-one joined #salt
10:42 gomerus[m] joined #salt
10:42 toofoo[m] joined #salt
10:44 gmoro joined #salt
10:46 sh123124213 joined #salt
10:54 cablekev1n joined #salt
10:54 om2 joined #salt
10:57 qwertyco joined #salt
10:58 sh123124213 joined #salt
11:05 pbandark1 joined #salt
11:12 Rumbles joined #salt
11:12 lorengordon joined #salt
11:24 exegesis joined #salt
11:28 netcho joined #salt
11:34 onlyanegg joined #salt
11:36 hlub joined #salt
11:41 Sarphram joined #salt
11:43 hlub left #salt
11:46 pbandark I am running multiple salt formulas to multiple servers. but, I dont see its parallel execution and hence, it takes lot of time to complete the task. is there any tuning parameter to enable parallel execution of salt formulas to multiple minions ?
11:46 zerocoolback joined #salt
11:46 zerocoolback joined #salt
11:49 lionel_ left #salt
11:49 lionel joined #salt
11:52 dev_tea joined #salt
11:53 k_sze[work] joined #salt
11:55 armin joined #salt
11:59 nku pbandark: what do you see anyway? the output is always per minion afaik, not a stream or somesuch
12:00 nku things should run in parallel by default
12:00 dev_tea joined #salt
12:01 pbandark i am deploying 9 different salt formulas to 9 different servers. which taks approx 1hour to complete and it seems to be 1 formula gets applied on 1 server at a time
12:01 armin seems like what you want is parallelization
12:03 nku oh, states in parallel..
12:03 pbandark armin: can you elaborate bit in detail? do I need to tune any parameter ?
12:03 pbandark from salt master
12:04 nku looks like there is https://docs.saltstack.com/en/develop/ref/states/parallel.html
12:04 nku not released yet it seems
12:05 mr_kyd joined #salt
12:08 felskrone joined #salt
12:10 ujjain joined #salt
12:10 ujjain joined #salt
12:11 om2 joined #salt
12:11 Naresh joined #salt
12:12 armin pbandark: uhm i don't know if that's even easily possible from just within salt. i was just mentioning the vocabulary.
12:12 armin pbandark: and yes, nku's warning applies.
12:12 armin :)
12:12 babilen joined #salt
12:12 pbandark ok
12:13 nku well, you could do multiple state.sls if the states don't depend on each other
12:14 babilen joined #salt
12:21 babilen joined #salt
12:21 fredvd joined #salt
12:23 felskrone joined #salt
12:26 thinkt4nk joined #salt
12:26 pbandark k
12:26 pbandark ok
12:28 asyncsec joined #salt
12:29 pbandark nku: so does that mean, the behavior  which I am getting now is expected?
12:31 spiette joined #salt
12:35 numkem joined #salt
12:43 onlyanegg joined #salt
12:47 coredumb is there to template only a certain part of a file ?
12:47 coredumb a way*
12:48 nku pbandark: yeah
12:48 pbandark ok
12:48 nku coredumb: include it?
12:48 nku coredumb: why though?
12:49 coredumb nku: guess that could work
12:49 coredumb nku: I don't want salt to change the entire content of the file
12:49 coredumb I hate doing that but ...
12:49 nku coredumb: well, you can append lines to files
12:50 coredumb nku: I'd rather not :P
12:51 gmoro joined #salt
12:51 snc joined #salt
12:52 coredumb nku: guess I'll try to concatenate blocks of it ...
13:01 NegiLXXXVIII joined #salt
13:11 sjorge joined #salt
13:20 XenophonF word
13:22 bluenemo joined #salt
13:23 noobiedubie joined #salt
13:24 hemebond left #salt
13:31 netcho joined #salt
13:31 racooper joined #salt
13:34 bowhunter joined #salt
13:40 aphor joined #salt
13:41 Yamazaki-kun joined #salt
13:43 gmoro joined #salt
13:43 onlyanegg joined #salt
13:48 mchlumsky_ joined #salt
13:49 cgiroua joined #salt
13:50 mchlumsky joined #salt
13:54 edrocks joined #salt
13:58 mchlumsky joined #salt
13:59 dyasny joined #salt
14:00 mchlumsky joined #salt
14:01 mchlumsky joined #salt
14:02 noobiedubie joined #salt
14:02 spiette I need to run a command on all minions and compare that to the output of a command run on the master. How I can do that with states (but I could create a grain for that)
14:03 lorengordon joined #salt
14:04 XenophonF Pillar data gets rendered on the master.
14:05 XenophonF Maybe you could use cmd.run in a Pillar SLS file?
14:06 spiette I'll have a look at this, thanks for the pointer
14:07 evle joined #salt
14:13 Inveracity joined #salt
14:28 keldwud joined #salt
14:31 mpanetta joined #salt
14:34 mpanetta joined #salt
14:34 NegiLXXXVIII can i define grains for minions without default values? so far i'm copying a grains file to the minion but im not quite sure if thats the right way...
14:37 dyasny joined #salt
14:38 fritz09 joined #salt
14:44 onlyanegg joined #salt
14:46 seanacais joined #salt
14:51 noobiedubie joined #salt
14:56 MTecknology NegiLXXXVIII: I don't think I understand what that question means.
14:56 qwertyco joined #salt
14:56 NegiLXXXVIII :D
14:57 amcorreia joined #salt
14:57 NegiLXXXVIII i need to have some grains for my minions which i require in custom modules ans states.
14:57 babilen NegiLXXXVIII: Most grains should be obvious on the minion
14:58 NegiLXXXVIII and i want to ensure that these grains are on the minion when i call a highstate
14:58 babilen NegiLXXXVIII: Could you start a bit earlier: What are you trying to do with that data and why did you decide to put it into grains? How are you managing those grains?
14:59 NegiLXXXVIII ok
15:00 NegiLXXXVIII i want to manage windows minions and one step in the highstate is to update the system
15:00 babilen okay
15:00 NegiLXXXVIII other states like configuration etc. shall only be run, if there are no further updates
15:00 babilen (I prefer highstates to not perform upgrades, but go on)
15:01 NegiLXXXVIII therefore i am currently storing in a grain on the minion if there are updates required
15:01 sarlalian joined #salt
15:02 NegiLXXXVIII however, if the grains im using in the custom module is not previously defined, the information will be lost
15:03 babilen How do you define it?
15:03 NegiLXXXVIII currenty with file.managed and copying a grains file to the minion...
15:03 babilen But how do you know if upgrades are necessary when you manage that file?
15:04 NegiLXXXVIII which has the problem of overwriting any information
15:04 NegiLXXXVIII i require the file to be absend currently
15:04 NegiLXXXVIII *absent
15:05 babilen Yeah, but at one point you somehow surmise that upgrades are necessary. What is that based on?
15:06 NegiLXXXVIII i check if the installed updates match the required updates
15:06 NegiLXXXVIII i have a list in my pillar which contains the required updates
15:06 babilen And that is from an external data source?
15:07 NegiLXXXVIII and after each update i store a list of the currently installed updates in a file on the minion
15:08 babilen Is that your own software and the states are configuring it, but you only want to "upgrade" the configuration after you installed the respective software version? (as you cannot upgrade the configuration during the installation)
15:08 NegiLXXXVIII no i want to configure and update windows
15:09 babilen Right, and the "upgrade required" flag is global or per-software?
15:09 babilen What's so bad about configuring and upgrading independently?
15:10 NegiLXXXVIII because some configuration require specific updates
15:10 NegiLXXXVIII e.g. lgpo setting
15:11 NegiLXXXVIII the upgrade required flag is per-software, if by per-software you mean individual for each minion
15:11 rihannon joined #salt
15:12 babilen No, I meant one flag for software-pkg-foo and andother for software-pkg-bar
15:13 NegiLXXXVIII ok, then no
15:13 babilen My basic idea would be to target the "configure" states based on a specific OK-GOGOGO value of your "upgrade" grain
15:14 babilen That way they wouldn't be part of the highstate if the grain is not set correctly
15:14 NegiLXXXVIII yes thats what i made so far
15:14 babilen The question is then: How do update that grain?
15:15 babilen One way would be to schedule an orchestrate run in which you first set the grain and then run a highstate on a suitable schedule (or just manually)
15:16 mikecmpbll joined #salt
15:16 babilen The "unset" grain shouldn't be a problem as that, for purposes of targeting, is the same as if the grain is set to something else
15:17 babilen (thanks for the explanation btw, it makes it much clearer as to what you are trying to achieve)
15:18 NegiLXXXVIII ok so what i have so far is that i made a wrapper module for the windows update module in salt
15:18 NegiLXXXVIII and in my wrapper i check if updates are required or not
15:19 NegiLXXXVIII the problem is how do i ensure that the grain i use is existent
15:20 NegiLXXXVIII ohh
15:20 Splix76 joined #salt
15:20 NegiLXXXVIII now that i think about it again i counld just check if __grains__['xxx'] is existent, can't i?
15:21 magz0r joined #salt
15:23 NegiLXXXVIII ok now much easier question. can i create a grain with the global variable __grains__ or do i need to execute a salt-call?
15:23 PatrolDoom joined #salt
15:23 NegiLXXXVIII ohh and thank you babilen for your help and your ideas
15:24 whytewolf NegiLXXXVIII: look into the grains state and module for better ways to manage grains instead of using file.manage
15:25 NegiLXXXVIII thx whytewolf i will do so
15:28 babilen NegiLXXXVIII: Why would you have to check if __grains__['xxx'] exists at all? The states that require the upgrades would never run if the grain is neither set nor set to a different value
15:28 babilen And yeah, look into grains.set / grains.append, ...
15:29 NegiLXXXVIII yeah but it will also never run once the updtate is completed.
15:29 NegiLXXXVIII so yeah, i will lokk into the grains module
15:30 NegiLXXXVIII totally forget that i could use this :D
15:30 NegiLXXXVIII *look
15:30 babilen "yeah but it will also never run once the updtate is completed" -- why is that a problem?
15:32 NegiLXXXVIII i need the minion also to be configured correctly
15:32 NegiLXXXVIII the configuration may change over time since it is a windows system...
15:33 NegiLXXXVIII ok i got to go now
15:33 NegiLXXXVIII thank you again for your help. bye!
15:34 babilen But if it is: Just target the states on "not grains:update_needed"
15:34 babilen argh
15:34 babilen ...
15:40 debian112 joined #salt
15:43 woodtablet joined #salt
15:44 dendazen joined #salt
15:48 aldevar left #salt
15:56 edrocks joined #salt
15:57 heaje joined #salt
15:57 tobiasBora joined #salt
16:08 POJO joined #salt
16:18 Praematura joined #salt
16:20 morissette joined #salt
16:26 onlyanegg joined #salt
16:35 bowhunter joined #salt
16:37 edrocks joined #salt
16:40 kipper joined #salt
16:43 ChubYann joined #salt
16:43 wendall911 joined #salt
16:48 cyborg-one joined #salt
16:58 drawsmcgraw joined #salt
16:59 Trauma joined #salt
17:27 easytyger joined #salt
17:29 Roh joined #salt
17:31 keltim_ joined #salt
17:40 nicksloan joined #salt
17:45 cliluw joined #salt
17:47 impi joined #salt
17:51 nixjdm joined #salt
17:54 onlyanegg joined #salt
18:06 pppingme joined #salt
18:08 XenophonF is there a way to tell salt how long to wait for a service to start?
18:10 Inveracity joined #salt
18:10 woodtablet yes
18:11 woodtablet init_delay
18:11 woodtablet https://docs.saltstack.com/en/latest/ref/states/all/salt.states.service.html
18:11 mt5225 joined #salt
18:12 woodtablet scroll down to salt.states.service.running =D, hope this helps my friend
18:12 tcolvin_ joined #salt
18:12 mihait_ joined #salt
18:12 doriftoshoes__ joined #salt
18:12 copelco_ joined #salt
18:12 cgiroua_ joined #salt
18:12 ople_ joined #salt
18:12 nickadam_ joined #salt
18:12 GnuLxUsr joined #salt
18:13 miruoy_ joined #salt
18:14 noobiedubie joined #salt
18:14 icebal- joined #salt
18:14 citaret_ joined #salt
18:14 mugsie_ joined #salt
18:14 cswang_ joined #salt
18:14 Mogget_ joined #salt
18:15 Shados_ joined #salt
18:16 XenophonF ah
18:16 XenophonF thanks!
18:16 pewpew_ joined #salt
18:17 peters-tx joined #salt
18:17 CrummyGummy_ joined #salt
18:18 jor_ joined #salt
18:18 ThomasJ|d joined #salt
18:18 rubenb_ joined #salt
18:18 rideh- joined #salt
18:18 absolutejam_ joined #salt
18:18 cb_ joined #salt
18:19 MeltedLux_ joined #salt
18:19 carmony_ joined #salt
18:21 fritz09 joined #salt
18:22 aarontc joined #salt
18:22 g3cko joined #salt
18:22 shalkie joined #salt
18:22 evilrob joined #salt
18:23 PatrolDoom joined #salt
18:23 psychi[m] joined #salt
18:24 onovy joined #salt
18:24 ujjain joined #salt
18:25 klaas joined #salt
18:25 jerrykan[m] joined #salt
18:25 ujjain joined #salt
18:25 monokrome joined #salt
18:25 dyasny joined #salt
18:25 packeteer joined #salt
18:26 fujexo[m] joined #salt
18:27 theblazehen joined #salt
18:30 ekkelett joined #salt
18:32 yoConQuimio joined #salt
18:35 yoConQuimio left #salt
18:36 nickadam joined #salt
18:38 major is there a way to specify config variables on the CLI?
18:38 major like .. point to a specific file_roots or some such?
18:41 netcho joined #salt
18:43 edrocks joined #salt
18:48 lorengordon `salt -h` should display all the cli opts
18:48 lorengordon there's also the web docs, but i find that they are sometimes do not have the complete cli opts, https://docs.saltstack.com/en/latest/ref/cli/salt.html
18:49 lorengordon major: ^
18:50 JPT joined #salt
18:51 aldevar joined #salt
18:52 major right, but that wouldn't really be inclusive to some sort of module which allowed config variables to be specified as arguments for local execution.
18:53 major sort of hoping there was such a module which I can't find
18:57 schemanic joined #salt
18:57 schemanic Hi
18:58 Praematura_ joined #salt
18:58 schemanic Im still trying to wrap my head around doing ext_pillar based roles. My ext_pillar is a git repo which hosts all of my pillars including my top file. If I wanted to do pillar-based roles by host, how would I go about assigning the pillar data to hosts when the top file is meant to do that?
18:59 major also .. the bootstrap-salt.sh needs some serious rework...
18:59 major bleh
18:59 Edgan major: Are you in AWS?
19:00 major no
19:00 bowhunter joined #salt
19:01 nixjdm joined #salt
19:02 schemanic Has anyone here worked with pillar-based roles?
19:03 whytewolf in states, but not in a ext_pillar to pillar sense
19:07 major seriously .. the amount of mixed syntax, unquoted variables, and redundant checks in this script is making me want to open the whiskey
19:08 tcolvin joined #salt
19:10 monjwf joined #salt
19:16 Edgan major: My solution is to bake the minion into my AMI, you could do the same with VM images, and not use bootstrap.
19:19 major maybe..
19:20 major this is more for bootstrappin random masters all over the network
19:20 major bootstrapping even
19:20 schemanic major are you trying to bootstrap masters?
19:20 schemanic sorry that was a malformed question
19:20 major ;)
19:21 Edgan major: I bootstrap masters with salt-ssh using salt code
19:21 J0hnSteel joined #salt
19:21 schemanic Edgan, do you do that from a locally installed master?
19:21 major hmm
19:22 schemanic My research into salt-ssh seems to indicate that a local master is required
19:22 major think he is saying bring up a new minion and just point it at an existing master
19:22 Edgan schemanic: no
19:22 Edgan schemanic: salt-ssh is completely independent of salt masters
19:22 * major ponders.
19:22 Edgan schemanic: works just like ansible
19:22 major Edgan, what about bringing up a new master+syndic?
19:23 Edgan schemanic: you can run it from anywhere salt-ssh is installed
19:23 major actually .. that wont work for this
19:23 osmola joined #salt
19:23 major monday morning brain just parsed the -ssh suffix ...
19:23 * major goes to get more coffee..
19:25 losh joined #salt
19:25 om2 joined #salt
19:26 coredumb I bootstrap masters with salt code either
19:26 Edgan coredumb: either?
19:26 SalanderLives joined #salt
19:26 coredumb Edgan | major: I bootstrap masters with salt-ssh using salt code
19:27 Edgan coredumb: ah
19:27 coredumb :P
19:28 Edgan I actually do it in two phases. I have a grain called bootstrapping. When in that mode I skip some parts of my base, like using Artifactory for apt repos.
19:28 whytewolf schemanic: just read your question about ext_pillar based roles. first. you would need a DIFFERENT ext_pillar then git_pillar that exacutes before git_pillar. so that the top file in git_pillar has the pillars defined alread
19:28 Edgan Then when I have all the bootstrapping cross dependencies sorted out, I switch it off and run salt again.
19:29 coredumb Edgan: interesting
19:29 coredumb I actually have a self contained state file I localy run that is doing the job
19:31 schemanic whytewolf, so if I understand correctly, I can set it up exactly the same way I have my existing git-based, ext_pillar, but all it is is a top file which maps role pillars to hosts
19:31 lorengordon joined #salt
19:31 coredumb master is actually the easy eay... how do you mass add minions in a secure way?
19:31 whytewolf schemanic: no, not all ext_pillars use top files
19:32 coredumb like pre-generating the keys
19:32 schemanic whytewolf, interesting, how does the ext_pillar understand what hosts get assigned the pillar dicts?
19:33 whytewolf schemanic: because the minion_id that is requesting the pillar is passed to the pillar function.
19:33 whytewolf https://docs.saltstack.com/en/latest/topics/development/external_pillars.html#ext-pillar
19:34 schemanic wait...
19:34 schemanic this is referring to a module of some kind
19:34 schemanic wait
19:34 whytewolf yes, what do you think ext_pillars are
19:34 mt5225 joined #salt
19:34 coredumb ext_pillar are modules
19:34 schemanic no
19:34 major my environment is .. special .. enough so that I think I have flavoured paste in my budget...
19:35 whytewolf https://docs.saltstack.com/en/latest/salt-modindex.html#cap-p yes
19:35 schemanic ext_pillar is a property of /etc/salt/master or a /etc/salt/master.d/*.config file that specifies where an external pillar store is located
19:35 whytewolf ext_pillar is a setting that tells the master what exT_pillar module to use
19:36 schemanic I see
19:36 schemanic so what's really happening is it's saying 'use the git ext_pillar module to load pillar data from a git repo'
19:36 major salt is like a giant cross-reference table...
19:36 coredumb schemanic: check https://github.com/saltstack/salt/tree/develop/salt/pillar
19:36 major this tag/element invokes this function "over here!!"
19:36 schemanic yes major, and the documentation is also quite arcane in some places
19:37 whytewolf or just look at the index starting at p that lists all the different ext_pillars
19:37 schemanic coredumb, what am I meant to understand from this link
19:38 coredumb schemanic: this is the pillar modules
19:38 coredumb looking at them is insighful :)
19:38 schemanic So, whytewolf, you're saying I can still do what you're talking about from a git repo?
19:39 whytewolf schemanic: no, you are already useing git_pillar for all of your other pillars.
19:39 schemanic so I can't use git to do an ext_pillar based role system
19:39 schemanic if I'm using git for my pillars
19:40 whytewolf git_pillar is already an ext_pillar that is already loaded.
19:40 whytewolf so you are already useing it
19:40 schemanic And I cant do both at once or something like that?
19:40 whytewolf there is no both
19:40 whytewolf you are using IT
19:41 schemanic yeah so why cant I use it to load role pillars?
19:41 whytewolf because you can't reference 1 item from with in it's self
19:42 schemanic You are saying that I can't use git_pillar to load role pillars, because I'm using git_pillar to load (other) pillars
19:42 schemanic yes?
19:42 whytewolf yes
19:42 schemanic And you are saying that I have to use a different pillar integration to map hosts to roles
19:43 whytewolf if you want to access those pillars in git_pillar you do
19:43 schemanic Oh I see why i was so confused
19:43 schemanic You said 'then' and not 'than'
19:43 whytewolf i do that a lot. my mistake
19:43 schemanic as in 'a separate ext_pillar coming in time before git_pillar'
19:43 schemanic (but not necessarily separate FROM git_pillar)
19:44 whytewolf well you can't load git_pillar twice
19:44 whytewolf so in essence the first sentance is correct :P
19:44 schemanic Okay, so now that I know that I need a separate kind of pillar, whats a good/EASY one to use?
19:44 schemanic am I right at least in that what I'm going to be doing is mapping hosts to roles?
19:45 whytewolf yes
19:45 whytewolf i heard pillarstack is popular. but if you have a database option i tend to like the database backed ext_pillars
19:46 babilen schemanic: Back in the day I was mentioning the pillarstack external pillar for a reason
19:46 babilen (as it is not the same as the pillar in which you want to use the roles you defined)
19:48 rihannon joined #salt
19:49 major never mind .. I think I have a workflow that can abuse salt-ssh
19:49 major Edgan, thanks for the idea
19:51 fracklen joined #salt
19:53 schemanic babilen, I don't understand - you're saying you recommend pillarstack, but NOT for roles?
19:55 schemanic Oh I think I understand
19:55 babilen schemanic: No, not at all. I am saying that when we had this conversation some time ago I mentioned pillarstack for a reason. That reason was that it is different from your the pillar module you actually use to define that data.
19:56 babilen I had never meant to imply to use the git pillar for roles itself
19:56 schemanic Ah I see. Thank you. I think I got hung up because i'd never seen ext_pillar anywhere outside of git pillar configs
19:57 schemanic so I thought it just meant git
19:57 whytewolf ahh. no git_pillar is one of many ext_pillars
19:57 whytewolf guess that would have saved you a lot of time a lot earlyer
19:58 babilen Yeah, sorry for that
19:58 babilen I was essentially referring to https://docs.saltstack.com/en/latest/ref/pillar/all/index.html
19:58 schemanic It's not your fault, It's just a bit of communicative statice
19:58 schemanic static*
19:59 swa_work joined #salt
19:59 schemanic Oh I see
19:59 schemanic This will be useful though
20:00 schemanic I intend at some point to explore vault for securing the pillar data
20:00 schemanic But I think I'm understanding correctly - 'Roles go in one kind of ext_pillar, Regular state pillars go in another'
20:02 viq Why vault and not gpg?
20:02 nixjdm joined #salt
20:02 schemanic I intend to go gpg for now
20:02 whytewolf yes. i do need to say that that is only if you want to use your roles in pillar. if you are just using your roles in states. then you can do both in the same pillar
20:03 schemanic with gpg-encrypted strings in the pillar data
20:03 schemanic I believe that's how that works
20:03 viq Indeed.
20:03 viq That's what I have currently for sensitive bits of pillars
20:03 schemanic is the decryption key help on the minion?
20:03 schemanic I had a colleague ask me if that was so, if if yes, what keeps an attacker from decrypting the pillar data
20:03 viq Up to you - you can have them decrypted on master, or on minion, depending where you want to put the decryption keys
20:04 schemanic mmm
20:04 schemanic viq, can you have multiple keys?
20:04 viq Uhm, it needs to be decrypted anyway, so the question is what the attacker has access to
20:04 viq schemanic: what do you mean?
20:04 schemanic I've heard people say not to just have one gpg key for encrypting those strings
20:05 schemanic like, have a gpg key for apache keys and another for tomcat or something
20:05 schemanic am I completely turned around?
20:05 viq And if attacker can control salt-master, then it's game over anyway, since even if only the minion has the decrypted values, then you can make it just send them to you
20:05 fracklen joined #salt
20:06 viq Because of what I just wrote I have gpg key on saltmaster that decrypts the secrets (which are also encrypted with keys of people that are to have access to them)
20:06 schemanic how about I put it like this, viq - what's the right way to do gpg encryption between salt-master and minions
20:06 filippos joined #salt
20:08 viq sec, refreshing my knowledge ;)
20:08 drawsmcgraw joined #salt
20:08 evidence joined #salt
20:09 s0undt3ch joined #salt
20:09 babilen We just use a single key on the master
20:09 babilen But that was more for convenience rather than anything else
20:11 viq schemanic: depends what's right for you. But, first answer this: why do you think master should not be able to decrypt the data?
20:11 AvengerMoJo joined #salt
20:11 schemanic viq, I'm not aware that I think that
20:11 schemanic I actually DO think it should be able to
20:12 schemanic I'm asking about implementation paradigms
20:12 viq schemanic: then you have the pillar values decrypted by master, and served in plaintext to minions that they are assigned to (protected by communication encryption)
20:12 viq https://i.redd.it/u7o0w68n7n4z.gif
20:12 viq grr, wrong link
20:12 viq https://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.gpg.html
20:12 schemanic right, that sounds correct
20:12 schemanic so they land on the minion properly
20:12 viq That's what I'm using.
20:13 viq With git_pillar
20:13 v0rtex joined #salt
20:13 viq So the data that I want protected in encrypted in git, but master and selected people can decrypt
20:13 viq Side note, salt does _NOT_ like signed git commits
20:17 schemanic yeah, this is what I intended to implement
20:17 schemanic however what I'm asking is
20:18 schemanic can saltstack handle multiple gpg keypairs
20:18 schemanic in the case that I want one for creds to one system and another for creds to the next
20:18 babilen viq: How do you guys deal with the making the key available to people working on the pillar?
20:18 schemanic viq: I also would like to know that
20:19 schemanic babilen, what if thats what 'selected people' is?
20:19 viq schemanic: that's asymetric, so why would you need to do that?
20:19 MTecknology they only need the public key to add data
20:19 viq babilen: that wasn't particularly complicated, seeing as there were approximately 1.2 people working with salt...
20:19 babilen MTecknology: Yeah, but that would be write only
20:20 babilen Which, I guess, is okay
20:20 MTecknology why do you need read?
20:20 MTecknology minions need read, not you
20:20 viq babilen: though exported key in the git repo where the pillar resides sounds like should work decently well.
20:20 babilen aye
20:21 babilen Pretty much in the same position and was just wondering how to scale .. but "read-only" is perfectly fine
20:21 schemanic babilen, MTecknology, viq - this last part of the conversation lost me - what are you referring to?
20:21 babilen schemanic: You just need the public key to encrypt the data
20:21 viq And you may encrypt with keys of a couple of additional people that are to be able to read the values - a couple salt shepherds, security officer, and the like.
20:21 babilen Members of the team won't be able to decrypt the data, so you essentially only grant "read only" access, by handing out the public key
20:22 viq schemanic: you only need to give people the public key to encrypt data with, and only master will be able to decrypt it. Why would you need to use different keys for that?
20:22 schemanic viq, you mean that multiple person's keys all 'count' for decrypting ciphers generated by the same private key?
20:22 MTecknology schemanic: at a high level, how familiar are you with how gpg works?
20:22 onlyanegg joined #salt
20:22 viq schemanic: no, if you want multiple people to be able to read values you have to explicitly tell gpg to encrypt with their keys
20:23 schemanic viq what are 'thier keys'?
20:23 schemanic their* rather
20:24 viq schemanic: the admins that are to have access to the values
20:24 schemanic MTecknology, I understand that there is a key that is used to encrypt and decrypt ciphers
20:24 viq schemanic: no, those are two separate keys.
20:24 schemanic You are saying a person is equivalent to a cryptographic cyper
20:24 schemanic cypher*
20:24 schemanic I'm confused. You said a key is a person
20:24 viq No, I'm saying a person has a key(pair)
20:25 fracklen joined #salt
20:25 babilen one public and one private
20:25 schemanic Okay. I follow that. I work with ssh so the public/private cryptopgraphy, I have the high level view of
20:25 babilen The private one can decrypt, the public one can encrypt
20:25 schemanic okay
20:26 schemanic So viq is saying there are multiple... private keys in thier setup?
20:26 woodtablet joined #salt
20:26 babilen You leave the private one on the master, so that only it can read the data .. the public one can be handed out freely to "selected people" to manage secrets
20:26 viq schemanic: so the master has it's keypair. You have your keypair. And your coworker has a keypair. Your coworker encrypts the data, eg with master's and your keys. Only master and you can read the values, even your coworker cannot read them after encrypting them that way
20:26 woodtablet left #salt
20:26 viq Yeah, master's public key could just be kept in git next to the rest of the data
20:27 coredumb viq: how does the salt master handle the GPG passphrase ?
20:27 woodtablet joined #salt
20:27 schemanic You mean to say that these keypairs are accessible to all users?
20:27 schemanic So person one may encrypt a secret using person two's key?
20:27 viq coredumb: it doesn't, you give it an unprotected private key
20:28 coredumb viq: oh ok
20:28 viq schemanic: yup, that's how asymmetric encryption works
20:28 schemanic I understand what babilen is saying, I don't follow what viq is saying.
20:28 babilen :D
20:28 viq schemanic: eh, I'll leave it in more capable hands then, especially as it's time for me to get moving :P
20:28 coredumb viq: do you manage root passwd that way?
20:28 schemanic I dont understand how something can be encrypted with two keys at once...
20:29 viq coredumb: I manage very little currently, but various secrets and API keys, yes
20:29 schemanic you're saying person 2 just decides 'well I only want people 3, 5, and 9 to be able to use this secret, so I'll select those keys, throw them in a pot, and cook them into the secret'
20:30 schemanic Well, thank you for explaining the term I can use for research then
20:30 schemanic I'll look up asymmetric gpg
20:31 coredumb viq: ok
20:31 viq schemanic: no, asymmetric encryption. gpg by default uses asymmetric encryption
20:31 schemanic okay.
20:32 viq RSA, DSA, EC25519 are examples of asymmetric encryption. AES, 3DES are examples of symmetric encryption.
20:33 viq https://en.wikipedia.org/wiki/Public-key_cryptography vs https://en.wikipedia.org/wiki/Symmetric-key_algorithm
20:34 * viq poofs
20:34 MTecknology The only people I know of using DSA are worried about NSA backdoors in RSA
20:34 brd MTecknology: even then, I think most people have given up since DSA is weak(?)
20:35 MTecknology you underestimate people's paranoia of the NSA
20:35 Rumbles joined #salt
20:35 MTecknology It doesn't matter how much information they volunteer to the entire internet, but don't let the NSA see any of it!
20:35 PatrolDoom joined #salt
20:36 pbandark1 joined #salt
20:36 major well .. sort of a huge difference between volunteered information and information taken w/out volunteering it
20:36 MTecknology I'm just poking fun at it because I've seen it relatively recently..
20:36 coredumb MTecknology: better use ecc25519 then instead of DSA
20:37 major certainly there are plenty of people who are pretty bad about what they volunteer .. or recognizing where privacy ends..
20:37 major I kinda like ecc25519 ;)
20:37 whytewolf using DSA because RSA by have a RSA backdoor? thats like using MD5 because SHA1 has a collision attack
20:37 coredumb but here I see devs generating dsa ssh keys because "it's written in the wiki"
20:37 MTecknology I also laugh when I see companies demanding 4096 bit rsa keys
20:37 coredumb wiki being written 15y ago doesn't help indeed
20:37 coredumb :D
20:41 ecdhe joined #salt
20:46 ecdhe joined #salt
20:48 noobiedubie ok i have to point this out it's ed25519
20:48 coredumb noobiedubie: thx for that :P
20:49 major noobiedubie, yer no fun ;)
20:53 noobiedubie someone has to be the rain to the parade
20:53 noobiedubie bring* not be
20:55 spartakos joined #salt
20:57 TheFlyingCorpse joined #salt
21:02 losh joined #salt
21:02 nixjdm joined #salt
21:02 ahrs joined #salt
21:03 nicksloan joined #salt
21:05 armonge joined #salt
21:10 asyncsec joined #salt
21:11 fredvd joined #salt
21:20 Rumbles joined #salt
21:21 Elijah joined #salt
21:21 Ouzo_12 joined #salt
21:21 Elijah joined #salt
21:21 coldbrewedbrew_ joined #salt
21:21 graffic joined #salt
21:21 oyvindmo_ joined #salt
21:21 jshm joined #salt
21:21 nineteen joined #salt
21:21 godlike joined #salt
21:21 godlike joined #salt
21:21 TomJepp joined #salt
21:21 edgr joined #salt
21:21 TRManderson joined #salt
21:21 iter joined #salt
21:21 sybix joined #salt
21:21 egilh joined #salt
21:21 socket- joined #salt
21:21 Dr_Jazz joined #salt
21:22 hexa- joined #salt
21:22 tbrb joined #salt
21:22 skrobul joined #salt
21:22 eichiro joined #salt
21:22 rathier joined #salt
21:22 linovia joined #salt
21:22 lorengordon joined #salt
21:22 jagguli joined #salt
21:22 Alan_ joined #salt
21:24 osmola joined #salt
21:25 hashwagon joined #salt
21:26 inire joined #salt
21:26 kalessin joined #salt
21:26 JPaul joined #salt
21:26 upb joined #salt
21:27 skeezix-hf joined #salt
21:27 ThomasJ|m joined #salt
21:28 major joined #salt
21:31 censorshipwreck joined #salt
21:39 shanth i have one minion and every command i run, it does it two times for the minion
21:40 shanth the heck lol
21:40 whytewolf shanth: log into the minion shutdown the minion software and check to make sure it isn't still running. kill it if it is. then restart the minion
21:41 shanth ill try it whytewolf
21:41 shanth it's coming back twice again
21:41 whytewolf was there an extra minion process after you stopped it?
21:42 shanth let me try it again
21:42 whytewolf also, make sure it is dead
21:42 whytewolf dead dead deadsky
21:43 shanth killing them allllllllllll whytewolf
21:43 shanth i think that's what it was
21:45 kalessin joined #salt
21:46 mt5225 joined #salt
21:48 upb joined #salt
21:48 viq shanth: I also had once someone clone a machine that had salt set up, which caused same thing
21:49 viq so check eg uuid grain
21:49 whytewolf that was going to be my next sugestion. but iirc shanth you currently only have 1 minion
21:49 shanth oh nice tip viq
21:50 shanth i even ran the grains.get uuid without having to refer to the docs
21:51 whytewolf shanth: gratz you are making progress.
21:51 shanth :)
21:57 onlyanegg joined #salt
22:01 cro joined #salt
22:02 nixjdm joined #salt
22:04 onlyanegg joined #salt
22:10 scottk_ joined #salt
22:10 edrocks joined #salt
22:11 Praematura_ joined #salt
22:11 scottk__ joined #salt
22:14 scottk__ left #salt
22:14 scottk__ joined #salt
22:15 scottk_ joined #salt
22:15 scottk_ Does anyone know if you can do a pkg.upgrade from a specific repo in aptpkg?
22:16 OliverUK joined #salt
22:17 whytewolf scottk_: does not look like aptpkg version of upgrade has repo info
22:17 OliverUK joined #salt
22:18 OliverUK joined #salt
22:19 whytewolf but let me duble check if it does use the -t option anywhere in the code and it just isn't listed
22:19 OliverUK joined #salt
22:20 OliverUK joined #salt
22:20 whytewolf nope. no repo info in the code
22:21 OliverUK joined #salt
22:21 scottk_ bummer. i'm trying to apt-get upgrade debian based systems, but i only want to upgrade packages that have security updates.
22:23 OliverUK joined #salt
22:23 scottk_ right now i'm doing: - name: apt-get update && apt-get upgrade -o Dir::Etc::sourcelist=/etc/apt/security.list -y
22:23 scottk_ any ideas on how to make that better?
22:23 scottk_ that's from a cmd.run
22:25 spartakos joined #salt
22:26 whytewolf sorry, that is about as good as it gets right now. although if you know python you could start submitting PR's to the aptpkg and pkg state modules to have been apt support
22:26 whytewolf s/been/better
22:45 sh123124213 joined #salt
22:47 nicksloan joined #salt
22:50 mikecmpbll joined #salt
23:07 nicksloan joined #salt
23:58 nicksloan joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary