Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2016-12-29

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:14 sh123124213 joined #salt
00:19 sh123124213 joined #salt
00:25 jmastron joined #salt
00:30 raspado joined #salt
00:30 raspado i have a file that is managed by salt, there is jinja vars in there, is there anyway i can get the file managed to add the proper vars
00:31 raspado instead of it copying the jinja template to the minions
00:31 jmastron All rendering happens on the minion side, so not really.
00:32 jmastron Are you concerned with performance?
00:34 raspado no, its just not rendering jinja
00:34 raspado its copying the jinja format to the minions when it should be a config file
00:34 raspado does the file.managed module not support jinja in files?
00:35 jmastron Do you have '- template: jinja' in the file.managed?
00:35 raspado oooo
00:35 PatrolDoom heh
00:35 XenophonF joined #salt
00:35 PatrolDoom raspado: yeah you have to remember the nuances ;)
00:35 PatrolDoom i was doing something backwards the other day
00:35 raspado ahhh crap thx jmastron
00:35 gnord joined #salt
00:36 raspado yup, that did the trick
00:38 jmastron nice!  you can pass extra values in with the defaults and context options too
00:39 madboxs joined #salt
00:46 uu joined #salt
00:49 jmastron Anyone good with saltenv?  Not sure what i'm trying to do is possible.  Basically I want my minions to all have a set environment they pull from (dev/test/prod) but I want to be able to dry run or apply the *same* formula from a different environment on demand.  The more I dig into salt environments the more they seem to be purely additive.  Is there something I'm missing on the practical side?
00:50 gnord joined #salt
00:50 madboxs joined #salt
00:51 hwtt joined #salt
01:00 madboxs joined #salt
01:07 madboxs joined #salt
01:09 oaklndr1 joined #salt
01:14 Edgan jmastron: Most of the wise people here will tell you don't bother with salt environments, and setup a salt master per environment.
01:14 DEger joined #salt
01:18 jmastron That's what I was afraid of.  Unfortunately the environments I deal with can be pretty fuzzy and it would be nice to have some other options.
01:19 PatrolDoom oh really
01:19 PatrolDoom dang
01:19 PatrolDoom i was hoping to do salt envs
01:19 PatrolDoom a master per env is a lot of overhead
01:21 jmastron Especially when you have to deal with geographic complexity on top of lifecycle.
01:21 PatrolDoom yeah
01:21 Edgan jmastron: I have two AWS accounts, and one salt master per region. Each AWS account has it's own git repositories for formulas and pillars. The development formulas is a fork of production formulas. The development and production pillars are independent git repositories.
01:22 madboxs_ joined #salt
01:23 Edgan jmastron: We also have dev and master branches in the development formulas and pillars. We merge from dev to master, and then master development to master production via a pull request.
01:24 Edgan jmastron: Salt formulas for code that was written in house lives in that code's git repository.
01:24 DEger joined #salt
01:24 madboxs_ joined #salt
01:25 Edgan jmastron: We use symlinks in the main formula repository to the directory in the code git repository.
01:25 dyasny_ joined #salt
01:25 Edgan jmastron: We test new code via salt-ssh, pre-commit.
01:27 jmastron That's probably the direction I'm heading.  I have to deal with a lot of "dev in prod" though, so I was wondering if there was a way to simplify and make develop available everywhere.
01:27 jmastron I like the salt-ssh idea, I'll have to try that.
01:31 shoemonkey joined #salt
01:52 hwtt joined #salt
01:55 DEger joined #salt
02:03 DEger joined #salt
02:09 catpigger joined #salt
02:09 udiabon left #salt
02:17 nZac joined #salt
02:31 sh123124213 joined #salt
02:32 raspado joined #salt
02:40 sebastian-w_ joined #salt
02:48 ilbot3 joined #salt
02:48 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2016.3.4, 2016.11.1 <+> Support: https://www.saltstack.com/support/ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ (please don't multiline paste into channel) <+> See also: #salt-devel, #salt-offtopic <+> Ask with patience as we are volunteers and may not have immediate answers
02:53 hwtt joined #salt
03:28 madboxs joined #salt
03:45 bastiandg joined #salt
03:50 madboxs joined #salt
03:50 evle joined #salt
04:03 madboxs joined #salt
04:13 darkpixel joined #salt
04:14 TylerJWhit joined #salt
04:15 TylerJWhit Got an issue here with salt-cloud and digital ocean. Trying to spin up a lemp server and I'm getting this error https://gist.github.com/TylerJWhit/5f348282159157ec8a09a58d6eada81b
04:18 zifnab joined #salt
04:19 DEger joined #salt
04:26 TylerJWhit Anyone see my post?
04:28 shoemonkey joined #salt
04:36 llua yep
04:37 TylerJWhit Thanks for responding
04:49 rdas joined #salt
04:55 hwtt joined #salt
05:16 TylerJWhit Repeat of an earlier message Got an issue here with salt-cloud and digital ocean. Trying to spin up a lemp server and I'm getting this error https://gist.github.com/TylerJWhit/5f348282159157ec8a09a58d6eada81b
05:20 PatrolDoom TylerJWhit: idk can you connect to the hosts mentioned
05:21 TylerJWhit Yes
05:21 TylerJWhit SSH works fine.
05:22 PatrolDoom lots of errors about not being able to connect
05:22 TylerJWhit Using a standard ubuntu image I have no problem. Lemp has an issue though. Haven't tried any other images
05:22 PatrolDoom indeed
05:30 TylerJWhit Yes, but it does eventually connect. Looks like it's rebooting
05:30 TylerJWhit Maybe that's expected behavior with that stack and salt hangs
05:31 TylerJWhit Is there a way create a wait time before continuing the bootstrap?
05:48 sh123124213 joined #salt
05:51 samodid joined #salt
05:55 bocaneri joined #salt
05:55 hwtt joined #salt
06:00 sh123124213 joined #salt
06:04 sh123124213 joined #salt
06:16 sh123124213 joined #salt
06:20 DEger joined #salt
06:28 mpanetta_ joined #salt
06:28 Xenophon1 joined #salt
06:33 sh123124213 joined #salt
06:38 DEger joined #salt
06:38 DEger joined #salt
06:50 ivanjaros3916 joined #salt
06:56 hwtt joined #salt
07:05 whytewolf humm came in late to that one... but looking at his error. looks like it isn't a problem at first. those connection errors at the start are normal. that is just salt trying to get in while the system is still coming up. the actually issue is that it DOES connect. and then the system reboots on it causing that set of connections to die.
07:06 whytewolf it would actually be better if ssh didn't come up before that reboot
07:14 flyboy joined #salt
07:21 icebal joined #salt
07:33 PatrolDoom joined #salt
07:50 sh123124213 joined #salt
07:56 preludedrew joined #salt
07:57 hwtt joined #salt
08:03 PatrolDoom joined #salt
08:08 madboxs_ joined #salt
08:13 fracklen joined #salt
08:35 armguy joined #salt
08:58 hwtt joined #salt
09:02 darioleidi joined #salt
09:05 madboxs joined #salt
09:08 cyborg-one joined #salt
09:11 sh123124213 joined #salt
09:33 __JZ__ joined #salt
09:53 J0hnSteel joined #salt
09:59 netcho_ joined #salt
10:01 madboxs_ joined #salt
10:05 aarontc joined #salt
10:07 sh123124213 joined #salt
10:22 JPau1 joined #salt
10:23 Ryan_Lane_ joined #salt
10:24 hillna_ joined #salt
10:24 phtes_ joined #salt
10:24 lkannan_ joined #salt
10:24 J0hnSteel joined #salt
10:24 kutenai_ joined #salt
10:24 Antiarc_ joined #salt
10:25 mrueg_ joined #salt
10:27 mmidgett joined #salt
10:29 dijit- joined #salt
10:31 demize joined #salt
10:31 windowsrefund joined #salt
10:31 stupidnic joined #salt
10:33 cebreidian joined #salt
10:33 kidneb joined #salt
10:36 mswart joined #salt
10:36 FreeSpencer joined #salt
10:36 nerdsville joined #salt
10:36 darix joined #salt
10:36 rai_ joined #salt
10:36 pmcg joined #salt
10:36 wm-bot4 joined #salt
10:36 coldbrew- joined #salt
10:36 Nebraskka joined #salt
10:36 Mads[m]1 joined #salt
10:36 jcl[m] joined #salt
10:36 Deliant joined #salt
10:36 monokrome joined #salt
10:36 inire joined #salt
10:36 Flying_Panda joined #salt
10:36 hax404 joined #salt
10:36 HRH_H_Crab joined #salt
10:36 khorben_ joined #salt
10:36 giany joined #salt
10:36 iggy joined #salt
10:36 jerrcs joined #salt
10:36 capn-morgan joined #salt
10:36 nineteen joined #salt
10:36 schinken joined #salt
10:36 chron0 joined #salt
10:36 hexa- joined #salt
10:36 robinsmidsrod joined #salt
10:36 cb joined #salt
10:36 Xevian joined #salt
10:36 Ashald joined #salt
10:36 esc\ joined #salt
10:36 pocketprotector joined #salt
10:36 sh123124213 joined #salt
10:38 bigjazzsound joined #salt
10:38 nZac joined #salt
10:38 hexa- joined #salt
10:39 monokrome joined #salt
10:40 honestly AndreasLutro (or anyone): Are custom execution modules supposed to be working at all with salt-ssh?
10:41 madboxs joined #salt
10:41 AndreasLutro yes
10:41 honestly are they?
10:41 AndreasLutro they have been for me
10:42 honestly I made a file _modules/userdata.py with a def get_userdata(uid) in it
10:42 haam3r joined #salt
10:42 honestly and then "salt-ssh localhost userdata.get_userdata foobar"
10:43 honestly and all I get is "'userdata.get_userdata' is not available."
10:43 honestly I think it looks at my file, because [DEBUG   ] In saltenv 'base', looking at rel_path '_modules/userdata.py' to resolve 'salt://_modules/userdata.py'
10:44 honestly (the _modules dir is in my "states" dir which is in file_roots)
10:44 AndreasLutro run `python _modules/userdata.py` and check for syntax errors
10:44 honestly oh :|
10:44 honestly well thanks
10:46 honestly ok, now how do I get debug output out of the module execution? I made a logger (logger = logging.getLogger(__name__)) but the log messages aren't showing up
10:48 honestly ok, the log is on the minion...
10:53 honestly great, it works, but I completely forgot I want to execute this on the master, not on the minion...
10:53 uu joined #salt
10:53 honestly :)
10:54 eseyman joined #salt
10:58 honestly let's see if ext_pillar works!
10:58 mpanetta_ joined #salt
10:59 hwtt joined #salt
11:04 madboxs joined #salt
11:06 DEger joined #salt
11:10 OliverX2 joined #salt
11:25 madboxs joined #salt
11:30 irctc523 joined #salt
11:31 irctc523 hi i am new to saltstack ,i want to setup saltstack gui saltpad on my local machine can anyone suggest me a detail tutorial for it
11:32 sh123124213 joined #salt
11:36 sh123124213 joined #salt
11:38 irctc523 ??\
11:41 sh123124213 joined #salt
11:44 abednarik joined #salt
11:46 madboxs joined #salt
11:51 sh123124213 joined #salt
11:51 muuse joined #salt
11:58 Trauma joined #salt
12:00 hwtt joined #salt
12:07 madboxs joined #salt
12:15 darvon joined #salt
12:16 Trauma_ joined #salt
12:21 m0nky joined #salt
12:21 antonw joined #salt
12:21 munhitsu_ joined #salt
12:21 copelco joined #salt
12:24 netcho_ joined #salt
12:28 madboxs joined #salt
12:28 fracklen joined #salt
12:39 nZac joined #salt
12:45 madboxs joined #salt
12:47 haam3r hi! Does anybody have any experience in moving minions from one master to another existing master?
12:53 the_lalelu joined #salt
13:01 hwtt joined #salt
13:03 Trauma joined #salt
13:03 rojem joined #salt
13:06 madboxs joined #salt
13:06 nahkiss Any idea why I am getting "error: [Errno 22] Invalid argument" when trying to file.append text to a file?
13:11 Neighbour haam3r: I don't have experience with that, but afaik it's a matter of adjusting the minion's configfile (which references the master) and accepting the minion's keys in the new master (and optionally deleting them from the old master)
13:18 jonher joined #salt
13:18 hwtt joined #salt
13:22 mpanett__ joined #salt
13:22 haam3r should be...just thought if there might be any gotchas to it
13:22 Trauma_ joined #salt
13:27 honestly AndreasLutro: neato, ext_pillar works too
13:27 madboxs joined #salt
13:27 nZac joined #salt
13:31 JohnnyRun joined #salt
13:31 ivanjaros joined #salt
13:37 netcho_ joined #salt
13:47 haam3r does anybody have an example for debian_ip.build_routes ... I can't for the life of me figure out the correct syntax?
13:52 nZac joined #salt
13:53 XenophonF joined #salt
14:03 Trauma joined #salt
14:06 anotherzero joined #salt
14:07 yawniek when i store a pubkey in a pillar how do i need to put it, getting rendering fails
14:08 madboxs joined #salt
14:08 yawniek i'm using  a:\n\tb: |\n\t\tMYDATA\nMYDATACONT
14:09 AndreasLutro use 2 spaces instead of tabs
14:10 yawniek yeah i have that actually
14:10 yawniek still fails
14:10 AndreasLutro put the sls on a pastebin or something
14:10 yawniek whats the rule after the pipe
14:11 yawniek https://gist.github.com/yannick/2fc8d803046e421682c32619f0d425e8
14:11 AndreasLutro well you've mixed tabs and spaces in your indentation
14:12 yawniek just realized, my editor screwed me over
14:13 yawniek still doesnt work though
14:13 _Cyclone_ joined #salt
14:13 AndreasLutro update your gist with the fixed sls
14:16 yawniek got it made
14:17 yawniek me against the editor
14:17 mirko can i somehow force salt-ssh using only ipv4, as i can do with ssh by passing '-4'?
14:19 hwtt joined #salt
14:23 jonher @haam3r To me it looks like it expects a pillar to be set. I found this when searching on the googlez https://fossies.org/linux/salt/salt/templates/debian_ip/route_eth.jinja
14:25 sh123124213 joined #salt
14:29 haam3r jonher: thanks..I'll take a look
14:29 madboxs joined #salt
14:30 jonher @haam3r But I believe the documentation is lacking, if you find out how to do it, it would be nice to add that info to the docs :)
14:30 haam3r mhm...will do
14:33 mpanetta_ joined #salt
14:41 abednarik joined #salt
14:41 catpiggest joined #salt
14:44 edrocks joined #salt
14:48 nZac joined #salt
14:50 madboxs joined #salt
14:51 netcho_ joined #salt
14:51 dyasny joined #salt
14:54 nZac joined #salt
15:06 nicksloan joined #salt
15:10 bbradley joined #salt
15:11 madboxs joined #salt
15:11 dlloyd joined #salt
15:11 hexa- joined #salt
15:15 sh123124213 joined #salt
15:17 Trauma joined #salt
15:18 fracklen joined #salt
15:19 edrocks_ joined #salt
15:20 edrocks joined #salt
15:21 johnkeates joined #salt
15:23 pipps joined #salt
15:23 edrocks__ joined #salt
15:23 pipps joined #salt
15:25 edrocks joined #salt
15:29 sarcasticadmin joined #salt
15:31 madboxs joined #salt
15:46 oida joined #salt
15:48 sh123124213 joined #salt
15:50 keltim joined #salt
15:51 abednarik joined #salt
15:52 madboxs joined #salt
15:56 johnkeates I'm getting SLS render errors on a perfectly fine State :( failed: Jinja error: get() got an unexpected keyword argument 'merge'
15:57 hwtt joined #salt
15:59 bbradley joined #salt
16:00 sh123124213 joined #salt
16:04 nZac joined #salt
16:09 nZac_ joined #salt
16:13 madboxs joined #salt
16:18 tiwula joined #salt
16:27 sagerdearia joined #salt
16:28 SamYaple is there a way to force a mine.update from within a state?
16:28 snc joined #salt
16:30 abednarik joined #salt
16:34 madboxs joined #salt
16:39 johnkeates joined #salt
16:39 tharkun joined #salt
16:40 dyasny joined #salt
16:48 CrummyGummy joined #salt
16:50 Trauma joined #salt
16:55 madboxs joined #salt
16:55 pcdummy gtmanfred: thanks for looking into this: https://github.com/saltstack/salt/issues/38440
16:55 saltstackbot [#38440][OPEN] Pillar is not overwriting lists, with pillar_merge_lists: False | Description of Issue/Question...
16:55 pcdummy gtmanfred: i double check
16:58 abednarik joined #salt
16:58 TyrfingMjolnir joined #salt
17:01 sarcasticadmin joined #salt
17:05 sh123124213 joined #salt
17:06 cyborg-one joined #salt
17:06 cyteen joined #salt
17:10 mpanetta_ joined #salt
17:10 windowsrefund joined #salt
17:10 whytewolf SamYaple: in thoery you could use mine.update in a module.run state
17:12 abednarik joined #salt
17:17 theblazehen joined #salt
17:20 cyraxjoe joined #salt
17:24 Praematura joined #salt
17:26 pcdummy whytewolf: you have git pillar in use?
17:26 whytewolf pcdummy: I do
17:26 pcdummy whytewolf: you have multiple git repos for a single env?
17:26 whytewolf a single one
17:26 pcdummy ok, ta
17:27 pcdummy I'm experiencing a merge bug with multiple repos, you got some time to test this with a single repo: https://github.com/saltstack/salt/issues/38440#issuecomment-269512899 ?
17:27 saltstackbot [#38440][OPEN] Pillar is not overwriting lists, with pillar_merge_lists: False | Description of Issue/Question...
17:27 pcdummy Please :)
17:28 pcdummy Just want to see if I'm allone with that one.
17:30 edrocks_ joined #salt
17:31 madboxs joined #salt
17:32 TyrfingMjolnir joined #salt
17:35 pcdummy I can confirm that one with a single gitfs pillar.
17:35 shbst joined #salt
17:36 edrocks joined #salt
17:36 whytewolf pcdummy: yes i see the issue on my system with git_pillar
17:36 pcdummy whytewolf: thank you
17:43 DEger joined #salt
17:44 whytewolf humm, looking at the code i don't get why it is happening.
17:45 bowhunter joined #salt
17:46 pcdummy whytewolf: i also look at the code.
17:46 pcdummy whytewolf: will find a fix.
17:46 pcdummy whytewolf: must be something salt.utils.dictupdate
17:47 pipps joined #salt
17:47 madboxs joined #salt
17:48 whytewolf if it was in dictupdate ... then everything would have problems with the merge.
17:50 cyraxjoe joined #salt
17:51 gtmanfred whytewolf: any idea if it occurs with any other external pillars?
17:52 * whytewolf shrugs. I don't use other external pillars
17:52 gtmanfred ok
17:53 toastedpenguin joined #salt
17:55 gtmanfred yall don't have anything set for pillar_source_merging_strategy right?
17:56 pcdummy gtmanfred: yes
17:56 pcdummy gtmanfred: i tested with "recurse" but doesn't help at all
17:57 whytewolf i might. but if so it would be smart
17:57 whytewolf which would be the default
17:57 gtmanfred yeah
17:58 gtmanfred which should also cause it to run through the merge_recurse function and not merge_lists
17:58 whytewolf gota love that a bootstrap master has all of the options set to the defaults ...
17:59 hwtt joined #salt
18:00 pcdummy Is there a pillar cache i can clear?
18:01 pcdummy I don't get this message at all in "salt-master -l debug" https://github.com/saltstack/salt/blob/develop/salt/pillar/git_pillar.py#L381
18:01 gtmanfred salt \* saltutil.pillar_refresh
18:03 madboxs joined #salt
18:04 r0mr0m joined #salt
18:06 r0mr0m Hi. Is there a configuration which allows a minion to execute an sls file upon joining the master automaitcally?
18:06 gtmanfred yes
18:06 gtmanfred r0mr0m: https://docs.saltstack.com/en/latest/ref/states/startup.html
18:06 jonher joined #salt
18:07 r0mr0m Great!!! and it pulls the configuration from the master, correct?
18:07 r0mr0m or I have to prepopulate it manually?
18:08 r0mr0m into the minion server
18:09 r0mr0m @gtmanfred
18:09 gtmanfred i believe you have to set those settings in the minion config
18:09 gtmanfred but if you use salt-cloud, you could just set it in the minion: dictionary of the cloud provider or /etc/salt/cloud
18:10 r0mr0m I have to keep the master internal to my cluster
18:13 mpanetta_ joined #salt
18:14 r0mr0m can u point me to that config please?
18:14 justanotheruser joined #salt
18:16 _JZ_ joined #salt
18:17 DEger joined #salt
18:18 madboxs joined #salt
18:19 bob_twinkles Is there a good way to have Salt manage temporary files?
18:19 gtmanfred r0mr0m: what config?
18:20 bob_twinkles "temporary" in the sense that they should only exist while performing a highstate
18:20 gtmanfred bob_twinkles: do a file.managed at the beginning of the highstate, and a file.absent at the end... I am not aware of another method
18:20 bob_twinkles darn =/
18:21 jonher joined #salt
18:21 bob_twinkles The two usecases I have are internally-developed Python wheels that need to be installed in to an app's virtualenv and config files that are are only needed to run management tools
18:21 bob_twinkles I suppose there isn't much harm in just leaving them lying around
18:26 jonher joined #salt
18:27 jonher joined #salt
18:28 r0mr0m gtmanfred: I want to set a state file in master for specific minion that once it connected, it pulls the sls file and highstate it
18:29 jonher r0mr0m: I can't see the history, but sound like a usecase for pillars?
18:30 r0mr0m jonher even for pillars use, someone has to execute the highstate command
18:30 r0mr0m I wander whether I can somehow configure that it works automatically
18:31 jonher r0mr0m: I think pillars + include, require or similar woulb be sufficiant. Even a "salt-call state.highstate" would work :)
18:31 gtmanfred so, you have to set startup_states in the minion config
18:32 r0mr0m jonher: I dont want to execute anything (salt-call)
18:32 r0mr0m I want everything to happen automatically.
18:32 r0mr0m minion added, pulls it config and executes it
18:33 gtmanfred use startup_states
18:34 gtmanfred https://docs.saltstack.com/en/latest/ref/states/startup.html , put it in the minion config
18:34 r0mr0m thanks gtmanfred.
18:34 madboxs joined #salt
18:34 Edgan joined #salt
18:49 oida joined #salt
18:50 madboxs joined #salt
18:53 edrocks joined #salt
18:59 pcdummy gtmanfred: that bug, does only happen with multiple top.sls files
18:59 hwtt joined #salt
19:00 pcdummy gtmanfred: I think it happens for all pillars that merge multiple top.sls files.
19:00 pcdummy what do you think?
19:01 edrocks_ joined #salt
19:06 madboxs joined #salt
19:06 Edgan pcdummy: can you describe the problem? I think I have had a like problem before.
19:07 pcdummy Edgan: https://github.com/saltstack/salt/issues/38440#issuecomment-269512899
19:07 saltstackbot [#38440][OPEN] Pillar is not overwriting lists, with pillar_merge_lists: False | Description of Issue/Question...
19:09 Edgan pcdummy: The way to do what you want is a map.jinja with merges at the end. You set a default in the map.jinja, and then you use pillars to override defaults. You don't use pillars to override pillars.
19:09 whytewolf pcdummy: multiple top files? I only have one top file in my setup and no enviroments.
19:09 pcdummy whytewolf: now I'm unable to reproduce that bug with single top.sls :/
19:10 johnkeates joined #salt
19:10 pcdummy whytewolf: in the same env i mean.
19:10 whytewolf what did you change?
19:11 whytewolf i only have 1 env. and only 1 top file in that env
19:11 Edgan pcdummy: a very simple example map.jinja, https://paste.fedoraproject.org/514820/30386681/
19:12 pcdummy Edgan: Thanks, but I'm not interested in a workaround, I want to fix that bug.
19:12 pcdummy whytewolf: not sure what I'm doing, not sure why i can't reproduce it with a single top.sls anymore.
19:12 Edgan pcdummy: I don't consider it a bug, but you using it wrong. I don't consider mine way the workaround, but the way to do what you are trying to do.
19:13 whytewolf Edgan: it is a bug. it is not operating in the correct manner. or a consitent manner. it works correctly in pillars outside of git_pillar.
19:13 pcdummy Edgan: not sure we talk about the same thing.
19:14 mpanetta_ joined #salt
19:14 Edgan pcdummy: https://github.com/saltstack/salt/issues/38440  the first comment is a method of using pillars that I wouldn't expect to work guaranteed consistency
19:14 saltstackbot [#38440][OPEN] Pillar is not overwriting lists, with pillar_merge_lists: False | Description of Issue/Question...
19:16 whytewolf acutally since the top.sls should be read top to bottom. and over right is the default. yes it should be consitent.
19:16 pcdummy Edgan: the first comment?
19:16 Edgan whytewolf: Assuming it is an overwrite and not a merge
19:16 pcdummy Edgan: yes, cause "pillar_merge_lists" is off
19:17 whytewolf Edgan: pillar_merge_lists defaults to False so yes it should be an overright
19:17 Edgan pcdummy: You don't say that in the first comment
19:17 pcdummy Edgan: in the title i do
19:17 pcdummy Edgan: and its the default
19:19 PatrolDoom joined #salt
19:20 fleaz1 joined #salt
19:21 Edgan pcdummy: When you say multiple top.sls files, do you mean salt environments?
19:21 pcdummy Edgan: no, top.sls in multiple git pillars
19:21 madboxs joined #salt
19:22 pcdummy Edgan: but I do something wrong here, as others confirmed it.
19:22 Edgan pcdummy: That sounds like the equivalent of salt environments, in that normally you only have one top.sls.
19:22 fleaz1 joined #salt
19:22 Edgan pcdummy: Are you giving it multiple git repositories for pillars and expecting them to merge?
19:23 pcdummy yes
19:23 PatrolDoom now thats something i was curious about
19:23 pcdummy Edgan: but as said others confirmed that it happens with a single git_pillar top.sls
19:23 PatrolDoom e.g. use consul for pragmatic pillars, git for gpg secrets
19:23 Edgan pcdummy: My experience with multiple branches and salt envs trying to merge top.sls is that you get unexpected results.
19:23 pcdummy PatrolDoom: i have 2 git pillars, one "base" and one per "customer" :)
19:23 PatrolDoom dang really
19:24 PatrolDoom Edgan: as in a "prod", "dev", "stage" method or how so?
19:24 Edgan PatrolDoom: consul, I understand why you say this, but not a fan
19:24 PatrolDoom yeah me either tbh
19:24 haam3r joined #salt
19:24 PatrolDoom but i need a simpler way to manage pillars, (havent used consul but thinking about it)
19:24 whytewolf Edgan: thing is he is doing it in a single enviroment. but with multiple repos in that 1 enviroment. the top files defintly merge correctly. but order is kind of a toss up in that instance
19:25 whytewolf but with a single top.sls i am seeing the same behavour. which defintly should not be happening
19:26 Edgan PatrolDoom: It was if you use gitfs you get saltenvs and it reads all branches, including feature branches as saltenvs. It then tries to merge all the top.sls files. I didn't notice this till I broke my top.sls files into pieces using includes and then got weird behavior, because of the merging across branches.
19:26 PatrolDoom oooo snaps
19:26 PatrolDoom i have a lot of "feature" branches... i use them to manage my mista... progress
19:26 Edgan PatrolDoom: I fixed it by locking down the branches to read
19:27 DEger joined #salt
19:28 Edgan PatrolDoom: iggy and others here activate not using saltenvs
19:28 whytewolf I am one of those against saltenvs. :P
19:28 PatrolDoom indeed, hrm - def. going to have to figure something out then
19:29 PatrolDoom we were hoping to be able to manage each env via single salt implementation
19:29 * PatrolDoom dies a little inside
19:29 Edgan PatrolDoom: The recommended method is one salt master per env, and I do one per region too
19:30 PatrolDoom awe man thats a lot of overhead
19:30 PatrolDoom i'd have what, 12 salt masters?
19:31 * PatrolDoom prod/stage/dev  then each of my regions :'|
19:31 pcdummy whytewolf: why not multiple envs?
19:31 pcdummy whytewolf: to much things to consider?
19:31 whytewolf envs = pain and suffering the likes you have only guessed at. you think you have merging problems now?
19:32 pcdummy So i makes things complicated, more bugs... :)
19:32 Edgan PatrolDoom: Technically I am doing it by account/region. Development is one account, and we have two regions. Production is another account and we have one region. So I have three salt masters.
19:33 PatrolDoom hmm i mean... technically i don't see it being a issue for the most part
19:33 Edgan PatrolDoom: We also use salt-ssh for deploys out of jenkins, which bypasses the salt master for a lot of stuff.
19:33 pcdummy PatrolDoom: you can't have hostnames like "host.env.region.yournetwork.com" ?
19:33 PatrolDoom i mean it would be great to be able to /not/ break prod salt
19:33 PatrolDoom pcdummy: thats what i'm planning on implementing, (they did a really backasswards scheme beforehand)
19:33 PatrolDoom Edgan: ok so you give me hope
19:34 PatrolDoom we're needing to do a lot more w/ jenkins
19:34 PatrolDoom been working on cleaning everything up
19:34 whytewolf pcdummy: since saltenvs are not really a hard defined limit. you run into issues with things across envs that conflict in name. as well as people forgetting that base is a default enviroment. not knowing how to target and end up with instances that pull from multiple enviroments
19:34 PatrolDoom man i praise you guys that can do this so simply
19:34 Edgan PatrolDoom: my scheme is component-cluster-role-host_id.env.region.provider.root_domain
19:35 Edgan PatrolDoom: jenkins-foo-master-01.inf.us-west-1.aws.acme.com
19:35 PatrolDoom ah nice
19:35 * PatrolDoom takes notes
19:35 Edgan PatrolDoom: Foo being the equivalent of the pet names, like bart.acme.com
19:36 PatrolDoom indeed the public facing name or w/e
19:36 stanchan joined #salt
19:36 Edgan no, just a unique name to tell things apart
19:36 Edgan public facing is a whole other story
19:36 whytewolf pcdummy: by splitting the env's into seperate masters you eliminate most of the problems by defining a hard split in enviroments
19:36 PatrolDoom ah yeah correct
19:36 PatrolDoom i see
19:36 Edgan but the rest of the name follows a programmatic style which fits very well with salt top files
19:36 pcdummy whytewolf: I understand, and will do the same
19:37 madboxs joined #salt
19:38 pcdummy To be true, currently i have one saltmaster for dev. and another for prod, but not cause of envs it was just easier to setup.
19:38 pcdummy One saltmaster gets the develop branch the other the master.
19:44 Edgan We have formulas-development with dev and master. The formulas-development is a fork of the formulas-production git repository. The formulas-production only has a master branch. We use salt-ssh to test pre-commit. Things get commited to dev. The dev branch gets tested with salt-ssh running deploys from jenkins. Then dev gets merged to master. Then we pull request from formulas-development to formulas-production.
19:45 PatrolDoom ah nice
19:45 PatrolDoom thats what i'd like to do
19:45 Edgan For pillars we have pillars-development with dev and master. We have pillars-production with just master. We merge from dev to master, but development and production are independent.
19:46 pcdummy Edgan: salt-ssh do you have vagrant/docker for that?
19:46 Edgan pcdummy: EC2 instances
19:46 abednarik joined #salt
19:47 Edgan pcdummy: docker is worthless for things need something like systemd, unless you use a special image that has systemd hacked in
19:47 Edgan pcdummy: LXD could be more useful for that
19:47 pcdummy I love LXD a lot :)
19:47 pcdummy Doing a lot with it.
19:48 Edgan pcdummy: We use vagrant to isolate development and production VPNs from each other on the same laptop
19:48 pcdummy Still i understand to the pre-commit thing, how do you test a single formula with pre-commit?
19:48 Edgan pcdummy: you can do state.sls
19:48 Edgan pcdummy: but we generally highstate to make sure everything is happy together
19:49 pcdummy So you have hosts for each formula and highstate it?
19:50 Edgan pcdummy: We are developing a lot of in house code, and the code is tied to hardware.
19:50 Edgan pcdummy: So we could use vagrant, but then if someone shuts down their laptop we can't collect the data from the hardware
19:50 Edgan pcdummy: So we have all-in-one instances for dev/testing in EC2
19:51 Edgan pcdummy: I can also spin up three of something for a cluster of some service like rabbit, redis, cassandra, etc
19:51 pcdummy Edgan: ok, when you test one formula you test a bunch, right?
19:52 pcdummy Edgan: i mean you test states, not a single formula.
19:52 Edgan pcdummy: yes, and we have a flow of deployments that it goes through before production
19:52 pcdummy kk
19:52 Edgan dev/ci -> qa -> staging -> preprod -> prod
19:53 pcdummy Edgan: and all highstate, the whole stuff, right?
19:54 Edgan pcdummy: in dev everything is on the same instance and anything that the code is directly dependent on is in salt-ssh. So rabbitmq, redis, service1, service2, service3, etc.
19:54 Edgan pcdummy: later in staging we break out rabbitmq and redis to their own instances and they switch to master mode.
19:55 Edgan pcdummy: master mode in dev is more user accounts, default packages, etc
19:56 edrocks joined #salt
19:57 pcdummy Edgan: thats why you need so much stages
19:57 pcdummy Edgan: wouldn't it be simpler to develop in many (with LXD perhaps)?
19:57 Edgan pcdummy: We nee so many because we deal with complex hardware
19:57 pcdummy Edgan: I have written a formual to spin up lxd instances
19:58 pcdummy It still needs some salt-cloud love
19:58 Edgan pcdummy: We are probably going to move to kubnetes for our code stuff.
19:58 Edgan pcdummy: already have my own python scripts that use boto to make instances, ALBs, and ELBs
19:58 Edgan pcdummy: more advanced than salt-cloud
19:59 Salander27_ joined #salt
19:59 PatrolDoom really
19:59 PatrolDoom dang
20:00 Edgan pcdummy: I have it adding route53 entries, and plan to add creating and attaching volumes.
20:00 hwtt joined #salt
20:00 Edgan you can do the same boto stuff through salt(not salt-cloud), but someone has to write salt to boto glue per boto function, and there were none for ALBs
20:01 Edgan pcdummy: but instead of dockerfiles(bash script like), we will probably continue to use Salt
20:01 pcdummy For me "state.sls -> spin up lxd -> copy script to lxd -> run script" works nice
20:02 pcdummy where run script installs salt
20:02 Edgan pcdummy: I considered it, and even switched from Fedora to Ubuntu to get LXD support, but ended up switching back to Fedora and continuing to use instances.
20:02 Edgan pcdummy: I have previously used vagrant.
20:03 Edgan pcdummy: Vagrant has too many vagrant-only-isms that make it a pain, and make it not equivalent to an instance.
20:03 Salander27Temp left #salt
20:03 pcdummy Its sad LXD is Ubuntu only
20:03 Edgan pcdummy: AWS also has many AWS-only-isms too, which makes anything else a 80/20 or 90/10 game.
20:03 Salander27 joined #salt
20:04 Edgan pcdummy: actually, people have been branching LXD out to other distributions
20:04 pcdummy Edgan: not realy, right?
20:05 madboxs joined #salt
20:05 * pcdummy should know that
20:05 pcdummy I'm an active LXD contributor/promoter/supporter/whatever.
20:08 Edgan pcdummy: https://linuxmonk.ch/wordpress/index.php/2016/getting-started-with-lxd-on-fedora/
20:09 Edgan pcdummy: a copr is like a ppa
20:09 pcdummy Edgan: wow, does it work?
20:09 Edgan pcdummy: haven't tried it
20:09 hwtt joined #salt
20:10 Edgan pcdummy: I had tried to compile LXD on Fedora a few months ago, and got fairly far without too much trouble. Looks like it finished it and packaged it, but works?
20:10 pcdummy I see big security risks without Apparmor and selinux disabled.
20:10 pcdummy Not sure this guy disabels selinux.
20:11 Edgan pcdummy: I see big security risks with containers in general. It doesn't stop developers. :(
20:11 pcdummy Edgan: thanks for that link, its great!
20:11 pcdummy Will give it a try
20:11 pcdummy Fedora LXD :)  :)
20:11 Edgan pcdummy: I see apparmor as a joke, and selinux(outside of it's use with containers) as near worthless.
20:11 * pcdummy is very happy
20:13 Edgan pcdummy: SELinux is another layer of security that is about as useful as ASLR, but includes a whole another layer of ACL. So not a lot of benefit, and a huge overhead.
20:16 Edgan pcdummy: security layers are good, it is how security works. We use ASLR, because even though it only provides a little benefit, the cost of using it is low enough that it is worth it.
20:17 madboxs joined #salt
20:25 writtenoff joined #salt
20:29 hwtt_ joined #salt
20:30 madboxs joined #salt
20:39 keimlink joined #salt
20:42 madboxs joined #salt
20:47 hwtt joined #salt
20:47 edrocks_ joined #salt
20:50 edrocks joined #salt
20:54 madboxs joined #salt
21:05 fracklen joined #salt
21:07 edrocks_ joined #salt
21:07 madboxs joined #salt
21:16 theblazehen71 joined #salt
21:22 ekristen joined #salt
21:27 jonher left #salt
21:27 jonher joined #salt
21:29 madboxs joined #salt
21:33 edrocks joined #salt
21:36 WesleyTech joined #salt
21:39 madboxs joined #salt
21:49 netcho joined #salt
21:50 madboxs joined #salt
21:58 pipps joined #salt
22:00 madboxs joined #salt
22:10 madboxs joined #salt
22:12 netcho joined #salt
22:14 _JZ_ joined #salt
22:19 catpiggest joined #salt
22:20 madboxs joined #salt
22:30 madboxs joined #salt
22:37 yetanotherzero joined #salt
22:40 madboxs joined #salt
22:42 morissette joined #salt
22:46 Praematura joined #salt
22:47 shoemonkey joined #salt
22:50 madboxs joined #salt
22:51 Praematura joined #salt
22:57 NeoXiD joined #salt
22:58 WesleyTech joined #salt
23:07 pipps joined #salt
23:10 madboxs joined #salt
23:17 jonher left #salt
23:18 sh123124213 joined #salt
23:20 abednarik joined #salt
23:20 madboxs joined #salt
23:25 KingJ joined #salt
23:31 madboxs joined #salt
23:36 hwtt joined #salt
23:41 madboxs joined #salt
23:45 citaret joined #salt
23:48 wangofett joined #salt
23:51 madboxs joined #salt
23:52 citaret What's the proper way to intall a package from source using salt?
23:54 rojem joined #salt
23:55 vodik citaret: don't know if there's a better way, but i've done this before (compile collectd from source):
23:57 vodik citaret: didn't really bother to clean it up: http://ix.io/1OEI - there's more stuff to push a rc.d init file  (i'm compiling from source on some really old boxes - for reasons) and the files (/etc/collectd.conf, etc) comes from another state
23:58 vodik oh, the service is there
23:59 whytewolf citaret: I would say something like using something like fpm to actually make your source into a package.
23:59 citaret vodik: Thanks.
23:59 vodik yeah, making a package is best

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary