Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2014-07-30

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 zandy joined #salt
00:00 Singularo joined #salt
00:01 rome joined #salt
00:03 kober joined #salt
00:04 kober Is there a way to depend on salt states?  For example could I say "WSGIApp" depends on "Python State"  and python state compiles and installs our custom python?
00:04 kober Or do I just write the python state installing stuff inside the wsgiapp state?
00:05 manfred kober: http://docs.saltstack.com/en/latest/ref/states/requisites.html
00:05 manfred that is the docs page you want
00:05 quanta_ joined #salt
00:05 manfred require the state that builds python in your wsgiapp state
00:07 kober Nice, just like require: pkg, there is require: sls :D
00:07 kober You are a gentleman and a scholar
00:07 manfred :)
00:18 kinetic_ joined #salt
00:18 rome joined #salt
00:26 Luke joined #salt
00:28 KyleG saltstack was added to the Palo Alto applications detection list
00:28 KyleG go salt
00:29 mosen yay
00:29 manfred link?
00:29 KyleG We have palo altos so I got an email
00:29 KyleG http://ss.digitalflydesigns.com/Screen%20Shot%202014-07-29%20at%205.30.03%20PM.png
00:30 manfred nice
00:31 rome joined #salt
00:31 Luke joined #salt
00:32 KineticSalt left #salt
00:32 KineticSalt joined #salt
00:33 rome joined #salt
00:34 rome_ joined #salt
00:36 rome joined #salt
00:41 robawt ping whiteinge
00:41 retrospek REJECTED
00:41 manfred lol
00:42 robawt hater retrospek :)
00:42 bezeee joined #salt
00:42 robawt retrospek doesn't know anything about external pillar soooo
00:42 retrospek whiteinge is robawt's first line supervisor
00:42 * robawt </end trolling retrospek>
00:42 retrospek i never troll
00:43 robawt i do
00:43 robawt digging around info about external pillars
00:43 retrospek neat. this process has a timestamp from 1927
00:43 robawt but it's not written in Perl so I dunno how useful you could be
00:43 robawt :P
00:43 retrospek says the guy that knows neither python nor perl
00:43 manfred retrospek: ... how...
00:43 retrospek quick write a script
00:43 robawt retrospek: you hurt you hurt
00:44 retrospek im no johnny cash but sure
00:44 * manfred learns that you can go negative epoch.... never go negative epoch
00:44 manfred date --date=@-1002045099
00:48 robawt retrospek: how are you Sir?
00:49 retrospek what backend are you trying for pillar?
00:51 robawt retrospek: using a little gitfs
00:53 badon_ joined #salt
00:56 gzcwnk im pre-epoch  :P
00:56 retrospek robawt: so what is broken?
00:57 robawt retrospek: nothing broken, looking to get fancy and need to see if i'm off base
00:57 retrospek well you are robawt so im going with default yes
00:57 robawt can I specify multiple types of extended pillar types
00:57 * retrospek runs away
00:57 robawt lol @ retrospek
00:59 TyrfingMjolnir joined #salt
00:59 robawt retrospek: ty :)
01:00 retrospek you mean mix and match gitfs+other pillars or just multiple remotes for gitfs?
01:01 retrospek http://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html (both are examples)
01:03 elfixit joined #salt
01:08 rome joined #salt
01:15 robawt retrospek: mix and match
01:19 bhosmer joined #salt
01:19 retrospek robawt: see the multiple backends?
01:20 retrospek 3.4.8.5
01:20 yomilk joined #salt
01:20 retrospek or do you mean like mixing gitfs+ldap or w/e?
01:21 robawt retrospek: ty for 3.4.8.5
01:21 robawt i don't think so, helping team mates
01:22 retrospek pfft. hindering is more fun
01:23 robawt cold blooded
01:29 zandy joined #salt
01:29 rome joined #salt
01:35 rome joined #salt
01:38 gzcwnk is there anyway on the minion to check it can talk to the salt master?
01:39 retrospek salt-call test.ping
01:39 retrospek salt-key -L to confirm key exchange happened
01:40 gzcwnk thanks, suspect its a firewall issue then
01:40 gzcwnk yeah that isnt happening
01:40 gzcwnk all i can think of is the firewall isnt open
01:41 gzcwnk can i telnet to a port?
01:43 jhauser_ joined #salt
01:45 gzcwnk does the minion talk to the salt master on 4505?
01:45 gzcwnk tcp?
01:45 gzcwnk hmm tcp, yes
01:45 robawt gzcwnk: 4505 and 4506 open on maser only
01:46 gzcwnk yeah im just trying to see if the company firewall is blocking the minion
01:49 gzcwnk not the firewall
01:51 gzcwnk for some reason the handshake isnt happening
01:51 retrospek yes you could telnet to test 3whs but not send anything useful
01:51 retrospek probably a packet filter between the two then (firewall)
01:52 gzcwnk the minion log is saying the master has cached?
01:52 retrospek depending on the message usually just means cached the credentials
01:52 gzcwnk but the master cant see it
01:52 gzcwnk says cached public key
01:52 retrospek yep
01:52 gzcwnk so taht should mean firewall is OK?
01:53 gzcwnk or not an issue?
01:53 retrospek it was ok at one point
01:53 retrospek or it wouldn't have that much
01:53 gzcwnk ok found it, duplicate dns entry
01:55 gzcwnk bugger the salt-master now seems confused
01:56 gzcwnk the master caches dns?
01:57 retrospek just clear the minion keys and restart the master to start over
01:58 robawt restart the minion, right?
01:58 robawt clear keys on master, restart minion, accept keys?
01:59 retrospek latter yes
01:59 retrospek there were issues related to minions changing dns but not keys at one point. dunno bout now
01:59 MatthewsFace joined #salt
02:00 gzcwnk its our DNS its a mess
02:00 TheThing joined #salt
02:03 gzcwnk looks like teh salt-master is dns caching
02:04 gzcwnk or something is....not the host
02:04 retrospek yes hence restarting it
02:04 retrospek minion is fine as the master key/dns isnt changing i assume
02:05 retrospek purge the keys on master, restart master and minion should reconnect
02:05 gzcwnk i keep doing taht and teh key is the wrong name
02:05 gzcwnk even restarting the master doesnt fix it
02:06 gzcwnk teh AD is flushed and a host lookup fails
02:07 gzcwnk the salt-master still sees teh old name even after a restart
02:07 retrospek that implies that the nameserver the master is using has the 'wrong' address in cache still
02:08 gzcwnk looks like teh issue is the minion
02:08 CeBe1 joined #salt
02:08 gzcwnk looks like I need to flush dns on the IPA masters
02:09 retrospek i thought it used to be only the minion lookup of the master was an issue. maybe they implemented reverse dns checking. *shrug*
02:09 xcbt joined #salt
02:10 retrospek you can run both -l debug and see exactly what is puking if you want
02:11 gzcwnk well the minion still thinks its something else via a dns lookup, ive edited /etc/hosts to see if taht fixes it
02:11 retrospek you can set the nodename in the minion config if you want
02:11 gzcwnk i nee dot
02:12 gzcwnk its still buggered
02:12 scoates joined #salt
02:12 gzcwnk where do i do that?
02:12 gzcwnk in /etc/salt/minion?
02:12 retrospek yea, id: whatever
02:13 retrospek http://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.html#minion-id-generation
02:13 TyrfingMjolnir joined #salt
02:14 gzcwnk yeah taht's fixed it, thanks
02:15 gzcwnk phew, i need this server done today....
02:22 otter768 joined #salt
02:23 gzcwnk dnt think the master is happy with something on the minion
02:25 gzcwnk it wont compile
02:26 oz_akan joined #salt
02:26 gzcwnk http://pastebin.com/3ck7LCvR
02:27 gzcwnk any idea on that failure msg?
02:28 jY you have another job running
02:28 jY ps aux | grep 2273
02:30 gzcwnk not as 2273
02:30 jY kill all the running salt minions
02:30 jY and re-try
02:33 gzcwnk cant see anything to kill
02:35 gzcwnk i cant even ping the minion
02:35 gzcwnk something is hung somewhere big time
02:36 gzcwnk rebooting the server
02:37 jY might have been a file in /var/cache/salt/minion/proc
02:37 jY matching the jid
02:40 gzcwnk well the master seems to be broken
02:42 gzcwnk where do i start looking to repair it?
02:43 jY stop the master
02:43 jY run in debug mode if you think its the master
02:43 vbabiy joined #salt
02:43 ml_1 joined #salt
02:43 gzcwnk there must be lock files on its disk
02:44 gzcwnk as they survive a reboot
02:44 jY i told you on the minion there is /var/cache/salt/minion/proc
02:44 gzcwnk im rinning debug but its just hanging
02:44 jY with jid files
02:44 jY no output?
02:45 gzcwnk im on the master it now cant run on other minions
02:45 gzcwnk either
02:46 jY you can try to delete everything in /var/cache/salt/master/jobs
02:46 gzcwnk this is as far as it gets, http://pastebin.com/2Bt9Fjif
02:47 aquinas joined #salt
02:48 jY did you delete all the jobs on the master then try to run it?
02:48 markhayden joined #salt
02:48 gzcwnk yep
02:48 gzcwnk still hangs in the same place
02:49 jY ohh
02:49 jY run master in debug
02:49 jY salt-master -l debug
02:57 Luke joined #salt
03:04 TyrfingMjolnir joined #salt
03:08 kermit joined #salt
03:17 kober joined #salt
03:18 kober Hey, wondering if it is possible to run salt without root.  It seems to want everything in /etc but our current process is with ansible and we are used to just cloning the config repo and then running the commands on the servers over ssh
03:19 kober Is salt-ssh well maintained?
03:34 kober I'm trying to run without root from a pip installed version of salt and its trying to get access outside the virtualenv
03:35 kober salt-ssh -c . '*' state.highstate   is trying to get to: OSError: [Errno 13] Permission denied: '/etc/salt/pki/master/ssh'
03:36 kober Can I tell salt to do its key management somewhere else?
03:46 kober I have tried salt-ssh -c etc/ '*' test.ping  as well as creating a Satlfile and putting the settings in there and it still tries to get /etc/salt/pki
03:47 kober Which leads me to believe these docs are invalid: http://docs.saltstack.com/en/latest/topics/ssh/#running-salt-ssh-as-non-root-user
03:48 manfred kober: salt-ssh is very new
03:48 kober I'm running salt==2014.1.7
03:48 manfred still new
03:48 manfred it was brand new in 2014.1
03:48 manfred it is better in 2014.7, including an ansible inventory roster
03:49 manfred have you looked at not salt-ssh
03:49 manfred ?
03:49 manfred it is so nice
03:49 manfred as for salt-ssh
03:49 kober manfred: I found that it came out in 0.17  (2013-9-26), so its almost a year old
03:49 manfred running as not root has been fixed in 2014.7 iirc
03:49 manfred it is still very new, cause most of the focus goes into zeromq
03:49 manfred and regular salt
03:50 manfred also the /latest documents come straight out of develop, so those may not be valid for 2014.1 branches
03:50 manfred i would need to check git
03:50 manfred where did you change your pki_dir and cachedir to point to?
03:50 hotbox joined #salt
03:51 kober Yeah, I totally plan on using salt in masterless mode but I need a way to get the states up to the server, which is where salt-ssh comes in
03:51 manfred where did you point your pki_dir and cachedir to for /etc/salt/master?
03:51 manfred those are usually the problem children cause they usually are only readable by root
03:51 manfred cause they contain all the information about the environment, as well as your salt encryption keys
03:51 kober I couldn't figure out what cli argument was needed for salt-ssh to change pki_dir
03:52 kober I changed it in my Saltfile to be ./pki
03:52 manfred iirc, it should be change in /etc/salt/master
03:52 kober I don';t have /etc/salt/master
03:52 kober I created a virtualenv and then pip installed salt
03:53 quanta_ left #salt
03:53 manfred try dropping assignment for those two variables to /etc/salt/master
03:54 manfred salt-ssh spins up a temporary salt-master instance, to pass the data through zeromq to the minion (which salt-ssh deploys a compiled binary that recieves those things on the minion... temporarily)
03:54 kober So the -c argument doesn't mean anything until after you've set stuff in /etc/salt/master?
03:54 kober Whats the Saltfile for then?
03:54 manfred oh
03:54 manfred i missed that
03:55 manfred i have no idea what Saltfile is
03:55 manfred sorry
03:55 kober Oh, I found in the docs if I had a file called `Saltfile` I could set configuration that I would normally pass on the command line
03:55 kober I don't have to use it if it doesn't work either
03:56 manfred i have no idea if it works, i usually just use salt-ssh using root
03:56 kober I'm starting to question some of these docs, are the docs from the sprint not up yet?  I want to make sure I'm reading the right stuff
03:56 manfred i think the saltfile is just used to configure the -c variable
03:56 manfred kober: the doc sprint didn't really cover salt-ssh
03:56 manfred we are still going through them all
03:56 kober Yeah, I can't get the -c to work either though
03:56 manfred and expanding and fixing the docs
03:57 manfred you shouldn't need -c if you put that config_dir in the SaltFile
03:57 manfred still need to have a $config_dir/salt/master file
03:57 manfred that configures pki_dir and cachedir
03:57 manfred i think
03:57 kober http://paste.ofcode.org/UnUq64D7t9Hj39XGVJ62Uk
03:58 manfred pki_dir and cachedir go in the master config
03:58 manfred inside $config_dir/salt/master
03:58 manfred not in the SaltFile
03:58 manfred that is what it looks like
03:58 manfred at least
03:59 manfred and i have no idea how it works using relative directories in the master config
04:00 kober oh, let me try that
04:00 mosen joined #salt
04:02 kober Like this? http://paste.ofcode.org/D2iA4tBFw2c4xYYFsLyMYD
04:02 manfred i would hope so
04:03 manfred either way, please make an issue about that, because the docs should be expanded
04:03 manfred and I will try to figure it out this weekend.
04:03 manfred or someone else will
04:03 kober It didn't help, still permission denied
04:04 rap424 joined #salt
04:04 manfred kober: can you make an issue request about it?
04:05 manfred http://github.com/saltstack/salt/issues
04:05 manfred and I will try and take a look tomorrow
04:05 ramteid joined #salt
04:05 gzcwnk i already have a request to expand docs
04:05 gzcwnk hehe
04:06 manfred gzcwnk: shush
04:06 manfred we have lots of requests
04:06 ckao joined #salt
04:06 manfred yours is silly because your state didn't even follow the example on that doc page
04:06 gzcwnk tahst because I find it hard to understand
04:06 gzcwnk i tried to follow it
04:07 thayne joined #salt
04:07 manfred you could have just copied that state
04:07 manfred and used it
04:07 gzcwnk hmmm
04:07 gzcwnk i had to change it
04:07 gzcwnk so i dont follow u
04:08 gzcwnk home time
04:08 gzcwnk :D
04:08 gzcwnk WoT here I come
04:08 andrej Œhow do I get a  custom grain into the mine?
04:08 gzcwnk catch you later
04:08 gzcwnk im not even goign to ask waht a mine is, LOL
04:09 manfred andrej: http://docs.saltstack.com/en/latest/topics/targeting/grains.html#writing-grains
04:09 manfred andrej: write that
04:09 markhayden joined #salt
04:09 manfred andrej: anything that is in __grains__ should go into the mine iirc
04:10 markhayden hey there. i am tryign to connect to an existing master that i was connected to in the past but am getting: [CRITICAL] The Salt Master has rejected this minion's public key!
04:10 andrej manfred: thanks, I'll have a read
04:10 markhayden i’ve already removed the bad key with salt-key -d <minion name>
04:10 markhayden but getting the same thing
04:11 andrej manfred : I read that, but it doesn't apply to my question (that, or I fail to see how)
04:12 andrej I did create a custom grain, and applied that ot 17 relevant machines
04:12 manfred andrej: if you write a custom grain, it gets run, and should add stuff to __grains__
04:12 manfred and that should go into the mine
04:12 manfred if not
04:12 manfred then i got nothing
04:12 andrej How do I make the machine stick that one grain in its mine?
04:12 andrej Oh ok
04:12 manfred andrej: you could try http://docs.saltstack.com/en/latest/topics/mine/#mine-functions
04:12 andrej I'll have a look
04:13 markhayden nm. i figured it out. just had to clear my minion keys locally and generate new ones.
04:13 manfred but i was under the impression that anything in __grains__ should get added to the mine
04:13 andrej manfred : that URL is the reason that I'm asking here :)
04:13 manfred yeah, that is the end of what I know :)
04:13 andrej the mine functions seem to pertain to built-in modules
04:13 manfred andrej: so
04:14 kober manfred: interesting enough I set a PDB and it takes the config directory properly and then returns self.config to be  'conf_file': '/etc/salt/master',
04:14 manfred andrej: if you write a custom grain, it should return data in minion.py iirc
04:15 andrej manfred: so, how would I put that in mine.conf in the proper parliance?
04:15 manfred just like the salt/grains/core.py
04:15 manfred every function in there should return something to be in the __grains__
04:15 manfred and then all of those should be in the mine
04:15 manfred you shouldn't need to from my understanding
04:16 kober Ok, looks like its just not good at giving you warnings about problems and falls back to defaults
04:16 manfred kober: got it working?
04:16 andrej Hmmm ... ok, I'll try some more (and harder :D)
04:16 kober I said salt-ssh -c ./etc   but put the master in ./etc/salt    so it didn't find a master
04:16 andrej thanks manfred
04:16 kober but now that the master is just in ./etc it finds it
04:16 manfred ahhhh
04:16 manfred ok
04:16 manfred andrej: good luck
04:17 manfred kober: that makes sense
04:17 kober I think I would expect an exception about the path I defined not being valid rather than a fallback to defaults
04:18 manfred i agree
04:18 kober only other gotcha looks like you are required to set root dir even if you are using relative paths
04:18 kober because it makes the relative paths join with root_dir
04:20 kober Final result: http://paste.ofcode.org/ydZA4yv7SdDzi6DXkktASP
04:20 manfred nice
04:20 manfred so it works now?
04:21 manfred kober: what if you add that root_dir to the SaltFile ?
04:21 andrej Meh.  The mine functions (and the modules doco) are doing my head in.
04:22 manfred does that bring it full circle?
04:22 andrej I can't seem to get ANY data out of the mine
04:22 manfred andrej: what command are you running for the mine?
04:22 andrej To retrieve data?
04:22 manfred yeah
04:22 andrej salt 'oob*' mine.get 'oob*' main_ip grain
04:22 andrej All i get is a list of 17 host names
04:23 andrej main_ip is my custom grain
04:24 andrej which I had to set because the boxes all have 3+ interfaces
04:24 manfred hrm
04:24 manfred one second
04:24 andrej They connect to the master on an arbitrary routing decision
04:24 andrej and I put their DNS IP into the main_ip grain
04:26 malinoff joined #salt
04:27 manfred oh
04:27 manfred salt-call mine.get \* grains.items
04:28 manfred your grain should be in there
04:28 andrej Oooops ... that looks quite different from what I tried :)
04:28 andrej where did you find that one? :)
04:28 manfred i dropped in
04:28 manfred mine_functions:
04:28 manfred grains.items: []
04:28 manfred so that it would drop the output of grains.items into the mine
04:29 kober manfred: salt-ssh doesn't listen to saltfile at all
04:29 andrej Aaah ... that's on the minion itself
04:29 manfred kober: werid
04:29 manfred andrej: yeah
04:29 andrej can I query them from the master?
04:29 manfred you cannot
04:29 manfred you can only query mine data from a minion ( there is a feature request to fix that)
04:30 andrej :/
04:30 andrej What's the point of having the mine, then? :)
04:30 manfred andrej: https://github.com/saltstack/salt/issues/12070
04:30 andrej I need to be able to generate a config file for icinga whether the minion is currently online or not
04:30 manfred andrej: to use it in states so that you don't have to do publish.publish?
04:30 manfred yes
04:30 andrej Heh
04:30 manfred the minion can use the mine
04:31 kober Hmm, it has the Saltfile mixin, maybe my saltfile isn't correct and its not telling me
04:31 manfred the master cant use it directly
04:31 kober Time for another pdb :)
04:31 andrej Aight ... that explains that.  I'll go home now - frustrated :D
04:31 andrej thanks manfred
04:32 manfred andrej: you can use the mine from your minion, you can also configure the minion mine_functions in /etc/salt/master or any minions /etc/salt/minion
04:33 kober Oh, saltfile mixin was added in June, probably not released yet?
04:33 manfred probably
04:33 manfred yeah
04:33 manfred the latest documents are strtaight from develop
04:33 manfred if it was in june, should be in 2014.7
04:34 kober Its not what I have installed but I see it in git
04:34 kober When is the next release planned?
04:34 manfred 2014.7 is tagged
04:34 kober Looks like I might have better luck with master
04:34 manfred 2014.7.0rc1 is in the works
04:34 kober perfect, I'll get master
04:35 manfred probably will have two of those
04:35 manfred i would expect it out in the middle of august
04:36 rgarcia_ joined #salt
04:36 kober hah, develop broke everything I just got working
04:36 manfred awesome
04:37 manfred but it has an ansible inventory roster thast I wrote :P
04:37 kober Does it support the dynamic ansible inventory stuff? :D
04:37 manfred https://github.com/saltstack/salt/blob/develop/salt/roster/ansible.py
04:37 manfred yes
04:37 manfred it should
04:37 kober Nice :)
04:37 kober ok, I'm going to brew some coffee, if this is already tagged I need to get to work!
04:37 manfred basic ones atleast that follows that
04:37 manfred with the _meta
04:38 manfred it isnt' tagged
04:38 manfred it is just branched
04:38 manfred still backporting patches
04:38 kober I keep not using salt because every time I try salt-ssh is broken
04:38 manfred and stablizing
04:38 kober Gonna bite the bullet and fix it
04:39 kober haha, guess what broke all my stuff?
04:39 kober I'll give you a hint, you just linked me
04:39 manfred how
04:39 manfred where is it broken at?
04:39 manfred i am looking at pushing it to use salt-ssh instead of deploy_script for salt
04:39 manfred -cloud*
04:40 kober In init it tries to open the inventory file
04:40 manfred https://github.com/saltstack/salt/blob/develop/salt/roster/__init__.py#L52
04:40 manfred should be caught there on line 52
04:40 kober and inventory_file is empty
04:40 manfred ahh shit
04:41 manfred ok
04:41 manfred one second
04:41 kober Because I haven't setup a roster yet
04:42 manfred ok
04:42 manfred i am ok with that
04:42 manfred with a rosterfile it should work correctly
04:42 kober You are ok with it raising IOError: [Errno 2] No such file or directory: ''
04:42 kober ?
04:42 manfred yes, cause you have no roster file
04:42 manfred should be fixed to display a more explicit error
04:42 kober Shouldn't it say "Hey, you don't have a roster file"
04:42 manfred but it rasing an error ther eis fine
04:43 manfred kober: what line is your error on?
04:44 manfred File "/root/salt/salt/utils/__init__.py", line 1110, in fopen
04:44 manfred fhandle = open(*args, **kwargs)
04:44 manfred ?
04:44 kober I think its a bug anyways, because its checking for different rosters, like an ansible roster
04:44 manfred it cycles through all the rosters
04:44 kober if the string isn't in there it just moves on
04:44 manfred --roster doesn't actually do anything anymore
04:44 kober but ansible.targets happens to be in there
04:45 manfred because it is the first alphabetically
04:45 manfred it is the first salt/roster/*.py file used to check
04:45 manfred so it failes
04:45 manfred it isn't ansible roster that is failing
04:45 manfred File "/root/salt/salt/roster/__init__.py", line 52, in targets
04:45 manfred targets.update(self.rosters[f_str](tgt, tgt_type))
04:45 manfred should have an IO Error around that
04:46 kober Yeah, thats what I was thinking
04:46 manfred gimme one second
04:46 kober but you can also remove your default settings for inventory_file because it'll never not get passed something I guess
04:51 manfred kober: http://ix.io/dEr
04:51 manfred [root@salt ~]# salt-ssh \* test.ping
04:51 manfred No Roster file found
04:52 kober I wonder if that is better than just returning empty and allow the rest of the rosters to do their thing?
04:52 kober Since its just cycling
04:52 manfred there is only one roster
04:53 manfred this one should only fail on it not being able to open the --roster-file
04:54 ajw0100 joined #salt
04:54 kober Oh, I see you are doing it outside of the ansible roster parsing stuff
04:54 kober Yeah, that works
04:54 manfred yes
04:54 manfred if it fails there
04:55 manfred it is because the file that you are trying to run through the rosters, doesn't exist
04:55 kober But would the other parsers raise IOError or is that only in the ansible one?
04:55 bhosmer joined #salt
04:56 manfred all of them would raise it
04:56 manfred ansible only raises it because it is the first one alphabetically
04:57 badon_ joined #salt
04:57 kober I think the rest are trying to default to /etc/salt/ if it doesn't exist
04:57 kober but I think that is worse
04:59 manfred it shouldn't
04:59 manfred it should check for the rosterfile location or --roster-file
05:00 manfred and if it fails, then meh
05:00 manfred we catch it
05:00 manfred but there isn't a fallback location
05:00 kober Yeah, they all return [] instead
05:01 kober so it ends up raising salt.exceptions.SaltRenderError: Unable to render any roster.
05:01 Tusker joined #salt
05:01 manfred yeah
05:01 manfred and if it does that
05:01 manfred then there is that last raise
05:01 manfred in there*
05:01 kober I think your way looks cleaner because its not an exception
05:01 kober but it would only handle ansible, not the rest
05:01 manfred if not targets:
05:01 manfred raise salt.exceptions.SaltRenderError('Unable to render any roster.')
05:02 manfred my change, handles everything, it only appears to be ansible, because that is the first one run, because alphabetical.
05:02 manfred oh wait
05:02 manfred yeah you ar eright
05:02 manfred but my way is cleaner, cause it will allow people to expand rosters more in the future
05:03 kober Yeah, I don't think each roster parser should be doing the file exist checking and everything
05:03 kober They should raise IOError
05:04 Tusker heya manfred, am trying now to use the tomcat module to deploy to tomcat, and was wondering how the tomcat module works in conjection with my own salt state called tomcat ?
05:05 manfred i have no idea
05:05 Tusker "Comment: State tomcat.wait found in sls crewproject is unavailable"
05:05 Tusker that's what I get as an error
05:05 manfred yeah i haven't messed with the tomcat state yet
05:05 manfred unfortunately
05:05 manfred sorry :)
05:06 Tusker but, my own state called tomcat, might likely interfere, right ?
05:06 Tusker ie,. salt.states.tomcat vs /srv/salt/tomcat/init.sls
05:07 allanparsons joined #salt
05:07 bmatt Tusker: it depends on the order specified in the master config
05:07 bmatt file vs gitfs roots, and then each file/gitfs root in turn, from top to bottom
05:07 bmatt the first match is the one that wins
05:08 Tusker ok, so if I want to use the deploy mechanisms in the salt.states.tomcat in the distro, I should call my own state something different ?
05:08 bmatt er, uhm, well, hold one
05:08 bmatt er, hold on
05:09 bmatt there's state modules (salt.states.tomcat) and state files (/srv/salt/tomcat.sls) and state declarations (foo: tomcat.whatever: - option: bar)
05:09 Tusker ok
05:09 bmatt state files contain state declarations which in turn invoke state modules
05:10 bmatt there's no namespace conflict between state modules and state files
05:10 bmatt which it sounds like you're worried about
05:10 Tusker ah ok
05:10 Tusker great
05:11 Tusker now to figure out why it isn't behaving :)
05:11 bmatt so in your state, you have something like
05:11 bmatt foo: tomcat.wait: - option: bar
05:12 kober manfred: so if there is only one roster file, how do you know what type of roster to use?
05:12 kober manfred: for example if its using a standard flat one, how does the ansible one know it shouldn't parse it / run
05:12 bmatt "tomcat" in this case refers to the state module (salt.states.tomcat), and wait is the function you're trying to call
05:12 bmatt but the tomcat state module has no wait function
05:12 bmatt :)
05:13 Tusker ok
05:13 Tusker :)
05:13 Tusker and another question... how do I move files around on a minion ?
05:13 Tusker ie, copy directory /opt/blah/dir to /var/lib/blah/dir ?
05:13 bmatt so you have to ask yourself the question
05:13 Ryan_Lane1 joined #salt
05:13 manfred kober: it runs it through all the types and figures it out
05:14 bmatt do you want to "go do the thing" or do you want to "describe the end result"?
05:14 manfred kober: otherwise salt-ssh --roster scan
05:14 manfred and specify which type using --roster
05:14 Tusker bmatt: well, I've just extracted the zip archive to a destination, but I want to have some of the files in one spot, and some elsewhere
05:14 bmatt because state modules (and, by extension, state files and state declarations) describe what the system looks like ex post facto
05:14 ghartz joined #salt
05:15 bmatt Tusker: the state.modules.file execution module is useful for moving things around
05:15 bmatt but use of execution modules directly isn't typically idempotent
05:15 Tusker can I extract a subset of files from an archive ?
05:16 bmatt sure, using cmd.run :)
05:16 Tusker :P
05:16 Tusker ah, but I could use excludes ?
05:17 manfred aight, I am going to bed
05:17 manfred nite
05:17 Tusker night night
05:22 rap424 joined #salt
05:22 badon joined #salt
05:24 Katafalkas joined #salt
05:26 vlcn anyone played with the new salt cloud vsphere module?
05:26 vlcn documentation is a bit thin
05:29 ramishra joined #salt
05:30 kober manfred: ok, now that we have all that figured out, when I try to SSH locally with salt-ssh: "Permission denied for host 127.0.0.1, do you want to deploy the salt-ssh key? (password required):"  I think its not using my user's ~/.ssh
05:30 kober is it trying to use keys in etc/pki instead?
05:35 Katafalkas joined #salt
05:39 kober manfred: How do I define what user salt-ssh uses?
05:42 aw110f joined #salt
05:48 Tusker hey guys, say I want to stop tomcat, copy a file over, and start tomcat in a state file, how can I do that in that order ?
05:48 Katafalkas joined #salt
05:51 m1crofarmer joined #salt
05:54 TyrfingMjolnir joined #salt
05:56 bmatt Tusker: http://docs.saltstack.com/en/latest/topics/tutorials/states_pt2.html
05:57 manfred it defaults to the user you run as, otherwise, specify the user: whatever and an passwd: if you are sshing, and then sudo:True to use the same password to sudo run everything
05:57 manfred i am going to bed
06:02 aquinas joined #salt
06:09 roolo joined #salt
06:22 ramishra joined #salt
06:33 nextdoorwarren joined #salt
06:35 Katafalk_ joined #salt
06:36 topochan joined #salt
06:40 oz_akan joined #salt
06:47 Katafalkas joined #salt
06:50 kjkoster5489 joined #salt
06:54 duncanmv__ joined #salt
06:57 TyrfingMjolnir joined #salt
07:03 Tusker ok, so, I've got tomcat deploying the application, and it's running now, but say I want to have different environment variables set for the running tomcat instance, for different environments ?
07:04 Tusker my generic tomcat state will deploy a generic tomcat instance
07:05 Tusker but, how to have custom environment variables passed in to the startup script, for different environments ?  ie, production-customername ?  these would be stored in pillar perhaps ?
07:06 duncanmv__ joined #salt
07:08 ml_1 joined #salt
07:10 m1crofarmer joined #salt
07:12 tyson_ joined #salt
07:12 matthiaswahl joined #salt
07:16 Katafalkas joined #salt
07:16 Tusker ok, so, after reading some more, it looks like I need to use something like jinja and modify the files based on pillar data ?
07:19 ramishra joined #salt
07:24 ramishra_ joined #salt
07:26 TyrfingMjolnir joined #salt
07:26 chiui joined #salt
07:41 oz_akan joined #salt
07:42 kober How do I tell what salt is doing?  I run state.highstate and I got this: http://paste.ofcode.org/35eFQr69cDYjLQRxU2kkTUN
07:42 malinoff kober, append "-l debug -t 30"
07:46 kober malinoff: thanks!
07:46 kober http://paste.ofcode.org/yqQ6ZZkWBeQ3gQjsYgVnpt
07:46 kober That is the output
07:46 kober not sure how to figure out what its trying to do from that
07:46 malinoff kober, so am i :)
07:50 go|dfish joined #salt
07:52 babilen kober: Which version of salt do you have on the master and minions? Could you run both in debug mode (stop the service and then run "salt-{master,minion} -ldebug" -- Does that shed some light on the situation?
07:54 ramishra joined #salt
07:55 topochan joined #salt
07:58 Katafalkas joined #salt
07:58 intellix joined #salt
07:58 kober babilen: I'm running develop branch of salt (salt-ssh) and on the client the minion is 2014.1.7
07:59 Katafalkas joined #salt
08:00 xsteadfastx joined #salt
08:01 kober babilen: This is the output with debug http://paste.ofcode.org/ET4iMN8e2uudfMjpiFP7cA
08:03 linjan joined #salt
08:06 babilen kober: I'd say it is most likely a bug in git HEAD and I haven't seen it before. Is the minion having a problem when you run it in debug mode or is it simply the master that is unhappy?
08:06 babilen s/bug/interoperability issue between HEAD and 2014.1 also
08:08 ramishra joined #salt
08:10 jY is there anyway to run an external pillar using salt-call --local ?
08:12 ramishra joined #salt
08:12 xsteadfastx joined #salt
08:16 darkelda joined #salt
08:19 kober babilen: I'll check running the minion in debug
08:23 muzammil i have a control.py script with needs name,port,memory (planning to use jinja template), this will come from range server. I need to copy over this control.py with values substituted to selected minions. Not sure were to start from
08:23 kober Interesting, now that I'm on the minion its saying: [ERROR   ] This master address: 'salt' was previously resolvable but now fails to resolve! The previously resolved ip addr will continue to be used
08:24 kober but I have file_client: local
08:24 kober its supposed to be masterless so I can use salt-ssh
08:25 wr3nch joined #salt
08:26 malinoff muzammil, start from the tutorial: http://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.html
08:31 muzammil malinoff: thx i tried that, but not able to figure out how to substitute those value and then do a copy to minions. this is my example http://paste.ubuntu.com/7902810/
08:31 malinoff muzammil, what is this?
08:32 muzammil malinoff:  the salt module is trying to write
08:33 muzammil malinoff:  this top part is the template(snippet) and below is the function
08:33 malinoff muzammil, I guess you missed the point of modules. You can do what you want without python at all
08:34 intellix joined #salt
08:35 malinoff muzammil, basically you need to put 2 files in appropriate locations with specific content
08:35 muzammil malinoff: sorry for missing that. I am trying this to learn more in salt modules and how i can it to leverage and do more complex this
08:35 muzammil malinoff: okey
08:36 malinoff muzammil, you really don't need to write a module if you don't want to use an external tool that does not have it's module already
08:36 malinoff for example, couchbase
08:36 malinoff otherwise you should stick with state files
08:36 prasanna joined #salt
08:37 _viq joined #salt
08:37 muzammil malinoff: can i use state file what will pull info from range server ? name, port, memory etc are all stored in range clusters
08:41 oz_akan joined #salt
08:42 malinoff muzammil, I don't know what is "range server". Does it have a minion?
08:44 muzammil no, its a remote server which provides meta data about a cluster, https://github.com/ytoolshed/range
08:44 prasanna hi..i am newbie to saltstack...i installed saltapi and while testing through curl....am not getting any results...
08:44 prasanna # curl -sS -i -H 'Content-Type: application/json' -d '[{"eauth":"pam","username":"saltdev","password":"saltdev","client":"local","tgt":"*","fun":"test.ping"}]' http://localhost:8000
08:44 prasanna HTTP/1.1 200 OK Date: Wed, 30 Jul 2014 08:54:10 GMT Server: Apache/2.2.15 (Red Hat) Content-Length: 16 Connection: close Content-Type: application/json  {"return": [{}]}
08:46 malinoff muzammil, it doesn't even has a README :) but in state files you can call an arbitrary salt module using {{ salt['cmd.run']('echo test') }} - so you have access to the cli
08:47 Katafalk_ joined #salt
08:48 muzammil malinoff: let me try that
08:51 zooz joined #salt
08:54 poogles joined #salt
08:54 matthiaswahl joined #salt
08:55 xsteadfastx joined #salt
08:59 Katafalkas joined #salt
09:02 N-Mi joined #salt
09:05 intellix joined #salt
09:09 ramishra joined #salt
09:11 feco joined #salt
09:12 ramishra_ joined #salt
09:20 xsteadfastx joined #salt
09:23 yomilk joined #salt
09:25 aw110f joined #salt
09:27 ramishra joined #salt
09:27 MrTango joined #salt
09:27 aw110f_ joined #salt
09:33 martoss joined #salt
09:35 urtokk joined #salt
09:37 giantlock joined #salt
09:39 badon joined #salt
09:42 oz_akan joined #salt
09:43 iggy_ joined #salt
09:43 eculver_ joined #salt
09:43 mackstic1 joined #salt
09:43 Heartsbane_ joined #salt
09:43 ashb_ joined #salt
09:43 Ahlee_ joined #salt
09:43 hoodow_ joined #salt
09:43 xzarth_ joined #salt
09:44 Corey_ joined #salt
09:44 imil_ joined #salt
09:44 Heggan_ joined #salt
09:44 Zuru_ joined #salt
09:44 SachaLig1hert joined #salt
09:44 ntropy_ joined #salt
09:44 emostar_ joined #salt
09:44 Valdo joined #salt
09:44 jayne joined #salt
09:44 roo9 joined #salt
09:44 evidence joined #salt
09:44 sifusam joined #salt
09:44 lude joined #salt
09:44 __number5__ joined #salt
09:44 Damoun joined #salt
09:44 twinshadow joined #salt
09:44 urtokk joined #salt
09:44 bitmand joined #salt
09:44 estherbester joined #salt
09:44 Kalinakov joined #salt
09:44 philipsd6 joined #salt
09:44 seblu joined #salt
09:44 andredieb joined #salt
09:45 flebel joined #salt
09:45 tyson_ joined #salt
09:45 seventy3_away joined #salt
09:45 xt joined #salt
09:45 InAnimaTe joined #salt
09:45 dcmorton joined #salt
09:45 markhayden joined #salt
09:45 pjs joined #salt
09:46 Deevolution joined #salt
09:46 ckao joined #salt
09:47 Katafalk_ joined #salt
09:48 blast_hardcheese joined #salt
09:49 Katafalk_ joined #salt
09:51 ramishra joined #salt
09:52 TheThing joined #salt
09:52 Katafalkas joined #salt
10:00 urtokk joined #salt
10:04 ramishra joined #salt
10:05 gywang joined #salt
10:06 gywang How can I use pillar.get() in custom grains ?
10:20 bhosmer joined #salt
10:22 Katafalkas joined #salt
10:25 Katafalkas joined #salt
10:31 intellix joined #salt
10:38 agend joined #salt
10:38 ramishra joined #salt
10:48 ggoZ joined #salt
10:48 che-arne|2 joined #salt
10:48 che-arne joined #salt
10:54 zions joined #salt
10:55 zions Hi there. I'm trying to use the publish.publish module to run commands on peer minion and get the result.
10:56 zions The minion which match the target filter execute the command very quicky
10:56 zions however, the publish module returns only after the set timeout
10:57 zions Is this normal behavior?
10:59 xsteadfastx joined #salt
11:01 martoss joined #salt
11:01 CeBe joined #salt
11:12 ramishra joined #salt
11:13 Katafalk_ joined #salt
11:22 ghartz joined #salt
11:34 TyrfingMjolnir joined #salt
11:43 ramishra joined #salt
11:46 intellix joined #salt
11:53 diegows joined #salt
11:56 martoss joined #salt
12:07 Katafalkas joined #salt
12:10 Katafalkas joined #salt
12:22 hobakill joined #salt
12:22 hobakill join /#kde
12:22 hobakill whoops
12:23 zions joined #salt
12:24 zions Hi, anyone here for help?
12:25 hobakill zions: hello. ask your question. sometimes it gets answered right away and other times you have to check back in the logs. log page is in the room description
12:25 SteveJ1729 joined #salt
12:25 6A4AALUJF joined #salt
12:25 ajw0100 joined #salt
12:26 rypeck joined #salt
12:30 toastedpenguin joined #salt
12:31 zions Thanks mate.
12:31 zions I'm gonna ask a question I asked a few hours ago...
12:31 zions I'm trying to use the publish.publish module to run commands on peer minion and get the result.
12:31 zions The minion which match the target filter execute the command very quickly.
12:31 zions however, the publish module returns only after the set timeout.
12:31 zions Is this normal?
12:32 scoates joined #salt
12:34 jpl1079 joined #salt
12:36 helderco joined #salt
12:43 rome joined #salt
12:44 zandy joined #salt
12:45 rome_ joined #salt
12:50 rypeck If you go to - http://salt.readthedocs.org/en/latest/ - and scroll down to the bottom of the page... looks like some CSS isn't getting properly applied.
12:54 rome joined #salt
12:56 mpanetta joined #salt
12:56 rome joined #salt
12:58 jslatts joined #salt
12:58 Katafalk_ joined #salt
12:58 MTecknology rypeck: very much true
12:58 rypeck Not sure where to report it.
13:00 rypeck Figured a message here would get the right person.
13:00 MTecknology https://github.com/saltstack/salt  <-- docs get generated from that repo
13:01 rypeck Ah I'll report it there. Thans
13:01 rypeck *Thanks
13:01 MTecknology more thanks for taking the time to do so
13:01 blarghmatey joined #salt
13:02 ericof joined #salt
13:03 Katafalkas joined #salt
13:03 mpanetta joined #salt
13:04 mpanetta joined #salt
13:05 vlcn anyone played with the new salt cloud vsphere module?
13:05 vlcn documentation is a bit thin, just looking for something more to go on
13:08 racooper joined #salt
13:12 brandon_ joined #salt
13:12 Katafalkas joined #salt
13:12 bhosmer joined #salt
13:13 elfixit joined #salt
13:14 superted joined #salt
13:14 rome joined #salt
13:15 rypeck MTecknology: https://github.com/saltstack/salt/issues/14631
13:15 rypeck Done
13:15 rome joined #salt
13:16 superted Don't suppose anyone is online who has some experience with the external_auth mechanism?
13:16 superted Looking for a little advice...
13:16 FeatherKing joined #salt
13:21 Ahlee_ I use it, what's up?
13:22 Ahlee_ I just plugged in my usernames and let pam auth do the magic (pam handles talking back to AD for auth)
13:22 mapu joined #salt
13:23 Ahlee_ neat, i see it now handles LDAP directly
13:23 superted Thanks Ahlee, well we've been using it for a few months. We have it setup to pam and that is indeed working.
13:23 superted I've been asked quite a few times though why it's challenging for user credentials
13:24 superted The users are already authenticated through LDAP on the salt server, looking to see if i can use that authentication instead of re-requesting...
13:25 Ahlee_ superted: set client_acl: for the users/groups instead of external_auth
13:25 Ahlee_ client_acl:\n  user.name:\n  - .*
13:25 superted Will client_acl work against ldap?
13:26 Ahlee_ then just call salt without -a
13:26 Ahlee_ client_acl will trust local users, which are authenticated external to salt
13:26 Ahlee_ assuming it matches
13:26 superted Aha, that sounds ideal
13:27 superted Would that be for all local users? We have granular permissions depending on team etc
13:27 Ahlee_ however you define the acl
13:28 Ahlee_ we don't care what users execute, as long as they exeucte as their username
13:28 Ahlee_ so we just allow all, but you can get as granular as you want, specifying specific
13:28 superted Sounds promising. We unfortunatley have to split out access a little as we have 3rd parties. I'll have a test
13:28 superted Thanks Ahlee
13:28 superted much appreciated
13:28 Ahlee_ good luck
13:30 to_json joined #salt
13:35 smcquay joined #salt
13:35 ajprog_laptop joined #salt
13:42 thayne joined #salt
13:42 nyx_ joined #salt
13:43 jalaziz joined #salt
13:44 zandy joined #salt
13:44 donnyk joined #salt
13:44 dude051 joined #salt
13:47 donnyk Hi, salt doesn't seem to want to run any of my scheduled tasks.  I put the schedule in a pillar to run a state every 5 seconds.  The state executes manually without issue - salt '*' state.sls test_schedule.  The pillar.items displays the schedule item.  The master is configured to loop_interval: 5.  There are no errors in the logs.  Can anyone tell me what I might have done wrong?  I'm using 2014.1.7
13:48 xcbt joined #salt
13:52 jalaziz joined #salt
13:57 bhosmer joined #salt
14:00 rome joined #salt
14:00 Katafalk_ joined #salt
14:02 jalaziz joined #salt
14:03 scoates joined #salt
14:07 oz_akan joined #salt
14:09 cpowell joined #salt
14:09 superted @Ahlee - Worked a treat mate. Thanks again
14:09 housl joined #salt
14:10 gothix joined #salt
14:11 aquinas joined #salt
14:12 gothix joined #salt
14:15 jalaziz joined #salt
14:18 gq45uaethdj26jw6 joined #salt
14:19 rallytime joined #salt
14:21 ipmb joined #salt
14:22 jalaziz joined #salt
14:23 donnyk Anyone use salt scheduled tasks?  How do I get salt to actually execute the schedule?  Not a single one of the following scheduled jobs executes automatically in the given time period: http://pastebin.com/by8KKwKv .  That output is from salt '*' pillar.items.  I copied the code straight from http://docs.saltstack.com/en/latest/topics/jobs/schedule.html
14:24 jalbretsen joined #salt
14:26 mgw joined #salt
14:26 aquinas joined #salt
14:26 wendall911 joined #salt
14:26 mgw Is there a way from within an SLS to detect that we're in test mode?
14:27 ujjain joined #salt
14:27 manfred mgw: it should be in opts[], maybe salt['config.get']('test') ?
14:27 honestly .ww
14:27 honestly whoops
14:27 mgw manfred: perfect, thanks
14:28 manfred https://github.com/saltstack/salt/blob/develop/salt/modules/state.py#L256
14:28 manfred mgw: not sure if that works, but i think it will
14:28 manfred mgw: state.highstate sets puts it into opts[]
14:28 manfred soooo, however you can get opts stuff :P
14:28 mgw manfred, thanks!
14:29 mgw I'll test in a second
14:29 quickdry21 joined #salt
14:29 mgw state.sls does the same it appears
14:30 mgw https://github.com/saltstack/salt/blob/develop/salt/modules/state.py#L366-L371
14:33 manfred year
14:33 manfred yar*
14:35 * Sacro spends ages reading the documentation and then realises the feature I need is in 2014.7 :(
14:36 jamesog_ joined #salt
14:39 arthabaska joined #salt
14:40 zions joined #salt
14:40 helderco joined #salt
14:41 Ozack1 joined #salt
14:41 vejdmn joined #salt
14:43 tyson_ joined #salt
14:48 rypeck Sacro: I feel your pain. It's coming soon
14:49 williamthekid joined #salt
14:50 nyx joined #salt
14:52 thedodd joined #salt
14:52 babilen donnyk: I don't immediately see something wrong with that :-/
14:53 rome joined #salt
14:54 duncanmv__ is there a way to say salt $wildcard foo and get back the list that matches without doing anything?
14:54 tyson__ joined #salt
14:54 Sp00n just do a test.ping?
14:57 donnyk I stepped up log level to debuggin and loop_interval is set to 5 seconds.  Every 5 seconds I see the master update the fileserver cache.  But nothing about schedules at all in the logs
14:58 tyson_ joined #salt
15:01 spiette joined #salt
15:08 rome joined #salt
15:09 _alpha_ joined #salt
15:10 ghartz joined #salt
15:11 dccc joined #salt
15:12 Luke_ joined #salt
15:14 markhayden joined #salt
15:14 cnelsonsic joined #salt
15:18 conan_the_destro joined #salt
15:20 geekmush joined #salt
15:21 ggoZ joined #salt
15:21 exanimo joined #salt
15:23 iMil joined #salt
15:25 jemejones joined #salt
15:25 gothix joined #salt
15:27 rgarcia_ joined #salt
15:28 geekmush joined #salt
15:30 penguin_dan joined #salt
15:32 ipmb is there an easy way to install master and minion (and have the minion key accepted) using salt-boostrap.sh?
15:32 ipmb I figured out -M to install the master, but I'm having issues scripting key acceptance
15:33 kiorky ipmb: generate the minion key and inter copy the keys (master one in pki/minion and minion in pki/master/minions/<mid>) before the minion launches.
15:34 ipmb ok
15:34 ipmb it's ubuntu so the minion launches automatically
15:34 ipmb so I think I want to stop it, delete the unaccepted key if it exists, generate keys, restart it
15:35 kiorky ipmb: or just mv the key to master/minions.
15:35 jliljenq joined #salt
15:35 kiorky (and restart minion, then.)
15:36 ipmb does this look correct? https://dpaste.de/Lawh
15:36 kiorky ipmb: only if id: `hostname` in /etc/salt/minion
15:37 ipmb yep
15:37 kiorky ipmb: cp -f /etc/salt/pki/master/master.pub /etc/salt/pki/minion/minion_master.pub
15:37 kiorky instead of the if rm
15:38 ipmb ok
15:38 jliljenq anyone know of a way to force the master to update its fileserver cache? I'd like to keep the interval at 60 seconds, but there are a few cases where I need changes to be available to minions immediately
15:38 kiorky rm -rf the cache folder content :)
15:40 patarr joined #salt
15:40 patarr joined #salt
15:40 vbabiy joined #salt
15:40 rome joined #salt
15:44 hobakill joined #salt
15:45 schimmy joined #salt
15:45 bezeee joined #salt
15:47 intellix joined #salt
15:48 jliljenq haha, you'd think that'd be obvious... thanks, kiorky
15:48 schimmy1 joined #salt
15:54 rome joined #salt
15:55 tyson__ joined #salt
15:57 khaije1 joined #salt
16:02 jeremyBass2 joined #salt
16:05 vejdmn joined #salt
16:06 tligda joined #salt
16:06 kiorky jeremyBass2: ?
16:06 kiorky jliljenq: ?
16:06 kiorky dont understood
16:08 ipmb darn, bumping into https://github.com/saltstack/salt/issues/12248
16:08 ipmb kiorky: looks like you had it too. did you figure out a workaround?
16:09 kiorky the problem solved by itself a long ago.
16:09 jliljenq kiorky: I was over thinking it. your solution works perfectly :D
16:09 kiorky jliljenq: ha :)
16:10 martoss joined #salt
16:10 ipmb pretty bad here: https://dpaste.de/w3CM
16:12 Comradephate joined #salt
16:13 Guest24635 I'm working on writing some simple states on top of the extfs module and I'm trying to determine a good way to know if a partition is already formatted.  I tried extfs.attributes - it actually crashes when used on a partition with no filesystem... so I think I'll have to hack on that to make it execute w/o error.  Are there other simpler approaches anyone could recommend?
16:13 KyleG joined #salt
16:13 KyleG joined #salt
16:13 KyleG joined #salt
16:13 KyleG joined #salt
16:15 kballou joined #salt
16:16 troyready joined #salt
16:17 jas-_ joined #salt
16:19 thedodd joined #salt
16:20 drybjed left #salt
16:21 bmatt Guest24635: I've abandoned partitions entirely
16:21 bmatt I use LVM on bare disk
16:21 bmatt and the blockdev state module
16:22 robawt bmatt++
16:22 Guest24635 bmatt: this is intriguing, I am new to LVM but aiming to use it. is there a good reference to this approach?
16:23 robawt Guest24635: http://tldp.org/HOWTO/LVM-HOWTO/
16:24 Gareth morning
16:24 bmatt Guest24635: https://gist.github.com/thenewwazoo/ff76c143de751a970c2e
16:24 bmatt combine that with robawt's link (and ignore my own custom iscsi module stuff)
16:25 bmatt I'm doing a similar thing with mdadm+lvm+blockdev, in fact
16:26 tligda1 joined #salt
16:27 jaimed joined #salt
16:27 robawt wuddup Gareth
16:28 Gareth robawt: howdy :) not too much.  starting the day.
16:28 estherbester joined #salt
16:29 * robawt highfives estherbester
16:30 estherbester joined #salt
16:31 estherbester \o robawt
16:32 robawt \m/
16:32 bmatt noobie question
16:33 bmatt I deleted a host (salt-key -d) from the master, reinstalled the minion, installed salt-minion, and now I'm getting a "master has rejected this minion's key"
16:33 bmatt oh. derp.
16:33 bmatt dhcp hostname
16:35 bmatt hmm. still an issue
16:35 bmatt ah. rm /etc/salt/minion_id
16:35 Outlander joined #salt
16:37 ksalman suddently all my salt-minions say "Minion failed to authenticate with the master, has the minion key been accepted?"
16:38 ksalman if i try it from the master, it says "Failed to authenticate, is this user permitted to execute commands?"
16:38 bmatt O_o
16:38 ksalman nothing has been changed recently
16:38 bmatt ksalman: that's... big
16:38 ksalman i verified that the minions can resolve the master
16:38 bmatt did you reissue your master key
16:38 ksalman so i dnt konw what is going on
16:38 ksalman bmatt: I did not. Additionally the master is configured to Accept all keys and configured in open_mode
16:39 ksalman so I should not see any auth errors, no?
16:39 ksalman i have 400+ minions and it was working all fine but now they all seem broken
16:40 ksalman master.pem, master.pub, last updated september 2013
16:40 joehillen joined #salt
16:41 Linuturk is there a way to specify personality or cloud init scripts via the salt-cloud openstack provider?
16:45 manfred Linuturk: yes
16:46 manfred Linuturk: lemme find it real quick
16:46 kober joined #salt
16:47 manfred Linuturk: actually no, only in aws...
16:47 m1crofarmer joined #salt
16:47 Linuturk :(
16:47 Linuturk :'(
16:47 manfred i wi8ll add it when I get back from lunch
16:47 Linuturk lol
16:47 Linuturk awesome
16:47 geekmush joined #salt
16:48 manfred i could have sworn i talked to someone about that being there... i must have just forgotten to do it.
16:48 Linuturk want a github issue to remind you?
16:48 manfred yes please
16:48 manfred tag @gtmanfred
16:52 davet joined #salt
16:53 thayne joined #salt
16:53 Linuturk done and done
16:54 ipmb can I call publish.publish inside a pillar?
16:54 ipmb like this https://dpaste.de/iU1c
16:54 rap424 joined #salt
16:55 ipmb if I call pillar.items with that, nothing shows up from that pillar file
16:55 manfred ipmb: pillars are rendered on the master, not the minion
16:55 Ryan_Lane joined #salt
16:55 retrospek joined #salt
16:56 manfred i would be supprised if you could, and you are probably going to get weird results
16:56 ipmb ok
16:56 ipmb so move to states
16:57 davet joined #salt
16:57 kballou joined #salt
17:00 m1crofarmer joined #salt
17:01 khaije1 hi all, I'm trying to construct an organization scheme making central use of nodegroups as a basic organizational unit, is there a way to query for a list of nodegroups and their members from configuration? I'd like to test and validate my configurations :)
17:01 _jslatts joined #salt
17:02 schimmy joined #salt
17:03 zandy joined #salt
17:03 schimmy1 joined #salt
17:04 vejdmn joined #salt
17:05 forrest joined #salt
17:06 ml_1 joined #salt
17:07 tyson_ joined #salt
17:08 gmoro joined #salt
17:09 CheKoLyN joined #salt
17:09 tyson_ joined #salt
17:10 ksalman =( https://github.com/saltstack/salt/issues/14645
17:11 Ryan_Lane ksalman: did you increase worker_threads, as suggested by the output?
17:11 forrest ksalman, which release is that on
17:11 forrest also yea, worker threads
17:11 scoates joined #salt
17:12 Ryan_Lane it simply may not be able to keep up with the auth requests
17:17 tyson_ joined #salt
17:18 ksalman forrest: master is 2014.1.1. Minions are either 2014.1.1 or 2014.1.0
17:18 ksalman I did not increase worker threads, but would it just stop working suddenly?
17:18 ksalman i can try increasing threads
17:18 ksalman Ryan_Lane: currently worker_threads is set to 4
17:19 Ryan_Lane oh, that's very low
17:19 ksalman it is?
17:19 ksalman hm
17:19 Ryan_Lane yes
17:19 Ryan_Lane that's 100 minions per thread
17:19 Ryan_Lane how many CPUs does your box have?
17:20 Ryan_Lane you want the thread count as a multiple of your CPUs
17:20 ksalman Ryan_Lane: it's a 4 core VM, with 2GB ram
17:20 Ryan_Lane right, so use 8 threads
17:20 ksalman okay, i'll try that
17:20 dstokes morning all
17:22 tyson_ joined #salt
17:25 ksalman what is a recommended number of minions per thread?
17:27 arthabaska joined #salt
17:29 gmoro joined #salt
17:29 jslatts joined #salt
17:34 rlarkin joined #salt
17:36 forrest ksalman, it depends on what is happening when your states run from what I've seen
17:36 forrest there isn't a 'hard' suggestion
17:37 Ryan_Lane I was running 1000 minions on 8
17:37 ksalman aww
17:37 Ryan_Lane but I was only doing remote execution
17:37 Ryan_Lane no state runs
17:37 aw110f joined #salt
17:37 ksalman state runs need more resources i presume?
17:38 forrest yea
17:38 cpowell joined #salt
17:39 Ryan_Lane they make a lot more connections
17:40 Ryan_Lane you may consider going to 16 at some point
17:40 Ryan_Lane I think each master process uses about 20-40MB of memory, last I checked
17:40 Ryan_Lane I haven't used a master in a long while, though
17:42 ksalman 16 threads on 4 core would be fine?
17:43 Ryan_Lane yeah, it's a multiple of 4
17:43 ksalman okay
17:43 Ryan_Lane until you're using ~80% of all your cores you should be fine
17:43 ksalman oh i see
17:44 Ryan_Lane you use multiples of your cores to avoid context switching
17:44 blarghmatey joined #salt
17:44 Ryan_Lane (this is a normal scaling approach for python apps)
17:44 evidence joined #salt
17:44 ksalman thanks
17:45 Ryan_Lane yw
17:45 urtokk joined #salt
17:45 ckao joined #salt
17:46 oz_akan hi all, I am looking at Python Client API, does anyone know a way to get get content of a state file that would be applied to a minion?
17:48 bmatt oz_akan: `salt-call state.show_highstate` does what you want
17:48 CeBe2 joined #salt
17:48 bmatt but it depends a bit on what you're looking for
17:49 kermit joined #salt
17:49 gmoro joined #salt
17:49 CeBe3 joined #salt
17:49 InAnimaTe joined #salt
17:51 pjs joined #salt
17:52 oz_akan bmatt: awesome, let me try that and then see how I can call it in a python app
17:52 Carl_ Question: I have a line that looks like this:
17:52 Carl_ {% if pillar['mapper'] is defined %}
17:52 Carl_ Is there a way I can replicate the behavior while using this line:
17:52 Carl_ I@mapper:
17:53 Carl_ So that I can match "is defined"
17:53 dccc joined #salt
17:54 bezeee joined #salt
17:56 scoates joined #salt
17:58 M0WGLI joined #salt
17:59 jpl1079 joined #salt
18:05 chiui joined #salt
18:08 Guest24635 is there a way to pause/unpause a minion?  I have a backup process that depends on unmounting a volume to copy data out, if highstate were to run while this is in progress it'd be bad.
18:09 Guest24635 I suppose I could entirely stop the minion and start it back up after the backup but that seems extreme
18:10 Gareth Are you running the highstate from a scheduled job or in cron?
18:10 manfred Linuturk: nevermind, it is in there
18:10 jslatts joined #salt
18:11 manfred Linuturk: https://github.com/saltstack/salt/blob/develop/salt/cloud/clouds/openstack.py#L524
18:11 Guest24635 Gareth: no but the backup will be scheduled regularly
18:11 Gareth Guest24635: Salt doesn't run the highstate until you tell it to, so unless you have it set to run on a schedule then it won't run until you manually run it.
18:12 Guest24635 Gareth: I know, just worried someone's up at 3AM and decides to run highstate and inadvertantly kills a backup
18:13 Guest24635 is it true that a minion is only doing one job at a time?  if so perhaps I can schedule the backup via salt's scheduler and thereby avoid this issue
18:13 thayne joined #salt
18:14 matthias_ joined #salt
18:15 linjan joined #salt
18:15 zandy joined #salt
18:16 armonge joined #salt
18:17 armonge left #salt
18:18 Gareth Guest24635: hm. good question.  That I'm not sure about, I *think* so long as the jobs don't conflict then salt will try and run them concurrently.
18:19 armonge joined #salt
18:19 Ryan_Lane1 joined #salt
18:20 chrisjones joined #salt
18:21 vbabiy joined #salt
18:21 m1crofarmer joined #salt
18:22 Carl_ I too was looking for a way to pause salt-minion schedule on a host. Preferred way is to touch a file and salt just exits the highstate run if file exist
18:22 Carl_ But was unable to find anything
18:23 manfred Gareth: it should only be able to run one state at a time.
18:23 manfred Guest24635: ^^^
18:25 gq45uaethdj26jw6 manfred: saw the sftp merge, thanks for that! i was reading the thread about this change and could not tell if sftp or scp was the new default?
18:25 maboum joined #salt
18:25 gq45uaethdj26jw6 i havent merged that change in yet, but i will
18:25 manfred gq45uaethdj26jw6: scp remains the default
18:26 gq45uaethdj26jw6 manfred: so you have to actually specify sftp in the config?
18:26 manfred i didn't have a compelling reason to change the default
18:26 manfred gq45uaethdj26jw6: yeah
18:26 gq45uaethdj26jw6 seems to me that sftp will work if ssh works, but scp does not necessarily work if ssh works. is that not the case?
18:27 manfred so
18:27 manfred sftp works no matter what, because it is built into the sshd daemon
18:27 gq45uaethdj26jw6 right
18:27 Gareth manfred: good to know.
18:27 manfred as long as ssh is up, sftp should work (unless you have the Subsystem sftp /usr/libexec/sftp-server...w/e removed from /etc/ssh/sshd_config)
18:27 manfred but
18:28 manfred but i didn't have a compelling reason to change the default
18:28 bezeee joined #salt
18:28 manfred it is more of a... hey, incase yours doesn't work, here...
18:28 gq45uaethdj26jw6 that sounds like a compelling reason right there, doesn't it? sftp works if ssh works, and ssh is a hard dependency. is there a downside to changing?
18:29 manfred gq45uaethdj26jw6: if ssh works and you have the subsystem configured
18:29 manfred you have to have that line in sshd_config
18:30 gq45uaethdj26jw6 hm, is that commonly not configured? as it stands, i literally just run a dirty hack of issuing a root_cmd to yum install openssh_clients or whatever
18:30 gq45uaethdj26jw6 its a really dirty hack
18:30 jslatts joined #salt
18:31 manfred is scp commonly not installed?
18:31 gq45uaethdj26jw6 on a number of redhat/centos/fedora boxes, scp is missing
18:31 manfred it shouldn't be.
18:31 manfred the base server with ssh should have it installed
18:31 CeBe joined #salt
18:31 gq45uaethdj26jw6 and if it's gone, it throws an exception while spinning up my map file, and it terminates the salt-cloud command
18:32 manfred that is when you should enable use_sftp: True
18:32 CeBe1 joined #salt
18:32 gq45uaethdj26jw6 sure, and I'll be doing that now, it's in about 10 different profiles I have though
18:33 manfred gq45uaethdj26jw6: you don't have to put it in the profiles
18:33 manfred just drop it in /etc/salt/cloud
18:33 manfred right in the root
18:33 gq45uaethdj26jw6 oh snap, you are right. okay, i dont know what i was thinking
18:33 manfred echo 'use_sftp: True' >> /etc/salt/cloud
18:33 gq45uaethdj26jw6 i dont really care in the case
18:33 manfred :P
18:33 manfred yeah
18:33 kober joined #salt
18:33 manfred that is what i was saying \o/
18:34 gq45uaethdj26jw6 derp
18:34 gq45uaethdj26jw6 okay, thanks
18:34 manfred np
18:34 kober Does this mean I setup my salt state wrong? http://paste.ofcode.org/EwWeMtrBBsCsW2yRkgcXdF
18:34 kober I have a top.sls file in the same directory I'm running that command
18:36 kober I can show my top.sls as well
18:36 kober http://paste.ofcode.org/7864DAdThrb7zQ3Xpzj2yJ
18:37 manfred kober: did you configure your master file with a seperate roots: to point to the current directory?
18:37 manfred does -l debug show it finding top.sls
18:38 nyx joined #salt
18:39 Guest24882 manfred: Gareth: thanks!
18:39 kober manfred: This is my etc/master http://paste.ofcode.org/Wu7g6xU9bwGM8zCMP8R2H9
18:39 kober You think that blank output is debug it didn't find a top.sls?
18:40 kober is because* it didn't find a top.sls
18:41 kober manfred: here is some of the debug output: http://paste.ofcode.org/Gc7xrKCZnLKJ2VTZdDwGHg
18:42 manfred interesting
18:42 roolo joined #salt
18:42 kober This is salt-ssh '*' state.highstate, so no minion or anything
18:43 kober Its ssh'ing as a normal user not root
18:43 manfred kober: i think you need to specify your file_roots in there so that it points to your current directory
18:43 kober so maybe its confused and not running sudo
18:43 kober It doesn't tell you if it can't find any files?
18:43 manfred that i have no idea
18:44 manfred i am wondering if that is the problem
18:45 matthia__ joined #salt
18:45 kober Is there a way to find out what file its trying to run to check?
18:46 zandy joined #salt
18:47 zandy_ joined #salt
18:49 kober manfred: this is my setup http://paste.ofcode.org/uwT2hmki8iXDmQJLkMa9dh
18:50 kober manfred: this is the etc/master file: http://paste.ofcode.org/ZWL68YeFDvKCqRMbtCTCve
18:52 manfred it looks like it should work
18:52 manfred soory, I am getting busy at work and don't have time to test it out
18:52 tyson_ joined #salt
18:53 forrest DAMN YOU BILL PAYING WORK, DAMN YOU!
18:53 kober Yeah, I'm just not familiar enough with salt to know what to do, if it gave me an error I could fix it, but its just a blank output
18:54 manfred kober: it looks like it just isn't finding top.sls
18:54 KaaK joined #salt
18:55 KaaK where is pillar data stored on minions?
18:55 manfred KaaK: /var/cache/salt/minion
18:55 kober Yeah, but how do I find out where its looking?  I set my master file_roots to a hard coded path just to test and it didn't help
18:56 manfred kober: # The state system uses a "top" file to tell the minions what environment to
18:56 manfred # use and what modules to use. The state_top file is defined relative to the
18:56 manfred # root of the base environment as defined in "File Server settings" below.
18:56 manfred should be looking in the base file_roots
18:56 manfred so
18:56 manfred should be correct now
18:57 kober Can I get debug output of what path its trying?
18:57 kober I guess I could find where it loads and set a pdb
18:57 manfred kober: i just start droping log.debug() everywhere when i can't see that stuff
18:58 manfred maybe drop a log.debug(pprint.pformat(__opts__)) somewher to see what the options are set to.
18:58 manfred and make sure your file_roots is set right... /shrug
19:00 vejdmn joined #salt
19:01 kober Do you know where it tries to load top.sls so I can check what file_roots is at that point?
19:02 tyson_ joined #salt
19:02 bezeee joined #salt
19:04 manfred kober: https://github.com/saltstack/salt/blob/develop/salt/state.py#L2127 that is when it opens/caches the file/ and puts it into the tops variable (or the second one, depending on environment stuff)
19:04 manfred you'll want self.opts for log.debug(pprint.pformat(self.opts))
19:05 ghartz joined #salt
19:05 manfred yup, already has log in that file
19:06 Comradephate joined #salt
19:07 vejdmn joined #salt
19:10 martoss joined #salt
19:11 snuffeluffegus joined #salt
19:11 vejdmn joined #salt
19:13 bezeee i’m installing pip using get-pip.py like this: https://gist.github.com/brianz/82e85ea94a6dcc235d7b
19:13 bezeee what would be the best way to require pip is present in my other state files?
19:14 bezeee requiring a package doesn’t make sense since it’s not installed via the package manager
19:14 forrest bezeee, as a question, why are you not using the package?
19:14 peters-tx joined #salt
19:14 manfred bezeee: just require that the state is present
19:15 kermit joined #salt
19:15 bezeee usually the package mangers have really old pip versions
19:16 manfred update pip using pip, once pip is installed
19:16 bezeee manfred: I’m still coming up to speed on salt…do you mean: require: -getpip ?
19:16 manfred are you using a state to install pip?
19:16 forrest bezeee, and yea, +1 for what manfred said, I've added a comment to your gist.
19:17 forrest manfred, I've already added a comment with an example :P
19:17 manfred cool
19:17 manfred i really should /quit irc
19:17 forrest or just minimize, works well
19:17 bezeee got it….that’s what i was wondering about…if the include + including the sls was the way to go
19:17 manfred dwm doesn't have minimize, and I have to check the internal chat for stuff.
19:18 manfred aight, i cm closign my freenode bouncer
19:18 manfred o/
19:18 forrest bezeee, yes, include the sls, and then you can see I have a require as well. Technically you don't need the require, but it helps explain the example
19:18 forrest manfred, later
19:18 bezeee thanks manfred and forrest
19:18 forrest bezeee, yar
19:19 bezeee i didn’t even think about installing via pkg then just self-dating
19:19 bezeee ha…s/dating/updating
19:19 forrest bezeee, yea I'd honestly just grab a more recent release, and then put it in your repo
19:19 bezeee i like that approach better since it removes the need for the include
19:19 forrest bezeee, or you could just store it locally and user the sources stuff for pkg.installed, etc.
19:19 jslatts joined #salt
19:19 forrest bezeee, well, you still need to do the include if you require a package that was installed in another state.
19:20 bezeee i see
19:20 kballou joined #salt
19:20 danielbachhuber joined #salt
19:22 beneggett joined #salt
19:23 DanGarthwaite joined #salt
19:26 retrospek joined #salt
19:27 tyson_ joined #salt
19:32 tyson_ joined #salt
19:33 aberdine joined #salt
19:34 KennethWilke joined #salt
19:45 tyson_ joined #salt
19:46 rypeck For salt - is there anything like pep8 or puppet-lint?
19:47 zandy joined #salt
19:48 forrest rypeck, not really, you can import yaml and render the state out, but it doesn't work very well if there is jinja. you can take a look at https://github.com/jesusaurus/salt-shaker but it hasn't been updated in a while
19:48 nahamu https://github.com/saltstack/salt/issues/802
19:49 forrest nahamu, heh
19:49 forrest really, we should have something that does the yaml linting, but also does a jinja check :\
19:49 forrest the problem is when you have stuff being pulled in from pillar. You're basically doing a state 'run' without doing a run, which is problematic.
19:50 smcquay_ joined #salt
19:50 tyson_ joined #salt
19:50 forrest I was thinking about just combining the yaml syntax checker with https://groups.google.com/forum/#!topic/salt-users/1ZYVGn5BnzQ
19:51 rypeck Alright so I'm gonna be a PITA and start comparing Salt to Puppet again. This is part of my learning process... How do I reuse an sls module while changing the values for a particular node or group of nodes?
19:52 rypeck In puppet I create a module with classes and provide parameters to it. I don't see an equivalent in salt
19:52 forrest rypeck, can you gist a simple example of that for me?
19:52 forrest it's been like 2 years since I used puppet
19:52 forrest so I don't remember the syntax very well
19:53 rypeck it's more the concept than a syntax. You build out code to control various resorces, files, services and then for each node you can define the parameters and what gets applied
19:53 rypeck let me give you a gist of what I expect to be able to do...
19:53 _jslatts joined #salt
19:53 forrest rypeck, gotcha, did you already look at the 3rd and 4th examples under this section? http://docs.saltstack.com/en/latest/topics/pillar/index.html#declaring-the-master-pillar
19:54 tyson_ joined #salt
19:54 cpowell joined #salt
19:54 rypeck Is that the right link? forrest
19:54 forrest yea
19:55 forrest there's an example there using matching based on the OS of the system (you could also base it off of other stuff)
19:55 forrest there's also node groups, and the associated targeting for groups of servers: http://docs.saltstack.com/en/latest/topics/targeting/nodegroups.html#node-groups
19:56 rypeck forrest: so I can set up a different set of data in a pillar and have a State file use that instead?
19:57 smcquay joined #salt
19:58 forrest rypeck, based on certain matching and ways of breaking it down yea. So you could say, have a different nameserver depending on a specific grain or something in the pillar data.
19:58 forrest or your custom grains or whatever.
19:59 rypeck I am going to have the same state files for my nodes - just each needs to have slightly different settings - and I want to be able to manage which settings are applied to each group of nodes easily.
20:00 forrest rypeck, what kind of settings? Settings in managed files?
20:00 rypeck Yes.
20:01 mackstick joined #salt
20:02 forrest rypeck, then you might need to do a combination of pillar data, and templatizing formulas
20:02 ipmb what's the trick to get minions created with salt-cloud to run state.highstate after the build?
20:02 rlarkin ryprypeck: I use a custom grain ( role ) , and also match subnet
20:02 rypeck forrest: yea okay. I'm gonna take a stab at it. Thanks.
20:02 ipmb I tried adding "start_action: state.highstate" to my provider in /etc/salt/cloud but it doesn't seem to do anything
20:02 forrest rypeck, np
20:10 kober manfred: I got it working just by tinkering, would be nice to get a way to debug that easily
20:11 Katafalkas joined #salt
20:16 patarr joined #salt
20:16 patarr joined #salt
20:16 mway joined #salt
20:16 allanparsons joined #salt
20:17 patarr joined #salt
20:17 patarr joined #salt
20:18 allanparsons (sorry if this was reposted - didn't look like it posted earlier):  ok so that bug (?) I logged yesterday - i think it's because publish.publish isn't returning on time.  is there anyway I can wait on a module to complete?  https://github.com/saltstack/salt/issues/14605
20:20 kober Now that I have a working system, whats the best way to manage "roles", I have "state.highstate" which installs everything in the root top.sls but I don't know if I would ever want to globally install everything
20:20 kober I think what I want more is to say "Install mysite.com role to this set of servers"
20:20 allanparsons kober:  you can use a top file to manage "roles"
20:20 allanparsons and target either on grain data or host glob matching.
20:20 allanparsons (we target on ec2_tags)
20:21 kober Ours would be similar, on openstack metadata
20:21 gothix allanparsons, are you deploying code from jenkins via saltstack ?
20:21 allanparsons so:  'G@ec2_roles:*webserver* and G@ec2_apps:*myapp*':
20:22 allanparsons gothix - no.
20:22 allanparsons gothix i'm deploying directly to the minions
20:22 allanparsons and i need minion B (in process of running it's state) to find minion A's IP based off of grain data
20:23 cyberjames joined #salt
20:23 allanparsons given that it sometimes works and sometimes doesnt, I suspect it's an issue w/ the rest of the state not waiting on publish.publish
20:23 gothix allanparsons, thats what im trying to do via a jenkins job being kicked off - not sure the best way to go about it
20:24 allanparsons gothix - maybe someone in the channel can recommend a solution other than publish.publish?  Or, recommend a way to have a state wait on the outcome of a module.
20:25 gothix thanks
20:25 bezeee joined #salt
20:25 kober allanparsons: Do you know of an example of that?
20:25 allanparsons of using openstack metadata?
20:26 allanparsons you'd have to probably write a custom grain (which is what we do for ec2 data).
20:27 jmccree joined #salt
20:29 kober allanparsons: I meant just roles based
20:29 carmony basepi: is there any documentation on outputters?
20:29 kober allanparsons: So I went to say something like salt-ssh 'webservers' <run this role>
20:30 allanparsons kober - you could do something like in your top file:  https://gist.github.com/allanparsons/b96881811c16198b0ed0
20:30 allanparsons and then run a high state
20:30 basepi carmony: not in a general sense.  each output module has its own documentation.  full list here (http://docs.saltstack.com/en/latest/ref/output/all/index.html) and docs for highstate here (http://docs.saltstack.com/en/latest/ref/output/all/salt.output.highstate.html#module-salt.output.highstate)
20:31 allanparsons or, kober, you could do:  ssh -G 'ec2_roles:*webserver*' state.highstate
20:31 ocelot joined #salt
20:31 allanparsons or, kober,  salt glob-hostname*.yourdomain.com state.sls <state_name>
20:31 carmony basepi: so it calls a output(data) for each host
20:31 allanparsons actually, you'd need quotes around that last one:  salt 'glob-hostname*.yourdomain.com' state.sls <state_name>
20:32 carmony is there a way we could add a final(data) or something with the data from all the hosts so you could do a summary line?
20:32 carmony or a start() and end() ?
20:32 _alpha_ joined #salt
20:33 basepi carmony: I thought we had solved that, let me take a peek
20:34 basepi carmony: it appears that `output()` is only called once, with a dictionary of host:hostdata
20:34 kober allanparsons: I was worried that state.highstate would run things I didn't want because its a generic top.sls
20:35 basepi carmony: but....i could have sworn we have summary data on the default highstate outputter
20:35 kober allanparsons: Running a specific .sls seems more my style
20:35 basepi carmony: let me dig a little more
20:35 kober allanparsons: Thanks!
20:35 carmony basepi: ok, because it feels like output(data) gets called more somehow
20:35 carmony because it doesn't wait for the minion to return
20:35 carmony or for ALL minions to return
20:35 carmony to start showing data
20:36 allanparsons kober - you could do something this in your top file.  https://gist.github.com/allanparsons/ddf2ab11cf1b2cc72f71
20:36 jslatts joined #salt
20:36 allanparsons if i have an openvpn server with tag:  ec2_apps: openvpn-server
20:36 allanparsons if i run:  salt '*' state.highstate    : anything in my infrastrcture will get all base packages.  and openvpn servers will get "openvpn.server" salt state
20:37 kober So you just basically always rely on grains, so you would never target arbitrary servers?
20:37 allanparsons kober, if i run salt 'G@ec2_apps:*openvpn-server*' state.highstate  , I'm only targeting anything with that ec2_apps tag.  So, my openvpn servers will get anything in that BASE config AND anything in the ec2_apps:*openvpn-server* section
20:38 kober That makes sense
20:38 Katafalk_ joined #salt
20:38 kober and so if you have grains in top.sls, you can limit them even more by targeting on the CLI still, right?
20:38 kober So I could deploy to only half of the webservers if I wanted
20:38 basepi carmony: oh, now i remember.  we have summaries per host, but not overall, and i think that's where the feature request is.  let me see if i can find the actual issue to verify
20:38 allanparsons if i don't feel like waiting forever to install base packages, I can "manually" install an SLS to my openvpn servers doing:  salt 'G@ec2_apps:*openvpn-server*' state.sls openvpn.server, anotherpackage, maybe.another
20:39 basepi carmony: it also appears that the highstate outputter is designed such that it can be given a number of hosts at a time, but in practice it's only given one host at a time (dictionary with a single host:hostdata pair)
20:39 allanparsons you can also target a specific machine, kober, using globs.  ex:  salt openvpn-server-001.myhost.com state.sls openvpn.server    -- this will install openvpn.server on just the minion with that id (openvpn-server-001.myhost.com)
20:39 basepi carmony: here's the issue:  https://github.com/saltstack/salt/issues/11199
20:40 allanparsons kober, if i want to install on all vpn servers (assuming i have a nice naming convention), i can do:  salt 'openvpn-server-*.myhost.com' state.sls openvpn.server   (notice the wildcard glob matching)
20:41 FeatherKing joined #salt
20:42 allanparsons kober, yes, you can limit even more on the cli.  you can override.  lets say i have an apache state, but for some odd reason i want apache on the openvpn server even though it's not defined in the top file:  salt 'openvpn-server-*.myhost.com' state.sls openvpn.server
20:42 carmony basepi: where in the code does it call the outputter?
20:44 Katafalkas joined #salt
20:44 aberdine_ joined #salt
20:44 poogles joined #salt
20:44 basepi salt/output/__init__.py is where the abstraction and some format-guessing happens, pretty sure it all goes through there.  in the case if highstate, the state system will just call display_output(data, out=highstate)
20:44 basepi carmony ^
20:45 carmony basepi: awesome, thanks
20:45 carmony I really need to get my local machine setup for saltstack dev
20:45 basepi =)
20:46 forrest carmony, are you on linux?
20:46 forrest if so, why not use lxc or docker?
20:46 carmony forrest: OS X
20:46 forrest makes it easy
20:46 forrest oh
20:46 forrest well, VMs at digitalocean are 5 bucks a month
20:46 carmony I know :)
20:46 allanparsons kober, if you are ever afraid, you can always run:  salt <target> test.ping to make sure your grains are targeting correctly.  and, you can also run test=True at the end.  eg:  salt <target> state.sls openvpn.server test=True
20:46 forrest carmony, well then no excuses for you! :P
20:46 basepi hahahahaha, "oh, you're on OSX?  rent a vm."
20:46 carmony haha
20:46 zandy joined #salt
20:46 forrest basepi, better than dealing with vagrant
20:46 basepi (it's funny because it's true)
20:47 basepi i do very little salt testing on my local machine
20:47 forrest yea I do 0
20:47 carmony my issue is... I like sublime text :P
20:47 forrest yea I don't blame you there
20:47 basepi That would do it.  =)
20:47 carmony so I like to edit code locally
20:47 forrest you can make vim look similar to sublime at least
20:47 allanparsons oh god, sublime text.
20:47 allanparsons <--- PyCharm
20:47 forrest bleh pycharm
20:48 allanparsons ha ha
20:48 forrest I refuse to pay a yearly license
20:48 carmony allanparsons: I use PhpStorm for PHP work I do :P
20:48 carmony haven't bought PyCharm yet
20:48 forrest pycharm is an awesome app
20:48 basepi That's one reason why I've never tried to properly learn sublime or PyCharm -- can't take them with me.  =P
20:48 forrest I just don't like the company
20:48 basepi (to my VMs, i mean)
20:48 forrest yea I am mostly using vim nowadays
20:48 forrest allanparsons, do you pay for your pycharm license?
20:48 allanparsons lol, no @forrest
20:49 forrest allanparsons, hah
20:49 allanparsons vim = such a steep learnign curve.  i'm always too busy to learn anything more than esc-i, esc wq, esc q!
20:49 forrest allanparsons, just use vim tutor
20:49 forrest type that into the console
20:49 forrest no space
20:49 forrest vimtutor
20:49 forrest sorry
20:49 forrest and it walks you through all the most useful stuff
20:49 forrest it's great
20:49 allanparsons sick!  my last day is friday
20:49 forrest uses examples and everything
20:49 allanparsons now i have something to occupy my time.
20:49 forrest it won't take that long
20:50 allanparsons oh :(
20:50 Gareth forrest: I completely blame carmony.
20:50 forrest you could go through the flask tutorial
20:50 forrest Gareth, hah
20:50 carmony ? :P
20:50 allanparsons forrest - i am
20:50 forrest the mega tutorial?
20:50 basepi forrest: let's face it.  just because you know vim basics doesn't mean you can be productive in it.  still have to wade through that learning curve
20:50 allanparsons no
20:50 allanparsons link?
20:50 forrest allanparsons, http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world
20:51 forrest basepi, that's very true. But I'm all for letting people dip their foot in the pool
20:52 basepi definitely.  worth it for sure, but just FYI, it's hard going.  stick with it.  =)
20:52 basepi and then you'll be happy to know that pycharm has the best vim emulation i've ever seen
20:52 forrest yep, I still suck in it
20:52 forrest Seth is a master at it
20:52 basepi so you can actually apply what you learn in vim
20:52 forrest ridiculous
20:52 basepi ya, seth is awesome at it.  i'm decent.
20:52 Gareth basepi: It's all about the addons and your .vimrc.
20:53 basepi i don't use its full power
20:53 basepi yes and no
20:53 allanparsons drewstokes is way too good at vim
20:53 allanparsons nm... dstokes
20:53 allanparsons like, has some odd freakish keyboard that makes him fast at vim.
20:53 allanparsons he has like 19 tmux windows open on 2 cinema displays at all times.
20:53 basepi haha, it's not the keyboard, it's the muscle memory
20:53 forrest lol
20:54 * basepi goes back to work
20:54 aberdine_ joined #salt
20:54 * dstokes tips hat, types furiously
20:54 forrest dstokes, no fedoras
20:55 dstokes never..
20:56 forrest just making sure there with the hat tipping
20:57 bmatt ooh, interesting. I'm seeing zombie salt-master processes
20:57 ksalman forrest: I increased worker_threads to 16 but the result is the same re: https://github.com/saltstack/salt/issues/14645
20:57 forrest ksalman, :\
20:57 viq_ joined #salt
20:58 forrest at this point I'd try to confirm if it's specific minions that are problematic, then see what version of salt they are running
20:58 forrest which might suck to do, lol
20:58 jut joined #salt
20:58 ksalman forrest: that's the thing, I picked some random minons and they are all having this issue
20:58 forrest weird
20:58 ksalman granted I haven't tested all 400+
20:59 forrest is there any correlation on those systems?
20:59 forrest I think you need to confirm first that it's totally random as much as that will suck to do
21:00 ksalman I suppose I can run through all of them with ssh in a for loop (poor mans config management)
21:03 bmatt that was fun
21:03 bmatt discovered a new salt-master failure mode
21:04 bmatt zombie salt-master processes holding tmpfs FDs open, filling the disk
21:05 rap424 joined #salt
21:05 jhauser joined #salt
21:05 allanparsons @dstokes i wonder if thats why we ocassionally have to bounce a salt master
21:06 bmatt this is actually choking the whole host
21:07 patarr joined #salt
21:07 patarr joined #salt
21:09 williamthekid_ joined #salt
21:10 bhosmer joined #salt
21:15 Katafalkas joined #salt
21:15 bezeee joined #salt
21:18 kermit joined #salt
21:29 forrest carmony, good tweet there
21:30 bmatt oh whoa
21:30 kober Link or it didn't happen
21:30 bmatt so /var/cache/salt/master/jobs is consuming 392866 inodes
21:31 bmatt it's not filling up tmpfs, it's consuming all the inodes on the filesystem
21:32 forrest lol
21:32 forrest bmatt, yea clean that directory up
21:32 forrest bmatt, someone else was having this problem before
21:32 bmatt hm. so known issue?
21:32 bmatt with long-lived salt masters? :)
21:32 bezeee joined #salt
21:33 bmatt can I just indiscriminately blow the contents away?
21:33 forrest bmatt, well, that's a pretty low number of inodes
21:33 bmatt yeah, I think we'll need to tune the fs a bit :)
21:33 forrest bmatt, and yes, you can just destroy the jobs in there from what I recall last time. It is just records
21:34 bmatt but it's an order of magnitude more than any other dir off /
21:34 bmatt wow, this disk really only have 500k inodes? that seems like a small number
21:35 forrest yea that's pretty small
21:35 forrest even this lxc I'm looking at has 1.3 million
21:35 forrest and that's still pretty low
21:35 gothix Are there any opinions on the best way to deploy a war file to a application server via a jenkins job?
21:36 forrest gothix, I'd use http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.copy
21:36 forrest to copy the file over and overwrite an existing one, then you can just do a cmd.run to deploy it
21:38 nyx joined #salt
21:38 gothix i was thinking write a custom grain with the url as it might change  per the code version and call a salt-call state on the server
21:39 forrest gothix, hmm, then you'd have to update the grain though right?
21:41 gothix forrest, yes, wrtite some text and call the state.sls to do the work
21:41 forrest Could you not store that data inside of pillar?
21:41 forrest just to make it more centralized
21:41 Ryan_Lane joined #salt
21:42 gothix how would i know when the version of the release changes or snapshot?
21:42 forrest I'm not sure, I just wasn't sure how the grain was planning on being updated
21:42 gothix so thats what i am trying to figure out how to do
21:42 kober Is there a good document for a minimal minion config?
21:43 forrest gotcha
21:43 kober I want to test out the vagrant salt integration
21:43 forrest kober, minimum config is just to install the minion, then you can run commands locally. If you want to connect to a master, just update that field in the conf and restart the service
21:46 zandy joined #salt
21:47 arthabaska joined #salt
21:51 KaaK can someone help me understand how/what pillar data gets on to a minion for security audit purposes?
21:53 forrest KaaK, did you read the first note here: http://docs.saltstack.com/en/latest/topics/pillar/#storing-static-data-in-the-pillar ?
21:53 agliodbs joined #salt
21:54 agliodbs I have a top.sls file where on one salt master, PCRE matches are just silently failing.  2014.01.4
21:54 agliodbs any idea what could cause that?
21:55 ksalman forrest: i verified that all of the minions are failing exactly the same way :/
21:56 forrest ksalman, that sucks
21:56 forrest all 400?
21:57 ksalman yea
21:58 forrest that sucks
21:58 ksalman yea..
21:59 forrest I'm not sure where to go from there, Ryan_Lane any other suggestions for ksalman? He's still having the auth issue even after upping the number of workers, and all the machines are failing in the same way
22:00 Ryan_Lane I don't. sorry. I haven't used a master in like a year
22:01 forrest lol
22:01 ksalman =)
22:03 herlo how does one check to see what salt says is the hostname of a machine?
22:04 laubosslink joined #salt
22:04 gzcwnk salt-key -L  ?
22:05 oz_akan joined #salt
22:05 jslatts joined #salt
22:06 rgarcia_ joined #salt
22:06 herlo gzcwnk: even if I'm only using the minion?
22:06 * herlo goes to check
22:06 herlo gzcwnk: not even installed. :(
22:06 gzcwnk huh? so you are on the minion?
22:06 herlo no
22:06 herlo all of my nodes are minions
22:06 herlo I have no master
22:07 herlo it's intentional
22:07 gzcwnk ah ok
22:07 herlo my minions run amok. :)
22:08 gzcwnk no idea, sorry
22:08 herlo gzcwnk: it's okay, I'll ask around.
22:08 gzcwnk i use master mionion model
22:08 herlo gzcwnk: I would, except the master doesn't gain me anything.
22:08 gzcwnk a master is very easy to use
22:08 herlo gzcwnk: I'm aware.
22:09 * herlo originally packaged salt for fedora/centos/epel.
22:09 herlo back in the day. :)
22:09 retrospek herlo: depends on the system
22:09 retrospek http://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.html#minion-id-generation
22:09 herlo retrospek: well, I've got a top.sls with my hostname in it, but it's only running things that match '*'
22:09 * herlo looks
22:10 retrospek well are you wondering how targeting happens or how minions choose their nodename?
22:11 herlo retrospek: thanks, that helped. I had changed my minion's hostname after running salt the first time, had to remove the /etc/salt/minion_id file
22:11 herlo retrospek: how minions choose their id
22:11 herlo but I think you gave me the right info
22:11 herlo targeting makes sense.
22:11 * retrospek nods
22:12 retrospek ltns btw :P
22:12 herlo ltns?
22:13 herlo oh, lol
22:13 herlo retrospek: same to you
22:13 herlo retrospek: what is your name IRL? I'm having a hard time putting things together today (and lately overall).
22:13 herlo or maybe I never knew it, but I think I did... :)
22:14 MatthewsFace joined #salt
22:14 retrospek oh just remember you from back when we only had 10 people in #salt a good while back
22:14 retrospek mostly bc of the aforementioned packaging haha
22:14 retrospek 'herlo hep uz'
22:14 herlo :D
22:15 retrospek need new build of 0.9.x halp
22:15 kober joined #salt
22:15 herlo retrospek: your nick is familiar, we should def meet IRL sometime then, for sure.
22:15 herlo retrospek: ahh, 0.9.x. Those were the days. Such naivete. :)
22:15 d_p joined #salt
22:15 kober Can you run your states as root?  I'm trying to install a package but its saying it failed: " The following packages failed to install/update:"
22:16 herlo retrospek: I took a long break from salt, but I'm back now.
22:16 herlo kober: I'd hope so.
22:16 kober I assume because of no root access?  But it doesn't tell me why
22:16 herlo kober: if you can't, most things won't work. :)
22:16 kober I've noticed salt assumes you are running root for everything everywhere in the docs
22:16 herlo you can run your master as non-root
22:16 herlo kober: right, because you kind of need it for many things, installation, system config, networking, etc.
22:17 kober Yeah, I'm just used to using sudo instead of straight root
22:17 kober How can you tell it to run with sudo for certain things?
22:18 herlo kober: well, as a rule, I don't
22:18 herlo kober: mainly, the point of configuration management is idempotency. Making things run the same every time. Users shouldn't access stuff as root, but cm is probably better if it can run as root.
22:19 retrospek herlo: yea same. only 17 major releases since then. nothing changed *cough*
22:19 kober Yeah, I just don't enable root ssh, I have users with ssh keys
22:19 herlo lol
22:20 herlo kober: don't use ssh
22:20 herlo kober: use either 0mq or raet
22:20 * herlo is hoping to use raet, since he feels 0mq is still insecure.
22:20 kober herlo: Well you are mr.not helpful aren't you
22:20 kober You sound like mdeehan from ansible "You are doing it wrong!"
22:20 herlo kober: actually, I'm not trying to be unhelpful. :)
22:21 retrospek kober: running it as nonroot for root tasks makes no sense
22:21 herlo kober: but using ssh inherently as root would be bad, I agree.
22:21 retrospek going through sudo doesn't solve that
22:21 retrospek just makes it worse
22:21 kober I'm using salt-ssh, not zeromq or raet, and I don't want to enable root ssh
22:21 Ryan_Lane herlo: for the most part you can't run without root
22:21 retrospek if you have tasks that run as normal users (virtualenvs, containers) you can run salt as those users
22:21 kober So I need sudo
22:21 herlo kober: but you can grant a certain user root rights, which works with an ssh key. Though it's essentially the same thing. But you can do things like limit ssh access to a single set of commands.
22:21 herlo Ryan_Lane: I agree.
22:22 retrospek then run it through sudo
22:22 herlo kober: have you ever setup a separate user with limited ssh access?
22:22 herlo it's awesome and probably what you want.
22:22 herlo kober: you can limit them to just running salt-ssh or what not.
22:22 herlo as root. Then it's the one sudo command you do do. :D
22:22 retrospek you can configure sudo to allow specific users to run specific commands (and patterns) with no password required if that's your goal
22:23 kober Yeah, all our SSH users have limited access.  We have a service oriented architecture, so certain teams get access to their stuff with their keys
22:23 herlo retrospek: exactly
22:23 Ryan_Lane alas, salt wasn't designed to work the way you'd like to run it :(
22:23 retrospek as it isn't a salt problem :P
22:23 retrospek it's a sudo one
22:23 herlo kober: no, not even that limited. No tty, for instance.
22:23 kober ahh, we haven't got that deep yet
22:23 * herlo finds a command
22:23 kober Ryan_Lane: you are saying salt isn't designed to have a multi-user administration team?
22:24 kober Its designed for a single admin with god powers?
22:24 Ryan_Lane if you're using salt-ssh? basically, yes
22:24 Ryan_Lane but ansible works that way too
22:24 herlo kober: http://serverfault.com/questions/441108/how-do-i-allow-users-to-execute-commands-via-ssh-without-allocating-a-pseudo-ter
22:24 retrospek there's a distinction. salt in normal mode. yea what ryan said
22:24 oz_akan joined #salt
22:24 retrospek you are asking for userland CM which is a different beast
22:24 herlo kober: the point of cm is that it's a GOD on a system.
22:24 kober Ryan_Lane: We only have 400 servers but we manage it all over ansible with ssh in seconds
22:24 kober I don't see the problem with ssh
22:25 retrospek again the issue isn't ssh
22:25 herlo it does the changes, but it's automated. And if someone compromises it, you are screwed. yes, that's known.
22:25 Ryan_Lane it's not an issue with ssh. it's an issue that salt-call assumes it's root
22:25 herlo kober: you can use ssh
22:25 kober Ryan_Lane: I can't have it run sudo salt-call?
22:25 herlo I just prefer the speedier options
22:25 Ryan_Lane kober: yes, you can do that
22:25 kober Ryan_Lane: I don't mind salt-call being ran as root, I just don't want the ssh connection to be made as root
22:25 retrospek add a sudoers rule for salt-call
22:25 herlo kober: you still have to have a user that can do all of the things root can do, so why not just use root?
22:25 Ryan_Lane I don't see why that wouldn't work
22:26 herlo Ryan_Lane: I think it would.
22:26 retrospek ssh user@foo sudo salt-call \\* test.ping
22:26 Ryan_Lane in what way does it not work?
22:26 retrospek voila
22:26 * Ryan_Lane runs salt-call via sudo all the time
22:26 kober retrospek: So do it manually instead of using salt-ssh? :P
22:26 Ryan_Lane bleh. does salt-ssh not support this?
22:26 retrospek that was an example
22:27 herlo Ryan_Lane: In the way that you'd have to allow the 'user' access to do anything that salt wants to do. for instance, partitioning disk or formatting won't work with sudo unless you give the user you are sshing in as root rights via sudo, then it's just a matter of allowing 'all all'
22:27 kober It might, thats what I was asking from the beginning
22:27 andrej I think I might be too stupid to use the mine.  I still don't get it, after hours of reading and trial and error
22:27 kober but then we got on the topic that I was doing everything wrong
22:27 Ryan_Lane it looks like it does
22:27 Ryan_Lane you need to specify it in the roster
22:27 Ryan_Lane http://docs.saltstack.com/en/latest/topics/ssh/#salt-ssh-roster
22:27 herlo kober: I didn't imply you were doing anything wrong. I'm sorry you got that impression.
22:27 Ryan_Lane note the sudo: True line
22:27 andrej assuming i'm on one minion - how do I get to a grain value of another minion via mine?  I have made sure (checked the minion dir on the master) that the values are present and up to date
22:27 Ryan_Lane http://docs.saltstack.com/en/latest/topics/ssh/roster.html#targets-data
22:28 herlo kober: sudo just makes things harder and is one step away from root anyway. You have to give the user the root rights for anythign you want to do, which means you have to configure sudo every time you make a change to your CM rules.
22:28 jut It's simple. Corporations have internal policies. Some dictate that ssh root logins are not allowed. Salt-ssh should understand this.
22:28 herlo kober: but you CAN do it, I just don't recommend it.
22:28 kober herlo: Don't worry about it, I just don't like people being like "ZERO MQ IS THE WAY TO GO!"... You have no idea what my corporate infrastructure or needs are, so its silly to jump to those types of conclusions
22:28 herlo kober: oh, I'm not a 0mq guy, that's for sure
22:29 herlo kober: it's why I'm running salt-minion only with a cron job and a git pull
22:29 herlo :)
22:29 Ryan_Lane I think sudo is a perfectly acceptable way to do it
22:29 Ryan_Lane in fact, that's definitely how I'd do it
22:29 herlo Ryan_Lane: it is, totally acceptable. Just a big PITA for CM.
22:29 Ryan_Lane nah. sudoers.d directory
22:29 Ryan_Lane and use groups
22:29 kober Yeah, so its not something you can turn on and off, you just always have to be root
22:30 herlo and then you have to add that every time you add something to CM. Not something I want to have to remember to do. At some point, you're going to give full access to it anyway methinks.
22:30 kober In ansible we can toggle specific tasks as sudo or not
22:30 retrospek bc the operations being done require root
22:30 Ryan_Lane yep. it's not per-task like ansible
22:30 herlo kober: you can do that in salt
22:30 kober 90% of our operations do not require being root
22:30 herlo kober: then go with sudo.
22:30 retrospek then dont run it as root and use sudo
22:30 kober Once system packages are installed and configuration is added, we chown everything as the service owner
22:30 Ryan_Lane you guys aren't actually listening to him :)
22:30 kober from there the service owner can control their specific pieces
22:31 Ryan_Lane ansible runs as non-root, and each individual task can say whether it's sudo or not
22:31 herlo kober: ahh, so a different tack from the standard control everything of CM
22:31 retrospek ryan: part of his problem space is system oriented. sudo for escalation. part of it is userspace. noop
22:31 andrej salt-call mine.get minion-name grains.items    just gives me local: and a line of dashes
22:31 andrej what am I missing?
22:31 Ryan_Lane in salt, you can't say "this state should be run as root, this one should be run as my user"
22:31 andrej ugh
22:31 kober Yeah, we are a service oriented architecture and allow teams to own their part of the system, rather than having a single ops team
22:32 andrej fat fingers .. sorry all, ignore me
22:32 Ryan_Lane kober: I do the same thing
22:32 retrospek not in the same instance no but nothing prevents running multiple (isolated as it should be)
22:32 andrej it's indeed grains, not grain
22:32 Ryan_Lane I also don't use 0mq (or a master at all)
22:32 kober So ops would do the sudo operations to install everything, but from there the service owners don't have sudo, they just have access to the parts on the system that are theirs
22:32 Ryan_Lane kober: ah. that's actually doable
22:32 retrospek i would question why you'd want that in the same salt instance personally
22:32 Ryan_Lane assuming none of their states require root, it's doable
22:32 retrospek if you want the logical separation make it so uniformly
22:33 retrospek one for sys one for users and never the twain shall meet. deescalate the userspace one. done
22:33 kober Yeah, if they require root then it messes salt up
22:33 andrej right ... so, what is the syntax to get one specific grain?
22:33 retrospek kober: are you running states as those users or one userspace (nonroot) user runs it all?
22:33 kober Is there a reason salt couldn't toggle back and forth?
22:33 Ryan_Lane kober: you'd want to run two highstates for this
22:34 retrospek isn't designed to out of the box but can be done
22:34 Ryan_Lane pointing at two different locations
22:34 retrospek far easier to just run multiple instances
22:34 Ryan_Lane retrospek: are you assuming a minion daemon?
22:34 Ryan_Lane I'm not
22:34 retrospek from a secpol standpoint you'd want isolation anyhow
22:34 retrospek i am making no assumption on transport or peerage
22:35 ajprog_laptop joined #salt
22:35 retrospek if anything it's a mutant form of a syndic in salt terms
22:35 retrospek if you REALLY wanted you could have a master (root) that connects to a minion (root) who is a syndic that controls a minion (userspace) to cascade all of this
22:35 Ryan_Lane it's easier to help someone work through their use-case than to try to talk them into a different model ;)
22:35 kober Yeah, I probably just need to wrap my head around the design decisions and maybe change the way we do things within the company
22:36 Ryan_Lane kober: I can explain how to do what you want
22:36 kober Prior to ansible, people were handed users with keyauth.  Those users had access to certain parts of /srv/
22:36 retrospek kober: well when you handle userspace changes are you doing that with a common user or as the user calling salt?
22:36 Ryan_Lane and you can decide whether or not you want to go with this model or not
22:37 Ryan_Lane (I operate in a similar way you're aiming for, but we let our devs have root on their services)
22:37 retrospek bc if you have userland changes like jenkins pushing data around its one thing. if you want the enduser invoking salt-call then that's a different issue
22:37 Ryan_Lane you'd have two repos: base and service (or whatever you want to name them). ops would run base on the nodes via sudo
22:37 kober retrospek: A task would be like "Update application", so currently they do  ansible-playbook  update_service.yml  and it ssh's into the box with their user and key and runs a git pull/pip install
22:37 retrospek or salt-ssh, whatever
22:37 Ryan_Lane devs would run service without sudo
22:38 kober So I want to change that to salt and just do something like   salt-ssh 'webservers' state.highstate
22:38 Ryan_Lane yeah. you'd need to set up minion options, though
22:38 kober or maybe salt-ssh 'specific-service' state.highstate
22:38 Ryan_Lane http://docs.saltstack.com/en/latest/ref/configuration/minion.html#user
22:38 kober and have it use that specific service user account
22:38 retrospek and these devs have root access on the target box in question?
22:39 kober retrospek: Some do have sudo, but not all
22:39 kober retrospek: sudo is nice because it triggers a log of everyime they use it, so we know what is going on in the system
22:39 retrospek well
22:39 Ryan_Lane kober: you'd also need to set some other settings, like cache_dir, log_dir, etc.
22:40 kober ok, so if I switched to agents instead of salt-ssh, I could just create multiple minions on the same server to do this?
22:40 Outlander joined #salt
22:40 Ryan_Lane yes. but I think it's also doable via salt-ssh
22:41 Ryan_Lane in fact salt-ssh may do all the options for you automatically
22:41 kober yeah, I basically just want to do what I do in ansible where I just say "run this task as sudo"
22:41 Ryan_Lane (to run as non-root)
22:41 kober but I'll live with what I can get
22:41 patarr joined #salt
22:41 Ryan_Lane well, you'd need to run an entire state file as non-root
22:41 patarr joined #salt
22:41 retrospek that's why i asked
22:41 retrospek there are 3 different use cases being thrown around ambigiously
22:42 Ryan_Lane no. there's one specific use case. ops runs one set of code and devs run another
22:42 Ryan_Lane devs run as non-root and ops runs as root
22:42 Ryan_Lane kober: correct?
22:43 kober Ryan_Lane: Those are the most important
22:43 kober Ryan_Lane: 3rd would be devs who have sudo
22:43 kober but thats not as important
22:43 Ryan_Lane kober: http://docs.saltstack.com/en/latest/topics/ssh/#running-salt-ssh-as-non-root-user
22:43 kober they can obviously ping ops instead
22:44 retrospek master--> ssh root@minion salt-call; master --> ssh user@minion salt-call; master --> ssh user@minion sudo salt-call;
22:44 Ryan_Lane you could also have a third state file for them
22:44 kober retrospek: yeah
22:44 retrospek given you said devs "may" have sudo access we're focusing on the third for devs?
22:44 kober This is why companies just go with an ops team, way easier to manage a set of people who all get root :P
22:45 retrospek or they properly partition their CM use for system and application use :P
22:45 Ryan_Lane or go with a model where all the devs that manage a service get root ;)
22:45 retrospek they aren't the same
22:45 kober The 3rd way would be the best way, because then it wouldn't matter, the other two ways would work
22:45 retrospek ouch ryan, true.
22:45 Ryan_Lane if everything is config managed, everything would ideally be code-reviewed
22:45 retrospek kober: third way you just run salt non-root and go about your business. provided that the user that you run salt as has full permission to do everything you'll need the CM functionality to provide
22:46 retrospek which means proper group usage and whatnot
22:46 retrospek doing all that nonroot is tricky and most people skip it hence using root
22:46 zandy joined #salt
22:46 yomilk joined #salt
22:46 retrospek if you need salt to be invoked as the calling user (like you're expecting salt to setuid/seteuid) then that's a different conversation
22:47 kober Yeah, the hard part is having to go back and fix the permissions if you are doing everything as root
22:48 kober if you just say "run these tasks as this user" and then special case the root actions, then all permissions are correct without having additional tasks
22:48 dcmorton joined #salt
22:48 kober Because 90% of our actions are non-root, they are application specific
22:48 kober its only when a new system dependency comes about that we would run a root action
22:48 kober but editing app configs, restarting app services, etc. is all by the service owner user
22:49 kober We don't want our applications running as root at all
22:50 kober one bad piece of developer code and a process running as root would be unfortunate ;)
22:50 Ryan_Lane they run under the account of the user sshing into the system?
22:50 aquinas joined #salt
22:51 kober Ryan_Lane: They currently do, we've created accounts for the specific services so that each team can manage them
22:51 Ryan_Lane oh, that's terrifying
22:51 kober I can switch that to group ownership instead of shared auth
22:51 herlo kober: oh, so you are using salt as a deployment mechanism more than for CM it seems.
22:51 Ryan_Lane just because salt runs as root doesn't mean your services need to
22:51 kober herlo: yeah, we use it for CM and deployment
22:51 kober herlo: well, currently ansible, but hopefully salt :
22:51 Ryan_Lane you can specify which user/group owns files and which user commands are run as
22:52 herlo kober: indeed.
22:52 herlo :)
22:52 Ryan_Lane the only services I have running as root as system-level services
22:52 Ryan_Lane everything else has service users added
22:52 Ryan_Lane and the services run under those users
22:52 kober I'll be first to admit that our process isn't standard, but change is harder to push than you would think ;)
22:52 Ryan_Lane your init (or upstart, or systemd, or supervisor) scripts should manage which user owns the service
22:53 herlo kober: no, I think I understand your dilemma. :)
22:53 Ryan_Lane right now if a developer leaves your org you have a pretty serious problem
22:53 kober Ryan_Lane: yeah, thats pretty much how we do it
22:53 kober Ryan_Lane: thats true, since they have the shared key
22:53 kober They would still need VPN and everything, but you are right that its probably bad practice
22:54 Ryan_Lane then salt will work perfectly fine and it's ok to let the users manage it via sudo
22:54 Ryan_Lane if your only worry is stuff running as root. that's avoidable
22:55 gzcwnk does salt have a salt state to ensure DNS1= etc for redhat servers is present?
22:56 kober Yeah, we'll just have to be good about chowning/chmodding but thats not a big deal
22:56 gzcwnk n/m found it...
22:56 kober I think this will work well
22:56 kober I do plan to introduce the minions and everything at some point but SSH is the easiest
22:56 kober We actually spin up every developer box in Vagrant with the same ansible scripts that we deploy our systems with
22:56 Ryan_Lane kober: see http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.managed
22:56 Ryan_Lane it has user/group params
22:57 Ryan_Lane kober: http://docs.saltstack.com/en/latest/ref/states/all/salt.states.cmd.html#salt.states.cmd.run cmd.run has a user param
22:57 mosen joined #salt
22:59 helderco joined #salt
23:04 buhman joined #salt
23:05 thedodd joined #salt
23:09 buhman joined #salt
23:09 rmnuvg joined #salt
23:09 xenoxaos joined #salt
23:11 herlo If I have '{{ grains['ipv4'][1] }}      {{ grains['fqdn'] }} {{ grains['host'] }}' in my /etc/hosts template and - source: salt://common/files/etc/hosts in my sls file, shouldn't things get rendered rather than just printing the literal characters into the /etc/hosts file?
23:12 herlo I am assuming I'm using the jinja2 renderer...
23:12 * herlo looks at basepi with red eyes...
23:12 Outlander_ joined #salt
23:12 andrej are there any mine gurus hanging out here today?
23:18 TheThing joined #salt
23:21 andrej Hmmm ... so to find out why I don't get a response for salt-call -l debug mine.get oob-01\* grains.item main_ip I enabled garbage output on the master ... looks like it never gets interrogated (and local debug on the minion doesn't suggest any problems).
23:21 andrej How does salt-call get mine data w/o talking to the master?
23:21 agliodbs I'm using match: pcre in top.sls, but it doesn't match anything
23:22 agliodbs for example:   '^(uat)?(hr|db)01(a|b)[\.$]':
23:22 nyx joined #salt
23:22 agliodbs *ought* to match the minion "uathr01a", and it does when I test it
23:22 agliodbs on the python command line
23:23 jslatts joined #salt
23:23 agliodbs are the rules for pcre somehow different from the rules for python re module?
23:23 andrej agliodbs , that last part makes no sense ... you're forcing a regex to include a literal backslash, period or dollar
23:23 andrej that can't match your host
23:23 agliodbs andrej: aha!  so I don't need to escape the "." inside brackets?
23:24 andrej No
23:24 andrej But even if you did, it still wouldn't match a period or a dollar
23:24 agliodbs andrej: how would you match "either a period or the end of line"?
23:24 retrospek you do if you expect it to be a literal .
23:24 retrospek you can't match eol via $ anchor inside a class operator
23:25 retrospek ^(uat)?(hr|db)01[ab]\.?$
23:25 andrej '^(uat)?(hr|db)01(a|b)\.*$' should work for what you're after
23:25 andrej heh
23:25 thehaven joined #salt
23:25 andrej or retrospek's version
23:26 agliodbs actually, I think I want this: '^(uat)?(hr|db)01(a|b)(\.*)?$'
23:26 retrospek \.* is gibberish
23:27 retrospek \. = literal period. .* = wildcard anything
23:27 agliodbs wait, fixing
23:27 agliodbs hmmm, can simplify that: '^(uat)?(hr|db)01[ab]\..*$'
23:27 agliodbs no?
23:27 retrospek or just truncate \..*$
23:27 retrospek .* with a terminal anchor is moot
23:28 retrospek if you're not including all the possibilities up to the next . (assuming you're supported subdomain hostnames or something)
23:28 andrej Why would \. with one quantifier be ok, but not with another?
23:28 agliodbs retrospek: except there might not be a period ...  so: '^(uat)?(hr|db)01(a|b)(\..*)?'  wait, no
23:28 agliodbs retrospek: so, here's the thing.  I want to match uathr01a.dot.com
23:28 retrospek he wants to match on uathr01a.sub.example.com
23:29 agliodbs but not uathr01a-test.dot.com
23:29 retrospek and only tokenize to this zone
23:29 retrospek yep
23:29 agliodbs and I want ot match "uathr01a" with no .dot.com
23:29 agliodbs but not uathr01a-test
23:31 agliodbs retrospek: is PCRE the same as the python re module?  Can I use that for testing regexes?
23:33 retrospek no
23:33 retrospek pcre = perl compatible regexes
23:33 vbabiy joined #salt
23:33 retrospek has more functionality (and confusion) than python re
23:36 retrospek python regex is kind of between re and pcre
23:37 Outlander joined #salt
23:38 agliodbs retrospek: oh!  I'll just test in in perl then
23:38 andrej This one really should match what you want, because it won't allow a hyphen before the subdomain ... ^(uat)?(hr|db)01(a|b)\..*
23:39 andrej So whether you have a FQDN with subdomains or not it should match
23:39 agliodbs except that doesn't match the case of their being no domain.  some of the minions I'm matching are just "hr01a" with no .dot.com
23:39 andrej But the - won't pass through
23:41 andrej ^(uat)?(hr|db)01(a|b)(\.)?[^-]* :)
23:45 agliodbs this works according to perl: ^(uat)?(hr|db)01[a|b](\..*)?$
23:45 agliodbs feh, pasteo: ^(uat)?(hr|db)01[ab](\..*)?$
23:46 zandy joined #salt
23:48 jpl1079 joined #salt
23:50 yomilk joined #salt
23:51 che-arne|2 joined #salt
23:51 toolman joined #salt
23:52 thedodd joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary