Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2017-07-03

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:21 edrocks joined #salt
00:24 justanotheruser joined #salt
00:25 justanotheruser joined #salt
00:28 darix joined #salt
00:33 socket- joined #salt
00:45 jeddi joined #salt
01:11 aneeshusa joined #salt
02:03 anotherZero joined #salt
02:12 juanito joined #salt
02:17 evle joined #salt
02:26 jeddi joined #salt
02:29 s joined #salt
02:40 socket- joined #salt
02:44 mosen joined #salt
02:47 JPT_ joined #salt
02:59 major joined #salt
03:01 citaret joined #salt
03:01 vishvendra joined #salt
03:06 icebal joined #salt
03:09 citaret joined #salt
03:14 donmichelangelo joined #salt
03:43 citaret joined #salt
03:47 anotherZero joined #salt
04:03 tobstone joined #salt
04:41 aleph- joined #salt
04:58 Zachary_DuBois joined #salt
05:07 salt-noob joined #salt
05:19 gnomethrower joined #salt
05:22 Lionel_Debroux joined #salt
05:32 tobstone joined #salt
05:38 hasues joined #salt
05:40 yuhl joined #salt
05:49 hasues If I'm building a configuration file with Jinja and salt, and I'm inputting data from pillar, why would it only operate on the first line and truncate everything else?
05:56 pcn joined #salt
05:59 hemebond hasues: We would need to see your state to help.
05:59 hasues Fair enough
06:00 do3meli joined #salt
06:00 do3meli left #salt
06:04 JPT joined #salt
06:04 hasues hemebond: http://dpaste.com/22KXQR0
06:05 hasues hemebond: So my concern is the output at the bottom.  If you look at the cp.get_template call, that is what I would expect to be placed, but note that it truncates every line after the first in the actual file placed.
06:08 Praematura joined #salt
06:11 colttt joined #salt
06:14 hemebond I can't see any problem there.
06:16 hasues Yeah, me neither.  Just don't understand why it isn't replacing the file,but it says from the info at the top "hey, these files are different
06:16 felskrone joined #salt
06:25 dfr joined #salt
06:45 cyteen joined #salt
06:52 JohnnyRun joined #salt
06:55 aldevar joined #salt
06:59 J0hnSteel joined #salt
07:02 _KaszpiR_ joined #salt
07:15 darioleidi joined #salt
07:22 xet7 joined #salt
07:22 Ricardo1000 joined #salt
07:23 ProT-0-TypE joined #salt
07:30 Puckel_ joined #salt
07:32 Mogget joined #salt
07:34 Heartsbane joined #salt
07:35 ccha joined #salt
07:47 Rumbles joined #salt
07:55 impi joined #salt
08:06 mikecmpbll joined #salt
08:13 pbandark joined #salt
08:21 Naresh joined #salt
08:21 jhauser joined #salt
08:26 coredumb Any way to have an encrypted value - pillar or whatever - that would only be decrypted by the minion process?
08:27 coredumb I'm looking for a way for sharing a password that even a root user on the minion's server couldn't retrieve
08:30 Mattch joined #salt
08:33 babilen coredumb: What would stop root from just calling whatever retrieval function you use on the minion directly?
08:33 hasues joined #salt
08:34 coredumb babilen: yep that's my problem
08:36 absolutejam Morning
08:37 babilen coredumb: More coffee might help, but nothing suitable comes to mind right now
08:37 babilen root has control of the minion and therefore access to all the information
08:40 coredumb babilen: yeah my first guess - and how I'm doing it right now with another tool - would be to use a temp script that contains the password that deletes itself upon completion
08:44 LotR a password for what? if it's for some login process, couldn't you just store the already hashed version? no need to have the plain-text version around ever
08:46 coredumb LotR: Joining AD
08:46 coredumb have to use it in the clear
08:46 candyman88 joined #salt
08:59 fxhp joined #salt
09:05 zulutango joined #salt
09:11 jhauser joined #salt
09:14 icebal joined #salt
09:33 OliverUK Does a Syndic node sync the base directories from the master or do minions go all the way to the master for their files?
09:37 sjorge joined #salt
09:42 icebal joined #salt
09:42 coredumb babilen: other option would make a static encrypted binary that does the join part...
09:43 coredumb but it's quite ugly
09:51 Praematura joined #salt
10:11 icebal joined #salt
10:15 yuhl______ joined #salt
10:16 preludedrew joined #salt
10:40 XenophonF coredumb: you can mitigate some of your security concerns by delegating the "join the domain" privilege to an otherwise unprivileged service account
10:42 XenophonF alternatively, you can handle domain joins at O/S install time, outside of salt (e.g., in WDS, in a VMware customization profile, etc.)
10:50 XenophonF I'm getting `SaltReqTimeoutError: Message timed out` when trying to apply even smallish states on my minions (like the epel state from saltstack-formulas/epel-formula).
10:50 XenophonF https://gist.github.com/xenophonf/8d978897166b3777e98070376ffb7bc2
10:50 XenophonF Has anyone seen that error before?
10:51 XenophonF All these minions access the master over the WAN, but I haven't had problems in the recent past.
10:51 icebal joined #salt
10:51 XenophonF They are all running 2016.11.6 on the master and minion.
10:54 XenophonF maybe I need to increase worker_threads on the master?
10:55 coredumb XenophonF: ot's already an unpriv dedicated account
10:55 _aeris_ joined #salt
10:58 XenophonF Maybe use orchestration and have the Salt master reset the account password after joining the domain?
10:59 XenophonF Treat it like a one-time password.
10:59 XenophonF Depending on what you're doing, you have to use orchestration anyway (rename the computer and reboot, wait, join the domain and reboot, wait, etc.)
11:00 coredumb I'm talking about joining a linux box
11:00 coredumb not a windows
11:00 coredumb I don't reboot :D
11:01 _aeris_ joined #salt
11:02 XenophonF same difference - use orchestration to join the domain and reset the password
11:04 absolutejam I know nobody's interested, but I got my 'copy file based on group membership' thing working
11:04 absolutejam Instead of fudging about with jinja, it was way easier to just write a module
11:04 absolutejam and I love that you can cross-call salt modules
11:05 Puckel_ joined #salt
11:06 LondonAppDev joined #salt
11:08 armin so what's the correct way to run a salt-call with state.apply, but in dry-run mode, so nothing actually gets touched?
11:09 armin salt-call -l debug state.highstate test=True ?
11:13 XenophonF yes
11:13 XenophonF that
11:19 hemebond left #salt
11:26 icebal joined #salt
11:31 armin salt-minion[45775]: AttributeError: 'list' object has no attribute 'iteritems'
11:31 armin hmhm, looks like a bug in salt to me if it tries to call .iteritems() on a list object, but i can't find anything related when googling.
11:33 armin maybe a very old bug and we're just using an old salt version while this was fixed ages ago? ;)
11:38 losh joined #salt
11:41 XenophonF well increasing worker_threads didn't help - now my master is unresponsive to even the local minion
11:42 XenophonF and is it normal to have a zombie salt-master process?
11:42 XenophonF root      7145  0.0  0.0      0     0 ?        Z    11:42   0:00 [salt-master] <defunct>
11:42 XenophonF that's after a reboot
11:45 evle1 joined #salt
11:50 XenophonF at least reverting worker_threads fixed local connectivity to the master
11:50 XenophonF still see the zombie process though
11:50 armin XenophonF: since it's impossible that a zombie process survives a reboot, i'm pretty sure you had a salt run after booting.
11:50 dstensnes no zombies survive a reboot
11:51 XenophonF well right - systemd starts salt-master at boot
11:51 XenophonF there shouldn't be a zombie process
11:51 armin true.
11:51 XenophonF it goes away if i stop the salt-master service
11:52 XenophonF this is CentOS 7 btw, fully patched
11:52 snc joined #salt
11:55 dstensnes zombie processes usually happens when when a process spawns a child, and then the child terminates
11:55 dstensnes i think the zombie process is there until the parent has checked the status of the child process
11:57 dstensnes with the "wait" command in C. Don't know how that is handled in python, but at some point "wait" must be called by python to reap the child process
12:00 dstensnes or, until the parent dies too, in which case, noone cares about the return status from the child
12:01 XenophonF wait is a system call, so it's the same in all language runtimes on Unix
12:01 XenophonF there shouldn't be a zombie process
12:01 XenophonF the parent salt-master process should reap it
12:02 icebal joined #salt
12:02 dstensnes is it overloaded somehow?
12:03 dstensnes sorry about the long description btw. lots of people here probably knows it already. I just got a bit carried away :)
12:04 fredvd joined #salt
12:08 goutham joined #salt
12:08 magz0r joined #salt
12:11 goutham hi all
12:12 goutham https://docs.saltstack.com/en/latest/ref/tops/all/salt.tops.reclass_adapter.html what are these ext_pillars in the reclass ??
12:13 goutham does that mean that pillar in salt are replaced by the files in "reclass" ??
12:13 swills joined #salt
12:21 XenophonF and now we're back to having SaltReqTimeoutErrors :(
12:22 XenophonF doesn't seem overloaded
12:22 XenophonF plenty of free memory
12:30 ProT-0-TypE joined #salt
12:43 NegiLXXXVIII joined #salt
12:53 saltnoob joined #salt
12:53 thinkt4nk joined #salt
12:53 saltnoob Hi guys , I have a question re join
12:54 saltnoob im trying to use a mine function to pull IPs of different nodes and write them in the config on a single line as "ipaddr","ipaddr"
12:55 coredumb How can I watch a command execution only if this command has been executed through mathcing it "creates" statement ?
12:56 saltnoob I currently am using this addrs[0]|yaml_dquote which is giving me "Ipaddr""ipaddr" I tried to use a join function afterwards with a comma but that seems to not put a comma between the "" and a comma after each digit of the ipaddress
12:56 saltnoob anyone have any ideas ?
12:57 coredumb https://pastebin.com/Q74bcVZB < the cmd.run is always executed because cmd.script returns True
12:58 coredumb any idea?
12:58 saltnoob your checking if the state ran
12:58 saltnoob its a cmd.shell so will alwasy run
12:58 saltnoob *alwasys
12:58 saltnoob *always
12:59 saltnoob you could search for the retcode of the script I guess
12:59 saltnoob if thats what your actually after
12:59 coredumb ah shoot I forgot about that
13:00 saltnoob {% if salt[cmd.retcode](mybashcheck.sh) %} will probs help you get what you want
13:00 coredumb I should use cmd.wait instead of cmd.run right?
13:00 saltnoob what are you trying to do
13:01 coredumb run the cmd.run only if the cmd.script is actually run
13:01 saltnoob also gist for my issue https://gist.github.com/anonymous/3eb0af5822b5fa014e448521854d2e1b
13:01 coredumb that should happen only once because of creates: xxxx
13:01 saltnoob it will always run though even if it just exits ?
13:02 absolutejam coredumb: use stateful: in cmd.run?
13:02 coredumb ah cmd.wait seems to have make work
13:02 absolutejam cmd.wait is deprecated now so be careful
13:02 absolutejam I mean, it still works
13:03 absolutejam I've been using cmd.run with stateful: True and make my command return a { 'changed':
13:03 absolutejam { "changed": True, "comment": "foo" }
13:04 coredumb oh yeah
13:04 coredumb but cmd.run + onchanges can't work there
13:06 coredumb so it's deprecated but not replaced by anything equivalent? O_o
13:07 coredumb well it would if I merge by two states though
13:09 saltnoob cmd.run was replaced with cmd.sheel wasnt it
13:10 saltnoob and you can pass a flag for old behaviour
13:10 saltnoob *shell
13:10 coredumb I still don't see it in official doc ...
13:12 saltnoob 1 mo :p
13:13 saltnoob no offical doc however is a git issue
13:13 saltnoob https://github.com/saltstack/salt/issues/30460
13:14 saltnoob anyone have any idea about my formatting issue :D
13:15 evle joined #salt
13:16 coredumb ok as you said guess I'll have to go with stateful script
13:52 wangofett joined #salt
13:53 xMopxShell joined #salt
14:12 ronnix joined #salt
14:15 LondonAppDev joined #salt
14:19 saltnoob ended up useing  loop.last to write commas in
14:23 saltnoob {{number | count() /2 + 1 | round|int   }} any reason why the output of this is 2.0 rather than just 2 ?
14:24 saltnoob wouldnt int change is to 2 ?
14:24 saltnoob or am i misunderstadining somthing ?
14:26 felskrone1 joined #salt
14:38 cgiroua joined #salt
14:40 whytewolf coredumb: cmd.run + onchanges vs cmd.wait and watch. they do the exact same thing.
14:41 whytewolf cmd.wait is a blind function that doesn't do anything. the logic is in mod_watch which just performs cmd.run functions
14:42 whytewolf which is why it is deper in favor of cmd.run with onchanges
14:43 whytewolf also since all your second script is doing is rming a directory. why not just use fil.absent
14:43 coredumb whytewolf: yes I've changed all that to be on cmd.run + onchanges
14:43 whytewolf err file.absent
14:43 coredumb the thing that bugged me is that I thought that onchanges wasn't working on separated state ID
14:45 coredumb whytewolf: is cmd.run + onchanges behaves differently or ?
14:46 whytewolf no, it doesn't behavour diferently then cmd.wait + watch
14:46 coredumb I mean differently than onchanges used in other state than cmd.run
14:47 whytewolf well, cmd.run returns changed because it runs. which is counted as a change.
14:47 coredumb wait
14:47 whytewolf same for cmd.script
14:47 coredumb As I understood documentation
14:47 coredumb watch can match changes in other states
14:48 coredumb while onchanges matches changes in same state
14:48 whytewolf um, no
14:48 whytewolf the difference between watch and onchanges is that watch runs mod_watch when it see a change. while onchanges just runs the state it is attached to when the other state changes
14:49 coredumb damn I've add onchanges statements that required to be changes by watch because of that ... Or at least I believed
14:50 coredumb whytewolf: you mean watch would be usable only for services?
14:50 whytewolf no, there are many states that have a mod_watch function
14:51 coredumb ahhhhh
14:51 coredumb but many that don't
14:51 whytewolf yes
14:51 coredumb arf
14:51 coredumb this is severely unclear
14:52 whytewolf this is why the watch documentation states that it is for "changing the behavour" of a state
14:53 coredumb well it says "watch statements are used to add additional behavior when there are changes in other states."
14:54 whytewolf ahh yes.
14:54 coredumb I understand that a bit differently than changing the behaviour
14:54 whytewolf i summerized a little too much
14:54 mikea I couldn't get onchanges to work for my usecase
14:54 sarcasticadmin joined #salt
14:55 whytewolf mikea: what was your use case?
14:55 coredumb whytewolf: no problem thanks for the clarifications
14:55 mikea I'm pulling up the state now
14:55 mikea I have ca-trust/init.sls with the cmd.wait
14:56 mikea and then I have a bunch of other files that push specific CA certificates and in the file.present I use watch_in
14:56 mikea I have to use watch_in because I don't know which CA certs will need to be provided
14:56 mikea so I can't just onchanges from the state in the init.sls
14:56 whytewolf ok, so why didn't onchanges_in work?
14:57 mikea because the command ran
14:57 mikea without changes
14:58 mikea because without the cmd.wait
14:58 mikea cmd.run with no onchanges is just a command that gets run
14:58 coredumb I had another kind of problem actually ...
14:59 mikea I tried specifying an empty onchanges:
14:59 mikea but it just gave me a syntax error
14:59 whytewolf how about a default state that doesn't change.
14:59 mikea yeah, but that's just tomfoolery to work around broken functionality
14:59 mikea when cmd.wait works
15:00 whytewolf cmd.wait won't be around forever
15:00 whytewolf againt it is deperceated
15:01 whytewolf also it seems like to have a cmd.run that shouldn't exist if the states the it neds don't exist
15:01 mikea it does need to exist there because it needs to fire if a update-ca-trust enable runs
15:02 mikea but I have some jinja that switches that specific state between a cmd.run and a test.succeed_without_changes based on if update-ca-trust is enabled or not
15:03 mikea https://gist.github.com/mikeadamz/ee3c7d2ddbd50227c176c2bda7294115
15:05 whytewolf so, you have two states that could trigger an onchanges_in
15:06 Praematura joined #salt
15:06 whytewolf [the cmd.run would always trigger onchanges because it will run which counts as a change]
15:06 jimklo joined #salt
15:06 mikea which is fine because the cmd.run doesn't even exist until it needs to run
15:06 mikea otherwise it's test.succeed_without_changes
15:07 mikea did you look at the gist I linked?
15:07 whytewolf yes
15:07 whytewolf i still don't see a need for cmd.wait.
15:07 mikea if I put onchanges_in those two states and change cmd.wait to cmd.run it won't fire unless there's a change?
15:08 whytewolf exactly
15:08 it_dude joined #salt
15:08 mikea I didn't test that specifically, but I did test it from another sls file
15:08 mikea and the cmd.run ran every time the state applied and reported changes
15:08 coredumb whytewolf: If I understand correctly, for services to run on file modifications I should use watch{_in}, for anything else, onchanges{_in} should work?
15:09 whytewolf test.succeed_without_changes won't trigger a change [no changes] and cmd.run will trigger because cmd.run that runs is all ways coundted as a change
15:09 mikea which is the behavior I am looking for
15:09 mikea if those three files aren't links, the cmd.run needs to run
15:09 mikea otherwise it doesn't
15:09 mikea and the jinja definitely accomplishes that
15:10 mikea so if the cmd.run update-ca-trust enable runs, I need to run update-ca-trust extract every single time
15:10 whytewolf mikea: could also be accomplished with putting the cmd.wait into the jinja block. but you said it might need to run if there are other files
15:10 mikea then again if i drop a ca file in the right directory
15:10 mikea yeah, let me update the gist
15:11 mikea whytewolf, reload the gist
15:12 whytewolf ahh yeah okay then yeah it should be out of the jinja.
15:12 mikea there are a few different versions of that msat.sls pointing at different certificate authorities
15:13 whytewolf but yeah. putting onchanges onto both of those states should have the desired effect. [you can try it with watch first. just to make sure.]
15:13 mikea what would fix this is if you could specify a state ID in onchanges
15:13 mikea so I could just
15:13 mikea onchanges: enable_update_ca_trust
15:13 mikea and it would track that regardless of if it's a cmd or a test state
15:14 whytewolf yeah, unforchantly the only requisite that supports that currently is require :(
15:14 whytewolf and it supports it by watching every module under a stand id
15:14 whytewolf state*
15:15 whytewolf coredumb: sorry basicly yes. there are edge cases of corse. and you use watch with service because you want a reload instead of running
15:16 whytewolf coredumb: to poor fuel onto the fire there is also listen
15:16 whytewolf which is like watch only instead of doing the mod_watch right then. it adds another state at the end of the run that runs the mod_watch function
15:17 coredumb whytewolf: yeah I've not yet digged this one up :D
15:17 coredumb whytewolf: like you modify x files but want to restart only once?
15:18 coredumb kind of behaviour
15:18 whytewolf coredumb: yes. [although watch would still do that becasue it puts the other things before it in order of states]
15:19 coredumb yeah :D
15:20 whytewolf listen works well for things like i want to make sure that if anything changed all of my services restart at the end. in case i was a doof and forgot something
15:23 dstensnes hello. I want to make a module for salt that waits for a unixsocket connection and relays the information transmitted there as an event
15:23 dstensnes what kind of module would that be? Beacon?
15:23 dstensnes to run on all minions
15:25 mikea probably an engine?
15:25 dstensnes okay, thanks
15:25 dstensnes trying to figure out the terms here :)
15:25 mikea I suspect you can do the same thing with a beacon, but I don't believe beacons are meant to be long running
15:26 dstensnes okay, thanks :)
15:26 mikea the difference between a beacon and an engine, if I am correct is a beacon is a quick check and a return, an engine starts with the minion/master and stays until it dies
15:26 mikea whytewolf, is that correct?
15:26 dstensnes yes, i see that here...
15:26 whytewolf close.
15:27 dstensnes beacons have a function that is called every so often
15:27 dstensnes that is supposed to return some data
15:27 whytewolf engines stick around and if they die the minion will restart them.
15:27 whytewolf it is kind of like a daemon inside of salt
15:27 whytewolf and they do a lot more then beacons do
15:27 mikea we're using an engine to pulp event data out to a database
15:27 dstensnes great. Exactly what I need i think
15:28 dstensnes i want some events from syslog to be relayed via the event system in salt
15:28 dstensnes not sure i'm going about this the right way, but i started playing with a daemon thingy
15:28 dstensnes so now i will try to convert that to an engine module
15:29 * whytewolf wishes the documentation on writing engines was better.
15:29 mikea if your daemon is python its pretty easy
15:29 * dstensnes too
15:29 defswork joined #salt
15:29 dstensnes mikea: i'm making it in python, since salt is python
15:29 dstensnes i'm not horribly good with python, but right tool for the right job and all that
15:29 mikea the engines loader will call a start function with the kwargs  fed to the engines: master opts paramater
15:30 mikea so if you've already got something written, remove any fork code and replace the __main__ stuff with a start function
15:31 whytewolf kinda glad they didn't overwrite __main__ :P
15:31 mikea yeah
15:31 mikea for a long time you had to put the engine file in /usr/lib/python/site-libs/salt/engines
15:31 mikea but now you can toss it in /srv/salt/_engines/ I think
15:32 dstensnes can i also tell salt to load it from state file?
15:32 mikea and it'll get picked up when you do a saltutil sync_engines
15:32 whytewolf yeap. they put it in the dynamics
15:32 coredumb Is there a way for the minion to _not_ cache some files?
15:32 dstensnes or does it need a config file on the minion to actually start it?
15:32 whytewolf config on the minion
15:32 whytewolf sorry.
15:32 mikea before it'll run you need to specify a engines config on the minion
15:32 dstensnes awh....
15:33 whytewolf [don't think the engines lib uses config.get]
15:33 Rumbles joined #salt
15:33 dstensnes i had problems with beacons like that too
15:33 mikea so you need to create a state to drop a file in /etc/salt/minion.d/ assuming that's the config directory you're using
15:33 dstensnes impossible to define beacons from the server like it said in the documentation
15:33 whytewolf huh
15:33 mikea you can define beacons and engines from the master, you just need to manage a file
15:33 whytewolf where did it say you could define beacons not in the config?
15:33 dstensnes at least one of the beacons failed because list-vs-hash functionality
15:34 dstensnes uhm
15:34 mikea ./etc/salt/minion.d/engines.conf: file.managed, foobar etc etc
15:34 dstensnes at least one of the beacons failed because list-vs-hash error stuff, that i didn't find a solution for
15:34 whytewolf you mean list-vs-dict?
15:35 dstensnes ah, yes
15:35 dstensnes it's called dict in python
15:35 dstensnes https://docs.saltstack.com/en/latest/ref/states/all/salt.states.beacon.html
15:35 dstensnes tried doing that
15:35 dstensnes but i think it was the load module that i couldn't manage like that
15:35 whytewolf so ... a list is items with a - next to them, a dict is the ones without
15:35 dstensnes because of list-vs-dict issues
15:35 whytewolf :P
15:35 dstensnes yes :)
15:36 whytewolf you just have to remove the dashes if it wants a dict.
15:36 dstensnes so i ended up having a file.managed thingy pushed to the minion, and then restart the minion with some salt-call stuff
15:36 whytewolf or add them if it wants a list
15:36 dstensnes the beacon.present stuff insisted on list, but the beacon insisted on dict
15:36 dstensnes could not figure it out
15:37 cyborg-one joined #salt
15:38 dstensnes if i understand the problem correctly, then the beacon stuff needs some code cleanup
15:38 dstensnes for consistency
15:39 whytewolf eh, better documentation actually.
15:39 dstensnes hmm, okay?
15:40 dstensnes i think i tried pretty much every permutation possible, but I would love for someone to prove me wrong
15:40 whytewolf which beacon were you trying?
15:40 dstensnes i think it was the load beacon that didn't work with beacon.present
15:41 dstensnes or possibly the inotify beacon
15:41 dstensnes cannot really remember
15:41 khaije1 joined #salt
15:42 dstensnes i just know i spent quite some time trying to use beacon.present, and either beacon.present was complaining because i gave it a dict or the beacon wouldn't run because i had a list to beacon.present
15:42 dstensnes if i remember correctly
15:45 hasues So when a state is ran, and I get blue text instead of green, what is that indicating?
15:46 dstensnes whytewolf: are you testing it?
15:46 whytewolf yeap
15:47 whytewolf hasues: dark blue or cyan?
15:47 hasues whytewolf: cyan
15:47 whytewolf it is a change
15:48 hasues whytewolf: Any reason why I would be getting a message saying a change occurred but it doesn't happen?
15:48 fritz09 joined #salt
15:48 coredumb yeah OK I can just remove the file from cache
15:55 whytewolf hasues: are they happening on something like cmd.run? [which are always counted as a change]
15:56 hasues file.directory
15:56 hasues I'm wondering if it is because I have a _ in the label/name of the delcaration.
15:57 dstensnes is there a command i can run to see what the state file looks like after jinja rendering?
15:57 whytewolf dstensnes: cp.get_tempalte on the minion. or you could use show_sls to see how salt sees it. there is also a module i wrote that is like cp.get_template but returns the data to the master
15:57 hasues dstensnes: You can use salt-call cp.get_template and reference the file you want to see and then give it a target of where you want it copied after rendered
15:58 hasues What whyte said :P
15:58 bakins joined #salt
15:58 dstensnes aha, thanks :)
15:58 JohnnyRun joined #salt
15:58 whytewolf hasues: no shouldn't be a _ if that was causing problems would be a lot worse then just reporting a change
15:59 whytewolf most likely something IS changeing over and over again.
15:59 whytewolf like a file getting put in place. then permissions updated
15:59 whytewolf and two states fighting over permissions of the file
16:00 hasues whytewolf: I'm asking to create a directory, it says it reports true, but it never makes it.
16:00 whytewolf strange.
16:00 dstensnes whytewolf: i think it was the diskusage plugin actually
16:00 whytewolf it should report an error if it can't create it
16:00 dstensnes now that i test it
16:00 dstensnes diskusage beacon i mean
16:01 khaije1 How can I reproduce the Jinja rendering process in a python REPL? Im seeking to learn about and test using jinja macros with Salt states.
16:01 hasues whytewolf: It seems to be on any of these with a _ in the name.  So I'll do some tests.
16:01 dstensnes i fired up my test environment again, because it was from around that time
16:01 whytewolf dstensnes: honestly i think the beacons loader needs some work. i would add some bug reports
16:01 whytewolf hasues: use heavy use of -l trace
16:02 khaije1 Seems like some filters are added and others are removed, is there a module I could import to get a facsimile environment, (or perhaps I'm thinking about this wrong) ?
16:02 dstensnes whytewolf: i would also add some bugreports, i just wanted someone elses input as well, to verify before i post
16:02 whytewolf khaije1: honestly you would be better off just using salts render engine directly instead of trying to reproduce it. there are a lot of little things they add and subtract from it
16:03 pbandark1 joined #salt
16:04 khaije1 whytewolf: OK, I'll pay attention to that. Sounds like cp.get_template would be helpful for me too.
16:04 whytewolf yeap.
16:04 whytewolf or https://github.com/whytewolf/salt-debug
16:06 khaije1 Seems handy.
16:07 whytewolf it is, saves me hours when debugging jinja things that pop up in this channel
16:08 khaije1 I'm trying to figure how to remove the u/unicode-string prefix from the dictionary keys returned by my macro. Any initial recommendations or thoughts?
16:09 whytewolf humm. i can't think of any. maybe try forcing it to ymal with salts built in yaml filter. not sure if it might toss in other issues though
16:09 dstensnes whytewolf: can you try this? https://pastebin.com/u46XnAhW
16:09 * khaije1 considers hardcoding the macro params and running cp.get_template ...
16:09 dstensnes whytewolf: that should work, shouldn't it?
16:10 dstensnes if i apply that state, i get an warning in my minion log: "/usr/lib/python2.7/dist-packages/salt/beacons/__init__.py:56: DeprecationWarning: Beacon configuration should be a list instead of a dictionary."
16:10 whytewolf dstensnes: one second i need to test that.
16:10 khaije1 I've not had success when trying to use |yaml, load_yaml, or import_yaml ... so feeling like a bit of a klutz while working on this task.
16:11 dstensnes and after that i get this every 10 seconds:  File "/usr/lib/python2.7/dist-packages/salt/beacons/diskusage.py", line 72, in beacon  \n mount = mounts.keys()[0]  \n AttributeError: 'str' object has no attribute 'keys'
16:11 aldevar left #salt
16:11 dstensnes whytewolf: thanks :)
16:12 khaije1 These are files being processed by the pillarstack system, so that may be a factor
16:12 Praematura joined #salt
16:12 whytewolf oh, that could be a factor pillarstack takes the salt jinja and does other strange things to it iirc
16:13 whytewolf i havn't worked directly with pillarstack so can't be 100% sure
16:14 khaije1 I'm doing lots with it lately, really liking it too it's just hard to know if it's meant to keep all the same promises as Salt proper. Some of the variables are explicitly different, for example.
16:14 wangofett joined #salt
16:15 * khaije1 concludes: more testing needed
16:18 _JZ_ joined #salt
16:19 whytewolf dstensnes: something isn't lining up. i have my minion set to log_level_logfile: all and i am only getting basic logs
16:20 whytewolf but what it does say is that it can't load the module
16:20 dstensnes whytewolf: oh, that's weird. But it should work, shouldn't it?
16:20 PatrolDoom joined #salt
16:21 dstensnes i mean... the yaml for that state looks correct, as does the parameters to diskusage, except for them being a list instead of a dict?
16:21 whytewolf in thoery. there is a couple of things that make me question it.
16:21 jauza joined #salt
16:22 whytewolf like i don't know if the / is making it to the other side
16:22 dstensnes i tried quoting that too, but didn't make a difference
16:22 dstensnes as far as i could tell
16:23 whytewolf in any case. i don't think you are running into a dict vs list issue as currently beacons also use lists.
16:23 dstensnes hmm
16:23 whytewolf unless beacons.present is doing something stupid like trying to convert a list to a dict
16:24 dstensnes hmm, i don't really know...
16:24 dstensnes i have given up on it for now
16:25 dstensnes but i want to write a bug report, but I'm not really sure where the problem lies
16:25 dstensnes i would like to provide some more info before i report it
16:25 nledez joined #salt
16:25 whytewolf you don't have to give line numbers in the code.
16:26 dstensnes true
16:26 dstensnes but ideally i would like to provide a patch, but i think i have a way to go there :)
16:26 whytewolf just give them what you are trying. the expected result. and what is happening. also make sure you have tried it with out useing the state. make sure you actually can get it to work in the config
16:26 psychi[m] joined #salt
16:27 hasues This is what I'm seeing when trying to use file.directory, http://dpaste.com/0PE4R45.  Changing the _ character out didn't make a difference
16:27 dstensnes yes, it's working fine when i run it from a minion.d config file
16:27 dstensnes anyway, i will look more at that later....
16:27 dstensnes it's late here, so i think i will head home
16:28 dstensnes anyway, thanks for cluing me in so far whytewolf
16:28 whytewolf have a good rest of your day
16:28 dstensnes thanks, same to you
16:29 onlyanegg joined #salt
16:30 whytewolf hasues: strange. directories around it are being created. and staying created. is it possable that it is creating it but something else is deleting it afterwords?
16:30 hasues No
16:30 hasues Well, maybe?  haha, I don't think so.
16:31 hasues I mean, I chunked all the other states I have running but that one, and it still is not creating it.
16:31 whytewolf cause i thought that it checked the state of the directory to make sure that it actually created it. other wise returns error
16:32 anotherZero joined #salt
16:33 whytewolf hasues: have you tried running the state with -l debug directly on the minion?
16:33 jauza joined #salt
16:33 jauza left #salt
16:33 hasues Yes.  That was the execution of -l trace.  I was doing -l debug before that
16:34 whytewolf there is a lot of data missing for a -l trace
16:34 hasues I'll grab debug's output
16:34 whytewolf like the info around when it actualy trys to create the directory
16:35 jauz joined #salt
16:35 hasues Yeah, I was trying to find relevant info out of -l trace.  I didn't include all of that as it was a book :(
16:35 jauz Anyone here use SaltPad, still functional on 2016.11.x?
16:35 impi joined #salt
16:36 whytewolf jauz: should be, salt api hasn't changed that much since saltpad was made
16:36 jauz Cool, thank you. :)
16:36 hasues Just says it is in the correct state :\
16:37 cmichel joined #salt
16:37 whytewolf in the correct state? that doesn't make sence if the directory doesn't exist
16:37 hasues I agree
16:37 hasues I broke salt :(
16:37 hasues Hang on, now it is green
16:38 hasues Uh, it put it there now?
16:38 hasues wth
16:38 whytewolf no, it had to be there already for it to be green
16:38 whytewolf cyan would be it putting it there
16:39 hasues Right, so apparently one of those times worked?  I don't understand.
16:39 hasues Have to step away from the keyboard.
16:39 whytewolf ok
16:43 ThomasJ|m joined #salt
16:43 freelock joined #salt
16:43 jerrykan[m] joined #salt
16:43 gomerus[m] joined #salt
16:43 theblazehen joined #salt
16:43 hackel joined #salt
16:43 toofoo[m] joined #salt
16:43 fujexo[m] joined #salt
16:43 it_dude joined #salt
16:47 noobiedubie joined #salt
16:54 Edgan joined #salt
16:56 Praematura joined #salt
16:59 do3meli joined #salt
17:01 wangofett joined #salt
17:03 failatsalt joined #salt
17:03 ChubYann joined #salt
17:07 do3meli left #salt
17:08 astronouth7303 hm, `npm.installed` is failing me. It looks like an internal command is spewing a very large json that's getting truncated
17:09 astronouth7303 which appears to be two bugs: 1. Why is it truncating the command output at 64k? 2. Why is it calling npm in the application environment instead of globally?
17:13 * whytewolf shrugs
17:32 jeffspeff joined #salt
17:33 Trauma joined #salt
17:34 noraatepernos joined #salt
17:36 astronouth7303 hm. It looks like cmd.run will AND `creates` and `onchanges`. I want it to OR.
17:37 astronouth7303 ie, run it if this thing doesn't exist, OR if this other state has changed things
17:44 tiwula joined #salt
17:47 whytewolf astronouth7303: that would be because creates is internal to cmd.run and onchanges is global.
17:47 astronouth7303 ok. How do I get the behavior I want? Do I have to specify it twice?
17:49 whytewolf humm. maybe use a onfail: fail.exists instead of creates.
17:49 whytewolf https://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.exists
17:49 infrmnt joined #salt
17:50 whytewolf not really sure.
17:51 astronouth7303 i feel like the global condition logic is going to prevent control flow to ever reach cmd.run, so I'm pretty sure I just have to specify the state twice (once for init, once for refreshing)
17:54 astronouth7303 it might run twice during stand-up (initial highstate), but it's an idempotent operation
17:54 astronouth7303 (django `manage.py collectstatic)
17:55 izibi joined #salt
17:57 LondonAppDev joined #salt
18:02 edrocks joined #salt
18:10 it_dude joined #salt
18:10 miruoy joined #salt
18:13 fredvd joined #salt
18:20 noobiedubie joined #salt
18:26 aldevar joined #salt
18:34 JJJJJJdglaslg joined #salt
18:44 Rumbles joined #salt
18:47 darix joined #salt
18:48 Slimmons joined #salt
18:53 iggy why not just run it every time if it's idempotent?
18:56 astronouth7303 it's somewhat expensive? Involves copying >100 files
19:02 sjorge joined #salt
19:03 smartalek joined #salt
19:21 raspado joined #salt
19:22 raspado hi all, i want to create a directory if it does not exist in /var/log/newrelic
19:22 raspado if i have salt create this with file.directory, will salt try to change permissions recursively in each directory, for example /var , /var/log, /var/log/newrelic, or just newrelic directory?
19:28 coredumb raspado: only if makedirs is set and that parent doesn't exist
19:28 raspado okay, is it safe to do /var/log/newrelic: with makedirs True, or False?
19:29 raspado hmm, probably false...
19:29 raspado var/log should always be there
19:29 raspado thx coredumb
19:30 oida_ joined #salt
19:34 whytewolf makedirs:true should have no issues with that. it will only change permissions on /var/log/newrelic and any subdirectories after that. it will only change permissions on anything before /var/log/newrelic if they don't exist already
19:35 whytewolf also astronouth7303 one question. why have creates at all in your cmd.run ... the only time that would come up is when you first install. which all your files would be changing then anyways so will run
19:35 whytewolf onchanges should be enough
19:35 astronouth7303 it also comes up when you're doing your initial development
19:36 whytewolf you means when the files are being put there to begin with so are counted as changed?
19:36 astronouth7303 i suppose i should just polish off my new-instance process and just spin up new boxes
19:37 astronouth7303 no, when i first add the state
19:37 whytewolf so it wasn't run before you put salt on the system?
19:37 sjorge joined #salt
19:38 whytewolf but everything else exists?
19:38 astronouth7303 it's all greenfield, but I'm building up a complex salt system from scratch, so I'm adding more states to handle more of the process. I'm just being really lazy about applying the states to fresh instances because I'm not on a public cloud.
19:40 astronouth7303 (although what i should probably do is to run the build over in CI and then deploy the built version through salt, but i'm not even sure how to handle that data shuffle...)
19:40 whytewolf can i say i hate the term greenfield [and by proxy brownfield]
19:41 astronouth7303 yes you can
19:41 astronouth7303 *shrug*
19:41 astronouth7303 so i added the state to grab the git repo a while ago, but now i'm working on actually turning that into something I can point nginx at
19:44 whytewolf i guess, i spoil myself sometimes. every time i test a change i spin up an entirely new instance to test against.
19:44 cyborg-one joined #salt
19:45 whytewolf i keep forgetting some don't have that luxury. they are adding salt to existing archtecture or using limited resources.
19:45 astronouth7303 yeah, i'm on vmware, so i've had to do a chunk of work to get the new-instance process going, and it's not smooth.
19:45 astronouth7303 if i was on aws, i would make an orch file and just do that
19:47 whytewolf I'm on openstack as well as VirtualBox for local dev work. vagrent helps with the local stuff
19:47 astronouth7303 i need to see if vagrant can build vmx's....
19:48 astronouth7303 i'm just glad we have at least a private cloud. I can't imagine trying to do this with all-physical instances
19:49 whytewolf lol, i do, do it with pysical systems. what do you think builds my openstack setup :P
19:49 whytewolf astronouth7303: https://www.vagrantup.com/docs/vmware/configuration.html
19:50 juanito joined #salt
19:50 astronouth7303 1. i'm on debian, not osx, 2. I'm using vSphere/ESXi, not local fusion
19:51 astronouth7303 (that's basically VMware's private cloud solution)
19:51 whytewolf I know what vSphere is. and i wouldn't call it a private cloud. more just virtualization :P
19:52 whytewolf we use vmware as a hypervisor for our openstack at work
19:52 astronouth7303 i'm treating it as such, since it lets me take a pile of hosts and treat them as a vm computation fabric
19:53 astronouth7303 or at least, generally, abstract away the VM from the machine running it, even if it's not automagic
19:54 whytewolf anyway vagrant as far as i can tell is only for local virtualization not report.
19:55 whytewolf terraform would be the remote tool
19:55 astronouth7303 yeah, i haven't found anything that'll push directly to vSphere. I'd settle for something that can build a VMX file.
19:57 whytewolf well salt does have a vmware "cloud" module not sure how well it works
19:57 whytewolf and it does require an already built template to copy from
19:58 astronouth7303 it works just fine, but you need a base to clone from
19:58 astronouth7303 or, that's the simple solution
19:58 astronouth7303 there's more options, but I don't understand how you actually get the initial disk for it
19:59 whytewolf which really.. is how clouds normally work anyway. they are just copies of already prebuilt images
19:59 astronouth7303 (maybe set up your PXE to handle that? :shrug:)
20:00 sjorge joined #salt
20:02 Lionel_Debroux_ joined #salt
20:16 hashwagon joined #salt
20:19 coredumb astronouth7303: look at packer
20:19 astronouth7303 coredumb: i will
20:19 coredumb it has build module for vmware
20:19 astronouth7303 cool.
20:20 astronouth7303 my other problem is that there's about 5 other things that need to be worked on/fixed
20:21 coredumb good luck then :D
20:23 astronouth7303 tomorrow. trying to get this node app to deploy has destroyed my brain and half the office has left anyway
20:23 astronouth7303 s/node/ember/
20:24 armguy joined #salt
20:27 juanito_ joined #salt
20:30 aldevar left #salt
20:30 shortdudey123 joined #salt
20:30 noraatepernos joined #salt
21:03 hasues Back
21:03 xMopxShell joined #salt
21:16 hemebond joined #salt
21:21 censorshipwreck joined #salt
21:34 devster31 joined #salt
21:40 it_dude joined #salt
21:47 flowstategames joined #salt
21:53 noraatepernos joined #salt
21:56 onlyanegg yeah, packer is pretty awesome. I use it to build AMIs for AWS and OVAs for vagrant so I can use the same image in my local env as I do in prod.
21:57 onlyanegg build from ISO -> provision with salt -> output to ova -> send to s3 for AMI conversion
21:57 whytewolf i use it for building my openstack images. pretty good product.
22:02 jeffspeff joined #salt
22:05 coredumb actually is there anything remotely equivalent?
22:20 whytewolf could maybe use salt
22:20 hasues whytewolf: I still see similar issues from earlier.  File.managed isn't creating a file with the proper mode.  I get the cyan writing where it mentions it, but it doesn't do it.
22:21 whytewolf humm. really need to find the info in the debug
22:24 hasues Oh weird, my original issue came back, too.  That other directive isn't creating the directory.
22:24 hasues Maybe my python is broken.  I'm out of ideas.
22:24 whytewolf try finding info in the debug. last time you tried it "worked"
22:26 hasues Okay, I found it in the debug in an INFO statement.  Says that it executed, and that it completed, but it isn't there.
22:27 whytewolf i think something is deleting it
22:28 whytewolf if you can try running just that state and see if anything else changes
22:29 hasues Yeah, so I'm thinking you are right.  I went back and commented back to only that state and it works correctly.
22:34 hasues So my top.sls calls a state, then it calls another state, and that state has an include for the first state.  Maybe that is causing issues?
22:35 hasues I was doing that to simply say that if you do call Y, make sure X is performed first.
22:36 whytewolf well an include doesn't really do much for that.
22:37 hasues So it is a bad idea?
22:37 hasues Are you saying that if I put the include in, it will by default always re-execute the state?
22:37 hasues (the first state)
22:37 hasues Or does Salt evaluate that and say it isn't needed.
22:37 whytewolf no I'm saying that an include doesn't mean that the states added at the include will be at the top or bottom
22:39 hasues Well, if I put state A, then State B in top.sls, and in State B I say to include State A, doesn't the order already ensure that State A will execute?  Also, in State B, since at the beginning of that state, will the include not make that happen first?
22:39 whytewolf the include does absolutly nothing to order
22:39 hasues What does it do??
22:40 hasues I'll re-read https://docs.saltstack.com/en/latest/ref/states/ordering.html
22:40 whytewolf are you using require {which is what the include section is about]
22:40 hasues I am not.  I'll read on that next.
22:41 whytewolf basicly the include makes it so that the state is avalible to be used in a requisite
22:41 hasues Oh, it is mentioned here
22:41 hasues Ah
22:42 digitalr00ts joined #salt
22:42 hasues I see, I should be adding require sls: state A
22:43 whytewolf if all of the state ids in state a should be required.
22:44 whytewolf other wise pick and chose with module: state id
22:45 hasues Well, I suppose the idea I was thinking was "hey, if you ever use this state for separate purposes, require this other state".  This way if I want to mess with it for some other reason then what I am doing, it pulls in what is needed.
22:45 hasues I'm probably re-re-inventing the wheel.
22:46 whytewolf well, is it round?
22:46 whytewolf right now i don't think the include is causing your problem
22:46 hasues Heh, the debugging I'm doing seems to be a bit round-ish.
22:46 hasues It probably isn't.
22:47 hasues But if I can pick up on doing something better, might as well incorporate it.
22:47 whytewolf true
22:47 filippos joined #salt
22:48 whytewolf so, i would be looking for a file.recurse or file.directory that has clean: true. or a command that runs that might change the layout of the directory
22:48 whytewolf defintly look for another cyan state.
22:52 hasues Well, those cyan statements disappear up intil I include more states up to a point, so I'm trying to figure out why.
22:54 whytewolf well, I'm saying watch for the cyan state cause what ever is causing it. is 90% most likely also going to be a cyan state
22:55 hasues Okay, so I included this state.  On the first run?  It says everything is fine.  I re-run the high state a second time, cyan statements appear.
22:56 CrummyGummy joined #salt
22:58 whytewolf okay. it was green because the directory existed at the point in time when salt checked to see if the directory already existed. it found it. it existed and was in the proper state. but if a later state deleted it it is only going to report what it know so it is still green even though a later state deleted it
22:58 hasues Makes sense.
22:59 whytewolf so when you run it the second time. directory is gone so it has to recreate it. and the same state that deleted it which seems to be after the state that creates it. deletes it again.
22:59 whytewolf you need to find the state that is deleting it
23:00 hasues I'm in the one that makes those problems occur.
23:00 whytewolf so you add this state to the highstate and it goes wonky?
23:00 hasues Yeah.
23:01 whytewolf can you share it? or does it contain private stuff?
23:01 hasues I added it.  Ran those commands like I said, and it works, then next one, it goes cyan
23:01 hasues I can share it.  Let me gather the data
23:01 whytewolf k
23:07 hasues http://dpaste.com/29ZHEYX
23:08 whytewolf so many file.directories
23:08 hasues So the portage.sls works, but when I introduce layman.sls, it causes the issues
23:09 hasues To create the needed configuration directories.
23:09 noraatepernos joined #salt
23:12 whytewolf not that it matters but you don't need a cmd.run to create a blank file
23:13 whytewolf file.managed can do it i think there is also a file.touch
23:13 hasues I'll take any tips you want to pass.  I can't get better if I don't learn it, so feel free to give notes.
23:14 hasues So, what was your suggestion to my calls to multiple file.directories?  Just making a loop with a dictionary or some such?
23:14 whytewolf I'm confused, nothing in layman.sls should cause this. you arn't even running a cmd to laymon that should update anything
23:14 whytewolf hasues: personally i would use a couple of file.recurses
23:15 whytewolf build the directory structures locally.
23:15 whytewolf and i was just commenting by the number of them. other wise it is fine.
23:16 whytewolf if you have just portage.sls and layman.sls does it happen?
23:16 hasues I'll read up on how to use it.  I did that originally as I tried making configuration for software via multiple software file.directory calls, but if two states call the same command, it complains, so I made sure some master salt state (portage) creates all that is what is needed for what I have
23:17 hasues Correct.  Both of them together cause the issue.
23:17 whytewolf with out any other state files?
23:17 hasues Correct
23:18 hasues (I just tested it.  I have three that are called, but if I comment out the second one, it does the same thing)
23:18 whytewolf humm. maybe it is portage it's self causing it
23:18 hasues I can capture that second one and send it if you want.
23:18 hasues Yeah, that's what I'm wondering.
23:19 whytewolf if you commant out the app-portage/layman state in the laymon.sls does it go away?
23:20 hasues I'll test
23:20 whytewolf also, to combat the conflicting names. use discriptive state id's and put the directory info in - name
23:20 hasues Oh no, it does. :(  that means portage must be doing it.
23:20 hasues (not the sls, but the actual utility)
23:20 whytewolf ew.
23:21 hasues Conflicting names?
23:21 hasues Oh I see, you are saying because they shared the same state id to my previous issue of having to make some master state to manage those directories being created.
23:21 whytewolf yes
23:21 hasues I guess I was being casual and relying on the name of the state id to be passed to the method.
23:22 hasues I suppose I don't have to do that.  If the state ID is what is really the issue, I can do that.
23:22 jholtom joined #salt
23:23 whytewolf unforchantly. it has been years since i have touched gentoo [about 9]. so i am going to be of little help going forward.
23:24 hasues No no, you have been extremely helpful.
23:24 hasues I wouldn't expect you to provide OS help.
23:24 hasues that's something I can ask on #gentoo and get a better understanding.
23:24 whytewolf well, i typically do if i know the os well ;)
23:25 whytewolf you might need to use -l trace to see if you can get the exact commands that salt is running through portage. if it even uses a cmd.run which it most likely isn't.
23:42 nicksloan joined #salt
23:44 hasues whytewolf: Okay, I'll see if I capture that and see what I find out
23:56 masber joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary