Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2016-04-12

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 nZac joined #salt
00:02 andrew_v joined #salt
00:03 teatime iggy: I am.
00:03 teatime obviously I am, it wouldn't make any sense any other way :)
00:05 iggy then I don't know why you're worried about indentation
00:06 kevinquinnyo i forgot that i never actually solved a problem, i cheated a while back and forgot -- i need a state or possibly orchestration that does the following:  1) generate ssh keypair on a master server, snag that public key, and put it into the minion's authorized keys file.  I'd prefer not to generate keys on the master and/or save their contents in a pillar
00:06 teatime iggy: I want to read it.
00:06 kevinquinnyo i forgot to add numbers besides 1) above... but yeah
00:07 kevinquinnyo is there a clean way for example to set the generated public key as a grain so i could use saltmine to snag it from the master for all the 'slave' minions?
00:07 kevinquinnyo if that makes sense
00:07 teatime kevinquinnyo: you can't just manually copy the master's pubkey into a pillar value?
00:08 teatime grains can't give you data that isn't already on the minion
00:08 teatime kevinquinnyo: double-check your problem description, something isn't lining up here.
00:08 kevinquinnyo i dont want to do anything manually obviously..
00:08 teatime well if you only have 1 master, it doesn't seem like a bad thing to do manually.
00:09 teatime but you can script putting it into a pillar easily enough.
00:09 kevinquinnyo i dont only have one master, i am automating the creation of clusters
00:09 teatime aha.
00:09 kevinquinnyo platform as a service type deal where a cluster can be created on demand
00:10 hemebond Could you have a script that generates the key into /tmp, brings up the minion and then pushes the key to the minion?
00:10 teatime you can put things into mine other than grains
00:10 teatime mine functions can be any salt module function
00:10 teatime but you can just as easily script / use a state to copy the master's pubkey over to a pillar file, which avoids the whole salt mine thing
00:11 teatime do it at the same time you generate the key
00:12 kevinquinnyo hemebond: could you elaborate?  /tmp where?
00:12 hemebond kevinquinnyo: On the master?
00:12 hemebond Maybe I've misunderstood the issue.
00:12 kevinquinnyo how would i get the key from the master to the minion's authorized_keys file?
00:13 hemebond Actually I don't understand the problem.
00:13 teatime kevinquinnyo: something like file.append ?
00:13 hemebond Is there not an execution module for adding SSH keys to minions?
00:13 kevinquinnyo there is
00:14 kevinquinnyo but is there one that allows me to get the contents of a file (/root/.ssh/id_rsa.pub on the master-minion) and then use that as a variable to the ssh keys module?
00:14 teatime I don't understand the "I'd prefer not to generate keys on the master" part, nor do I understand why you wouldn't want to save public keys to the pillar (or even private keys, but I understand wishing to generate private keys on the actual host where they're used.)
00:15 teatime which is why I asked you to double check the problem-statement, as it seems, taken altogether, as a non-sequiteur
00:15 kevinquinnyo well im open to anything that will get the job done i guess
00:15 teatime you're only talking about 1 keypair, right?  on the master?
00:16 kevinquinnyo no
00:16 kevinquinnyo 1 keypair per cluster
00:16 teatime well ok, but still
00:16 kevinquinnyo so if i have 50 clients, then 50
00:16 teatime ok
00:16 teatime but within each cluster
00:16 kevinquinnyo yes just one keypair
00:17 teatime and it's for use in logging into minions from th emaster?
00:17 teatime then where else would you want to generate it but on th emaster?
00:17 kevinquinnyo rsync
00:17 kevinquinnyo from the server1 to the other 9 or so in the cluster, yes
00:18 kevinquinnyo i think i should have used a better term, because i was trying to avoid generating them on the "salt-master", not the per-cluster, cluster-master
00:18 kevinquinnyo but honestly at this point i dont care
00:18 teatime there's a lot of ways to accomplish, "if key does not exist, ssh-keygen, and copy pubkey to a pillar data file targeted to all minions" in your master highstate, and then "if pillar value exists, append it to authorized keys" in the client highstate
00:18 kevinquinnyo it just seemed to me better to generate the keypair on each "cluster-master" since that's how you would do it manually
00:19 teatime you could put it in the file_root instead if you wanted, since ssh pubkeys are not particularly sensitive.
00:19 kevinquinnyo right
00:20 mr_chris joined #salt
00:20 kevinquinnyo well
00:20 kevinquinnyo i'm using git so i dont want to be creating files in the file_roots
00:20 spuder joined #salt
00:20 kevinquinnyo i guess that's not that big of a deal
00:21 teatime if it's a dynamic, per-deployment value/file, it makes perfect sense to generate and place it and not check it into git
00:21 kevinquinnyo but i'm also using an ext_pillar that polls a remote api internally so i'm trying to avoid using pillar files as well
00:21 baweaver joined #salt
00:21 teatime I would put it into pillar just for ease-of-use, fwiw
00:21 kevinquinnyo since i just transferred all of that to a database
00:21 teatime you can use both, again since it's per-deployment generated, it wouldn't both me
00:21 teatime but if you prefer you can upload it to your database
00:22 teatime again, there's a lot of ways to accomplish it; I'm not seeing which part you're finding difficult; maybe telling that would help?
00:23 kevinquinnyo my difficulty was in framing the problem such that i would generate the keypair on the cluster-master, then i would need a way to grab the contents of the publick key i just generated on that minion and use it as a variable to put it on the other minions
00:23 kevinquinnyo but if i generate it on the salt-master like you say, it should be pretty easy
00:24 teatime it's a general best-practice to generate keypairs on the host where they're used, and never move the private keyfiles off of that host, anyway.
00:24 kevinquinnyo well then i'm really confused lol
00:24 kevinquinnyo becasue that's what i wanted to do
00:25 kevinquinnyo and i thought you were telling me to generate them on the saltmaster and distribute the private key to the cluster-masters and the public keys to the cluster-slaves...
00:25 teatime but generating on the top-level master and distributing to the site-masters is not fundamentally harder / much different from ssh-keygen'ing on the site-masters
00:25 teatime kevinquinnyo: we should standardize our terminology here
00:25 kevinquinnyo i'm notoriously bad about that.. sorry
00:25 teatime kevinquinnyo: I was never referrring to your top-level master-of-masters; only to the master in any given cluster.
00:25 kevinquinnyo ok
00:26 kevinquinnyo let me re-read what you said then
00:28 kevinquinnyo alright, so since you are *not saying* to generate the keys on my actual saltmaster, but on the cluster-master for each cluster, i still don't understand how i get the public key i just generated so i can append it to each cluster-slave's authorized_keys file..
00:29 kevinquinnyo is there a __salt__['file.get_contents'] maybe
00:29 mpanetta_ joined #salt
00:30 teatime don't think it would help you much.
00:30 teatime since you'll generate the file in the cluster-master state, but want to use its contents in the minion state
00:30 mpanetta_ joined #salt
00:30 teatime but I don't do orchestration, so perhaps it would, on that level
00:31 teatime but yes, there's something like that somewhere, and that's one (probably the cleanest) way to get the data to copy into a pillar file :)
00:31 teatime which is what I would do
00:32 teatime I think the artificial limitation of "and I don't want to copy this file to a pillar file, or file root, or my cluster database" is what's making this hard for you
00:32 kevinquinnyo well it's a public key that i need, so is there a way to expose it as a grain?
00:32 teatime there's also file_tree ext pillar, which if you configured that, would make this as easy as a file.cp state
00:33 kevinquinnyo there's literally nothing sensitive about a public key
00:33 teatime a grain on the master doesn't help you in a minion state
00:33 kevinquinnyo it does with saltmine
00:33 teatime if you want to use saltmine (which dreadfully overcomplicates this simple task IMO), again no need for it to be a grain
00:34 teatime put it into mine from master, query mine data in minion state
00:34 kevinquinnyo but again i still have to "get the contents" of cluster-master:/root/.ssh/id_rsa.pub and take that and put it somewhere (a pillar file for instance as you say) and i dont know how to "get the contents"
00:35 bltmiller joined #salt
00:36 blarghmatey joined #salt
00:37 mavhq joined #salt
00:37 teatime just a matter of searching the docs
00:37 kevinquinnyo ok
00:37 jjasinski joined #salt
00:37 teatime (or doing it with a cmd.run which I would probably do but I'm lazy like that)
00:37 teatime there seems to be a file.seek_read but no file.read or file.slurp
00:38 iggy it's not meant to be easy to get random files from the minions to the master
00:39 teatime iggy: he just wants to get an arbitrary, small file from the master and append it to a file on the minions
00:41 kevinquinnyo on the one hand it makes sense that salt isnt designed for this since it's supposed to be the source of truth for files that exist on the minions, so getting content from a minion is odd
00:41 teatime difficulty level: for some reason he doesn't want to copy it to file_roots or a pillar data file.
00:41 teatime kevinquinnyo: you're not getting content from a minion in any way
00:41 kevinquinnyo i feel like i'm going crazy here
00:41 teatime unless I still don't udnerstand the problem.
00:41 kevinquinnyo if i generate the keypair on the minion, how do does the saltmaster know what it is?
00:42 teatime omg this terminology
00:42 teatime again, I have never meant to refer to your master-of-masters
00:42 kevinquinnyo everything is a minion besides the one saltmaster
00:42 kevinquinnyo right?
00:42 teatime you described creating a file on the cluster-masters and distributing it the cluster-minions
00:42 kevinquinnyo cluster-masters is also just a minion though
00:43 kevinquinnyo it's not like a saltmaster in some master of masters setup if that's what you thought?
00:43 teatime yes, that's what I thought
00:43 kevinquinnyo ok that's the confusion
00:43 kevinquinnyo maybe this will help
00:43 teatime perhaps your error was calling a machine in each cluster the 'master' that doesn't, in fact, run salt-master
00:44 mpanetta_ joined #salt
00:44 teatime that was destined to confuse the hell out of us :)
00:44 kevinquinnyo {web1-phx-1001.whatever.com, web2.phx-1001.whatever.com cache1-phx-1001.whatever.com ...} is a single cluster, one of many clusters, another might be 1002 instead
00:44 mpanetta_ joined #salt
00:44 kevinquinnyo and web1 is what i was calling the "cluster-master" in the since that it needs to rsync to the other ones in it's cluster
00:45 teatime ok
00:45 teatime in that case, yeah salt mine seems the easiest route
00:45 teatime if you want to generate the keypair on the "cluster master" which is waht I would do
00:45 kevinquinnyo ok so
00:46 teatime or you can generate the key on the actual master, and distribute the parts of it to the cluster master and other cluster members as appropriate
00:46 kevinquinnyo then i need to figure out a way to "Get the contents" of the puiblic key after i generate it on these "cluster masters"
00:46 kevinquinnyo so generating it on the actual salt master is starting to seem 1000x easier
00:46 teatime yeah.  I'll help you search the docs now.
00:46 teatime you have the same problem either way.
00:47 kevinquinnyo if i generate it on the saltmaster, i could just create it directly in pillar_roots in /srv/pillar/1001/pki/key.sls /srv/pillar/1001/pki/pub.sls
00:47 kevinquinnyo or something
00:47 jjasinski left #salt
00:48 teatime yes; in that case you need to transform it slightly into valid YAML
00:48 kevinquinnyo oh right
00:48 teatime which wouldn't be hard with e.g. sed.  alternately you could use file_tree (which if you have other files to distribute, is quite useful)
00:48 kevinquinnyo or use that file tree ext_pillar you mentioned
00:49 kevinquinnyo ok given that i cleared up my issue, is that what you would do?  generate them on saltmaster, use file_tree, refresh_pillar, then push those public keys to the minions that need them?
00:49 kevinquinnyo and push the private key to the "cluster master" of course
00:50 teatime note there are an ssh cmd module and state; you should check what's avilable in those to see if they help / are preferrable to shell commands
00:50 teatime like I said, I'd probably generate them on the cluster-master, because this is a personal pet-peeve of mine
00:51 teatime but I don't really see a problem doing it the way you just described, and it does avoid the complication of salt mine
00:51 teatime iggy: I find it quite strange for there to be a file.seek_read but not a file.read / read_all / slurp
00:52 teatime iggy: also it appears {{ show_full_context()|yaml }} used to work
00:54 teatime there is a cp.get_file_str but it's not clear if it can get arbitrary files or only from salt://
00:55 teatime ah, tested; works for arbitrary files.
00:55 kevinquinnyo nice
00:55 teatime kevinquinnyo: so there's that, for future reference; it's cp.get_file_str; it will always get the contents of a file on the master, not the minion
00:56 teatime no, that's a lie
00:56 teatime but, should be true in a state
00:56 kevinquinnyo for getting files from the saltmaster, i could just use python directly
00:56 teatime when we were needing that function, I was still thinking your cluster-master was an actual salt-master
00:57 kevinquinnyo ah right
00:58 jinkyu joined #salt
00:58 teatime this looks like an alternative to salt mine, if you want to generate on cluster-master; I have never used it:  https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.cp.html#salt.modules.cp.push
00:58 teatime sounds slightly iffy
01:00 teatime cmdmod.run (module) and cmd.run (state) exist, so I was thinking originally of just using these with a command of 'cat /path/to/file' *shrug*
01:02 teatime well 'cat' for the former and 'cp' whatever for the latter.
01:02 teatime kevinquinnyo: but anyway, I think you're sorted now, yes?
01:02 kevinquinnyo i have a lot of options
01:02 kevinquinnyo heh
01:02 kevinquinnyo thanks for the help teatime
01:02 teatime kevinquinnyo: for executing cmds/states on different minions in order, use a script or orchestration.
01:02 teatime sorry for the confusion!
01:03 kevinquinnyo was my fault
01:04 spuder joined #salt
01:04 kevinquinnyo i wonder how hacky this would be -- use file_tree, but also use that cp.push, and use it's option to change the upload path to the file_tree directory /srv/pillar/files/<minion_id>/pki/
01:04 kevinquinnyo then refresh pillar and i'm good to do whatever i need with it
01:04 teatime I don't care for it...
01:05 teatime letting your minions write arbitrary files on master sounds like a terrible idea.
01:05 teatime and all it gets you is avoiding mine?  mine isn't that hard.
01:05 kevinquinnyo yeah i guess so
01:05 kevinquinnyo but it also gets that file, which was the only hard part
01:06 teatime (protip:  I think there's currently a bug where after configuring mine for a minion via pillar, you have to refresh_pillar (specifically, not any of the other functions that also refresh_pillar) and then mine.update to get the first mine collection to happen immediately (alternately, you can wait mine_interval, or restart the minion)
01:07 kevinquinnyo i noticed that some weird combination of refresh_pillar and mine_update seemed to do the trick, i didnt realize it was a bug
01:07 teatime for mine, you can just use cp.get_file_str for the mine function, since the mine functions run on the minion
01:07 kevinquinnyo ah
01:07 kevinquinnyo yeah
01:09 * teatime wonders why |yaml is calling .sort() on dicts
01:09 teatime python dict's don't have a .sort()
01:13 mapu joined #salt
01:16 hemebond teatime: Where are you seeing that?
01:16 catpigger joined #salt
01:17 teatime {{ show_full_context()|yaml }} failing with AttributeError: 'dict' object has no attribute 'sort' in represent_mapping in dist-packages/yaml/representer.py
01:18 teatime want the whole traceback?
01:19 teatime or any other info
01:20 teatime http://paste.debian.net/432280/  <-- traceback
01:21 teatime as it happens, I don't care about |yaml, I just want to pretty-print show_full_context(), which happens to be an object (maybe previously it returned a dict or something?  not that that explains the |yaml thing.)
01:23 hemebond Isn't it just |yaml and not |yaml() ?
01:23 hemebond The () can contains a boolean argument.
01:24 teatime although this works fine:
01:24 teatime {{ [{'a': 1, 'b': 2}, 1, {'x': 42, 'y': 99}]|yaml() }}
01:24 teatime hemebond: I've tried it both ways.
01:24 hemebond does yaml(False) still fail?
01:24 teatime hemebond: I think the way yaml filter syntax works they're equivalent
01:24 teatime yes, it does
01:24 hemebond Okay.
01:26 teatime just tried every combination of (False), (false), (True), (true), (), and no parens
01:26 hemebond mapping should be a list.
01:26 hemebond mapping = mapping.items()
01:27 teatime ah, hmm..
01:28 teatime where is the word 'mapping' coming from here
01:28 teatime oh is that in the representer code, or..
01:28 hemebond yeah
01:29 teatime aye, found it
01:29 donmichelangelo joined #salt
01:29 auzty joined #salt
01:29 teatime aha, so probably something in this object has a items() method that doesn't return a list.
01:29 teatime I can pastebin the stringification of this object, but it's all on one line (which is my original problem :)
01:30 hemebond Looks like it. Does show_full_context() work by itself?
01:30 teatime ^^
01:30 teatime 1 sec will upload
01:30 teatime but yes it does
01:30 hemebond That's easy to parse out.
01:30 teatime http://paste.debian.net/432281/
01:30 mavhq joined #salt
01:31 teatime this is easier:  http://paste.debian.net/plain/432281
01:31 hemebond go go Sublime text "Python Output" filter!
01:31 teatime oh damn I just uploaded the pillar data... I hope there's no sensitive data in it grr
01:31 teatime hemebond: if you will upload a pretty-printed version of that, I would love you
01:32 hemebond http://paste.debian.net/432282/ ?
01:33 teatime ahhh thank you
01:34 antpa Could someone take a look at this?  I'm having issues calling to my smtp password grain data on line 13 from pillar. https://gist.github.com/anthonyscolaro/8b6ced166876ca3065263499d8ed2726
01:37 hemebond antpa: sasl_passwd is a list.
01:37 hemebond A list of tuples/dicts.
01:39 nZac joined #salt
01:39 teatime I see several problems with this file.
01:39 teatime that is the main one
01:40 teatime in addition:  I'm fairly sure this will never fail:  {% if applications is not none %}
01:40 hemebond That's true.
01:40 teatime (maybe you wanted grains.get('applications', none), or more likely just remove the if altogether; the for loop will never execute when you default value of {} is used)
01:41 teatime 3rd problem: if your loop runs multiple times, you will make conflicting keys named 'postfix'
01:41 toastedpenguin joined #salt
01:46 antpa hemebond: I'm not sure how to call to the list in grain.
01:46 teatime hemebond: maybe if this is still the problem on develop, I could figure it out, and also make a PR to give that particular show_full_context() a better (pretty-printed) .__str__() ?
01:46 antpa hemebond: I thought using a . would work though it doesn't see it.  Neither does :
01:46 teatime antpa: just take out the dashes
01:46 teatime it looks like you want a dict/mapping there, not a list of dicts
01:48 teatime antpa: do you mean to be able for this to loop multiple times, also?  you'll have an error about duplicate 'postfix' keys, if so.
01:48 teatime antpa: and finally, 4th (possible) problem: I find it odd this is grain data (managed on minion), it looks like archetypical pillar data (managed on master).
01:52 toastedpenguin joined #salt
01:55 toastedpenguin joined #salt
01:57 cpowell joined #salt
01:57 DammitJim joined #salt
01:58 teatime hrm... does repos.saltstack.com always contain all dependencies even when they're in the base distro repos
01:58 kevinquinnyo this might have an obvious answer, but how can i get the minion_id from within a salt module
01:59 teatime I notice python-yaml 3.11-2 is available from both the debian and salt repos; I hope it's the same package exactly, since it's the same versino :)
01:59 teatime kevinquinnyo: from within the python code of the module?
01:59 kevinquinnyo yes
01:59 ZaK718 joined #salt
01:59 onlyanegg joined #salt
02:00 brianfeister joined #salt
02:04 teatime assuming it always runs on the minion, there seems to be __opts__['id'], or perhaps something in __salt__ ...
02:06 teatime of course, if it was always running on the minion, you could trust the id grain.
02:07 teatime kevinquinnyo: still working on the ssh key thing?  how come you decided to go w/ a custom module / what's your plan
02:08 kevinquinnyo i already have a custom module, so i'm just adding a function to it
02:08 pppingme joined #salt
02:09 kevinquinnyo so i can do __salt__['mymodule.get_master_pub_key']()
02:09 teatime word.
02:09 kevinquinnyo but i'm using saltmine
02:09 kevinquinnyo in the module
02:09 kevinquinnyo i actually needed to get the minion_id because of something else i noticed in my custom module, peripherally related
02:10 onlyanegg joined #salt
02:12 teatime if it always runs on the minion / can trust grains, use __grains__['id'] .. I grep'd through the existing modules, it doesn't seem to be a thing that is done much at all
02:13 gerhardqux joined #salt
02:13 justanotheruser joined #salt
02:13 teatime in the match module, for example, it gets passed a minion_id parameter, and if it's None then it falls back to __grains__['id']
02:14 racooper joined #salt
02:14 kevinquinnyo ahhh grains id
02:14 kevinquinnyo forgot about that
02:14 kevinquinnyo thanks
02:14 Ludo- joined #salt
02:14 chmod666org joined #salt
02:16 onlyanegg joined #salt
02:16 euidzero joined #salt
02:19 mavhq joined #salt
02:20 spuder_ joined #salt
02:21 mavhq joined #salt
02:22 teatime kevinquinnyo: heh, if the file did exist on the master, I just happened across this {% import_text "completeworksofshakespeare.txt" as poems %}
02:22 teatime I know you don't need it any more (and I dunno if lets you go up to /) but interesting
02:22 teatime if it lets you go up to /, you could then do {% poems|yaml_encode %} later
02:22 teatime {{ }} rather
02:23 kevinquinnyo does it just split the file into a list on \0
02:23 kevinquinnyo because that's actually useful in its own right
02:23 teatime I assume it's a big string
02:23 kevinquinnyo im not sure for what
02:24 kevinquinnyo ah
02:24 teatime did you mean \n ?
02:24 teatime or really meant NUL
02:24 kbyrne joined #salt
02:24 teatime that's an interesting idea kinda :)
02:24 kevinquinnyo i meant on null byte
02:24 kevinquinnyo like:  While IFS= read -r line; do echo "$line" < /tmp/somefile.txt
02:25 Ssquidly joined #salt
02:26 drags joined #salt
02:26 eightyeight joined #salt
02:26 pid1 joined #salt
02:27 teatime you could also use something like form feed or vertical tab or 0x1c (INFORMATION SEPARATOR FOUR aka file separator), or some of the new unicode whitespace like paragraph separator, or something
02:27 teatime since \0 can be hard to work with sometimes
02:27 foundatron joined #salt
02:27 capricorn_1 joined #salt
02:29 teatime also kevinquinnyo, I'm pretty sure that reads, well, lines
02:29 teatime not \0-separated chunks.
02:29 DammitJim joined #salt
02:34 kevinquinnyo you might be right, i'm not sure to be honest
02:38 mpanetta_ joined #salt
02:40 kshlm joined #salt
02:46 tawm04 joined #salt
02:48 bltmiller joined #salt
02:48 bltmiller joined #salt
03:04 ramteid joined #salt
03:06 rem5 joined #salt
03:08 teatime hey hemebond, guess what
03:08 hemebond wut?
03:08 teatime still looking through to find the thing that breaks it, but as a helpful aside I did find that this works:  {{ show_full_context().items()|pprint }}
03:09 teatime slightly uglier than what you gave me, but at least I do have a general solution/workaround for now :)
03:09 hemebond Nice.
03:10 teatime imo it's python-yaml bug / potential improvement, though... it should either check if items() returned a list, or perhaps it should be checking isinstance(blah, collections.Mapping) instead
03:10 hemebond Well it does check for the method.
03:11 hemebond And that should return a list.
03:11 hemebond I think it might be something to do with the keyname.
03:11 teatime but it's checking it on arbitrary objects to decide if they quack like a duck
03:11 hemebond That's true.
03:11 teatime *quack like a dict
03:11 hemebond pyyaml doesn't actually seem to be actively developed now.
03:12 teatime can salt optionally use either of 2 YAML implementations for python, or did I imagine/dream that
03:13 hemebond Is there another YAML library?
03:14 teatime at first glance, it looks like there are multiple implementations, but PyYaml wraps them all
03:15 teatime and specifically, perhaps PyYAML is a pure python implementation + libyaml bindings
03:15 teatime (I need to confirm that though, that's just initial impression)
03:18 teatime a quick grep of salt looks like it never does anything more complicated than 'import yaml' though
03:21 teatime ahh, but it does replace yaml.Loader and yaml.Dumper with CLoader/CDumper, if they exist
03:21 teatime so it seems, you get a different yaml implementation depending on if you have libyaml installed or not
03:22 teatime I wonder if that would affect this.
03:22 hemebond Really? That's a bit of a worry. Or is that lower than pyyaml?
03:22 mosen i think in PyYAML it uses libyaml if available otherwise pure python
03:22 hemebond Ah
03:22 teatime mosen: well that's the effect yeah, but salt has to explicitly make that happen
03:23 teatime hemebond: but CLoader/CDumper are provided by PyYAML when libyaml is installed
03:23 evle joined #salt
03:24 teatime this takes place in salt/config/__init__.py and salt/utils/yaml{loader,dumper}.py, so I guess that effect is confined to those files (plus whatever imports yaml from them, plus whatever uses functions etc. from them that parse/dump yaml), yeah?
03:27 teatime also, at least on debian 8, python-yaml depends on libyaml.  so I shouldn't have to worry about different behavior in environments with/without libyaml
03:27 hemebond Yeah I have libyaml installed.
03:28 _beardedeagle joined #salt
03:30 teatime heh, aside: why do I suspect that this is a handy bit of info: http://pyyaml.org/wiki/BugsInTheYAMLSpecification
03:30 teatime I rather hate the yaml spec, it's so difficult to read
03:30 hemebond cripes
03:31 hemebond It's a shame it's not just a simpler JSON
03:32 teatime perhaps.
03:32 teatime there are 'fancy' parts of it I like
03:32 teatime I don't know enough / haven't worked with it enough to say overall which I think is better / would prefer (simpler JSON or as-exists)
03:33 teatime LOL, the last entry on that page is kindof funny
03:35 teatime also weird, the spec versions don't seem to include any kind of changelog / overview of new features in this version
03:35 teatime maybe there's one elsewhere
03:39 teatime btw, Q:  Am I correct in understanding that you cannot use yamlex renderer to merge values across pillar files (only within a single file) ?
03:40 teatime so does that mean there's no way to merge lists with parts defined in multiple files other than to globally turn on list merging?
03:40 teatime (I guess you could give them distinct names in each file and then manually merge each of those into one list later, but that is kinda icky)
03:49 brianfeister joined #salt
03:49 hemebond yamlex renderer?
03:49 teatime hrm, checked all of the top-level things in the context; all either don't have an .items() or it returns a list.
03:50 teatime which narrows it down to almost definitely being something in salt.* (this 'salt' is the 'salt' member of the context)
03:50 teatime hemebond: https://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.yamlex.html
03:51 hemebond Oh, never used that one.
03:51 teatime hrm, I think it does some more stuff so there might be a better description somewhere else; not sure
03:51 teatime me either.
03:51 hemebond I did recently fix anchors and references in regular YAML renderer.
03:51 teatime heh, a lot of things that's frequently used, I'm afraid to use for fear they'll be buggy
03:51 mavhq joined #salt
03:52 teatime that's interesting, hemebond
03:53 hemebond I just realised how crap YAML anchors and references will be in Salt.
03:53 hemebond Since they only work within a document.
03:54 hemebond And if you include some default Pillar into multiple files you'll get an error.
03:54 hemebond (duplicate entry error)
03:55 hemebond Which means my Jinja-based importing and merging will have to be my method even if anchors will work in the future.
03:55 hemebond At least when I have multiple documents.
03:57 treaki__ joined #salt
03:58 teatime what do you mean by "my Jinja-based importing and merging" ?
03:59 hemebond I wanted to have some default values that I could then extend/override in many Pillar entries.
03:59 hemebond But YAML anchors and references weren't working so I used Jinja to work around it, merging a Pillar entry using a macro.
03:59 teatime where do you set your defaults?
04:00 hemebond And it allows me to do it without having the default values YAML in the document.
04:00 hemebond It's in its own file, def_datasource.yml
04:00 teatime I had thought it was perfectly valid to use data defined in an earlier file_root in a later one, but iggy was telling me this is not supported / doesn't work for some people.
04:00 hemebond So the macro reads in that YAML, takes my new values and merges it all.
04:00 teatime ah.
04:01 teatime also, and this is not very important but I've been curious about it, I have this utils.jinja file: http://paste.debian.net/plain/432295
04:01 hemebond That lets me use it in multiple sls Pillar files.
04:01 hemebond Ah yeah, I've seen that before.
04:01 hemebond Pillar probably won't work though.
04:01 teatime I can't think of an easy way to include this in every .sls file, which I guess is OK.  but also it seems you can't just {% import "utils.jinja" %}, I have to either list each of the symbols I want to import, or import them under a namespace like {% import "utils.jinja" as util %}
04:02 teatime hemebond: you mean pget()? it works fine for me.
04:02 teatime hemebond: or do you mean using earlier pillar_roots data in later ones?  which also seems to work fine for me
04:02 teatime if it's not a supproted thing tho, I'm glad I found out now because I was just about to use it extensively for some stuff
04:02 hemebond I didn't think you could use Pillar data at all in pillars.
04:03 teatime within the same pillar_root, I can definitely understand that
04:03 teatime it also seems valid for ext_pillar to refer to pillar_root data (or the reverse with ext_pillar_first); otherwise, I don't know why ext_pillar_first would exist as an option.
04:04 josuebrunel joined #salt
04:05 hemebond So if I have multiple entries for an environment in pillar_roots I can reference them in other Pillars somehow?
04:06 teatime yes, via salt['pillar.get'] just like in state .sls files
04:06 hemebond Which root can I access?
04:06 teatime and it does work.  but I need to find out if it is supported / assured to work.
04:06 hemebond I have a base root and a specific environment root.
04:06 hemebond In which root can I use it?
04:06 teatime I don't know if it works there (probably does, but:) and I don't know if those get executed in a well-defined order?
04:07 teatime oh, nm
04:07 teatime in the same environment?
04:07 hemebond Yeah
04:07 teatime I can't remember if it's earlier (higher) or later (lower) items in the list that seem to get processed first
04:07 teatime let me look at my testcase
04:08 msn joined #salt
04:08 Stinky joined #salt
04:08 bltmiller joined #salt
04:12 brianfeister joined #salt
04:21 mavhq joined #salt
04:23 jerredbell joined #salt
04:25 teatime hemebond: lol nevermind it no longer seems to work.
04:26 hemebond LOL, oh :-(
04:26 hemebond It should actually be fairly easy to create a renderer that can be added in after YAML that does variable interpolation.
04:28 Stinky Is there a way to disable the ssh connection step in salt-cloud? I am unable to allow ssh communication in my current scenrio and I dont need bootloader. My images already has salt-minion baked in. Currently I have ssh_connect_timeout:1 defined in my profile but of course I get an error message stating ssh failed to connect. I was just wondering if there is an arg I can disable the ssh step.
04:29 hemebond Stinky: Does disabling the installation of bootstrap not stop the SSH connections?
04:29 teatime hemebond: ehhh that doesn't sound worthwhile.
04:30 teatime hemebond: however, the answer for you and I both is probably pillarstack?
04:31 hemebond teatime: What does pillarstack do?
04:32 hemebond I'm reading the github page but can't figure out what it's for.
04:32 hemebond Why wouldn't a variable interpolation renderer be worth it?
04:32 hemebond I wrote something like it in my Powershell config management.
04:33 hemebond Powershell is revolting but my recursive variable interpolation was great to use.
04:33 teatime 1) it seems to let you basically stack arbitrary number of file_roots, ext_pillar, and maybe more kinds of pillar? and later ones can *definitely* refer to earlier ones' data.
04:33 Stinky joined #salt
04:34 teatime hemebond: 2) because that's like, a whole other layer / new-and-different thing, and we already have jinja for that, would it not be so much better to just make salt work the way we want rather than add yet-another layer of templating and interpolation?
04:34 hemebond But that is how Salt works, by chaining renderers together.
04:34 hemebond It just so happens that the default renderers are Jinja > YAML.
04:34 teatime yeah, I don't argue with that
04:34 teatime that you could fairly easily implement it as a renderer
04:35 teatime I just don't think it is a desirable thing to add, from an end-user perspective / conceptual-integrity perspective
04:35 hemebond Jinja is nice for templating but variable interpolation is different.
04:35 hemebond Well it's not like it would be default or mandatory.
04:35 hemebond It's just another renderer.
04:36 hemebond There are people who have requested it as a default function but I disagree with that.
04:37 hemebond In fact that might be my project for the week; create a renderer for variable interpolation.
04:39 teatime how is it different
04:39 hemebond hmm?
04:39 teatime if you're thinking a variable can include jinja syntax, you can already kindof do that
04:39 hemebond So jinja > yaml > jinja?
04:40 jerredbell joined #salt
04:40 teatime you could stack yaml renderer multiple times, yeah.. that would probably get nasty
04:40 teatime I was specifcially thinking of {% include %} which contains a {% raw %} block of yaml
04:41 teatime but actually, I'm not certain that actually works to interpret the yaml in the current template
04:41 teatime now that I think about it
04:41 teatime hemebond: you might be interested in the ticket babilen & UtahDave were talking about earlier
04:42 cpowell joined #salt
04:42 hemebond Do you know which ticket it was?
04:43 hemebond Oh, #23910 ?
04:44 teatime this one https://github.com/saltstack/salt/issues/23910
04:44 saltstackbot [#23910]title: Please implement static pillars | Hi,...
04:44 teatime heh yes
04:45 hemebond That seems like a very odd request.
04:48 teatime maybe.
04:48 teatime since you can't stack pillar_roots, I totally understand it, I think.
04:48 hemebond It seems like you could already do this though, depending on how you've organised your data.
04:49 hemebond top.sls is run through Jinja too. Jinja can import a YAML file.
04:49 hemebond *import and parse a YAML file.
04:49 hemebond Ah, msciciel mentions it in the issue comments.
04:50 teatime yeah, true.
04:50 teatime very true.
04:50 teatime did anyone object to that suggestion?
04:51 hemebond Which suggestion? load_yaml?
04:52 teatime yeah
04:52 hemebond I don't see any. In fact it appears to have been ignored or missed.
04:52 hemebond I hope the idea doesn't get implemented.
04:53 hemebond msciciel has the right idea in my opinion.
04:53 hemebond One of the things I like about Salt is its simple and consistent workflow.
04:53 hemebond e.g., everything is run through Jinja and YAML renderers.
04:54 teatime and I don't necessarily like how quickly things get merged/coded when requested
04:54 hemebond There have been a lot of funny new things added which seem odd to me.
04:55 teatime it's awesome if they're awesome things, but it seems like there's a lot of questionable features, semi-abandoned features, and also a lot of bugs that development time might could be better spent on rather than implementing new features?
04:55 hemebond I'm not sure I have a problem with that. By making it very active and dynamic _hopefully_ means it can also correct less-than-optimal decisions sooner.
04:55 teatime my impression there could be wrong, and I would not really want to try to tell volunteers to scratch my itches instead of their own, but that's the feeling I have
04:56 hemebond It's nice that many new features, thanks to Salts modularity, won't affect people by default (hopefully).
04:56 hemebond That's why I like the idea of a renderer to do variable interpolation. Easy to enable for those who really want it. Complete non-issue to anyone who doesn;t.
04:57 teatime fair enough
04:57 teatime you've managed to convince me btw that it's a good idea
04:57 hemebond :-)
04:58 teatime btw you could use jinja still for it, w/ a different value for the delimiters
04:58 hemebond I'm not very good at convincing people so that's good to hear. It's a very nice thing to have with large configurations.
04:58 hemebond You mentioned that before but I don't understand how.
04:58 hemebond Did you really mean jinja to yaml to jinja?
04:59 teatime no, I didn't really mean that
04:59 hemebond Actually the last bit wouldn't get YAML it'd get a big dict, so not sure if Jinja can do it.
04:59 teatime and now I'm talking about something different
04:59 hemebond Oh, sorry.
04:59 hemebond Talking about that issue or variable interpolation?
04:59 kshlm joined #salt
05:00 teatime I'm talking about implementing your custom renderer, but it would still be very like the yaml renderer, it would just 1) use different delimeters than { }, which are configurable w/ Jinja, and 2) load its context from wherever static data you want, rather than pillars etc.
05:01 teatime hemebond: also, you really might be able to just use pillarstack; not sure..
05:01 teatime I need to look at it in more detail also
05:01 hemebond pillarstack just seemed to be a place for pillar files. I couldn't work it out.
05:01 hemebond Do you mean create a renderer that would replace Jinja or YAML or both?
05:02 hemebond Because I think they both take single documents.
05:02 teatime hemebond: http://jinja.pocoo.org/docs/dev/api/#jinja2.Environment
05:02 hemebond Whereas recursive variable interpolation would take a dict and check each value for a pattern (like a jinja {{ }}) and recursively resolve it.
05:03 teatime pillarstack is indeed just for pillar; I guess if you want this in states you need something different.
05:03 teatime hemebond: I was thinking it would be mystaticjinja|jinja|yaml
05:03 hemebond re: pillarstack I meant I just couldn't understand the use case. I didn't see any clear examples of why I'd use it
05:04 hemebond re: var interp: oooooh, no it has to be the final step.
05:04 hemebond It's basically variables in your variables.
05:04 hemebond And needs the full pillar before it can resolve anything.
05:05 teatime hemebond: you can define a stack of pillars like, "pillar roots /srv/salt/pillars/defaults, ext_pillar file_tree /srv/salt/pillars/files, pillar root /srv/salt/pillars/base', and later ones can refer to pillar data defined by earlier ones.
05:05 teatime hemebond: I see 2 problems with that
05:06 hemebond re: pillarstack: I see, that is interesting.
05:06 teatime hemebond: 1) you can STILL USE JINJA FOR IT (when I say use jinja, I mean use the jinja API, I don't mean the salt Jinja renderer, you would make your own render) with different delimeters like <% %> instead of {% %}
05:06 hemebond Yes, I agree.
05:06 kawa2014 joined #salt
05:06 teatime 2) what do you do when your fully-rendered pillar data has strings that just happen to coincidentally be syntax your variable replacer will replace
05:06 hemebond But you can't use the existing Jinja renderer as far as I can tell.
05:06 cliluw joined #salt
05:07 teatime you need some kind of escaping, and you'd have to apply to pretty much every string you interpolate
05:07 hemebond You have ways of escaping it, like any templating.
05:07 Stinky I am unclear on how to disable the bootloader then. I see that I can pass a -N on the script-args but thats expecting to still run the bootloader to tell it to not install the minion. I see in /etc/salt/cloud there is a #script: bootloader.sh. I have tried putting None, or leaving blank but it still attempts to ssh into instance.
05:07 hemebond The renderer could very easily use Jinja for it, but it can't, as far as I can see, use the existing Jinja renderer as-is.
05:07 teatime although I suppose, if you only use it on specific files, you don't have to worry about that in all of the other files
05:07 teatime hemebond: I never said it could ;)
05:08 teatime hemebond: my very very first comment on it, quite a while ago, was a bit mis-guided
05:08 hemebond Ah. It seemed like you were saying I should just use the existing Jinja renderer.
05:09 hemebond Rather than a new renderer that used Jinja to do the string replacements.
05:10 teatime nah
05:10 teatime but you could probably reuse a lot of it, maybe even subclass it
05:10 hemebond Even using Jinja would probably be overkill for it to be honest.
05:11 hemebond I'd be passing a small string as a template.
05:11 hemebond (most of the time).
05:11 hemebond Still, using Jinja would take a lot of work out of it.
05:11 hemebond (a small string many many times)
05:15 mavhq joined #salt
05:18 rdas joined #salt
05:19 nidr0x joined #salt
05:24 bltmiller joined #salt
05:25 hemebond Would it be bad form to leave a -1 on that issue?
05:25 hemebond I don't like the +1 comments but I don't know where else to "vote" as such.
05:26 linjan joined #salt
05:27 hemebond Also file_tree doesn't seem as great as I thought it was. I thought it was a pillar:// protocol so I could move static files out of the states tree but it isn't really.
05:29 sauvin joined #salt
05:30 Vaelatern joined #salt
05:36 felskrone joined #salt
05:38 teatime bad form?   I dunno.  I think you should at least try to express your reasons why, if only because it will probably give your vote more weight.
05:38 teatime I was working on something I called adv_file_tree, but I kindof quit, I'm not sure how useful it actually is.
05:40 graffic joined #salt
05:40 roock joined #salt
05:43 jhauser joined #salt
05:45 impi joined #salt
05:45 mavhq joined #salt
05:47 hemebond It seems odd that file_tree is so specific in its structure.
05:47 hemebond So fixed in its usage.
05:52 teatime yeah, that was basically my reasoning for re-making it
05:52 teatime in a much more flexible form
05:52 teatime and especially to include some more-sane whitespace trimming and newline trimming options.
05:53 teatime I may get back to it at some point, but I want to do some more real-world work w/ salt first and also try out pillarstack
05:53 hemebond https://github.com/saltstack/salt/issues/1064
05:53 saltstackbot [#1064]title: Pillars should be able to cross-reference to other pillars | My setup contains a global pillar that applies on `*` and a per-role one applied via `role*.domain.tld`....
05:53 tharkun joined #salt
05:54 hemebond A veeeeery old issue that is the same request for Pillar references within Pillars.
05:55 teatime I can understand if referencing data defined in the same pillar_root is hard to implement, at least in a user-friendlish way
05:55 teatime but I don't understand why ordering multiple pillar roots and letting you refer to earlier ones is nigh-unpossible
05:56 teatime but I guess it's a case of, if you want it that bad, you (by which I mean me) should code it :)
05:56 hemebond That approach seems very odd to me.
05:56 teatime perhaps
05:56 teatime the former definitely seems more desirable, and I guess the later does have limited cases where it makes sense.
05:57 hemebond I think this is one of the features that I was thinking about earlier, where there seem to be better ways of implementing it.
05:57 hemebond And implementing it in a more general manner.
05:57 teatime like what
05:57 hemebond Instead of all these special data paths.
05:58 hemebond Like having recursive variable interpolation.
05:58 teatime like just defining the order pillar files are processed and then letting later-processed ones see pillar data from former ones?
05:58 hemebond Once you have that it doesn't matter where the data comes from, it'll be resolved.
05:58 hemebond Well we already have that.
05:58 teatime we do?
05:58 hemebond I do that with my pillar_roots already.
05:59 teatime could you explain/expound/clarify
05:59 hemebond Anything not specified in the specific environment/install/product pillar directory gets found in the base/default/fallback directory.
05:59 hemebond I'll paste an example of my pillar_roots.
06:00 hemebond This is how I had my pillar_roots for a while http://paste.debian.net/432305/
06:01 hemebond I removed the local environment because I didn't really want/need that much fallthrough.
06:01 favadi joined #salt
06:02 teatime ok, hemebond, what the actual fuck
06:02 hemebond lol
06:02 teatime oh wait, but you don't pillar.get in later ones?
06:02 hemebond no
06:02 teatime you're just making use of the default merging strategy?
06:02 hemebond Yes.
06:03 teatime ah, ok, yes of course that works.
06:03 favadi joined #salt
06:03 teatime my meaning was using pillar[] / salt.pillar.get
06:03 hemebond Sure, variable interpolation.
06:03 teatime yeah, you can't do that apparently
06:03 hemebond No, which is why I'm looking at making a renderer or something.
06:03 teatime even if you define the value in one pillar root and try to look it up later.
06:04 hemebond I suspect renderers get one document at a time which would suck.
06:04 hemebond (for my renderer I mean)
06:05 teatime hemebond: I would very much like to be able to pillar.get in a later pillar root.  It would be even that much better if I could pillar.get in a pillar .sls file processed after another in the same pillar root
06:05 teatime obv. an ordering would have to be defined for processing of .sls files within a pillar root, if it doesn't already exist
06:05 teatime that seems pretty easyish though, at least having not looked at the code
06:06 teatime it also sucks that ext pillars must all be before or all be after pillar roots, and they cannot be interleaved,
06:06 teatime however I think ALL of the desired functionality for me and probably you too is provided by pillar strack
06:06 teatime so I need to try that out ASAP
06:07 ivanjaros joined #salt
06:08 hemebond "and even use other pillar values that has already been merged by PillarStack (from previous yaml files in PillarStack configuration) through the stack variable"
06:08 hemebond So it's not recursive, but does allow access to variables already defined.
06:08 hemebond s/variables/values
06:09 mavhq joined #salt
06:10 teatime explain recursive
06:10 teatime do you really want to evaluate repeatedly until no more substutions happen?
06:10 teatime how does escaping work in that case?
06:10 hemebond If you have myvar: "testing saltstack in {{ domain }}"
06:11 hemebond It would check if "domain" is already in the properties/values dict.
06:11 teatime (and yes, the one thing I realized already pillarstack would not do for you, is repeated evaluation, if you really want that)
06:11 hemebond It would read that value and check if that value had {{ propertyname }} in it.
06:11 hemebond If so it would then lookup propertyname and resolve that and so on.
06:12 teatime ah, ok, so that's a little smarter than repeated evaluation, then?
06:12 hemebond Resolving all the way down until a property returned a regular string.
06:12 hemebond And do it once.
06:12 hemebond repeated evaluation? Not sure what that is, sorry.
06:13 teatime as in, given shell, I was thinking you wanted to take a string like (assume backslashes in '' don't need escaping themselves) '\\$\$$var' and you'd eval this repeatedly until it returned the same value twice, then use that value
06:13 djgerm1 joined #salt
06:14 teatime probably just a broken/stupid/naïve way to implement what you just described in a way that sounds much less broken
06:14 djgerm1 Howdy! Is there a way to "orchestrate" the application of a state prior to an event? I want to execute a state on a minion before salt-cloud -d kills it
06:14 djgerm1 I know… it sounds crazy
06:15 teatime phrasing it that way makes it weirder than the thing you actually want, which seems quite reasonable :)
06:15 teatime but I do not know the answer; perhaps someone else will.
06:15 hemebond teatime: Ah, it kind of would, yes.
06:15 hemebond Maybe.
06:15 teatime hemebond: my stupid way fucks up escaping, though; what you described seems doable.
06:15 hemebond Okay.
06:16 teatime hemebond: I wonder if you could do it w/ objects in your jinja context
06:16 hemebond That's how I do it currently.
06:16 teatime so that {{ blah }} would actually call a method on object blah in the context, that would inspect / attempt to do another layer of substitution on blah
06:17 hemebond Oh.
06:17 teatime of course it'd be recursive, but if you run out of stack depth due to variable stack complexity you probably have other problems.
06:18 teatime hemebond: you know, I think you and I are wanting to acheive the exact same goals, we just have different preferred syntax/methods for doing it / different intuition about how you'd go about doing it.
06:18 hemebond It sounds like it.
06:20 teatime djgerm1: I'm assuming what you want to do has to be done from the minion and/or while it still exists?
06:20 djgerm1 yeah… i mean, not technically.
06:20 djgerm1 that's just the easiest way I can figure
06:20 teatime hemebond: my use-case is very similar to the one in the ticket you linked
06:21 teatime djgerm1: gotcha
06:21 djgerm1 i use the minions host and domain grains for a state targeted at other servers
06:21 * teatime is really beginning to dislike a couple of minor UI/layout things about docs.saltstack.com
06:21 hemebond teatime: The only thing my method wouldn't allow was making the Pillar data available to Jinja.
06:23 djgerm1 if I understood salt mine, I bet I could leverage that somehow and then if the data for the minion is gone because it was deleted, I could still use salt mine for a bit to get the host and domain grains for a particular recently deceased minion… but I don't know how to do that. Endstate: I want to delete ddns entries on destruction via salt-caloud -d
06:23 teatime hemebond: if you could figure out a way to handle escaping, the repeated-eval might be a really easy way to implement, but I suspect repeated-eval is just a failed idea completely.
06:24 teatime maybe if I had more cs-fu / parser experience the truth there would be more obvious to me
06:24 teatime djgerm1: salt-mine actually turns out to be quite a bit easier than I initially thought when first looking at it
06:25 hemebond teatime: Using the Jinja parser/templating would likely be a good choice, but I do wonder about the overhead.
06:25 hemebond It would eventually create a template for every Pillar value.
06:25 teatime true, I hadn't considered that fact
06:25 teatime not sure if searching the values for { first would be a useful optimization :)
06:26 teatime (if doesn't contain '{', must be a terminal)
06:26 teatime hemebond: it's probably a lot more complex than what you would have implemented if you'd never spoken to me :)
06:28 teatime djgerm1: you basically give your minions a list of salt module functions and their arguments, which you can do via minion config file OR by putting it in pillar which is handy, and then every mine_interval or minion restart (or manual mine_update, with the first one requiring a refresh_pillar) causes the minion to run the functions and add their return values to the salt mine.  salt mine data is available to
06:28 teatime all minions/master, but can have ACLs to restrict which can see what functions.
06:29 teatime djgerm1: I've been planning to have each of my minions export their hostname/ip to salt mine, and then having a state on my dns server collect all of that data from salt-mine and make a zone file out of it, to avoid using ddns
06:29 djgerm1 well where's that?! :)
06:29 djgerm1 that'd solve all my woes
06:29 teatime which is basically doing what puppet's exported resources do.
06:29 djgerm1 ddns is working fine so far.
06:30 teatime djgerm1: lol, if I tell you what is holding me up, you'll laugh at me.
06:30 djgerm1 if every minion was checking in its host and domain grains to a mine, then I could have ddns.delete_host trigger with a reactor
06:30 teatime haven't yet quite worked out how to update the serial# in the zone file only when the rest of the content changes, and not updating the serial (and thus triggering changes) on every state run
06:30 teatime but I haven't thought much about it, that doesn't seem too hard; it's more, I haven't decided the best/cleanest way.
06:31 teatime discovered the data module the other day; considering using that to store the record entries, and generate a new zone file when that data has changed.
06:31 teatime djgerm1: that would be really easy to configure.
06:32 djgerm1 really… well first I need to get every minion reporting its host and domain grains into mine…
06:32 toastedpenguin joined #salt
06:33 teatime that's the part I meant would be easy :)
06:33 mavhq joined #salt
06:33 iggy powerdns with records updated by reactor events for minion coming and going?
06:33 favadi joined #salt
06:34 djgerm1 bind
06:34 djgerm1 using ddns module
06:34 djgerm1 i have the reacter + state for creation
06:34 djgerm1 and state for deletion
06:34 djgerm1 just need to trigger it.
06:36 teatime djgerm1: to get that data into salt mine (this is just an educational example; I'm not suggesting this is the best way to solve your current actual problem)
06:36 josue joined #salt
06:37 teatime djgerm1: you would need only add this to the pillar data:  http://paste.debian.net/432312/   and then saltutil.refresh_pillar and mine.update
06:38 teatime afterward you can mine.get('*', 'host', 'domain')
06:39 teatime djgerm1: I figure there's a way to do exactly what you first asked though, which seems simpler.
06:39 djgerm1 you'd think, right?
06:40 djgerm1 maybe… I should just try to do it the same way I did the add :)
06:40 djgerm1 see what happens.
06:40 ubaumann joined #salt
06:41 djgerm1 add a "salt/cloud/*/deleted' and a uh… state.sls.dns.ddns_delete
06:41 djgerm1 or something
06:41 ribx joined #salt
06:42 teatime I don't see a deleted event at https://docs.saltstack.com/en/latest/topics/cloud/reactor.html#available-events
06:42 djgerm1 - 'salt/cloud/*/destroyed':
06:42 teatime doesn't mean it doesn't exist.  but iirc you can't really customize the data these events include
06:43 teatime heh, their examples do include that
06:43 teatime (destroyed)
06:43 djgerm1 mighty past tense ho :/
06:43 djgerm1 *tho
06:45 teatime I assume your minion_id's are not conveniently their fqdn's
06:45 dmaiocchi joined #salt
06:45 KermitTheFragger joined #salt
06:46 djgerm1 as a matter of fact, they are…
06:46 djgerm1 which makes their host and grain perfectly suited for adding/deleting DDNS entries
06:46 teatime heh, all event's data contains the minion_id IIRC
06:46 ivanjaros3916 joined #salt
06:46 djgerm1 i'd have to… munge it up.
06:47 djgerm1 if minion id is minion.id.example.com, I need minion and id.example.com
06:47 teatime meh, not too badly, since you have one or at least finite possible domain parts
06:47 teatime (zones, I should say)
06:47 teatime djgerm1: there's also apparently a destroying event
06:48 djgerm1 how would one munge up the data coming back from an event, and stick it into a state?
06:48 teatime (I don't use reactors:) will salt-cloud pause what it's doing until the destroying event reactors have all completed, or is it asynchronous?
06:48 djgerm1 that's what I'm about to figure out :)
06:48 djgerm1 if I can write the reactor syntax correctly
06:51 teatime do you have multiple zones in a hierarchy?
06:51 djgerm1 i feel like this is something somebody should have solved already…
06:51 djgerm1 i do.
06:51 djgerm1 but the ddns zone is only the domain grains of my minions :)
06:52 teatime but there are multiple ones of them in a hierarchy?
06:52 teatime djgerm1: if it makes you feel better, I remember someone asking pretty much exact same question 1-2 weeks ago.  I don't remember his solution, if he found one :)
06:52 djgerm1 yes
06:52 djgerm1 I think that was probably me :)
06:52 djgerm1 I've bene asking for like a month
06:52 djgerm1 at least.
06:53 djgerm1 I keep getting distracted with other tasks
06:54 opdude joined #salt
06:59 babilen djgerm1: Haven't read all backlog, but you might want to consider calling a custom execution function to which you hand over the event data and that does the right thing™
07:00 ubaumann joined #salt
07:02 djgerm1 gosh I wouldnt think it was that unique a thing
07:02 djgerm1 just to delete a ddns entry upon minion destruction
07:05 babilen What do you need to do to achieve that?
07:06 djgerm1 the host and domain grain of the minion being destroyed
07:06 dmaiocchi joined #salt
07:06 babilen Didn't you say earlier that you could use the id?
07:07 djgerm1 i'd have to munge the data, break it apart into host and domain
07:09 babilen That doesn't sound too hard. In fact a "foo".split('.', 1) should do it, shouldn't it?
07:10 djgerm1 I've never used that.
07:10 babilen Otherwise: Pass the id, get the other data from the salt mine for that minion and continue with that
07:21 mavhq joined #salt
07:23 josuebrunel joined #salt
07:25 Guest79 joined #salt
07:26 djgerm1 where is mine data stored on the master?
07:27 lero joined #salt
07:27 djgerm1 memory on the master?
07:27 punkoivan joined #salt
07:29 hvn joined #salt
07:29 punkoivan joined #salt
07:35 babilen djgerm1: In the cache
07:35 rogst joined #salt
07:36 djgerm1 ah. should that be synced across masters in a multimaster setup?
07:37 Faris joined #salt
07:37 babilen djgerm1: You obviously want the same mine data on every master the minion might end up talking to.
07:38 djgerm1 obviously. but since it reports on an interval, it'll just report when it's talking to that master next right?
07:39 djgerm1 or do extra steps need to be taken (like for syncing keys across masters)
07:39 babilen No idea
07:39 impi joined #salt
07:39 djgerm1 yeah the docs are slim
07:41 babilen Might be a topic for the mailing list
07:43 cpowell joined #salt
07:44 dmaiocchi joined #salt
07:47 ronnix joined #salt
07:48 dgutu joined #salt
07:48 antpa joined #salt
07:50 JohnnyRun joined #salt
07:58 Miouge_ joined #salt
08:00 mavhq joined #salt
08:00 lero joined #salt
08:01 lero joined #salt
08:01 Rumbles joined #salt
08:02 Guest79 joined #salt
08:03 keimlink joined #salt
08:04 favadi joined #salt
08:06 djgerm1 ok so… i've got the mine populated.
08:09 djgerm1 now to use the id from the event to get that mine data from the mine.. would that be something like salt['mine.get']('data['id']', 'host', 'domain')
08:10 pcdummy joined #salt
08:10 pcdummy joined #salt
08:11 babilen [C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[B[Aq[C[C[A[Adjgerm1: I'd call a state, hand over the ID in the pillar data and do the work there
08:11 babilen yay
08:13 djgerm1 so, the ID would be coming from the event
08:14 mavhq joined #salt
08:14 djgerm1 i guess I dont understand what the pillar and mine have to do with eachother here.
08:15 kbaikov joined #salt
08:15 djgerm1 i have a pillar entry does does the mine_function
08:17 slav0nic joined #salt
08:18 babilen djgerm1: You would get the id out of the event and pass it via pillars to the state you call. A similar approach is exemplified in https://docs.saltstack.com/en/latest/topics/reactor/#passing-event-data-to-minions-or-orchestrate-as-pillar
08:19 babilen In the state you call there you get the rest of the data you need about the minion from the salt mine, but calling mine.get. That allows you to then do whatever you have to do with the data at hand.
08:19 geomacy joined #salt
08:20 kbaikov joined #salt
08:20 babilen You could, naturally, simply parse the id in the reactor and pass in the hostname and domain directly to a state. This shifts more logic into the reactor configuration which I would suggest to keep as simple as possible (see it as wiring rather than logic)
08:21 babilen An alternative approach would be to make sure that the event includes all data you need and then just pass it on.
08:21 djgerm1 it doesnt
08:21 djgerm1 i mean, it does, but i'd have to munge the id about
08:22 babilen Which also isn't tricky
08:22 djgerm1 nothing is, if you understand it
08:22 s_kunk joined #salt
08:23 babilen It literally boils down to: {% set host, domain = salt['pillar.get']('your:very:unique:key:from:reactor').split('.', 1) %}
08:23 babilen At least I hope so .. that's completely untested
08:24 Guest79 is this pythony type language?
08:24 djgerm1 very unique key from reactor?
08:24 babilen That is jinja with a larger sprinkle of pure Python, yeah
08:24 djgerm1 oh the id?
08:25 babilen djgerm1: Well, whatever you use to pass that data as kwarg: pillar into the state (cf. https://docs.saltstack.com/en/latest/topics/reactor/#passing-event-data-to-minions-or-orchestrate-as-pillar → newminion)
08:25 babilen But yeah, that's the id
08:25 babilen Use whatever name/key you see fit (it corresponds to 'newminion' in the reactor example)
08:28 djgerm1 the more i read the documentation and read what you're saying, the more confused I get. and I'm not even drinking.
08:29 djgerm1 i dont think i want any kwargs
08:29 djgerm1 the args already have key words.
08:36 babilen arg: would be the state you call, kwargs allows you to pass in pillars
08:38 djgerm1 like this? http://paste.debian.net/432321
08:38 babilen add_new_minion_to_pool calls (local.state.apply) the "haproxy.refresh_pool" state (this is the arg to state.apply) on minions targeted by 'haproxy*'. It also passes in additional data via pillars (the kwarg "pillar" to state.apply)
08:38 djgerm1 my reactor calls that state
08:38 babilen No, the kwarg of state.apply is "pillar"
08:39 babilen Do you know Python and the meaning of "kwargs" in this context?
08:39 djgerm1 i dont know python at all
08:39 djgerm1 or krwargs
08:39 Miouge joined #salt
08:40 babilen Okay, it isn't tricky. You can write functions in Python and hand over arguments like f(a, b, c)
08:41 ronnix joined #salt
08:42 babilen You have to pass all these arguments in order to call the function. In addition to that you can also include keyword arguments (kwargs) which are being handed into the function as a dictionary. In this case you want to pass "pillars" to the state.apply function.
08:43 om joined #salt
08:45 djgerm1 jeez, I thought the reactor was applying a state
08:46 djgerm1 i need a state.apply inside the state that the reactor is applying?
08:46 djgerm1 my heads gonna fall off.
08:48 _Dave_ djgerm1: python's not hard and there are tutorials out there
08:49 djgerm1 gosh I didnt know i had to learn python.
08:49 djgerm1 this concept seemed so simple when I started
08:49 traph joined #salt
08:50 djgerm1 so python is prereq for being able to use salt?
08:50 AndreasLutro no
08:51 AndreasLutro it sure helps though
08:51 AndreasLutro but as long as you know how function calls and arguments work you'll be fine
08:52 elsmo joined #salt
08:53 mavhq joined #salt
08:53 babilen djgerm1: You can literally copy the example in the documentation and add the id parsing to the state you call
08:54 djgerm1 so it'll be reactor > calls new state > calls my original state ?
08:55 mavhq joined #salt
08:55 impi joined #salt
08:56 teatime babilen: wb btw; you didn't sleep long?  :)
08:56 mavhq joined #salt
08:57 teatime babilen or anyone:  (because I'm curious) I don't suppose there's any chance salt-cloud would pause its execution after firing the 'destorying' event until the reactors listening for that event have finished execution?
08:58 teatime I can't find anything that indicates that happens, but if not it makes me wonder why have both 'destroying' and 'destroyed'.  Although, I suppose, if the destruction process begins but fails mid-way, you'd see one but not the other and maybe be able to do something useful with that.
08:59 babilen djgerm1: Rather "reactor > calls your original state"
09:00 babilen But then .. what exactly do you refer to by "state" here? Can we make this explicit? Could you paste what you have to one of http://refheap.com, http://paste.debian.net, https://gist.github.com, http://sprunge.us, … ?
09:01 AndreasLutro teatime: source code reveals no, salt-cloud doesn't block on "destroying"
09:01 RandyT joined #salt
09:01 AndreasLutro though it could technically vary from driver to driver
09:01 djgerm1 sure babilen, just a mo
09:01 west575 joined #salt
09:02 teatime AndreasLutro: thx
09:04 djgerm1 http://paste.debian.net/432324
09:04 Miouge joined #salt
09:04 djgerm1 my reactor, the new state I created just now for it to call
09:04 capricorn_one joined #salt
09:04 djgerm1 and the state that the reactor state calls
09:05 NightMonkey joined #salt
09:06 losh joined #salt
09:06 babilen djgerm1: Make that http://paste.debian.net/432325/ or somesuch
09:06 babilen You pass in the pillar as you would do on the command line in salt
09:09 djgerm1 I didnt realize you could pass pillar data on the command line :)
09:09 djgerm1 before tonight
09:09 josue joined #salt
09:10 djgerm1 i've bene restarting salt-master everytime I change reactor.conf, is that necessary?
09:11 mavhq joined #salt
09:13 djgerm1 well… that didnt seem to work… I am guessing since I dont see anything about dns states in the output from "salt-cloud -d -m mymap.map -l debug" that it's not reactoring properly?
09:19 djgerm1 is there a place that events are documented? maybe since destroyed isn't documented here https://docs.saltstack.com/en/latest/topics/cloud/reactor.html#available-events any more maybe it cant be used in a reactor?
09:19 mavhq joined #salt
09:19 djgerm1 even though tons of examples use it…
09:20 djgerm1 ah maybe something else
09:20 djgerm1 https://docs.saltstack.com/en/latest/topics/event/master_events.html#event-master-events
09:20 wendall911 joined #salt
09:24 dheeraj joined #salt
09:26 dheeraj Hi, I have a peculiar problem regarding scheduling of a highstate run. I have a custom grain "sensong_interface" which is set using a state (as _grains/ doesn't have access to existing grains which is needed for setting this grain). Then my other states use the "sensing_interface" grain in templates.
09:26 dheeraj Running a highstate on a new machine fails as "sensing_interface" is not set and templates fail to compile
09:27 dheeraj I need to run the state manually (state.sls custom_grains) first and then state.highstate suceeds
09:27 dheeraj I was looking at automating the deployment of new machines and would like to put highstate into a scheduler so that it runs every 30 minutes on all the minions
09:28 dheeraj but I need the custom_grains SLS to run before I run the highstate scheduled job
09:28 dheeraj any idea how I can do this?
09:28 dheeraj or are there alternate strategies I should look at?
09:32 babilen dheeraj: https://docs.saltstack.com/en/latest/topics/reactor/index.html#syncing-custom-types-on-minion-start
09:33 babilen And what kind of grains do you need for your custom grain that aren't available?
09:33 babilen (just curious)
09:33 dheeraj a list of interfaces on the system
09:33 dheeraj I can;t access __grains__ or other dunder dicts in _grains/foo.py
09:33 dheeraj so can't write a custom_grain based on existing grains
09:34 dheeraj I guess reactor might work in so far as I run the state on minion start
09:34 babilen Ack. You could use the same method the core grain uses for retrieving that information
09:34 dheeraj but will the run be  "held" till I accept the key on the master (for a new minion)
09:35 babilen Take a look at the order of events when you accept the key
09:35 dheeraj babilen: I thought about that...but it seems too involved...and that might change in a future release
09:35 babilen Sure
09:35 dheeraj babilen
09:35 dheeraj :babilen: thanks, will do
09:36 babilen You should get the auth events first and then the minion start
09:36 babilen But I haven't memorised everything :)
09:37 dheeraj babilen: heh....thanks for the reactor tip....it should get me started
09:37 babilen But I would also consider writing some robust code for retrieving that information in your custom grain. That code should still work even if saltstack uses something else in the future.
09:37 babilen It would make it a "stand alone" grain.
09:38 dheeraj yes...i'll also look into that
09:39 scc joined #salt
09:41 M-MadsRC joined #salt
09:42 mavhq joined #salt
09:45 cpowell joined #salt
09:53 Rumbles joined #salt
10:00 M-liberdiko joined #salt
10:02 mowntan joined #salt
10:02 mowntan joined #salt
10:07 djgerm1 any advice on where to troubleshoot when you don't see any reactor entry after "Compiling reactions…."?
10:08 djgerm1 no "Rendered data from" in the log
10:08 djgerm1 running salt-master -l debug
10:09 mavhq joined #salt
10:10 djgerm1 babilen: that pastebin you sent with the fixed state made things make sense a bit more. Thanks!
10:11 babilen Good
10:14 djgerm1 i don't understand why the reactor isn't triggering though. i should at least get some broken rendering.
10:14 djgerm1 or a complaint that it can't find the reactor state…
10:17 favadi joined #salt
10:17 kshlm joined #salt
10:18 _JZ_ joined #salt
10:18 babilen Restart master, paste output, prepare offerings for the spaghetti monster
10:27 djgerm1 i notice that no environment is being specified with the files under file_roots example
10:27 djgerm1 https://docs.saltstack.com/en/latest/topics/reactor/#mapping-events-to-reactor-sls-files
10:27 djgerm1 i wonder if that's it
10:28 babilen I don't do environments as they make everything pesky
10:28 babilen (rather stand-alone masters)
10:29 coolads joined #salt
10:32 djgerm1 i am putting the reactor state not in file_roots and testing again.
10:32 mavhq joined #salt
10:32 ravenx joined #salt
10:32 djgerm1 i cant find any docs that say saltenv is supported in reactor
10:32 crooton joined #salt
10:33 ravenx i only have this in my sls file:  https://paste.debian.net/432336/    and it keeps throwing "ERROR: Minions returned with non-zero exit code"
10:33 ravenx despite it succeeding
10:33 ravenx anyone know why?
10:36 babilen ravenx: Start the minion in debug mode (salt-minion -ldebug) and check what happens
10:37 Faris joined #salt
10:39 josuebrunel joined #salt
10:40 josuebrunel joined #salt
10:43 dmaiocchi joined #salt
10:44 quasiben joined #salt
10:47 R0N joined #salt
10:48 ravenx what is supposed toh appen, happens
10:48 ravenx only info and a few debugs
10:48 ravenx nothing of interest ...
10:48 ravenx i did tget this:
10:48 amcorreia joined #salt
10:48 ravenx [WARNING ] /usr/lib/python2.7/dist-packages/salt/minion.py:594: DeprecationWarning: Master pub message signing is disabled but we received a signature for this message.  Most likely this means that your masters and minions are not the same version.  After Salt 0.17.6 this s
10:48 ravenx ituation will throw an exception.
10:48 ravenx salt.utils.warn_until((0, 17, 6), 'Master pub message signing is disabled but we '
10:48 djgerm1 i've seen similar exit states that look like an error, but arent really. I usually write a test in to cover those
10:49 ravenx djgerm1: so you get those too?
10:51 cyberviking joined #salt
10:53 cyberviking joined #salt
10:54 djgerm1 yeah i think it's because "nothing changed" so the state "fails to apply"
10:54 djgerm1 even though… everything is good
10:54 djgerm1 i get them with like the partition module
10:55 djgerm1 not usually with pkg.installed
10:56 babilen A state succeeds if it has been achieved already
10:56 babilen (so that would be a bug)
11:04 DaveQB joined #salt
11:08 antpa joined #salt
11:08 hajhatten joined #salt
11:19 mortis does salt have any way of ensuring a file is present on only one of its matching minions without matching again on i.e. hostname in the sls?
11:20 Rumbles joined #salt
11:20 ravenx djgerm1: wow..so it returns that message
11:20 ravenx when nothing chagned
11:20 ravenx that's kinda stupid.
11:21 djgerm1 yeah. it's gotta be abug….
11:21 fredvd joined #salt
11:21 djgerm1 babilen: I am so close! I could use a little more of your python fu. I got my state to be acknowledged, but it's not able to render
11:21 djgerm1 barfing on {% set host, domain = salt['pillar.get']('minion_id').split('.', 1) %} because "need more than 1 value to unpack"
11:22 _Dave_ do you have a '.' in minion_id?
11:23 djgerm1 yes
11:23 pulp7 joined #salt
11:23 djgerm1 which i see when the reactor is rendered
11:23 djgerm1 (it's the fqdn)
11:25 djgerm1 http://paste.debian.net/432341/
11:25 djgerm1 is the state.
11:28 djgerm1 and here is what the reactor state is rendering as
11:28 djgerm1 http://paste.debian.net/432342/
11:30 josuebrunel joined #salt
11:31 punkoivan joined #salt
11:32 Muchoz joined #salt
11:36 cyberviking joined #salt
11:37 teryx510 joined #salt
11:42 source47 joined #salt
11:43 nZac joined #salt
11:47 djgerm1 maybe if I set them separately? with a rsplit for domain
11:48 djgerm1 hmm no
11:50 bluenemo joined #salt
11:50 djgerm1 maybe i need to put "salt['pillar.get']('minion_id')" in it's own special single quotes or something?
11:56 josuebrunel joined #salt
11:59 babilen djgerm1: Try {% minion_id = salt['pillar.get']('minion_id') %} and {% set host, domain = minion_id.split('.', 1) %}
12:00 josuebrunel joined #salt
12:03 djgerm1 ok…
12:03 djgerm1 here goes!
12:04 cyborg-one joined #salt
12:05 josuebrunel joined #salt
12:05 djgerm1 dang
12:06 djgerm1 Encountered unknown tag 'minion_id'
12:06 djgerm1 oh
12:06 djgerm1 set minion_id
12:07 babilen err, set yeah
12:12 djgerm1 darn back to "ValueError: need more than 1 value to unpack"
12:12 djgerm1 for {% set host, domain = minion_id.split('.', 1) %}
12:12 djgerm1 well i am calling it quits for the night
12:12 babilen djgerm1: What is the value of minion_id at that point?
12:13 djgerm1 let me check again. it was rendering correctly last I pasted
12:14 ronnix joined #salt
12:14 djgerm1 bah. i took the master out of debug so it didnt show me the reactor state render
12:16 djgerm1 maybe I need to put the minion_id in single quotes back in the reactor state
12:17 pid1 joined #salt
12:17 zphantom joined #salt
12:19 AndreasLutro more likely your minion_id does not have any . in it
12:20 djgerm1 when i had the master in debug earlier, it was rendering as expected.
12:21 djgerm1 http://paste.debian.net/432342/
12:22 west575 joined #salt
12:24 djgerm1 anyway. thanks for all your help.
12:24 djgerm1 we're so close it hurts to go to sleep…. but I've been awake too long and there's work in a few hours.
12:24 djgerm1 see ya on the flip side.
12:25 babilen saltenv=stage would be a kwarg btw
12:25 babilen But good night
12:27 zphantom Hi people, I have some problem in a state with a cmd.run step, if I call it with salt (using minion and zmq) the state work correctly but if I use salt-ssh I have this error: "State 'cmd.run' was not found in SLS 'test2'\nReason: 'cmd.run' is not available"
12:28 zphantom is cmd.run not supported in salt-ssh? I looking in docs but found nothing about it :(
12:28 teatime babilen: what's making you guys think minion_id should be in pillar
12:28 teatime did you do something to put it there?
12:28 babilen https://docs.saltstack.com/en/latest/topics/reactor/#passing-event-data-to-minions-or-orchestrate-as-pillar
12:29 babilen djgerm1: btw, you are missing two spaces of indentation on line 9
12:29 babilen (and, conversely, line 10)
12:30 honestly why would file.recurse copy a file called ".dummy.hg" if I set exclude_pat: E@^\. :/
12:30 teatime honestly: hrm, is E@ how you tell it' it's a regex?  I've only seen that syntax for compound target matching so far
12:31 honestly https://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.recurse
12:31 Hydrosine joined #salt
12:31 honestly according to the docs...
12:31 honestly I also tried just using exclude_pat: .*
12:31 honestly same result
12:32 babilen https://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.recurse → "exclude_pat .... exclude_pat: E@(APPDATA)|(TEMPDATA) :: regexp matches APPDATA or TEMPDATA for exclusion"
12:32 teatime fair enough
12:32 teatime babilen: heh, I saw
12:33 teatime honestly: basename/relative-to-dir issues?  what does '**/.*' do ?
12:33 honestly teatime: it complains if I put a / into the pattern
12:34 teatime one would it imagine they should be relative to source though
12:34 teatime ahh
12:35 babilen honestly: Are you sure that the state you apply contains that pattern?
12:35 babilen (pushed the git, updated fileserver, ...)
12:35 evle1 joined #salt
12:36 babilen Also try excluding '*' :)
12:36 honestly babilen: well salt tells me it's adding files that match the pattern...
12:36 babilen Which pattern?
12:37 honestly ...the exclude pattern...
12:37 honestly 14:30:08 < honestly> why would file.recurse copy a file called ".dummy.hg" if I set exclude_pat: E@^\. :/
12:37 babilen It is genuinely saying that it is adding files that match exclude_pat ?
12:37 babilen Could you paste that output?
12:37 TooLmaN joined #salt
12:37 honestly no, it is saying that it is adding files, and those files match the exclude_pat
12:38 babilen I think you misunderstood my question. I simply want to make sure that the state you execute corresponds to the state you assume you execute
12:39 honestly -l debug shows me the rendered sls that is executed.
12:39 honestly It is what I expect it to be.
12:40 geomacy joined #salt
12:41 garphy joined #salt
12:43 teatime honestly: do you also have an include pattern?
12:43 honestly no.
12:43 honestly that shouldn't matter anyway.
12:43 babilen yeah
12:43 mavhq joined #salt
12:44 teatime was just curious.
12:44 babilen include_pat is being applied first and the results filtered with exclude_pat
12:44 _Dave_ I suspect that regexp is wrong
12:44 babilen honestly: Could you provide some actual information (i.e. states, output, ...) ?
12:44 honestly tell me a better then.
12:44 honestly a better one*
12:45 teatime how does \ in unquoted strings... try 'E@^\.' ? or even 'E@^\..*$' on the theory that the regex may be implicitly anchored (not likely)
12:45 teatime er, how does YAML handle \ in unquoted strings, I meant
12:45 honestly http://paste.debian.net/plain/432356
12:46 honestly http://paste.debian.net/plain/432357
12:47 cpowell joined #salt
12:48 babilen /srv/ldap/compat-in/.dummy.hg doesn't begin with .
12:48 honestly ...
12:48 honestly so what?
12:48 honestly it's not a full path match.
12:48 babilen *shrug*
12:49 teatime honestly: 'dummy' and '\.dummy' might be worth trying to get more insight
12:50 DammitJim joined #salt
12:51 teatime sorry, I meant with E@ in the front of those
12:52 honestly sigh, E@.*/\./* works.
12:52 honestly so for regex it's a full path match apparently.
12:52 teatime clear docs ftw, lol
12:53 babilen Perfect
12:54 kows joined #salt
12:54 honestly no idea what glob matching does.
12:54 honestly ¯\_(ツ)_/¯
12:55 teatime probably fnmatch which doesn't do whole paths just path segments
12:56 teatime file.recurse seems to pass all that logic off to salt.utils.check_include_exclud
12:57 quix joined #salt
12:58 teatime which does regex match against the whole path, or runs fnmatch against the whole path... which actually should work with /'s although "Note that the filename separator ('/' on Unix) is not special to this module.", so no **/* style globs.
12:58 * teatime loses interest, goes back to perl6 stuff.
12:58 subsignal joined #salt
12:58 DammitJim LOL @ perl6... I know nothing about perl
12:59 honestly version 6 is to perl as version 3 is to python
12:59 honestly or so I've heard
13:00 edrocks joined #salt
13:01 akhter joined #salt
13:03 west575 joined #salt
13:03 teatime yeah it's not a terrible analogy
13:03 teatime Perl6 is more different from Perl5 than Python3 is from Python2, though.
13:04 teatime Perl6 will possibly taste bad to both non-perl-programmers and Perl5-programmers, heh.
13:04 teatime but I am enamoured, at the moment.
13:04 teryx510 joined #salt
13:04 xmj Is there a way to download the entire salt docs?
13:05 capricorn_1 joined #salt
13:06 subsigna_ joined #salt
13:06 babilen xmj: They are part of the source -- Might find a tarball somewhere, but I'd have to check
13:06 xmj babilen: the freebsd port doesn't ship them to /usr/local/share/doc, what's necessary to make it?
13:07 xmj I'm going to spend some qualitytime with AirCanada thursday & friday, and have a hunch the entire saltdoc would be *really* sueful
13:07 xmj useful, too.
13:07 babilen xmj: https://docs.saltstack.com/en/latest/topics/development/hacking.html#editing-and-previewing-the-documentation
13:07 Faris joined #salt
13:07 sk_0 joined #salt
13:07 xmj nice
13:08 babilen That's what I use .. no idea if there is better "offline" infrastructure
13:10 cro joined #salt
13:10 AndreasLutro Total run time: 86412.251 s
13:10 AndreasLutro lolwut
13:11 babilen Taking slim infrastructure to the max!
13:11 xmj 1day?
13:12 xmj i wonder what comes out if i run gmake epub or gmake pdf
13:14 AndreasLutro 12 seconds + 24 hours.. no idea how that happened
13:14 xmj sleep(1day) ?
13:14 xmj AndreasLutro: trick question, what were you doing and should it have taken a day? :)
13:15 AndreasLutro I was running it in my terminal, if it took 1 day I would've noticed
13:15 AndreasLutro something must've screwed up with the system clock... for that one particular state run >_>
13:17 CampusD joined #salt
13:17 xmj AndreasLutro: did it contain ntpdate or the likes?
13:18 CampusD Hi All, had a question about jinja variable checking
13:18 AndreasLutro my highstate does include both ntp configuration and timezone management, but this is still the first time I see this happening
13:18 CampusD I know I can do {% if variable is defined %} to check if a var exists/is defined
13:18 CampusD http://jinja.pocoo.org/docs/dev/templates/#defined
13:19 CampusD is there a way to do a negate of that
13:19 CampusD ?
13:19 AndreasLutro stick a "not" in there
13:19 mowntan joined #salt
13:19 AndreasLutro if variable is not defined
13:19 mowntan joined #salt
13:19 mowntan joined #salt
13:19 CampusD so {% if variable is not defined %}
13:20 xmj it can be defined but None
13:20 xmj i ran into *this* once. it was embarrassing.
13:20 CampusD is that like empty?
13:20 dmaiocchi joined #salt
13:22 renaissancedev joined #salt
13:22 kus joined #salt
13:23 cyborg-one joined #salt
13:24 babilen {% if variable is defined and variable %} ?
13:24 andrew_v joined #salt
13:25 CampusD would this yield a defined variable if the pillar value wasn't there or was empty? {% set variable = pillar['to']['some']['path'] %}
13:25 babilen I'd always use salt['pillar.get']('foo:bar:baz', DEFAULT_VALUE)
13:26 xmj babilen: if {% variable is not None %}
13:26 xmj is what i went with
13:26 teatime CampusD: That wiil just error if a parent doesn't exist
13:26 xmj it wasn't a pillar, that's the thing
13:26 _Dave_ reading the docs, I haven't come across  {% yet. where will I meet this?
13:26 flowstate joined #salt
13:27 teatime I am doing:  {% set foob = salt.['pillar.get']('foo:bar:baz', none) %}  {% if foob is not none %}
13:27 babilen *None
13:27 teatime babilen: none
13:28 teatime babilen: None works too, but docs recommend none
13:28 babilen Why?
13:28 xmj Ha! I have a 2200page Salt handbooks
13:28 babilen xmj: \o/
13:28 xmj yay for writing invoices in tex and having everything installed.
13:28 babilen teatime: Serious question .. I have used None so far ..
13:28 xmj yeah, i had issues with "none" (lowercase)
13:28 babilen xmj: xelatex all the way!
13:28 teatime the above ^^ assuming you really want to operate any time the value is defined, even if it's "" or {} or [] or such... usually you can use something quite a bit simpler since usually you don't want to operate on those values.
13:29 xmj because Jinja doesn't ... weird... things
13:30 dkrae joined #salt
13:30 teatime http://jinja.pocoo.org/docs/dev/templates/#literals
13:30 nZac joined #salt
13:30 teatime apparently the recommendation to use none (since the ver where they added None; didn't used to exist) is merely "for consistency" with other Jinja identifiers.
13:31 teatime so, screw that then, I prefer consistency with Python
13:31 xmj indeed.
13:31 babilen wtf?
13:31 babilen jinja is so speshul
13:31 babilen Thanks for finding it in the documentation, teatime
13:32 babilen I have not read that before and would simply have assumed that "normal" Python literals work just like they do in Python
13:32 xmj actually
13:32 xmj I'll embarass myself but teatime is right again
13:32 evle1 joined #salt
13:33 xmj in the template I was referring to, {% if foo is not none %} is the one that works
13:38 teatime xmj: lol, your sacrifice is appreciated ;)
13:39 teatime arrrgh, I forgot to plug in and let my stupidly-slow laptop hibernate by accident.
13:40 CampusD using the default value might do for my needs, I had forgotten about that option
13:40 CampusD thanks for the tips and thoughts
13:42 teatime CampusD: you can for example give [] or {} as the default for something that should be a list/dict, and then for-looping over it will not execute the loop body in the default case.
13:42 teatime so you can skip if's and set's
13:42 aanriot joined #salt
13:43 CampusD in this case it wouldn't be a list, just a single value, so comma and value in quotes?  , 'my_default_value' %}
13:44 aanriot is it possible to use salt-cloud to create a VM without any OS ? I mean a blank VM. It seems that the "clonefrom" argument is mandatory
13:44 JohnnyRun joined #salt
13:44 kows joined #salt
13:45 teatime CampusD: yeah
13:47 CampusD {% set variable = pillar['to']['some']['path'], 'my_default_value' %} , am I missing some parentheses here?
13:52 ferbla joined #salt
13:53 babilen CampusD: {% set variable = salt['pillar.get']('to:some:path', 'my_default_value' %}
13:54 CampusD i see, thanks babilen:
13:54 AdamSewell joined #salt
13:54 xmj teatime: haha
13:55 babilen CampusD: Alternatively: {% set variable = pillar.get('to', {}).get('some', {}).get('path', 'my_default_value') %}
13:55 teatime CampusD: the other form is dangerous/annoyign because if pillar['to'] or pillar['to']['some'] is undefined, it throws an exception.
13:56 teatime you can specify the delimiter I think, if : is inconvenient at some point
13:57 CampusD i see, many ways to do a task I guess
13:58 xmj it depends how many exceptions you want to catch
13:59 kows joined #salt
13:59 asoc_ joined #salt
13:59 ronnix joined #salt
14:00 teatime I was thinking there was some other way pillar.get was superior, but I can't think of anything now.
14:00 teatime oh, the defaulting behavior, I guess.
14:00 babilen Unfortunately Python doesn't allow for easy lookup in nested collections so Saltstack had to implement that themselves
14:00 jerredbell joined #salt
14:01 teatime hrm... I wonder if I could re-work/re-implement the 'defined' jinja test to actually work for pillar.get...
14:01 akhter joined #salt
14:01 akhter joined #salt
14:01 teatime hrm, probably not w/o changing pillar.get's default return, which woudln't really be workable :(
14:04 AndreasLutro {{ pillar.foo.bar.baz | default('default') }} should work in most cases
14:04 mapu joined #salt
14:04 babilen I don't really like defining defaults inline, but that might be a personal preference
14:05 mortis is there anyway of "breaking out of" a matching? if i want to match a set of minions, but only apply something to the first match?
14:06 mortis sounds silly, but there are some cases where you want to run a job on only one minion without maintaining some sort of fqdn-matching in the sls or similar
14:06 babilen mortis: You could just target the first minion
14:06 mortis babilen: in an sls for the whole set of minions i cant really do that
14:06 mortis afaik
14:06 babilen Well, you would target a SLS for all to all and then maintain the "first" one to the single minion
14:07 babilen You could, naturally, test the value of grains and then run into different branches
14:07 mortis say i have lolserver1,lolserver2 and their setup is the same, just that i want to have a file managed on the first one of them no matter if i add another 100 lolservers
14:07 mortis weird case hehe
14:08 mortis yeah, i can test on minionid or fqdn or whatever, but i would like it do be dynamic
14:08 babilen You could to {% if grains['id'] == "lolserver1" %} foo {% else %} bar {% endif %}
14:08 mortis yeah
14:08 mortis but then i would have to change lolserver1 to lolserver2 if lolserver1 is lost
14:09 andrew_v joined #salt
14:09 babilen Get a list of all "lolservers", sort it, compare current id against first id in list
14:10 babilen (from the mine for example)
14:10 mortis yeah
14:10 mortis something like that
14:10 mortis is what i need to do
14:10 mortis i could dynamically build a pillar with all the lols
14:10 mortis and have each minion query that
14:10 mortis i guess
14:10 mavhq joined #salt
14:11 babilen What is the bit you do want to do or don't want to do on the first?
14:11 mortis its gonna run a curl :x
14:11 mortis every 10 mins
14:11 mortis and can only run on 1 ..........cause well ...devs have been trolling me
14:11 pfallenop joined #salt
14:11 mortis i thought of using the salt scheduler
14:12 mortis but then i would have to batch-run it somehow and escape it
14:12 mortis hehe
14:12 hasues joined #salt
14:12 hasues left #salt
14:14 kows joined #salt
14:14 mortis since all lolservers would have that pillar
14:14 babilen mortis: How do you decide who is the leader?
14:14 mortis anyone can be the leader, but only one can be the leader at the time
14:15 jf_ joined #salt
14:15 mortis it can easily be fixed with stringmatching, but that means i will have to remember that if the leader goes down
14:15 mortis would be great to not have to change config when that happens
14:15 mortis heartbeat is a bit overkill
14:15 babilen Well, how do you know who the leader is (programmatically that is) ?
14:16 kaptk2 joined #salt
14:16 mortis i dont, really, so it wouldnt even be possible from the minion anyway i guess
14:16 scoates joined #salt
14:16 mortis unless you hack some peers stuff into it hehe
14:17 tharkun joined #salt
14:18 mortis i should look for a more correct fix, like telling them to fix the code
14:18 babilen Well, you could implement the heuristic of "first in an alphabetically sorted list of online minions"
14:18 mortis something like that yeah
14:18 babilen Or fix the thing
14:18 mortis its kinda hard for the minion to determine
14:19 mortis unless i do lots of stuff
14:19 mortis "lots"
14:19 mortis or use the api ..
14:19 mortis hm
14:21 mavhq joined #salt
14:24 akhter joined #salt
14:26 mavhq joined #salt
14:28 quix joined #salt
14:28 Hydrosine joined #salt
14:31 fer_bla joined #salt
14:37 mavhq joined #salt
14:39 fredvd joined #salt
14:40 flowstate joined #salt
14:45 racooper joined #salt
14:48 geomacy joined #salt
14:51 fxhp joined #salt
14:52 fxhp joined #salt
14:52 hightekvagabond joined #salt
14:56 mavhq joined #salt
14:57 Faris joined #salt
14:58 kows joined #salt
14:58 quix_ joined #salt
14:58 ronnix joined #salt
15:01 catpig joined #salt
15:02 josuebrunel joined #salt
15:08 Brew joined #salt
15:08 beardedeagle joined #salt
15:08 mavhq joined #salt
15:08 Eureka703 Quick but basic question. Can a salt-syndric act as a "master" for a group of systems controlled by a team of admins. I.E. If there is a master of masters that hands down "policy based states", Can a syndric override any of those states based on team-based needs?
15:13 akhter joined #salt
15:14 LondonAppDev joined #salt
15:14 ntropy Eureka703: i don't think that's possible
15:15 Eureka703 @ntropy I thought that was the case. As far as I can tell to get something like that would require using something like gitfs with multiple repositories/environments?
15:17 ntropy i suppose thats one way
15:18 Eureka703 I suppose another way would be to run multiple masters that are also minions of a primary master or group of masters to enforce specific things on each team's master?
15:18 ntropy another would be to distinguish those servers from the rest, like give them a unique role, and then just provision them with your standard top.sls setup
15:19 Eureka703 I know its a strange configuration, this comes from a place where people dont like new things and want to be able to override anything from the primary master (kind of defeats the purpose of CM/Masters/etc I think).
15:19 Eureka703 Right, that would work as well.
15:20 spuder joined #salt
15:20 onlyanegg joined #salt
15:21 Eureka703 I think it might be easiest to convince them to use the git/environments setup. Then they still have to conform to "base" but can tack on other stuff as they choose to.. Not sure how well that would work at any sort of scale though. Might be easiest to just have them run masterless.
15:24 akhter joined #salt
15:27 brotatochip joined #salt
15:29 rburkholder joined #salt
15:31 zak joined #salt
15:36 akhter joined #salt
15:41 cliluw joined #salt
15:42 Faris joined #salt
15:42 cpowell joined #salt
15:42 djgerm1 babilen: it was that white space on the reactor. saltenv isn't kwarg
15:49 ahammond is there an idiom for generating a timestamp in salt / jinja?
15:52 onlyaneg1 joined #salt
15:52 Trauma joined #salt
15:56 DammitJim joined #salt
15:56 Fiber^ joined #salt
15:56 flowstate joined #salt
15:56 whatever_sd_ joined #salt
15:56 dmaiocchi joined #salt
15:57 _Dave_ ahammond just call out to a date command
15:57 ivanjaros joined #salt
15:58 ahammond er... {% set timestamp = ??? %}
15:59 _Dave_ sorry n00b here. I've seen shell stuff out there.
15:59 _Dave_ also, there must be python
15:59 teatime heh there's a module to exec arbitrary python ... ...
15:59 beardedeagle use a cmd.run, example inbound
15:59 ahammond teatime that's evil. I like evil.
16:00 ahammond beardedeagle how's that going to get it into my jinja variable?
16:00 beardedeagle give me a moment
16:00 babilen djgerm1: So, it works with proper whitespace?
16:00 beardedeagle {% set timestamp = salt.cmd.run('date')
16:00 beardedeagle or whatever command you need to run to get your timestamp formated the way you want
16:00 ahammond beardedeagle dude. that makes perfect sense.
16:01 ahammond thanks!
16:01 beardedeagle np
16:01 beardedeagle forgot the %} , my bad
16:01 _mel_ joined #salt
16:02 edrocks joined #salt
16:05 mpanetta joined #salt
16:05 linjan joined #salt
16:08 onlyanegg joined #salt
16:11 nZac joined #salt
16:11 stinky joined #salt
16:11 alemeno22 joined #salt
16:12 teatime hehe, 2016.3.0 adds status.time
16:12 teatime which takes a format arg
16:14 heaje joined #salt
16:17 ahammond beardedeagle works like a charm. Thanks again!
16:17 beardedeagle npnp
16:17 ahammond will you be at saltconf? I'll buy you a beer. :)
16:17 beardedeagle I will actually
16:17 beardedeagle And I do love my beer
16:18 writtenoff joined #salt
16:18 ahammond ok, I'll be the guy with pink hair. :)
16:19 kows joined #salt
16:19 beardedeagle lol alright.
16:19 grumm_servire joined #salt
16:22 aw110f joined #salt
16:23 pwhack joined #salt
16:24 av_ joined #salt
16:24 brotatochip joined #salt
16:28 ronnix joined #salt
16:31 Faris Hey guys. When should I consider Masterless  mode using salt and what are its perks? :)
16:33 bltmiller joined #salt
16:34 brianfeister joined #salt
16:36 onlyanegg joined #salt
16:37 gtmanfred when you just want to?
16:38 gtmanfred in my experience i have seen it used when you want your minions to live seperately, or you are only managing one box, so you don't really need a second box to run salt master commands
16:39 gtmanfred if you don't need any of the orchestration or remote execution stuff that you get from having a master
16:39 gtmanfred (though, soon, not 2016.3, but the next major release, the reactor will be available as an engine on the minion)
16:43 debian112 joined #salt
16:43 akhter joined #salt
16:44 aganders3 joined #salt
16:45 impi joined #salt
16:47 pfallenop joined #salt
16:47 cberndt joined #salt
16:48 mTeK joined #salt
16:49 cliluw joined #salt
16:52 ronnix joined #salt
16:53 disbound joined #salt
16:55 Guest79 joined #salt
16:59 amcorreia joined #salt
17:03 Netwizard joined #salt
17:04 hightekvagabond joined #salt
17:04 sslb joined #salt
17:04 quaie joined #salt
17:04 quaie left #salt
17:05 scoates joined #salt
17:09 Heartsbane joined #salt
17:11 hightekvagabond joined #salt
17:12 lero joined #salt
17:16 rm_jorge joined #salt
17:17 ahammond so... I generated a timestamp and am trying to pass it as a pillar. I get SaltClientError: can't serialize datetime.datetime(2016, 4, 12, 10, 9, 11)
17:17 ahammond O
17:18 ahammond I'm guessing pyyaml is cleverly recognizing it as a timestamp and parsing it as a datetime... and then msgpack is too stupid to serialize it usefully?
17:19 teatime that's exactly what it sounds like
17:19 teatime and yes, YAML definitely does parse timestamps
17:19 ribx joined #salt
17:19 teatime if you don't want that behavior, quote it.
17:22 ahammond I
17:22 hvn joined #salt
17:22 hvn joined #salt
17:22 ahammond well, I'd prefer it was a datetime and that datetime didn't suck, but... :)
17:24 teatime it would be nice to pass around python datetime objects, but if it doesn't work (never tried it), RFC3339 (subset of ISO8601) strings are pretty much just as good
17:24 teatime info on the YAML type: http://yaml.org/type/timestamp.html
17:27 kevinquinnyo joined #salt
17:29 ajw0100 joined #salt
17:29 mavhq joined #salt
17:31 kevinquinnyo so i'm playing with the salt-api and just as a test i'm doing something like this:  https://gist.github.com/kevinquinnyo/de853a13e3fc57b8bad881d32a610e81 now when i do curl 10.170.246.24/hook/services/restart -H 'Accept: application/json' -H 'X-Auth-Token: auth-token-here' -X POST --data "tgt=web1-my-host.com&service=nginx" i get {"success": true} but nothing is happening
17:31 ahammond teatime yeah, that's what I'm using.
17:32 kevinquinnyo what should i look for to try to troubleshoot it?  i'm running salt-api -l debug and i can see the request is a 200 http status, and i'm running salt-minion -l debug and salt-master -l debug respectively
17:32 ahammond ok, in an orchestration is there something analogous to sleep?
17:32 kevinquinnyo the service on the remote minion does not restart
17:32 ahammond I hate to ask, I'm adding a new pillar and it seems to need a couple of seconds to propegate.
17:33 subsignal joined #salt
17:34 akhter kevinquinnyo: You don't see what happens when the service is restarted?
17:34 mavhq joined #salt
17:34 kevinquinnyo well on the remote minion i'm doing a:  ps auxf | grep nginx
17:34 kevinquinnyo and the start time for the service remains the same
17:36 kevinquinnyo i dont know where this should be being logged -- i'm thinking it's not even executing the cmd.service.restart because that should be logged to salt-master (i'm watching the debug log on it)
17:36 akhter I'm not very familiar with salt-api, so this is a learning experience for me.  Where are you passing the nginx service?
17:36 kevinquinnyo in the post data
17:37 ageorgop joined #salt
17:37 teatime is /hook/services/restart correct?
17:37 kevinquinnyo the salt-api supposedly (i'm learning too) will pass thhe http request info as a data dict so my 'reactor file ', restart.sls gets this data dict for free
17:37 kevinquinnyo then i pull it from data['post']
17:37 kevinquinnyo what do you mean is it correct?
17:38 akhter Do you see your reactor initiating?
17:38 akhter I believe you'll need to listen to the salt socket stream to see if it is running.
17:38 teatime I've also not used api.  but it seems like a kindof funny combo of the sls file path and the hook name thing in reactor.conf, thought maybe it was mistaken
17:38 akhter But you said you're returning a True bool, so it should be working.
17:39 akhter kevinquinnyo: Can you share the docs you read?
17:39 west575_ joined #salt
17:39 akhter So that I can understand from your point of view.
17:40 kevinquinnyo i actually borrowed some from http://bencane.com/2014/07/17/integrating-saltstack-with-other-services-via-salt-api/ but i was also reading the official docs for both the reactor system and the salt-api at https://docs.saltstack.com/en/latest/topics/reactor/ and https://docs.saltstack.com/en/latest/ref/netapi/all/salt.netapi.rest_cherrypy.html and trying to piece it all togetyher
17:41 kevinquinnyo oh teatime you might be right
17:41 akhter I was just reading that same article.
17:43 aganders3 left #salt
17:43 kevinquinnyo hmm the annoying thing is that is returns a 200 for any URI...
17:43 kevinquinnyo curl 10.170.246.24/hook/restartsdfsdfhsdkfhkjsadfhsad -X POST
17:43 kevinquinnyo is a 200 OK
17:44 teatime lol
17:46 mavhq joined #salt
17:48 akhter kevinquinnyo: Did you find the issue?
17:48 kevinquinnyo nope
17:48 akhter Just going by the sls, it should work.
17:48 akhter I'd open a ticket in github then :(
17:48 hal58th joined #salt
17:48 kevinquinnyo i threw:  `print data` into the restart.sls file and it printed out what i expected into the salt-api log
17:49 kevinquinnyo http headers, and a dict called post that has my post data
17:49 west575 joined #salt
17:49 kevinquinnyo tgt, and service
17:49 kevinquinnyo so i dont know why it's not working
17:50 armguy_ joined #salt
17:50 hvn joined #salt
17:50 hvn joined #salt
17:50 kevinquinnyo [DEBUG   ] Compiling reactions for tag salt/netapi/hook/services/restart
17:50 kevinquinnyo i see that in the salt-master log
17:50 kevinquinnyo when i hit it with curl, but it does nothing after that
17:52 mavhq joined #salt
17:57 kevinquinnyo hmm and i can see it registering the event salt/netapi/hook
17:58 josuebrunel joined #salt
18:01 barajasfab joined #salt
18:01 edrocks joined #salt
18:01 subsignal joined #salt
18:01 Muchoz joined #salt
18:02 mavhq joined #salt
18:03 flowstate joined #salt
18:08 west575 joined #salt
18:09 rmnuvg joined #salt
18:11 flowstate joined #salt
18:12 djgerm1 babilen: it does
18:16 akhter joined #salt
18:18 baweaver joined #salt
18:18 aw110f joined #salt
18:19 dmaiocchi joined #salt
18:21 cliluw joined #salt
18:24 kevinquinnyo i had a typo in my /etc/salt/master.d/reactor.conf
18:24 kevinquinnyo it's always a typo
18:24 kevinquinnyo Always.
18:24 akhter kevinquinnyo: Glad you got it sorted out.
18:25 akhter You're the first I've seen using the py renderer, any particular reason?
18:25 jfroot_ joined #salt
18:27 quasiben joined #salt
18:28 kevinquinnyo i hate yaml
18:28 kevinquinnyo i find it much more confusing than just using python data structures
18:28 kevinquinnyo akhter: in answer to your question
18:29 kevinquinnyo whenever i see yaml in salt context, i convert it to json so i can understand it
18:29 akhter kevinquinnyo: How do you write multiple states within one file then?  Do you just include the function in and run the run function last?
18:29 kevinquinnyo yeah usually
18:30 akhter That's not bad.  I might start doing that to confuse people :)
18:30 akhter Thanks.
18:30 kevinquinnyo haha
18:30 kevinquinnyo i mean you can just add to the states[] list as many times as you want
18:31 fredvd joined #salt
18:31 Muchoz joined #salt
18:31 kevinquinnyo but you can also separate a state into logical functions, then call them one at a time in your default run() function, passing the states[] to each one
18:31 kevinquinnyo which i like doing
18:31 renaissancedev joined #salt
18:31 akhter One more question though.  On modules like file.managed, if you have multiple environments, how would you pass the environment the source file belongs to.
18:32 kevinquinnyo i've never used multiple environments, i probably should
18:32 akhter I'll need see an example of how you use the states list.
18:32 kevinquinnyo but if it's an option in file.managed, you have access to it just like you would with yaml renderer
18:32 kevinquinnyo i'll show you
18:32 kevinquinnyo one sec
18:33 akhter For environments, the doc says not to change the __env__ list.
18:33 akhter And just execute it on shell with saltenv, I didn't know saltenv passed the environments to modules like file.managed.
18:34 kevinquinnyo i'm guessing that, just like __pillar__, __grains__, and __salt__ are available to you in your state files regardless of which renderer you use
18:34 kevinquinnyo so would __env__ but that's a guess
18:34 akhter Yeah, I'll play around with it.
18:37 kevinquinnyo https://gist.github.com/kevinquinnyo/8c50f49196a07ca69da6853771bef7b0 akhter that's a very simple example of the way i write states
18:39 akhter Ahh, so states is a dict and you return the whole dict.
18:39 akhter Which in-turn returns all of the functions.
18:39 kevinquinnyo yes
18:39 akhter That makes sense.
18:39 kevinquinnyo not sure if that's the best way to do it, but I like how it keeps it very clear
18:39 kevinquinnyo also if you use the py renderer, you can build a python class if you need to which might be helpful
18:40 akhter Yeah, not only that, also be able to import other modules when needed.
18:40 kevinquinnyo yep
18:41 akhter Cool, I'll give this a shot.
18:41 akhter Thanks.
18:42 kevinquinnyo np
18:42 brotatochip joined #salt
18:45 UtahDave joined #salt
18:47 felskrone joined #salt
18:47 salty_solution joined #salt
18:47 SheetiS joined #salt
18:50 _armguy_ joined #salt
18:56 jfelchner joined #salt
19:00 brianfeister joined #salt
19:03 scoates joined #salt
19:07 DammitJim joined #salt
19:12 source47 joined #salt
19:12 akhter kevinquinnyo: You still there, got one more question.
19:15 mavhq joined #salt
19:16 huddy joined #salt
19:17 kevinquinnyo akhter: sure
19:17 akhter On your reactor, you're calling cmd.service.restart.
19:17 akhter Is that something you wrote.
19:17 akhter I don't see on the docs anywhere.
19:17 brotatochip joined #salt
19:17 kevinquinnyo no i copied that from that blog post i think it might be deprecated or something.  i changed mine
19:17 kevinquinnyo let me paste you what i did
19:18 akhter Okay, thanks.
19:18 akhter I just created a crontab sls for the fun of it.
19:22 catpig joined #salt
19:23 kevinquinnyo https://gist.github.com/kevinquinnyo/4f05fd0f77348f0c3c8cbb2196ef2db1 akhter:  i made a general purpose state called reload_service.sls that i call from the reactor, and i pass the service name via a pillar (kind of a hack, but a good way to pass arbitrary data to a state)
19:24 kevinquinnyo there are probably better ways to do this, for instance, i bet you could call the salt service module itself/directly from the reactor, instead of using a state
19:24 edrocks joined #salt
19:24 kevinquinnyo the reason i have the aut disabled is because i handle that upstream in an haproxy instance that sites on my salt-master listening on port 443 by the way
19:24 kevinquinnyo auth*
19:25 ajw0100 joined #salt
19:26 akhter I like how you're passing the pillar..
19:26 brianvdawson joined #salt
19:27 akhter That's not something I've seen before.
19:27 akhter passing the pillar from state to state (if I'm not mistaken).
19:27 JoeJulian You have to. Reactor states have no access to the pillar data.
19:28 akhter Ahh, is that why you have to call local.state.sls?
19:29 mavhq joined #salt
19:30 JoeJulian akhter: I just gave this presentation at Incontro Devops International that looks like it might interest you: https://docs.google.com/presentation/d/1ke-fbNSiZQyF7201pt7BEllB77b6JlTWKain7EX60pk/edit#slide=id.g10ed149ead_0_15
19:31 akhter JoeJulian: Thanks, I'll go through that.
19:31 JoeJulian local is an async version of the salt class LocalClient which is used to perform commands on the client. This is used when called from the cli, like: "salt-call --local state.sls gluster.peer"
19:31 jfroot_ joined #salt
19:31 JoeJulian So local.state.sls calls state.sls on the client specified.
19:32 akhter That makes sense.
19:34 baweaver joined #salt
19:34 akhter I find it weird how the value 'pillar' is loaded into __pillar__ when passing the pillar.
19:34 akhter Is it true for grains in that case?
19:35 akhter Can you guys tell I don't deal much with reactors.
19:35 armguy_ joined #salt
19:35 andrew_v joined #salt
19:36 kevinquinnyo it's the same as when you run salt \* some_state.something pillar="{'foo': 'bar'}"
19:36 kevinquinnyo i believe
19:36 kevinquinnyo syntax may be wrong
19:36 dayid joined #salt
19:37 akhter Nice, I'll see if it's possible to pass the grains in a similar fashion.
19:37 catpig joined #salt
19:39 source47 joined #salt
19:40 Heartsbane joined #salt
19:40 pfallenop joined #salt
19:41 quasiben joined #salt
19:42 cberndt joined #salt
19:44 catpig joined #salt
19:49 jfelchner joined #salt
19:51 morissette joined #salt
19:52 kevinquinnyo does the reactor have access to __salt__ modules?
19:52 mavhq joined #salt
19:53 ajw0100 joined #salt
19:53 teatime surely.
19:53 teatime probably in a salt[] dict like in pillars and states, right?
19:54 kevinquinnyo yeah well i'm trying to access __salt__['mymodule.some_function']()
19:54 kevinquinnyo inside a reactor and i get an error
19:54 kevinquinnyo File "/usr/lib/python2.7/dist-packages/salt/utils/lazy.py", line 93, in __getitem__
19:54 kevinquinnyo raise KeyError(key)
19:54 kevinquinnyo but i can run that module function from the command line and it works
19:54 teatime do other, standard modules work?  maybe your module just hasn't been sync'd to the host where that runs
19:55 west575 joined #salt
19:55 kevinquinnyo i did sync_all
19:55 teatime but I don't know anything about reactor, so I should probably stfu
19:55 AndreasLutro kevinquinnyo: possibly not inside reactors... possibly
19:55 teatime kevinquinnyo: still, so standard ones work?  troubleshooting == dividing the space of potential problems into what it might still be related to and what has been ruled out
19:55 teatime *do standard ones…
19:56 AndreasLutro reactors are meant to be dumb and simple, I wouldn't be surprised if modules are disabled
19:56 quasiben joined #salt
19:57 kevinquinnyo it's not complaining about test.ping
19:57 AndreasLutro does your custom module function work when you use it standalone?
19:57 bltmiller joined #salt
19:57 AndreasLutro salt-call mymodule.some_function etc
19:58 AndreasLutro check your minion logs, salt tends to swallow module import errors
19:58 kevops joined #salt
19:58 source47 joined #salt
19:58 kevinquinnyo AndreasLutro: yes
19:59 kevinquinnyo it works with salt-call
19:59 kevinquinnyo it also works with salt minion-tgt-here mymodule.myfunction
20:00 kevinquinnyo so maybe with reactors, it lazyloads the default salt modules, but does not load custom user modules
20:00 AndreasLutro github issue time?
20:01 kevinquinnyo maybe
20:01 AndreasLutro oh
20:01 AndreasLutro one more possibility...
20:02 AndreasLutro since reactors are rendered by the master, not the minion, I think you have to use the extension_dirs or whatever instead of /srv/salt/_modules
20:02 kevinquinnyo ohh
20:03 kevinquinnyo ugh
20:03 hightekvagabond joined #salt
20:04 hrumph2 joined #salt
20:04 hrumph2 yo
20:05 hrumph2 'sup?
20:05 jab416171 how long does salt cache grains?
20:05 hrumph2 i'm trying to understand teh winrepo a bit better
20:05 iggy yeah, I filed a ticket about that _ages_ ago
20:05 iggy I think there might be some changes there in 2016.3
20:05 hrumph2 so i want to make sure that the pkg_refresh_db has been done before state.apply done
20:05 mTeK joined #salt
20:06 jab416171 and how do I force it to refresh the grains cache
20:06 hrumph2 its a bunch of windows workstations. not always on
20:06 hrumph2 so if i call salt '*' pkg.refresh_db it won't refresh all of them
20:06 hrumph2 only the ones that are on
20:07 hrumph2 so what should I do with the state to be certain that the db's have been refreshed before it looks for installed stuff
20:07 hrumph2 ?
20:07 mavhq joined #salt
20:09 jab416171 ugh, I freaking hate amazon AMIs
20:09 jab416171 or at least how we manage them.
20:10 hrumph2 should I just use "refresh: True" in my use of the pkg.installed ?
20:10 hrumph2 is that the best thing?
20:11 UtahDave reactors run on the salt master
20:11 UtahDave hrumph2: Yeah, I think so
20:12 hrumph2 UtahDave: ok it's not a problem. it's a fast operation anyway
20:16 heaje joined #salt
20:16 hrumph2 UtahDave: when you mentoned reactors were you talking to me?
20:18 kevinquinnyo i think he was talking to me based on my previous questions
20:19 hrumph2 kevinquinnyo: that's what i thought too but i was just checking
20:19 bowhunter joined #salt
20:19 source47 joined #salt
20:19 hrumph2 kevinquinnyo: or at least i thought he was talking to someone else anyway even if i didn't know who
20:21 UtahDave hrumph2: no, sorry!  I was replying to a previous conversation
20:21 eightyeight joined #salt
20:21 UtahDave sometimes I jump in way too late, lol
20:24 punkoivan joined #salt
20:24 punkoivan left #salt
20:26 baweaver joined #salt
20:32 hvn joined #salt
20:32 hvn joined #salt
20:33 hightekvagabond joined #salt
20:35 _armguy_ joined #salt
20:42 hoonetorg joined #salt
20:45 Gibzon joined #salt
20:45 DammitJim joined #salt
20:46 Gibzon hello!
20:47 Gibzon i have this issue on windows, the master loses connection to the minion after 10min and there is nothing on the log, anyone experienced the same problem?
20:47 DammitJim can I not have a -  in pillar?
20:47 DammitJim something like: ForceCommand: 'internal-sftp'
20:48 DammitJim is there a way I can get around this?
20:48 teatime there's nothing invalid about that all
20:48 teatime if a value starts with - you have to quote it; that's all i can think of about -
20:48 DammitJim TypeError: unsupported operand type(s) for -: 'StrictUndefined' an
20:48 DammitJim d 'StrictUndefined'
20:49 teatime can you show more of your file or whatever you're doign
20:49 DammitJim yeah, hold on
20:50 armguy_ joined #salt
20:50 DammitJim ha
20:50 DammitJim nevermind
20:51 DammitJim I still haven't defined that value... right now it's a hard coded variable... yuck
20:51 DammitJim thanks
20:51 teatime heh figured it out I guess?
20:53 ahammond I'm having a problem with the following not working: {%- set most_recent_stamp = salt.cmd.run('cd /srv/private_backups; FILENAME="$(ls -1r | head -1)" echo ${FILENAME%%.*}')
20:53 ahammond I'm guessing it's because salt.cmd.run isn't running a shell?
20:53 cpowell joined #salt
20:54 ahammond because most_recent_stamp is an empty string.
20:54 cpowell joined #salt
20:56 teatime ahammond: that doesn't even work for me in a shell
20:56 teatime also note, cmd.run will use /bin/sh by default.  but that line doesn't work for me even in zsh.
20:56 source47 joined #salt
20:57 ahammond well, it expects a bunch of files in /srv/private_backups with names like x.tar.bz2
20:57 teatime lol, I was smart enough to account for that
20:57 ahammond [root@shss98 private_backups]# cd /srv/private_backups; FILENAME="$(ls -1r | head -1)" echo ${FILENAME%%.*}
20:57 ahammond 20160412T102132
20:57 ahammond works for me... :)
20:58 teatime ahammond: instead of a cd, use the cwd= argument to cmd.run, and just do "ls -1r | head -1"
20:58 teatime ahammond: what shell?
20:58 ahammond teatime and... remove the .tar.bz2 in jinja?
20:58 teatime ahammond: see if this works:  sh -c 'cd /srv/private_backups; FILENAME="$(ls -1r | head -1)" echo ${FILENAME%%.*}'
20:59 teatime ahammond: well, forgive me for not knowing what it does, since it doesn't work for me :)
20:59 ahammond nope, that doesn't work
20:59 ahammond strang
20:59 ahammond works when I execute it from CL, but not when I -c it
20:59 teatime ahammond: oh, because echo is a shell built-in... put a ; before echo
21:00 ahammond teatime thanks! :)
21:00 cberndt joined #salt
21:00 ahammond and... the state works. Awesome. :)
21:01 subsignal joined #salt
21:01 flowstate joined #salt
21:01 s_kunk joined #salt
21:02 Brew1 joined #salt
21:02 TyrfingMjolnir joined #salt
21:03 source47 joined #salt
21:03 Brew1 joined #salt
21:03 v12aml joined #salt
21:09 akhter joined #salt
21:09 keimlink joined #salt
21:10 hal58th joined #salt
21:12 flowstate joined #salt
21:14 quasiben joined #salt
21:19 akhter joined #salt
21:20 _armguy_ joined #salt
21:24 source47 joined #salt
21:28 scoates joined #salt
21:28 TheoSLC joined #salt
21:29 TheoSLC Greetings.  I just added an integration for saltstack and JetBrains TeamCity -> https://github.com/JetBrains/meta-runner-power-pack/tree/master/saltstack
21:30 kus joined #salt
21:30 TheoSLC This is like the Jenkins plugin with a few more features.
21:35 armguy_ joined #salt
21:37 bowhunter joined #salt
21:38 UtahDave nice!
21:44 ribx joined #salt
21:47 lero joined #salt
21:48 gtmanfred nice!
21:48 gtmanfred that is awesome
21:49 baweaver joined #salt
21:49 source47 joined #salt
21:50 liskl joined #salt
21:50 Faris joined #salt
21:57 Brew1 joined #salt
22:03 edrocks joined #salt
22:05 _armguy_ joined #salt
22:08 hightekvagabond joined #salt
22:10 west575 joined #salt
22:15 baweaver joined #salt
22:19 Faris joined #salt
22:22 inire joined #salt
22:30 AdamSewell joined #salt
22:33 love2scoot joined #salt
22:34 love2scoot hey all, running into a problem on 16.04 with salt-master not starting once python-pygit2 is installed.  Using standard repos.
22:35 armguy_ joined #salt
22:39 jfelchner joined #salt
22:39 cliluw joined #salt
22:40 zenlot6 joined #salt
22:43 sjmh joined #salt
22:44 sjmh hm.  how do you target based on a compound matcher that has a pillar value with spaces in it?
22:44 sjmh ie, along the lines of:
22:45 sjmh salt -C 'min* and G@osfinger:CentOS Linux-7' test.ping
22:45 love2scoot joined #salt
22:45 sjmh er, grain, sorry, not pillar.
22:45 jfelchner_ joined #salt
22:46 love2scoot had to upgrade browser- any word on pygit2+salt-master+16.04?
22:49 sjmh ugh, you have to use \s
22:49 quasiben joined #salt
22:58 barajasfab joined #salt
23:00 jab416171 how can I set a variable (in jinja) with an if statement, and then use the variable later on in a state?
23:01 teatime yes
23:01 jab416171 like, {% if foo %} {% set var = 'foo' %} {% else %} {% set var= 'bar' %} {% endif %} mystate: \n  - pkg.installed: {{ var }}
23:01 teatime if it's not in the same file there are limitations.
23:01 teatime but yes, that works.
23:02 teatime http://jinja.pocoo.org/docs/dev/templates/
23:02 jab416171 teatime, I get a jinja error, var is not defined
23:03 jab416171 var's scope ends after the {% endif %}
23:03 kevinquinnyo1 joined #salt
23:03 teatime (in addition to the jinja docs, there are a couple of niceities added by salt described at https://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.jinja.html )
23:03 teatime (but not relevant here)
23:04 hightekvagabond joined #salt
23:05 teatime jab416171: hmm.
23:07 source47 joined #salt
23:07 nZac joined #salt
23:09 teatime I wonder if you can do {% set var %} or {% set var = none %} above/outside the if, and then inside it do something like {% do var = 'foo' %} ... or something similr.
23:10 teatime I did not realize it did scoping like that
23:11 brotatochip joined #salt
23:11 teatime you could also use the block form of {% set %}, but that would be slightly annoying
23:14 teatime you can also probably do {% set var = 'foo' if foo else 'bar' %} .. syntax might be slightly off there
23:15 Heartsbane joined #salt
23:15 Heartsbane joined #salt
23:15 CampusD joined #salt
23:17 mosen joined #salt
23:20 _armguy_ joined #salt
23:26 keimlink_ joined #salt
23:27 jab416171 teatime, setting it outside doesn't work, because when you set it inside, it creates a new variable
23:29 RandyT joined #salt
23:30 bowhunter joined #salt
23:30 mosen hi saltines
23:30 RandyT joined #salt
23:31 jab416171 is that what they call us now?
23:31 teatime it really does not seem like if should make a scope
23:31 RandyT joined #salt
23:31 jab416171 ok, what's the difference (from the salt master) between: salt-call state.highstate and salt $HOSTNAME state.highstate
23:31 jab416171 assuming the master is also a minion
23:32 RandyT joined #salt
23:33 eightyeight joined #salt
23:34 teatime salt-call runs the module function (state.highstate) directly, locally, without involving the master
23:34 teatime you can turn off the master and still salt-call
23:34 jab416171 what would be the expected difference in state execution?
23:34 mapu joined #salt
23:34 teatime ideally none
23:34 jab416171 well, there is a difference
23:35 teatime certain features of saltstack require the master
23:35 teatime but for straightforward states there should not be a difference
23:35 jab416171 salt-call state.highstate provides the expected behavior (one branch of an if statement in jinja), wheres salt $HOSTNAME state.highstate executes the other branch.
23:35 armguy_ joined #salt
23:35 jab416171 I could do: salt '*' cmd.run 'salt-call state.highstate' and that would do what I want, but if I do salt '*' state.highstate that will do what I don't want
23:36 teatime jab416171: btw, there's all kinds of examples via google (and even salt docs https://docs.saltstack.com/en/develop/ref/renderers/all/salt.renderers.jinja.html ) showing setting vars inside if and using them later... I really don't think if makes a scope?
23:36 teatime I'm fairly sure I've used it myself but I couldn't find an example in my current stuff
23:37 teatime jab416171: is this for the same host in both cases?
23:37 teatime and what is the if statement...
23:40 teatime ^^ would likely be obvious if you show your branching logic.
23:41 jab416171 yeah, same host
23:42 jab416171 this is inside a file.managed
23:42 flowstate joined #salt
23:42 jab416171 https://gist.github.com/jab416171/f37c4dd20c8af664884c9c6b4ddedf72
23:44 jab416171 the whole array thing is a hack to make it work outside of the scope of the if/for
23:47 teatime so it's the grain 'master' that varies between runs?
23:48 bbradley joined #salt
23:48 teatime between salt-call and salt, I mean.
23:49 jab416171 yes
23:50 jab416171 salt-call grains.item master returns a list of all 3 masters
23:50 jab416171 whereas salt $HOSTNAME grains.item master only returns $HOSTNAME
23:50 jab416171 as a string
23:50 jab416171 but that really shouldn't matter
23:50 _armguy_ joined #salt
23:51 teatime where $HOSTNAME is the minion id of the same master you ran salt-call on ?
23:51 jab416171 yes
23:51 jab416171 it's also literally $HOSTNAME (the environment variable)
23:51 jab416171 in this case, it would be salt01.example.com
23:52 kevinquinnyo1 so this is a weird question:  given a glob target, like 'web*-1001.myplatform.com', is there an internal salt function that will evaluate that glob target and return a list of matches that i could use arbitrarily?
23:52 teatime you're running salt-call as root?
23:52 teatime kevinquinnyo1: I think they tend to go the other way, from a list + a glob → a subset of the list; but if you start w/ something like manage.up or otherwise have a list of all minions, you could manage it.
23:53 teatime jab416171: if you have multiple masters, I don't know much about that.  but I think the master grain just comes from the.. config.. file... specifically, the minion config file... I bet that's it.
23:53 kevinquinnyo1 teatime: ok i think i see what you're saying
23:54 kevinquinnyo1 and actually i just realized i'm solving a problem in the wrong way by even needing that
23:55 kevinquinnyo1 teatime: by the way i shoudl seriously start paying you for the last few days of 24/7 tech support ;)
23:57 jab416171 teatime, yes, there's multiple masters, and it comes from /etc/salt/minion (or minion.d)
23:57 keimlink joined #salt
23:58 teatime kevinquinnyo1: lol.  I'm glad to have been able to help.

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary