Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2018-04-24

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 onslack joined #salt
00:33 JacobsLadd3r joined #salt
00:42 noraatepernos joined #salt
01:14 DammitJim joined #salt
01:17 justanotheruser joined #salt
01:24 shred joined #salt
01:28 hemebond joined #salt
01:56 ilbot3 joined #salt
01:56 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2017.7.5, 2018.3.0 <+> Support: https://www.saltstack.com/support/ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ <+> See also: #salt-devel, #salt-offtopic, and https://saltstackcommunity.herokuapp.com (for slack) <+> We are volunteers and may not have immediate answers
02:01 shiranaihito joined #salt
02:02 ymasson joined #salt
02:46 dxiri joined #salt
03:06 shred joined #salt
03:23 dxiri joined #salt
03:44 shred joined #salt
03:45 shred joined #salt
03:46 shred joined #salt
03:47 shred joined #salt
03:48 shred joined #salt
04:15 jerematic joined #salt
04:16 jerematic joined #salt
04:16 jerematic joined #salt
04:17 jerematic joined #salt
04:18 jerematic joined #salt
04:40 AvengerMoJo joined #salt
04:54 briner joined #salt
05:02 oida joined #salt
05:07 jerematic joined #salt
05:18 sh123124213 joined #salt
05:21 justanotheruser joined #salt
05:47 jerematic joined #salt
06:00 brokensyntax joined #salt
06:03 sh123124213 joined #salt
06:04 chesty joined #salt
06:11 awerner joined #salt
06:20 wongster80 joined #salt
06:25 jerematic joined #salt
06:38 Elsmorian joined #salt
06:44 jrenner joined #salt
06:49 cbosdonnat joined #salt
06:49 cbosdonnat joined #salt
06:49 Bochi_ joined #salt
07:00 CrummyGummy joined #salt
07:00 Elsmorian joined #salt
07:00 Tucky joined #salt
07:03 jerematic joined #salt
07:05 briner joined #salt
07:09 Elsmorian joined #salt
07:12 Waples_ joined #salt
07:13 crux-capacitor joined #salt
07:15 toanju joined #salt
07:17 Hybrid joined #salt
07:19 Elsmorian joined #salt
07:27 orichards joined #salt
07:28 Elsmorian joined #salt
07:28 Hybrid joined #salt
07:38 jerematic joined #salt
07:44 Ricardo1000 joined #salt
07:46 rollniak joined #salt
07:47 J0hnSteel joined #salt
07:56 Elsmorian joined #salt
07:57 jerematic joined #salt
08:01 DanyC joined #salt
08:02 tyx joined #salt
08:10 DanyC joined #salt
08:11 Elsmorian joined #salt
08:23 briner joined #salt
08:25 Naresh joined #salt
08:30 Mattch joined #salt
08:34 jerematic joined #salt
08:45 awerner joined #salt
08:57 tys101010 joined #salt
09:00 briner joined #salt
09:11 jerematic joined #salt
09:20 evle1 joined #salt
09:33 evle1 joined #salt
09:36 squig joined #salt
09:38 squig Hi, I wondered if any one had seen a good example of using jinja in managed files?
09:38 squig trying to find examples, tutorials for my colleagues
09:42 jfindlay {%- if grains['mygrain'] %}
09:42 jfindlay myconfig: yes
09:42 jfindlay {%- endif %}
09:43 babilen heh
09:43 * zer0def claps
09:43 babilen That would fail if 'mygrain' isn't defined though
09:43 zer0def it could cause a migrain
09:44 babilen Let's adapt it to: {%- if grains.get('migraine', 'headache') %} then
09:45 jfindlay squig: my general approach to templating is to avoid it if you can.  I find that generating a file from programming elements is more readable/maintainable.  Otherwise I can't really think of a good example
09:49 jfindlay actually, I just setup two minions at either end of a site-to-site strongSwan VPN using the old ipsec.conf config format.  It was kind of tricky to mix and match left and right for each host, but that could be a good example for templating
09:49 jfindlay here's what I wrote about it: http://findlay.space/posts/strongswan-vpn-between-two-vms/
09:50 jfindlay the example is not salted, but you could imagine what a template would look like
09:51 squig yeah, I know how to do it but I am out of the office for a week and hoping to only review PR's :)
09:51 squig my staff are new to automation
09:52 squig so any examples I can find them is great
09:52 squig 500 manually installed linuxmachines we are slowly salting
09:54 jfindlay it's a good time to be in 'devops'.  Sadly, most corps I've interviewed at undervalue salt experience :(
09:54 squig puppet?
09:54 squig where do you live?
09:54 snath joined #salt
09:54 jfindlay I'm in Seattle
09:55 squig oh right we are in europe and looking :/
09:55 jerematic joined #salt
09:55 snath I'm still new to salt.  I'm trying to figure out how to escape characters in '- contents: |' under file.managed
09:56 jfindlay snath: what kinds of chars?
09:56 zer0def snath: well, that raises a flag - is there a reason you're not using a separate template file?
09:57 jfindlay the obvious alternate, if you're prepared to manage the entire file is to pack it into your state tree
09:58 snath A separate template file is extra complications.  It's just a 4 line script for /etc/update-motd.d/
09:58 snath and 2 of the lines are just setting color
09:58 squig ofcourse though then you dont have to deal with the escaping :D
09:59 zer0def i generally favor `source` and `template` over `contents`, helps making sls files readable and not like kubernetes yamls
10:01 zer0def of course, if $software takes a known format supported by salt's serializers, i prefer `file.serialize`, which may be dualistic of me ;)
10:01 snath putting it inline makes it obvious what the stanza does.  The meat is just an echo statement
10:02 jfindlay snath: try `- contents: "first\nsecond"`
10:02 snath it wouldn't even be that if Ubuntu supported /etc/motd.d/
10:03 jfindlay squig: if you're ever looking for remote work, let me know :)
10:06 exarkun joined #salt
10:10 toanju joined #salt
10:12 DammitJim joined #salt
10:13 squig we cant because of our clients ;/
10:15 hax404 joined #salt
10:27 crux-capacitor joined #salt
10:29 cablekevin joined #salt
10:37 jerematic joined #salt
10:58 briner joined #salt
11:04 xist joined #salt
11:11 Micromus joined #salt
11:17 tyx joined #salt
11:22 jerematic joined #salt
11:23 crux-capacitor joined #salt
11:25 Micromus joined #salt
11:26 jhauser joined #salt
11:29 evle1 joined #salt
11:38 briner joined #salt
11:44 tyx joined #salt
11:48 dendazen joined #salt
11:50 rollniak joined #salt
11:50 Waples_ joined #salt
12:02 evle1 joined #salt
12:03 briner joined #salt
12:03 daxroc joined #salt
12:07 dxiri joined #salt
12:13 Elsmorian joined #salt
12:14 pmcnabb joined #salt
12:21 Nahual joined #salt
12:22 jerematic joined #salt
12:28 jerematic joined #salt
12:33 mchlumsky joined #salt
12:34 DammitJim joined #salt
12:37 zulutango joined #salt
12:37 crux-capacitor joined #salt
12:41 crux-capacitor joined #salt
12:45 motherfsck joined #salt
12:50 evle joined #salt
12:51 viccuad joined #salt
12:51 Elsmorian joined #salt
12:52 CrummyGummy joined #salt
12:52 viccuad Hello folks. Is there a way to execute something in `states.x509` (states, not module) with `salt '*' state.apply` or the like? Particularly this one: https://docs.saltstack.com/en/latest/ref/states/all/salt.states.x509.html
12:52 viccuad thanks in advance
12:53 viccuad basically, is there a way to execute builtin states from the command line?
12:55 AngryJohnnie joined #salt
12:55 whytewolf viccuad: https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.state.html#salt.modules.state.single
12:58 ddg joined #salt
13:01 Micromus joined #salt
13:06 viccuad whytewolf: thanks!
13:07 DammitJim joined #salt
13:26 racooper joined #salt
13:32 Ricardo1000 joined #salt
13:33 evle joined #salt
13:39 Gareth joined #salt
13:42 nixjdm joined #salt
13:49 CrummyGummy joined #salt
13:52 magnus1 joined #salt
13:52 magnus1 joined #salt
13:53 toddejohnson joined #salt
13:56 s0undt3ch joined #salt
14:00 evle1 joined #salt
14:04 alkyl joined #salt
14:08 cgiroua joined #salt
14:08 magnus1 joined #salt
14:09 gareth__ joined #salt
14:12 viccuad left #salt
14:35 pbuell joined #salt
14:35 Elsmorian joined #salt
14:39 Elsmorian joined #salt
14:42 Elsmorian joined #salt
14:44 tiwula joined #salt
14:48 noraatepernos joined #salt
14:50 IPvSean_ joined #salt
14:51 DammitJim joined #salt
14:51 Bochi_ joined #salt
14:51 sceptiQ joined #salt
14:55 Elsmorian joined #salt
14:57 pbuell joined #salt
14:58 DanyC joined #salt
15:01 noraatepernos Does anyone know if there’s a salt yarn module in the works?
15:04 hop left #salt
15:07 Bochi joined #salt
15:14 nebuchadnezzar hello
15:15 evle joined #salt
15:20 Elsmorian joined #salt
15:21 sceptiQ Hi! Im trying to make first steps with "https://github.com/saltstack-formulas/apache-formula" but it looks like im missing something. I cloned the git and added the path to my env like it is described in https://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html#adding-a-formula-directory-manually maybe some one can tell me what is necessary from that point on to get the module to do *something*
15:21 sceptiQ on my minion?
15:22 sceptiQ a simple site configuration would be nice - the apache is installed and the service is beeing restarted but i can not manage to get any site configuration on the host
15:24 whytewolf sceptiQ: see the next section after that about the pillar.example information
15:24 whytewolf in this case read https://github.com/saltstack-formulas/apache-formula/blob/master/pillar.example
15:25 Bochi joined #salt
15:26 sceptiQ i did. and i created a apache.sls pillar in my env with a minimal set of information but ..i cannot see anything happen on my minion (https://gist.github.com/sceptiq/2754293f18558207511c645791736299)
15:28 whytewolf did you run the state afterwords?
15:30 whytewolf apache.config being the state to run
15:32 Hybrid1 joined #salt
15:34 AngryJohnnie joined #salt
15:40 sceptiQ not sure... what is missing that a "state.highstate" hits?
15:41 mavhq joined #salt
15:44 inad922 joined #salt
15:44 gtmanfred did you assign the pillar data to the minion?
15:44 gtmanfred and then refresh the pillar data
15:44 sceptiQ that was it
15:45 sceptiQ the apache.config thing in the top.sls
15:45 sceptiQ now things happen!
15:45 sceptiQ thx!
15:46 nebuchadnezzar playing with salt returners in PostgreSQL, I see that the job has no “scheduled timestamp” but the jid looks like a timestamp, should I consider it a good way to get the scheduled timestamp?
15:47 gtmanfred if you do not pass a jid to use, it uses the timestamp
15:47 gtmanfred return '{0:%Y%m%d%H%M%S%f}'.format(datetime.datetime.now())
15:48 gtmanfred https://github.com/saltstack/salt/blob/2017.7/salt/utils/jid.py#L17
15:49 sceptiQ is there something to read about how to work/use saltstack modules ... best practice...  some thing like that...
15:49 sceptiQ looking for details and examples ;)
15:50 Skree joined #salt
15:50 gtmanfred you can also convert the jid back to a time stamp using jid_to_time in salt.utils.jid https://github.com/saltstack/salt/blob/2017.7/salt/utils/jid.py#L35
15:51 Skree *ahem* this clearly wasn't the channel i was looking for typing in random names looking for a channel in which to vent my rage. Have a good day all :)
15:51 Skree left #salt
15:52 VR-Jack2-H joined #salt
15:57 eMBee babilen sorry i was off work before i saw your responses: very old means one 2014.1 minion and a few 2014.7. i can probably upgrade the old minions first though. but thanks for confirming that the old minions work fine. that is helpful! not every software package is so supportive of old versions. i'll see about upgrading the master first then. (i have been meaning to move the master to its own container. this might be the time to do that, t
16:00 eMBee babilen (to long message again) i'll see about upgrading the master first then. (i have been meaning to move the master to its own container. this might be the time to do that, then i can get the new version without problems)
16:00 DanyC joined #salt
16:06 nbari hi all, it is posible to retrive the current file_roots using jinja, for example when dealing with files, source: salt://foo/bar.txt where that maps to /salt/base/foo/bar.txt
16:07 nbari so within jinja how to expand something like {% 'salt://foo/bar.txt' %}
16:07 nbari it is posibel ?
16:08 MTecknology that is the expansion; it's telling salt to use the salt file server on the master to retrieve the file. I don't imagine there's any way to ask the master which root it got the file from.
16:10 nbari ok
16:12 hoonetorg what is wrong with this command on 2018.3.0: salt-call --state-verbose=False state.highstate test=True
16:13 hoonetorg it says: Passed invalid arguments: 'bool' object is not iterable.
16:14 MTecknology Using tcpdump, I see chatter between the syndic and MoM, mostly on 4506 but also a little on 4505.  The only error I see is '' Error while bringing up syndic for multi-syndic. Is master at <master-fqdn> responding? ''  I'm not sure what the heck I'm missing.
16:15 whytewolf hoonetorg: idk, it works perfectly fine for me in 2018.3.0
16:15 whytewolf let me try it on a minion with values in the top
16:16 whytewolf althought technically all of my minions have items in top ... counting the dynamicly generated error if they don't]
16:17 hoonetorg whytewolf: maybe master still at 2017.7.4 -> problem
16:18 whytewolf ... that would be a problem. you should never run a master lower then the version of the minion
16:21 whytewolf althought with salt-call it shouldn't matter
16:21 hoonetorg whytewolf: mmmmh will make snap before
16:24 hoonetorg thx
16:25 whytewolf np
16:26 nebuchadnezzar gtmanfred: thanks
16:31 MTecknology For a syndic setup, the master needs salt-master and "order_masters: True" and nothing more, ya? Then the syndic needs to run salt-master and salt-syndic and the salt syndic process behaves as a minion connecting to the master and just needs syndic_master configured?
16:42 MTecknology hmm... if I try to make the minion process on the syndic connect to the master of masters, it can connect just fine and great. :S
16:46 Elsmorian joined #salt
16:46 Trauma joined #salt
16:49 MTecknology ah, the key showed up so it can be accepted, but the minion responds to nothing. This is starting to feel like a bug.
16:53 Trauma joined #salt
17:00 NEOhidra joined #salt
17:03 AngryJohnnie joined #salt
17:03 Nahual salt-syndic for me has been extremely hit or miss, especially in a multi-master setup.
17:04 Nahual There's a forwarding events option you should set as well.
17:08 armyriad joined #salt
17:17 DanyC joined #salt
17:19 DanyC joined #salt
17:25 JacobsLadd3r joined #salt
17:27 Yamakaja joined #salt
17:33 DanyC joined #salt
17:34 Hybrid joined #salt
17:36 JacobsLadd3r joined #salt
17:44 jhauser joined #salt
17:44 jhauser left #salt
17:47 JacobsLadd3r joined #salt
17:50 crux-capacitor when using file.replace, what is the syntax when using a pillar value in the repl argument. Ex: -repl: '^master: ' {{ pillar['master']['id'] }}
17:53 MTecknology This just gets more and more frustrating. It seems like the minion can talk to the master. The syndic process is generating traffic, but no events are showing up. It's all encrypted so wireshark isn't revealing much information other than communication happening over 4506 and syn, syn-ack, ack, fin-ack, fin-ack. It seems like it's right about the smallest three way handshake possible to
17:54 MTecknology respond with a middle finger from salt, but no logs or events.
17:56 noraatepernos joined #salt
18:00 MTecknology Is it possible to decrypt captured packets with the master keys?
18:01 MTecknology crux-capacitor: It's usually better to either manage a file or not manage it, and I always recommend using .d style directories.  echo 'master: foo' >minion.d/master.conf
18:02 AngryJohnnie joined #salt
18:03 MTecknology Oh weird... the syndic is closing the connection, and then sending a second data packet. This has to be a bug...
18:04 crux-capacitor MTecknology: I'm using file.managed to place the master.conf in minion.d/, and at first tried to use pillar to populate that file, but I either did it wrong, or that doesn't work
18:04 crux-capacitor so I thought I'd use file.replace to edit the contents
18:10 XenophonF crux-capacitor: don't do that
18:10 ddg joined #salt
18:10 XenophonF use salt-formula
18:10 babilen +1
18:11 XenophonF also the master config overrides go in $prefix/etc/salt/master.d/
18:11 XenophonF don't put minion configs into a file named "master.conf" b/c that violates POLA
18:12 whytewolf what if the setting is master :P
18:12 XenophonF violates POLA
18:12 crux-capacitor yea im configuring the 'master:' setting on the minion
18:13 XenophonF don't do anything that's going to cause the sleep-deprived/super-stressed-b/c-the-CEO-is-screaming-at-you-to-get-this-fixed-NOW!! version of you any confusion
18:13 * MTecknology votes -1 to salt-formula
18:13 XenophonF yeah, MTecknology doesn't like formulas.
18:13 XenophonF he's wrong but it's OK, he's still one of us gobble gobble
18:13 XenophonF ;)
18:13 whytewolf also not a fan of formulas
18:14 MTecknology formulas are great if you want to have no clue what's going on
18:14 whytewolf also a fan of master.conf having the master config in it for the minion
18:14 whytewolf as i am least surparised by it ;)
18:14 pbuell joined #salt
18:16 JacobsLadd3r joined #salt
18:17 Hybrid joined #salt
18:17 nixjdm joined #salt
18:17 sh123124213 joined #salt
18:18 DammitJim joined #salt
18:19 XenophonF hehe
18:21 babilen MTecknology: I still have a come up with a different pattern that allows for easy definition of default configuration for different releases or even distributions, user settings and default overrides
18:23 babilen Formulas are simply a design pattern that allows the developer to organise things in a consistent way for many services
18:23 babilen It's therefore easy to understand new formulas as you are familiar with their patterns and functionality
18:24 babilen ...
18:24 MTecknology formulas are more than just a design
18:24 babilen Obviously not all formulas on GH adhere to these best practices (assuming there is a single set of 'best practices')
18:25 babilen Which brings us to the software development side of formulas and the inconsistent support/implementation of code in the wild
18:25 babilen I consider the patterns exemplified in "best practices" doc and template-formula to be quite flexible and easy to adapt
18:26 babilen Should probably use more consistent patterns for deep merging values with defaults.merge in map.jinja and clear patterns for defining release and distribution defaults, but that can be addresse
18:26 babilen d
18:27 babilen I much rather have settings in defaults.yaml + pillars than hardcoded configuration in states, pillars and all over the place
18:28 babilen The ecosystem would work well if consistent standards were enforced and if we'd allow inter-formula dependencies
18:28 noraatepernos joined #salt
18:30 babilen I mean .. criticise specific formulas, but I found the patterns the community came up with to be quite usable if I have to adapt salt code to a new Debian/Ubuntu/... release
18:32 Edgan How do you get overrides of settings without formulas? Do you create a state for every possible combination of options?
18:32 pbuell joined #salt
18:32 MTecknology huh?
18:33 MTecknology there's about a million billion different ways to feed data into states
18:33 Edgan MTecknology: But what defines "formula" to you
18:33 MTecknology https://github.com/saltstack-formulas/
18:33 MTecknology anything listed there, for starters
18:33 Edgan MTecknology: But I am more looking for the constrast between that and what you do
18:34 MTecknology I don't use formulas?
18:34 MTecknology I don't understand your question.
18:34 babilen So .. how do you it?
18:35 MTecknology with jinja
18:35 babilen Say you configure foo-service for Debian 8/9 and Ubuntu 14/16/...
18:36 babilen How do you manage the differences in configuration files/templates (if they exist), settings that should be used by default, ...
18:36 MTecknology depends on what the setting is
18:36 Edgan MTecknology: If you are using jinja, how is that not a formula? You don't use a map.jinja?
18:36 MTecknology formulas: https://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
18:36 zer0def wouldn't formulae equate to cookbooks and playbooks in the sense that you download a blackbox, feed it inputs and expect outputs? :P
18:37 MTecknology zer0def: +1
18:37 babilen zer0def: Well, same holds true for any software
18:37 babilen (such as salt itself)
18:37 zer0def babilen: that's in opposition to writing your own SLS files to deploy things
18:38 babilen Why would I want to do that if I can use community maintained states that do exactly what I want and that follow a clear and consistent pattern?
18:38 MTecknology I've never argued against the logic found in some formulas. I use some of that same logic myself.
18:38 MTecknology Using jinja does not automatically make something a formula. (see that previous link)
18:38 zer0def babilen: that's exactly the point, on top of arbitrary directory structure for components in chef or ansible, which validates the use of cookbooks and playbooks in the first place
18:39 babilen I'm simply saying that you can't just say "formulas are bad" as that conflates two things: 1. Formulas as design pattern of code/state organisation and 2. The quality and usefulness of specific implementations in the wild
18:39 Edgan Ok, lets make an important distinction. I think the public formulas you will find in the url above are crap. But that doesn't mean the idea of formulas is bad. You have to just take it further. I don't use public formulas, because they aren't advanced enough, and also don't have a shared style.
18:39 zer0def which, at least imho, means formulae are more about "i don't want to learn or write my own SLS, so i'll download some publicly available ones" than design
18:39 babilen Edgan: Exactly
18:40 * MTecknology +1's zer0def again
18:40 Edgan zer0def: It is like saying I had a bad steak once, so I will never make or eat another one.
18:40 zer0def well, that's not saying this at all
18:40 zer0def i'm basically equating using formulae to laziness
18:40 babilen Using SaltStack is laziness
18:40 Edgan zer0def: yeah, but formulas are more than public code
18:41 MTecknology using salt is management, not lazy
18:41 Edgan babilen: I would say it can be efficency
18:41 MTecknology I was drowning at $previous_job until I managed to roll salt, and then we doubled the number of systems I was managing.
18:41 Hybrid joined #salt
18:41 babilen Okay, let's not get sidetracked. I just think its important to differentiate between proven code patterns and their specific implementation
18:41 Edgan zer0def: The style of formulas are more than they are public. If it was just public code, you could just right a sls that says pkg.installed nginx, and call that a "formula"
18:42 Edgan babilen: nod
18:43 zer0def i agree with proven code patterns and implementation, as long as they don't force you to bang your head against a wall
18:43 babilen IMHO SaltStack failed in not building infrastructure to manage "third-party" states easily while enforcing certain coding and testing standards
18:43 Edgan I also think if we got mature enough, public code would be good. I actually think the community suffers heavily from lack of public code. People do 10x more advanced stuff privately.
18:43 MTecknology I think my head is starting to bleed..
18:43 zer0def writing formulae is just fine, but mindlessly downloading and using them without at least a review is lazy :P
18:43 babilen zer0def: Sure, but again that holds true for everything
18:44 ymasson joined #salt
18:44 Edgan zer0def: That you even download Saltstack and use it shows that is all about quality and standards
18:44 Edgan zer0def: Have you read all the Saltstack python code?
18:44 babilen I'd love to have a trustworthy repository to which main players in the salt userbase would contribute. Coding standards that are (automatically) enforced, tests being run and the ability to easily run "salt-formula enable foo-formula" on the CLI and similar things
18:44 MTecknology Edgan: are you attempting to remove logic from any discussion?
18:45 zer0def not all of it, but went though state decomposing to lowstate and requisite enforcement
18:45 Edgan MTecknology: No, I am trying to make a point
18:45 babilen IMHO formulas would also benefit from being forced to use inter-formula dependencies, so that not everyone has to write similar states
18:45 MTecknology It doesn't seem like you're making any point
18:45 Edgan babilen: There is a problem with that road though
18:45 zer0def what else… cloud implementation, a bunch of modules and states, but the entirety of salt would be an exercise
18:46 babilen Edgan: Please elaborate
18:46 zer0def Edgan: so to answer you question, i've at least skimmed most of the components i use, mostly because i managed to break each at a time or another ;)
18:47 ecdhe joined #salt
18:47 Edgan babilen: It would be a lot of work to cover every possible option of most software, and then the salt code would be very complex and hard to understand. Then there are more extreme cases like mongodb, nginx, and apache where it would be a constant battle to keep up with the latest options. You also get into a problem where you are really just writing a wrapper alternative language of the configuration format.
18:48 zer0def cases of mongodb, nginx and apache are relatively straight-forward, just yesterday openstack and it's configuration hell was brought up
18:48 Edgan babilen: But then to some extent you have to just accept a certain amount of that. Look at trying to do boto stuff through Salt. It is really just a slightly more user friendly way of doing it.
18:49 Edgan zer0def: I have seen the Chef nginx. There are thousands of options, and they are constantly changing. Yes, OpenStack is an even more extreme case.
18:50 zer0def as far as i'm concerned, all the forementioned options have relatively straight-forward options that rarely force you down specific configuration paths
18:50 zer0def well, "forementioned" as in "apache, mongodb, nginx"
18:50 babilen Edgan: I agree - I'm not saying that there has to be a single formula for every service, but that users could easily specify dependencies on other formulas in their own to install "additional" software. That way you could easily organise your code per-service and, say, write another formula that contains the "orchestration" to configure a setup consisting of these services
18:50 wongster80 joined #salt
18:51 babilen Which soon brings us to versioning and versioned dependencies, semantics associated with version numbers, ...
18:51 zer0def basically everything related to a programming language ecosystem ;P
18:51 noraatepernos joined #salt
18:51 pbuell joined #salt
18:51 babilen Indeed
18:52 zer0def i'm not sure how i feel about this
18:52 Edgan zer0def: infrastructure as code, the industry won't let you ignore it
18:52 zer0def the industry won't let me ignore a wider and more significant selection of pathologic buzzwords
18:52 Edgan babilen: I define paths in map.jinjas, and import them into each other. The only corner case is you can't have a circluar dependecy.
18:53 zer0def s/buzzwords/buzzphrases/
18:56 zer0def i like the direction in which we got totally derailed from "what constitutes a formula?"
18:56 coredumb \o/
18:56 crux-capacitor sorry to interrupt the discussion....can you add additional parameters onto an include: such as onchanges? I haven't gotten this to work yet
18:57 zer0def crux-capacitor: there's an `extend` keyword, which you can use to bolt on additional requisites on included states
18:57 zer0def probably more than just requisites
18:57 babilen crux-capacitor: Do you want to change included states?
18:58 crux-capacitor Yes. Just add on a requisite and order: last
18:59 zer0def crux-capacitor: https://docs.saltstack.com/en/latest/ref/states/extend.html
18:59 crux-capacitor yup, got it. thank you
18:59 babilen Edgan: Interesting .. I'll have to look into that at one point. Still think that better tooling around shared salt states would have made/make a massive different in Salt-Land. Easy integration of third-party code, easy uploads to a shared repository, automatic unit tests (or negative 'karma' if you don't have them), ...
19:00 briner joined #salt
19:01 babilen Maybe some well-maintained cookiecutter templates for typical use cases ..
19:01 zer0def while the idea is neat, i find it neater how frequently new modules and states get included upstream, which slightly undercuts the idea of using formulae, at least imho
19:02 MTecknology babilen: it sounds like what you really want is chef
19:03 babilen Formulæ should make use of all states that are available to them and developers should™ strive to integrate their code as modules/states rather than extensive SLSs, sure
19:03 zer0def i somehow find it easier to traverse the breadth of currently available modules than depth of formulae
19:03 babilen And, why is that?
19:05 zer0def that i don't have an answer to yet, but i'm pretty sure you're leading me onto something
19:05 babilen The pattern is always: You do something manually or with cmd.run, you integrate that it into your SLS, you do that more often and jinja gets complicated, so you write an execution module to do it for you and a matching state module ...
19:06 MTecknology or you build something clean and flexible before you build a mess
19:06 babilen In the end what we want is an easy way to consistently get the same effect on a huge number of platforms
19:07 babilen modules and states are a good starting point and formulas provide the building ground for defining default values and to incorporate differences between platforms
19:07 zer0def but then wouldn't you want to squash formulae down to modules, if possible?
19:08 MTecknology There we go... got the issue created.
19:09 zer0def ok, i think i understand your thought process, babilen
19:09 babilen To a certain degree, yes. Let me give an example: You have foo-service with a single configuration file: I'd be in favour of writing a renderer that is able to render the configuration file format and integrate that into SaltStack itself. The content of the configuration should not be defined in Salt, but could be defined (per os_family, os, osmajorrelease, ...) in defaults.yaml
19:10 babilen foo-service also has a, well, service/daemon ... you naturally want a state to say "daemon is running", but the different daemon names on various platforms should - again - live in the formula
19:10 zer0def inb4 some hipster busts in and goes "why don't you contain your app, so that distro differences don't matter?"
19:11 MTecknology snappy!
19:11 zer0def MTecknology: excuse me, sir, would you like to be shown the door? ;P
19:11 MTecknology snappy+docker are teh bestest! :P
19:11 babilen Now foo-service has the ability to enable "plugins" by creating symlinks and so on .. We write horrible code in Jinja using file.symlink to address all different possibilities ..
19:11 coredumb babilen: isn't that just a formulae with its map.jinja?
19:11 coredumb ah
19:11 coredumb mmmh
19:11 babilen That is a bad thing .. so we write a module and state module foo_service and, again, allow the user to define the plugins in the formula or via pillars
19:12 coredumb dunno getting back to my popcorn don't mind me
19:13 babilen In the end it really depends on the ability of the developer ... not everyone is able to write state and execution modules or new renderers/...
19:13 zer0def babilen: so you're basically talking of higher-order states, in essence
19:13 babilen aye
19:14 MTecknology I'm starting to feel like I'm the only person using a master hosted in aws. I know I'm not, but it feels like it.
19:15 jfindlay babilen: I've thought for a long time that it would be worth the investment if someone wrote a new data language for salt that, based on real experience with salt use cases, eliminates the need for the separated yaml/jinja/formula complex
19:15 babilen Like a domain specific language?
19:15 jfindlay indeed
19:15 jfindlay but it could be just yaml
19:15 * MTecknology throws a tuna can at jfindlay
19:15 babilen Hmm .. finally another chance to put my Clojure knowledge to good use ;)
19:16 MTecknology actually.. I've heard that argument before and I've seen some initial attempts at it
19:16 jfindlay we already have py, pydsl, etc. maybe I should just start using one of those
19:16 zer0def reading what babilen described somehow gives me flashbacks to object-oriented inheritance hell
19:17 MTecknology zer0def: I suspect that's exactly what you'd end up with
19:17 babilen I think the emerging pattern of defaults.yaml + osmap.yaml + {{ grains[????] }}.yaml + deep-merging of defaults with user provided pillar values is not a bad approach
19:18 babilen NOOOO, don't mute me! ;)
19:18 jfindlay babilen: something that handles 0.8 of common use cases with simple directives like `grains: *-web` but still allows you to program out custom features if you need them. Probably still you'd want to put anything official into a custom module
19:18 zer0def babilen: i've seen this pattern in Terraform, it was horrible to use :D
19:19 zer0def and even more horrid to implement
19:20 jfindlay but I'm no expert on such things so don't take too carefully anything I say :)
19:21 zer0def well, it all just ends up being a logical spiral, just depends whether said spiral has an upward or downward inclination ;)
19:22 MTecknology for fuck sake.. this is pissing me off. I can't find any reason at all for this crap to not work.
19:22 zer0def would you like a debugging duckie?
19:23 MTecknology I've been asking for a while now... yes, I would
19:23 babilen zer0def: I'm essentially just describing the pattern that is being used right now. I mean .. in a complex system you *have* to encode the differences somewhere, and I *much* rather have those all in one place, in easy to read YAML files than hardcoded in various SLS with if-else blocks
19:23 rollniak joined #salt
19:23 zer0def that's still that syndic problem?
19:23 MTecknology yup
19:23 babilen Okay .. back to work
19:24 zer0def MTecknology: have you restarted syndics *and* minions on syndic-masters? last time i've read you mention this, you managed to get syndic-master's key, but ended up unable to talk to it
19:25 zer0def s/syndics \*and\* minions/salt-syndic *and* salt-minion/
19:27 MTecknology After swapping the EIP, if I run salt-minion on the syndic, the minion acts exactly how I expect it to. If I try to run the syndic, I see a 3-way handshake, a packet, and a closed connection, followed by another packet.
19:27 MTecknology https://github.com/saltstack/salt/issues/47282
19:29 MTecknology This is the error that's popping up- https://github.com/saltstack/salt/blob/15895c4ff8d58bf7a86623cd99a57dabdd084bb3/salt/minion.py#L2942
19:31 zer0def a bit late for me to replicate, but i'll hold onto the issue, might find some time to replicate and poke at this tomorrow
19:31 zer0def (getting slightly late around these parts)
19:32 MTecknology Running salt-syndic -l trace didn't show me anything more exciting than exactly that error. Watching for events on the master showed nothing at all happening.
19:40 pbuell joined #salt
19:42 MTecknology DANGIT!!!
19:42 MTecknology │salt.exceptions.SaltClientError: Could not access                                      │
19:42 MTecknology Command line: [│/var/lib/salt/pki/minion. Please give salt read permissions.                           │
19:47 MTecknology It seems I've managed to find myself two bugs. One in salt and one in the packaging for ubuntu.
19:53 boopie joined #salt
19:54 stoogenmeyer joined #salt
19:55 stoogenmeyer Hi folks
19:55 stoogenmeyer I'm having some trouble doing something in Salt
19:55 stoogenmeyer {% set addrs = map(lambda s: s + ":4161", ["1", "2", "3"]) %}
19:56 stoogenmeyer This doesn't seem to be valid jinja and not sure how to achieve the same (I'd like to output a space delimited list of 1:4161 2:4161 3:4161
19:56 MTecknology that's python, not jinja
19:56 stoogenmeyer right, that's python syntax that I tried using... it's not valid in a set block?
19:58 MTecknology nope- what is it you're trying to accomplish?
19:59 jfindlay stoogenmeyer: you could use a list comprehension: `{%- addrs = [addr + ':4161' for addr in ['1', '2', '3']] %}`
19:59 stoogenmeyer I'm simplifying but I have a variable in pillar that looks like: ["1", "2", "3"] and I'd like to render in my jinja template `addrs: 1:4161 2:4161 3:4161`
20:00 jfindlay {%- set addrs = [addr + ':4161' for addr in ['1', '2', '3']] %}
20:00 onslack <tmacey> So, I saw the issue here <https://github.com/saltstack/salt/issues/46971> about the current issues with pip 10 but I wanted to check in whether there is a fix to be had other than pinning to pip 9 or manually patching salt?
20:00 whytewolf since when did list comprehension start working in jinja?
20:02 stoogenmeyer jfindlay: Hmm for some reason salt complains https://pastebin.com/GP6m84TP
20:02 stoogenmeyer I would think that would work
20:02 whytewolf it won't
20:02 stoogenmeyer Ah sad
20:02 whytewolf jinja2 does not support list comprehension
20:06 whytewolf it isn't pretty but this is kind of what i use in my ETCD config file https://gist.github.com/whytewolf/520cf5832babf29dac1cee900300037c
20:06 whytewolf alternative. use #py
20:06 whytewolf err #!py
20:06 wongster80 joined #salt
20:07 MTecknology wouldn't the join filter give stoogenmeyer what they need?
20:07 whytewolf it wouldn't put it on the last item
20:08 MTecknology "addrs: {{ lst|join(':4161') }}:4161"
20:08 MTecknology "addrs: {{ lst|join(':4161 ') }}:4161"  *
20:09 MTecknology well.. fellers.. I highly recommend avoiding 18.04 on syndic masters.
20:09 whytewolf that would work. but would get ugly with more complex examples
20:11 whytewolf really wish jinja supported list comprehension. a lot of things would be a lot prettier
20:11 jfindlay totally
20:11 stoogenmeyer Yeah that works for me. I'll let people vent in the PR review.
20:11 stoogenmeyer Thanks guys.
20:12 jfindlay stoogenmeyer: you could get around it by using a jinja for loop and appending the port where you use it
20:13 jfindlay {%- for addr in addrs %} {{ set full_addr = addr + ":4161" }} ... {% endfor %}
20:17 stoogenmeyer Is there a way to do that to just build a temporary list with ["1:4161", "2:4161"] and then just join on that list?
20:17 whytewolf stoogenmeyer: that is what the gist i posted earlyer does
20:18 gswallow joined #salt
20:18 whytewolf I just also show where i got my info from
20:20 briner joined #salt
20:20 stoogenmeyer Ah perfect, let me try that. It's longer but better because it's clearer what's going on.
20:21 ddg joined #salt
20:23 stoogenmeyer Hmm am I doing this wrong? https://pastebin.com/W9vZqvV1
20:23 stoogenmeyer Output is empty for some reason
20:23 stoogenmeyer oh god nvm
20:23 stoogenmeyer I am a doofus
20:24 stoogenmeyer That works great. Thank you very much.
20:24 whytewolf no problem
20:25 stoogenmeyer What's the difference between `~` and `+` btw?
20:25 MTecknology concatenate strings or python's +
20:26 stoogenmeyer Ah, `~` is jinja specific and `+` just fallsback to python?
20:27 MTecknology I don't know jinja internals well enough to know about +, but ya.. ~ is specific to jinja
20:30 whytewolf ~ = convert to string and concat while + will keep them the same format they are
20:30 whytewolf which can lead to problems when trying to concat and you end up adding
20:30 whytewolf http://jinja.pocoo.org/docs/2.10/templates/#other-operators
20:34 AngryJohnnie joined #salt
20:34 whytewolf Man, I feel so lazy today.
20:35 noraatepernos joined #salt
20:35 rivyn joined #salt
20:41 jfindlay at least you don't have dubious jinja advice :)
20:57 noraatepernos I can’t find an existing salt module for yarn though it is quickly surpassing npm.  Should I just execute the raw commands?
21:00 dendazen joined #salt
21:00 AngryJohnnie joined #salt
21:01 MTecknology better yet, abandon that garbage!
21:03 MTecknology or writ a yarn module..
21:03 zmalone joined #salt
21:14 sjorge joined #salt
21:24 noraatepernos Can two minions have the same id?
21:25 ddg joined #salt
21:26 noraatepernos I can’t seem to figure this out.  I have aws AMI that I would like to pre-bake.  Especially when it comes to aws ELB, I can’t be around to authorize minions on master because of the loadbalancer scale volatility.
21:28 dmcnabb joined #salt
21:28 jfindlay noraatepernos: minion ids should be unique
21:29 noraatepernos jfindlay: Is this a requirement? I must pre-bake AMIs.
21:30 noraatepernos https://docs.saltstack.com/en/latest/topics/tutorials/preseed_key.html preseeding in this way would be impossible as instances come and go.
21:31 noraatepernos I guess my question is more broad: Is Salt not the best solution when you have scale-out volatility?
21:31 tzero noraatepernos: you can use autosign + some cloud_init stuff to set the minion id
21:31 noraatepernos tzero: Ok let me google autosign I didn’t know that was part of salt
21:32 InsigkneeUh joined #salt
21:32 xet7 joined #salt
21:34 whytewolf noraatepernos: see https://github.com/JensRantil/saltstack-autoscaling
21:35 noraatepernos Okay — also am I on the right path here: https://docs.saltstack.com/en/latest/ref/configuration/master.html#autosign-file ?
21:35 noraatepernos “never autoaccept a minion which has not been started through autoscaling” <— smart.  whytewolf thanks!
21:38 inad922 joined #salt
21:49 noraatepernos https://docs.saltstack.com/en/latest/topics/tutorials/autoaccept_grains.html#tutorial-autoaccept-grains I think I can make something just as secure using this
21:49 noraatepernos I will now update everyone to 2018.3.0 and cross my fingers
21:51 xet7 joined #salt
21:52 wongster80 joined #salt
21:53 MTecknology noraatepernos: unless you really absolutely need that, I would avoid it. You shouldn't trust any grain that isn't 'id'
21:55 noraatepernos I guess the problem is that if a minion is compromised we’re fubar
21:56 MTecknology that's the problem with auto-accepting keys.. unless acceptance is controlled, it's possible for a minion to join and act like any other minion, depending on how you assign data.
21:56 noraatepernos But also, if a minion is compromised they’ll have all sorts of fun stuff…hmac keys to make signed requests, etc.
21:59 MTecknology oh, neat. That sqs_engine is neat.
22:00 MTecknology neat enough to make me say neat twice..
22:01 whytewolf neat
22:02 MTecknology heh.. I feel like using salt-cloud would simplify this thing a lot, since it handles keys.
22:04 sjorge joined #salt
22:04 whytewolf there used to be a reactor thing that would fire based off of autoscale. but i can't find it
22:05 jfindlay when I was at SaltStack, I used salt-cloud and salt-ssh to create a salt-native orchestration tool that bypassed preseeding and boostrapping
22:05 MTecknology yup- salt-cloud would cut out most of what that stuff does, and it wouldn't have to do any matching on grain id.
22:06 onslack_ joined #salt
22:06 jfindlay I used the cloud roster for salt-ssh
22:07 jfindlay which reminds me that there were some bugs in the cloud roster I need to send in a pr for
22:08 inetpro joined #salt
22:12 jeremati_ joined #salt
22:21 Edgan jfindlay: Have you see the ansible roster ec2.py is supported in salt-ssh?
22:26 jfindlay Edgan: cool
22:58 jhauser joined #salt
23:00 noraatepernos Why does apt-get sometimes think 2017.7.5 is the latest and other times 2018.3.0?
23:02 noraatepernos I followed this: https://repo.saltstack.com/#ubuntu
23:02 noraatepernos Pin to major version.
23:04 noraatepernos nm missing some apt-get updates
23:07 justanotheruser joined #salt
23:14 noraatepernos Well, I have salt-master and salt-minion upgraded on all instances but my minions are not reaching my master.  minion logs don’t show anything 2018.3.0
23:15 MTecknology noraatepernos: "apt-cache policy $pkg"
23:15 noraatepernos MTecknology: I got that part worked out finally, thanks.
23:16 noraatepernos My test.ping is unresponsive on all minions.
23:16 MTecknology doesn't mean that's not good for future troubleshooting..
23:16 noraatepernos rebooted, restarted, etc…I feel like I missed something.  Do I need to rewrite the minion config files?
23:16 noraatepernos I opted to keep the old configs.
23:17 noraatepernos perhaps this was a huge mistake
23:17 noraatepernos I expected to see something in the minion logs like “can’t reach host at…” etc…but nothing.
23:17 noraatepernos MTecknology: For sure.  I save it, thanks.
23:18 noraatepernos salt-key -l all shows all my unresponsive minions, stilll.
23:19 noraatepernos Ahh, service salt-minion status…error processing configuration file
23:22 noraatepernos joined #salt
23:23 noraatepernos This is so weird.  My minion configs have duplicate entries for “master:”
23:24 Kelsar joined #salt
23:26 MTecknology how are you managing the config?
23:26 noraatepernos Something in the apt-get update overwrote master: in my config files or something.  It printed it everywhere.
23:26 armin_ joined #salt
23:27 exarkun joined #salt
23:27 noraatepernos I don’t.  I haven’t touched it in a year.
23:27 noraatepernos For instance, after the line “# Use if master_type is set to failover.” is master: the.id…
23:27 noraatepernos domain, I mean.
23:27 MTecknology you don't manage it, at all, yet there's configuration in what is normally only comments that is duplicated?
23:28 noraatepernos Today I used apt-get to update salt-minion.
23:28 noraatepernos There was a prompt to keep the salt-minion config…the original.  I chose that option.
23:29 mrueg joined #salt
23:29 noraatepernos Does anyone know where I can get a copy of the default apt-get salt-minion conf?
23:29 noraatepernos I have a feeling this file is corrupted beyond repair.
23:29 * MTecknology sighs and wanders off
23:30 noraatepernos Um, also, side note: there’s a process that can manage minion conf?
23:30 noraatepernos I wish I could walk off but all my minions are offline heh
23:31 noraatepernos https://github.com/saltstack/salt/blob/develop/conf/minion yeah something was doing replacements in my file.  Aye.
23:49 dendazen joined #salt
23:58 zulutango joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary