Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2017-01-25

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:05 xet7 joined #salt
00:07 madboxs joined #salt
00:20 DEger joined #salt
00:31 prg3 joined #salt
00:37 mikecmpbll joined #salt
00:39 Tanta joined #salt
00:43 alvinstarr1 joined #salt
00:50 Deliant joined #salt
00:53 XenophonF jgarr: there's a bunch of files in /var/cache/salt that you need to move, too
00:56 XenophonF maybe
00:56 XenophonF hm, i guess not
00:57 XenophonF did you copy $PREFIX/etc/salt/pki from the old master to the new?
00:57 jgarr XenophonF: oh, I forgot that's under $PREFIX Thanks I think that server has it moved
01:03 secrgb joined #salt
01:08 madboxs joined #salt
01:08 madboxs_ joined #salt
01:14 weylin joined #salt
01:27 debian112 joined #salt
01:38 keldwud joined #salt
01:40 cswang joined #salt
01:41 MeltedLux joined #salt
01:42 jas02 joined #salt
01:54 druonysus_ joined #salt
02:07 netcho joined #salt
02:08 Nahual joined #salt
02:17 edrocks joined #salt
02:24 catpigger joined #salt
02:30 Hipikat joined #salt
02:31 swills joined #salt
02:31 druonysus_ joined #salt
02:35 raspado joined #salt
02:46 jas02 joined #salt
02:48 ilbot3 joined #salt
02:48 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2016.3.4, 2016.11.1 <+> Support: https://www.saltstack.com/support/ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ (please don't multiline paste into channel) <+> See also: #salt-devel, #salt-offtopic <+> Ask with patience as we are volunteers and may not have immediate answers
03:06 mpanetta joined #salt
03:07 mpanetta joined #salt
03:14 djgerm1 i am getting the following error today with some new instances in ec2 and I don't understand why?  * ERROR: Failed to run install_ubuntu_stable_deps()!!!
03:14 djgerm1 I can run the script after the fail at it works fine… any clue?
03:19 evle joined #salt
03:26 John_Kang joined #salt
03:26 John_Kang hi there
03:27 John_Kang could someone let me know where i can find any performance tunning guide ?
03:29 John_Kang it's extremly slow when apply state :(
03:30 yuming joined #salt
03:30 yuming left #salt
03:31 djgerm1 there's no hard and fast requirements I don't think. salt is mostly memory and CPU bound
03:33 djgerm1 https://github.com/saltstack/salt/issues/19926
03:33 saltstackbot [#19926][MERGED] please add some info on system hardware requirements (salt at scale) | I understand it's tricky to put a number on on this, but I need to have some numbers on what hardware should be used to manage ~1k minions....
03:39 John_Kang djgerm1: because it takes too long time to run a state, approximately 3min
03:39 John_Kang and i tried to reduce the keysize to 1024, it didn't help
03:43 Tanta "a state"
03:43 Tanta oh ok we can debug that
03:43 pipps joined #salt
03:44 jas02 joined #salt
03:52 onlyanegg joined #salt
04:12 * MTecknology sad..
04:12 MTecknology http://dpaste.com/23H44TX
04:13 jagguli how do I schedule a runner ? I have a schedule pillar in my master config and my runner module in _runners but my master logs contains an exception saying that its not availabel
04:19 whytewolf jagguli: do you have a minion on the master?
04:20 edrocks joined #salt
04:20 onlyanegg joined #salt
04:21 whytewolf if not that is okay. [unless you have an older version]
04:22 jagguli yes
04:22 jagguli im on carbon
04:22 jagguli maybe i need to reload modules on the minion
04:22 whytewolf oh with a minion doesn't matter the version. just run salt-call saltutil.sync_runners
04:23 whytewolf but since you are on carbon you can call salt-run saltutil.sync_runners
04:23 jagguli ok
04:23 whytewolf https://docs.saltstack.com/en/latest/ref/runners/all/salt.runners.saltutil.html#salt.runners.saltutil.sync_runners
04:24 jagguli yea that seems to have loaded the new runners modules
04:24 jagguli greate
04:25 jagguli thanks
04:25 whytewolf no problem
04:27 MTecknology whytewolf: don't suppose you have any guesses why I'm broken, do ya?
04:29 whytewolf MTecknology: virtual_memory missing from psutil? swap space missing?
04:29 * whytewolf shrugs
04:29 jagguli MTecknology: try http://stackoverflow.com/questions/20027440/psutil-module-not-fully-working-on-debian-7
04:29 jagguli u on debian7?
04:31 MTecknology ubuntu 12.04, I think, but that might already have given me my answer
04:31 hrumph hi
04:32 hrumph cp.cache_dir seems to only work with salt:// urls
04:32 hrumph is that intentional or is that a bug?
04:33 hrumph it looks like cache_file will take other kinds of url's
04:34 MTecknology jagguli: well, maybe not.. I can't actually modify the version of this package that's installed because it's tightly integrated with the other software that runs on here. There's a strong chance it'll wreck something on this delicate flower.
04:34 jagguli lol i feel for ya mate :D
04:35 jagguli you could 'patch' the module
04:35 MTecknology that sounds exceptionally painful
04:35 jagguli na seems like a one liner
04:36 MTecknology I've been fighting a couple fires and haven't looked just yet
04:36 ruxu joined #salt
04:37 jagguli getattr(psutil.virtual_memory(), 'total', None)
04:37 jagguli oh not that sry
04:38 jagguli if hasattr(psutil, 'virtual_memory'):
04:38 aw110f joined #salt
04:38 whytewolf hrumph: the cp module is meant for working with the salt file system. so i don't think it takes other urls
04:38 hrumph cache_dir worked with file:/// (or it appeared to)
04:38 hrumph cache_file i mean
04:39 hrumph cache_dir gave an error
04:40 MTecknology jagguli: heheh... check out the source... one instance handles that bug
04:41 jagguli ah a chance for you to contribute ;D
04:42 MTecknology indeed, this will be a simple PR
04:42 MTecknology I should probably get it pushed out tonight
04:43 hrumph whytewolf, i've almost finished a module taht installs printers (and their drivers) on windows
04:43 whytewolf hrumph: cache_file is setup to work with remote files. cache_dir is not.
04:44 hrumph whytewolf, you can install ipp printers with url ports (like https://myprintserver/myprinter) and specificy a driver name and an inf file
04:45 whytewolf and looking at it doesn't look like cache_dir is meant to touch files not in salt://
04:45 hrumph it caches the directory of the inffile if supplied
04:45 hrumph would be preferable if it was more relaxed in what it would take but it doesn't matter. personally i only need salt://
04:45 * whytewolf shrugs. I'm like a bad maid. I don't do windows
04:45 MTecknology dangit... obviously it couldn't be that bloody simple
04:46 hrumph whytewolf, i'm basically using salt to bridge windows and linux in a way
04:46 hrumph with these new printer modules you can specify the driver with the printer, sort like what you can already do with an smb printer
04:46 jagguli MTecknology: why ?
04:47 hrumph I'll pr my new modules when i think they're good enough
04:47 jagguli MTecknology: are you on Carbon ?
04:48 MTecknology jagguli: 2016.11.1; it's not a simple oopsie
04:49 jagguli hmm ..
04:50 jagguli lol  # Oh good, we have psutil. This will be quick.
04:50 MTecknology GAH!!!!
04:50 MTecknology jagguli: yup... there's one place in the module missing the check, that place is as well, and now....
04:51 MTecknology http://dpaste.com/0HNRPM1
04:53 MTecknology all sorts of little regressions apparently
05:08 MTecknology jagguli: I need to stop talking on IRC when I start diving into that stuff. My emotions are just constantly up/down.
05:08 aw110f_ joined #salt
05:12 madboxs joined #salt
05:12 MTecknology jagguli: https://github.com/saltstack/salt/pull/38929
05:12 saltstackbot [#38929][OPEN] Fix psutil regressions in 2016.11 | What does this PR do?...
05:19 madboxs joined #salt
05:32 stewgoin joined #salt
05:33 jholtom joined #salt
05:37 madboxs_ joined #salt
05:39 madboxs joined #salt
05:46 jas02 joined #salt
05:46 impi joined #salt
05:53 sh123124213 joined #salt
06:03 gnomethrower joined #salt
06:03 gnomethrower Hey guys
06:03 gnomethrower I'm trying to follow along with https://docs.saltstack.com/en/latest/topics/tutorials/pillar.html#pillar-walk-through - and have completed it
06:04 gnomethrower but I want to take the user creation stuff from "More Complex Data" and extend it with things like homedir, ssh keys etc
06:04 gnomethrower and not really sure how to start!
06:04 sh123124213 joined #salt
06:08 MTecknology gnomethrower: https://docs.saltstack.com/en/latest/ref/states/all/salt.states.user.html  <-- figure out how the state works, then... move that data to pillar and make a template from the states.
06:09 gnomethrower MTecknology: perfect... will give that a shot
06:09 gnomethrower i'm still learning so this is all still a little tough :)
06:11 netcho joined #salt
06:11 MTecknology salt's learning curve is not shallow, but still better than most, and very much worth the reward
06:11 gnomethrower I'm used to Ansible
06:11 gnomethrower but so far Salt seems way more powerful and closer to how I think
06:11 gnomethrower learning curve is awful though, imo
06:12 gnomethrower then again, it's easy to learn how to do Ansible the "wrong" way
06:12 gnomethrower and hard to unlearn that later
06:12 MTecknology salt is incredibly flexible so it's best to pick a style and keep with it
06:13 MTecknology meh.. any non-opinionated thing I can say is gonna be obvious
06:15 MTecknology Just today, I re-deployed my backup server from scratch using salt because I broke it and wasn't sure in what way I broke it.
06:16 MTecknology grabbed a copy of the host ssh keys first, though
06:17 netcho moin all
06:18 * MTecknology looks at the clock
06:18 * MTecknology grumbles
06:19 bwellsnc joined #salt
06:21 bbradley joined #salt
06:22 DanyC joined #salt
06:25 MTecknology gnomethrower: this is a pillar file I /just/ wrote.. https://gist.github.com/MTecknology/ecdafed00d7f6e70f46603507a127438
06:25 MTecknology copy/pasted/modified from working on cleaning up my backup states today to get less "everythign" and more only what I need
06:26 gnomethrower MTecknology: interesting, thanks'
06:27 gnomethrower you misspelled pillar as pilllar though :P
06:28 MTecknology oops
06:28 MTecknology I don't plan to keep it there anyway, just sharing an example
06:31 amy_ joined #salt
06:33 gnomethrower :)
06:33 ivanjaros joined #salt
06:35 DanyC joined #salt
06:36 mk-fg joined #salt
06:38 cyborg-one joined #salt
06:40 gnomethrower MTecknology: this is what i'm working with atm
06:40 gnomethrower https://gist.github.com/Zorlin/a7dcd8ab6bb19514db5847acf6180841
06:40 gnomethrower I'm 99% sure I'm not on the right track
06:44 MTecknology gnomethrower: want a hint?
06:44 gnomethrower MTecknology: Definitely :)
06:44 MTecknology bookmark this - http://yaml-online-parser.appspot.com/
06:45 MTecknology ... and then use it
06:46 netcho or sintastyc yaml checker :)
06:46 gnomethrower I'm not sure i follow
06:46 gnomethrower sorry, haven't had much coffee today :P
06:46 MTecknology yaml is (just) a(nother) markup language. It doesn't do magic, only jinja does
06:46 netcho gmoro: ur yaml syntax is wrong
06:47 jas02 joined #salt
06:47 netcho gnomethrower:
06:47 MTecknology err.. just (yet)... was supposed to be funny
06:47 MTecknology his yaml /syntax/ is fine, but the structure is not
06:47 gnomethrower Oh, hang on
06:47 netcho that's what i meant :)
06:48 * MTecknology doesn't wanna confuse gnomethrower
06:48 netcho indentation ... sorry gnomethrower
06:48 gnomethrower https://gist.github.com/Zorlin/a7dcd8ab6bb19514db5847acf6180841
06:48 gnomethrower is the comment any better?
06:48 MTecknology gnomethrower: put your yaml into that site, then take the python equiv and drop it into a python interpreter, then you can futz around with the yaml data in a shell
06:49 MTecknology gnomethrower: there's an obvious flaw in your yaml structure (fwiw)
06:49 MTecknology structure is the word netcho was going for :P
06:49 gnomethrower when you say my yaml
06:49 gnomethrower you mean ==== /srv/pillar/users/init.sls ==== ?
06:50 MTecknology what is yaml?
06:50 gnomethrower it's a markup language?
06:50 MTecknology look up what yaml is... you've been writing it
06:50 gnomethrower and SLS files are written in YAML..
06:51 MTecknology sls = SaLt State
06:51 MTecknology sls is jinja templated yaml ... by default
06:51 DEger joined #salt
06:52 ruxu joined #salt
06:52 gnomethrower right
06:52 MTecknology at some point, you'll do jinja + yaml + gpg encrypted blobs
06:52 MTecknology you'll understand the distinction when you get there
06:53 netcho 3 steps to glory :D
06:53 gnomethrower oaky
06:53 gnomethrower okay*
06:53 gnomethrower for now - i'm having trouble finding the obvious flaw, even with the parser :(
06:54 MTecknology gnomethrower: think about what structure your data is in, then think about looping through each iteration, what does {{user}} equal?
06:54 MTecknology that's why I said take the http://yaml-online-parser.appspot.com/ python output, feed it into ipython, and write quick little loops to try out your logic
06:54 sh123124213 joined #salt
06:55 gnomethrower I don't know python yet
06:55 gnomethrower but i'll try that i guess
06:55 netcho don't scare him with python yet :)
06:56 netcho plenty of stuff can be done out of the box with salt, witouth using custom things with python
06:56 MTecknology lol...
06:56 netcho give him a week or two, he will write his own engines :)
06:56 MTecknology unless you pick up salt at 0.15
06:57 MTecknology I put that version down, thanked this channel for their time, and walked away.
06:57 MTecknology came back a year later to see 0.17 and started getting excited about it, but I had to do a *LOT* of python debugging up front.
06:57 netcho yeas ago :)
06:57 MTecknology FRICK! I *LOVE* the progress salt has made!!!
06:58 netcho couldn/t agree more
06:58 netcho damn keyboard
06:58 MTecknology regressions aren't really slowing down, but that's because most people don't write test cases for their patches
06:59 MTecknology It would be cool if every patch accepted mandated that... I'd know how to do it by now! but, that would deter most of the contributors
06:59 netcho i'd rather write tests than rewriting weeks of code
06:59 netcho but thats me
07:00 netcho MTecknology: i have a question... asked it yesterday but did not get response
07:00 netcho what's the best way of propagating a file among minions beside having it on master and applyng around to minions
07:01 MTecknology what do you mean?
07:01 MTecknology salt-cp?
07:01 netcho for example i would like to change file directly on a minion and would like that change to be seen on all other minions
07:02 netcho salt-cp wpuld chane it on a master and ther run state right?
07:02 netcho s/chane/change
07:02 doubletwist joined #salt
07:02 MTecknology I don't know what your setup looks like so I have no clue
07:02 netcho regarding setup
07:04 netcho i have file.txt on 10 minions. file.txt contains XXXX inside. I ssh to one of the minions and change XXXX to ZZZZ and save it. All other minions should have that change
07:04 MTecknology that's a broken workflow
07:04 MTecknology don't change it on a minion
07:04 netcho but is it possible?
07:05 netcho i know salt is made to prevent stuff like this
07:05 netcho but just wodering
07:05 netcho wondering
07:06 MTecknology it can be done, yes
07:07 netcho also i am considering multiple top.sls files, any cons of doing that?
07:08 MTecknology lots, yes
07:08 MTecknology what is it you're actually trying to achieve? Why such lunacy?
07:09 netcho i have pretty tricky infra to setup. lot's of environemnts, and different application with diferent technologies
07:09 MTecknology what makes it tricky?
07:09 netcho bunch of jinja conditions :D
07:10 lord2y joined #salt
07:10 MTecknology then you're probably abusing jinja a bit much
07:10 netcho or maybe i just cannot figure out how to do stuff the right way :)
07:11 MTecknology pillar drives states
07:11 MTecknology don't put logic into states that should be in pillar
07:13 netcho i have around 40 apps, 4 envs per app + per-developer dev environment. 6 different techologies. java,php,node,rails,c++ etc
07:13 MTecknology still sounds pretty simple...
07:14 netcho to someone who is using salt since 0.17
07:14 netcho :D
07:14 MTecknology I /could/ ...maybe see two masters on that
07:15 netcho but u r right, i need to cosider using more logic in pillars, now i am mostly concentrated on states
07:17 MTecknology break apart your setup logically, then figure out the most terse way to describe it via tree form
07:17 netcho actully i am tryig to setup old chef workflow in salt which is wrong :)
07:18 MTecknology NO!
07:18 MTecknology NO! NO! NO!
07:19 Miouge joined #salt
07:21 MTecknology you can do absolutely whatever you want in your pillar structure, so break it apart by that tree logic, and then organize your pillar data with that structure in mind.
07:21 MTecknology Then, as your vision changes, so can your data
07:21 edrocks joined #salt
07:23 MTecknology I'm actually planning on trying to come up with a salt talk to go over some of these snafu's that beginners make that creates an unmanageable system like $client has
07:23 sh123124213 joined #salt
07:23 Trauma joined #salt
07:24 MTecknology OMG!!! I JUST THOUGHT OF THE PERFECT TITLE!!!!!!
07:25 MTecknology netcho: you use a lot of recipes?
07:25 netcho yes
07:25 MTecknology don't
07:25 netcho u mean chef recipes? :)
07:25 MTecknology salt
07:26 netcho not sure what salt recipes are.. formulas maybe?
07:26 MTecknology ya, those
07:26 netcho naah
07:26 netcho i find those too comp[licated
07:27 netcho and most of the are overkill
07:27 MTecknology I've looked at them to understand things, but they force you into one design and I find that design very non-scalable
07:28 netcho i used couple of those but removed like 90% of stuff from there
07:28 gnomethrower MTecknology: Thanks for all the help
07:28 netcho in th end i wrote my own
07:28 gnomethrower My coworker ended up helping me out and solved the issue
07:28 gnomethrower i'm doing a brief git of it now
07:29 netcho gnomethrower: i noticed u tried to use uid for usernames in that pillar file :)
07:29 MTecknology you*
07:30 gnomethrower netcho: he pointed that out too ;)
07:30 MTecknology this isn't the network for T9 typing
07:30 netcho sorry :)
07:30 jas02 joined #salt
07:31 MTecknology wow... I really must be high
07:31 gnomethrower MTecknology, netcho: https://gist.github.com/Zorlin/010c5f242f9c7966445a8adc2e55137e
07:32 MTecknology I made a nice salad and salmon fillet, poured a beer, and I'm almost done eating with beer no the more empty
07:32 MTecknology and it's 01:30
07:33 MTecknology gnomethrower: much better
07:35 MTecknology I have some stuff shared that might be interesting - https://gist.github.com/MTecknology
07:35 gnomethrower MTecknology: awesome, will take a look. Thank you!
07:35 sh123124213 joined #salt
07:38 MTecknology netcho: also, do *NOT* arrange your states per your pillar structure. They are completely different. The states should represent the structure of your deployment.
07:39 MTecknology https://gist.github.com/MTecknology/17b020c2ff572ab2be8d535cbddb2f7e
07:39 rideh joined #salt
07:41 teclator joined #salt
07:41 Inveracity joined #salt
07:42 sh123124213 joined #salt
07:43 MTecknology heh...
07:44 MTecknology I don't want to log into a vm host to resize one of the disks so my solution seems to be do it with salt. :S
07:45 Yee joined #salt
07:45 Yee Hi All
07:45 netcho MTecknology: here is my nginx state
07:45 netcho https://hastebin.com/alazozeyax.js
07:46 Yee goo time to ask question?
07:46 MTecknology when is a goo time for any question?
07:47 netcho since i have like 40 apps i need 40 nginx configs
07:48 MTecknology ya.. that was a fun activity. The reason I didn't wanna log into my vm host is because I have a bastion host on every vlan that you have to go through to connect to anything, and I was already connected to the one that lets me into my salt master.
07:48 netcho on machine create i set custom grains per each app or if itsh shared one i set multiple values
07:48 joshin joined #salt
07:48 MTecknology hastebin doesn't work for me
07:48 netcho how come
07:48 MTecknology javacrap
07:48 MTecknology javascrap*
07:49 MTecknology use gist man!
07:49 MTecknology all the cool kids use gist
07:49 MTecknology or dpaste
07:49 MTecknology (.com)
07:50 Yee Data failed to compile:
07:50 Yee Rendering SLS '5.1_1-8:system' failed: Jinja variable 'dict object' has no attribute 'tra'
07:50 netcho MTecknology: i am not a cool kid :D https://paste.debian.net/910457/
07:50 Yee i get this error when i run state.highstate
07:50 MTecknology Yee: ^ see the pastebin discussion..
07:50 MTecknology lol... paste.debian.net asks me for SSO login
07:51 netcho whooooot?
07:51 * MTecknology is a debian dev
07:51 Miouge joined #salt
07:51 MTecknology netcho: very first jinja {% if 'production' in grains['id'] %}  <-- this is bad
07:52 netcho ok, why?
07:53 Yee MTecknology: asking to copy the whole error in to the pastebin?
07:53 netcho i have a nomenclature like this: {{ cloud_resource }}-{{ environment }}-{{ service }}
07:53 MTecknology netcho: this should be a pillar thing and you don't check if prod, you check if system should have certs and, within pillar, you say what the certs are, not if certs exist, then your state checks to see if certs are defined and then uses the value of certs to do things
07:54 aw110f joined #salt
07:54 MTecknology Yee: and ya know... the actual data you're working with, rather than making us guess
07:54 MTecknology Yee: the error is self-explanatory, the dictionary you accessed did not have the attribute tra...
07:55 MTecknology and it tells you where it came from...
07:55 Yee ok not sure where should define the attribute then
07:56 netcho MTecknology: i don't quite understad... it checks if minion is a prod machine, if it is copy cert, otherwise skip it
07:57 netcho but yeah i might set that in pillar
07:59 joshin joined #salt
07:59 MTecknology Yee: you tried accessing an attribute that is NOT defined
07:59 MTecknology netcho: ya.. that logic doesn't scale
07:59 scristian joined #salt
08:00 MTecknology it's why different environments and applications scares you
08:00 netcho yes :)
08:00 Yee MTecknology:it should be defined in pillar?
08:00 Yee http://pastebin.com/Qe8PFqeB
08:00 MTecknology Yee: the dictionary you tried to access should have that tra attribute defined
08:00 Yee here is the full message
08:00 aw110f_ joined #salt
08:00 * MTecknology sighs
08:01 netcho actually dynamic nginx configs are smth that i find most important to do right, everything else should be easy
08:01 MTecknology Yee: READ the message...
08:01 MTecknology netcho: it's all the exact same logic... no need to change it all around for different apps
08:01 Yee what do you mean dictionary?
08:01 MTecknology a python object
08:03 * netcho is thinking...
08:03 MTecknology Yee: from that error, which you've now provided... "{% set hostName = pillar['tra']['my_self']['name'] %}    <======================"  <-- this tells you what line broke rendering.
08:03 MTecknology Yee: do your minion have a 'tra' pillar attribute?
08:04 Yee in my case master and minion are running in same server
08:04 MTecknology Yee: congrats .. doesn't even kinda matter
08:04 madboxs joined #salt
08:05 Yee its a two nodes (servers) like a multi-master setup
08:05 MTecknology I have my salt master configure itself via states pulled from git
08:06 CEH joined #salt
08:08 MTecknology netcho: I /might/ be willing to share my nginx states if you can impress me enough.
08:08 MTecknology I wanna know what ideas you come up with first, though
08:08 netcho i am listening... :)
08:08 netcho ok
08:08 netcho fair enough :)
08:08 MTecknology partly, because I know they're not perfect and I need to know you can take what I give you and not blindly copy it
08:09 netcho haha u can be sure i won't :)
08:09 netcho how i thougt it might work (forget about the cert part)
08:10 MTecknology u?
08:10 MTecknology ^ not a good start
08:10 netcho zou
08:10 netcho you :D
08:10 joshin joined #salt
08:11 dariusjs joined #salt
08:11 MTecknology I think it's time for a nice glass o' tequila, a nice puff, a couple bits o' long-aged parmesan, and then nap-nap
08:12 MTecknology netcho: do you have /any/ pillar data?
08:12 netcho i am having an appname.conf.jinja template for each app. as you can see in my state i have a custom grain called appname which is set on instance creation
08:13 Straphka joined #salt
08:13 netcho no pillar for nginx so far
08:13 MTecknology I don't like the .jinja extension, but that's a personal preference
08:13 MTecknology you can't write nginx states without pillar data
08:13 MTecknology unpossible
08:14 netcho if i hardcore all the values i can haha
08:14 MTecknology that's *exactly* the problem, though
08:15 netcho yeah so that is why i am looking for a better solution. so far i have 2/40 apps and aleardy having a problem with my setup
08:15 MTecknology your states shouldn't have to change if you create a new set of web servers, only the pillar data (and templated files) changes
08:15 netcho i am aware of that
08:15 MTecknology so, let's get your setup to pillar, just a basic bit
08:16 MTecknology You told me you configure custom grains on hosts when you deploy them?
08:16 netcho is it worth to try creating pillar for each app?
08:16 Yee yes it has
08:16 netcho correct
08:16 MTecknology why?
08:16 MTecknology why is that the best way you have to detect what a node's role is?
08:16 Yee https://github.com/saltstack/salt/issues/16656
08:16 saltstackbot [#16656][MERGED] failed: Jinja variable 'dict object' has no attribute | After a `apt-get upgrade` I'm having problems with pillar variables and I can't see any error on my side....
08:17 Yee the issue is related to this?
08:17 netcho i spin them up with salt-cloud and i have template for each app/env
08:17 Yee my salt version is salt 2015.5.10 (Lithium)
08:17 MTecknology Yee: look up salt docs for pillar data
08:17 MTecknology netcho: you shouldn't need different templates (usually)
08:18 MTecknology you /should/ be able to have a base template and just deploy from there, let salt do the customizations
08:18 netcho sorry not templates, profiles
08:18 MTecknology same story
08:19 MTecknology In most environments, things are keyed off of the hostname
08:19 netcho true but my qa-staging-prod machines are wuite diffrenet in mem, cpu, volume size
08:20 MTecknology so?
08:20 MTecknology that's what grains are fore
08:20 netcho different subnets, vpc
08:20 netcho yes and i set them in profiles
08:20 netcho cloud profiles
08:20 * MTecknology grumbles
08:20 netcho we are not on a same page :)
08:20 netcho i think
08:22 mikecmpbll joined #salt
08:22 o1e9 joined #salt
08:22 netcho what i think i could easily pun in pillar, regarding nginx configs, is server_name, http_port for upstream, and ssl stuff
08:22 netcho s/pun/put
08:23 AndreasLutro templated nginx/apache virtualhost configuration is a pain
08:23 mikecmpbll joined #salt
08:24 bluenemo joined #salt
08:24 joshin joined #salt
08:27 madboxs joined #salt
08:28 netcho saltstack is salt open right?
08:28 MTecknology netcho: seems like a sensible statement
08:28 MTecknology AndreasLutro: I disagree
08:29 ronnix joined #salt
08:33 JohnnyRun joined #salt
08:35 MTecknology netcho: here.. https://gist.github.com/MTecknology/a7138375b14ea9c9561eee659114c00b
08:37 rpb joined #salt
08:37 joshin joined #salt
08:38 netcho MTecknology: here is my jinja temp for one of the apps https://paste.debian.net/910461/
08:38 netcho thanks for share, let me take a look
08:43 toanju joined #salt
08:44 jeddi joined #salt
08:46 impi joined #salt
08:47 MTecknology netcho: *THAT* is being used for managing web apps running on lots of servers in multiple environment, with multiple top level domains, etc. all hosted from the same master
08:47 MTecknology running in my basement
08:48 JohnnyRun joined #salt
08:51 colttt joined #salt
08:54 bbradley joined #salt
08:56 DanyC joined #salt
08:56 Rumbles joined #salt
08:57 armin joined #salt
08:58 jhauser joined #salt
08:58 dariusjs joined #salt
09:01 joshin joined #salt
09:02 alxf joined #salt
09:06 cyborg-one joined #salt
09:07 netcho MTecknology: i can see what it does, correct me if i am wrong but it copies over every site.conf on every machine?
09:08 MTecknology no
09:08 MTecknology Why would it copy all of them?
09:09 MTecknology I don't want all websites running on all boxes
09:09 netcho correct
09:09 MTecknology they grab what they need
09:09 netcho i can't see your pillar so i can only guess :P
09:10 MTecknology no need to guess... you can see the structure of the pillar data, how you contruct it is up to the design of your environment
09:11 netcho so pillar per app?
09:12 MTecknology irunno... does that work best for you/
09:12 MTecknology ?*
09:12 netcho i really need to dive into pillars, because obviously i am missing something :) nvm :)
09:12 MTecknology I shared a sample of how pillar data works in *MY* environment earlier
09:13 joshin joined #salt
09:13 MTecknology remember... yaml stuff gets merged
09:14 netcho currently i store only tokens, packet versions, sha, passwords and stuff like t hat
09:15 netcho merged?
09:15 xmj left #salt
09:18 muxdaemon joined #salt
09:18 sfxandy joined #salt
09:19 muxdaemon 'ning. I'm trying to get salt-ssh to run locally with out root access. I've set root_dir to a path my user owns but salt-ssh is still trying to write to /var/log/salt/ssh. How can I fix this so it writes under root_dir?
09:21 amy_ joined #salt
09:23 joshin joined #salt
09:23 bbradley joined #salt
09:24 MTecknology netcho: I imagine you can take what I've given you and get a lot of research done through it
09:24 MTecknology dang... 03:25 already
09:25 DanyC joined #salt
09:25 netcho yes, thank you
09:26 netcho now go to bed
09:27 MTecknology my laptop is about to crash anyway...
09:28 MTecknology battery hit 0%.. adding charge now kills it
09:28 MTecknology new laptop shows up tomorrow
09:28 MTecknology g'night
09:42 Rumbles joined #salt
09:44 catpig joined #salt
09:45 [CEH] joined #salt
09:49 s_kunk joined #salt
09:54 joshin joined #salt
09:56 amcorreia joined #salt
09:57 jhauser joined #salt
10:02 stooj joined #salt
10:03 Straphka joined #salt
10:08 Trauma joined #salt
10:15 amy_ joined #salt
10:18 N-Mi_ joined #salt
10:23 jhauser joined #salt
10:26 stooj joined #salt
10:30 antpa joined #salt
10:39 stooj joined #salt
10:40 ravi____ joined #salt
10:42 netcho joined #salt
10:42 saltnewb joined #salt
10:45 aidin joined #salt
10:48 antpa Is there a GUI for Saltstack Community Edition?
10:51 muxdaemon not officially antpa
10:51 muxdaemon antpa there is https://github.com/Lothiraldan/saltpad
10:52 stooj joined #salt
10:53 antpa Thanks,  I played with Saltpad a year ago but found it a little buggy so I dropped it after upgrading my Master.
10:54 coredumb Hey folks is there a way to blacklist env per nodegroup/group/minion ?
10:55 muxdaemon can pillar data contain calls to other data in the pillar? That is, can I write {{ pillar.get['some data'] }} with in the pillar? I tried it and came up blank but that could well be my code.
10:59 Firewalll joined #salt
11:01 muxdaemon the reason I want to do this is to set defaults for a salt formula I'm using... maybe play jinja2 vars are what I need?
11:05 AndreasLutro muxdaemon: no, you can indeed import jinja variables from other pillar SLSes though
11:06 cuxtud joined #salt
11:12 muxdaemon ah, ok, so import the files as one might with a map.jinja ?
11:13 AndreasLutro yep
11:15 muxdaemon great, thanks!
11:24 Antiarc joined #salt
11:40 Tanta joined #salt
12:12 lempa joined #salt
12:17 bookwar joined #salt
12:25 ugolino joined #salt
12:28 kettlewell joined #salt
12:31 bookwar hi, all. If i am using py renderer, does it matter what is there outside of run() function? Can I add a simple wrapper to make this file into a runable python script for testing purposes, smth like https://paste.fedoraproject.org/536178/ ?
12:32 justanotheruser joined #salt
12:35 swills joined #salt
12:45 toastedpenguin joined #salt
12:45 jhauser joined #salt
12:47 jhauser_ joined #salt
12:51 Guest58339 joined #salt
13:03 spirit__ joined #salt
13:05 sysadmin75 joined #salt
13:06 krymzon joined #salt
13:07 joshin joined #salt
13:07 joshin joined #salt
13:13 jhauser joined #salt
13:20 numkem joined #salt
13:21 mritchie joined #salt
13:27 smkelly joined #salt
13:29 babilen If anybody who is on the formulas team and also uses the mysql formula would take a look at https://github.com/saltstack-formulas/mysql-formula/pull/160 I'd be quite grateful
13:29 saltstackbot [#160][OPEN] Hardcode 'mysql-server' as debconf base path for root password | This is needed for correctly setting the root password when installing mariadb and addresses #43. ...
13:35 aidin joined #salt
13:45 ssplatt joined #salt
13:47 Sammichmaker joined #salt
13:57 \ask joined #salt
14:13 brokensyntax joined #salt
14:15 mritchie joined #salt
14:15 LostSoul joined #salt
14:18 evle1 joined #salt
14:21 rpb joined #salt
14:22 guerby joined #salt
14:22 guerby joined #salt
14:28 Tanta joined #salt
14:37 mritchie joined #salt
14:40 brousch__ joined #salt
14:42 bowhunter joined #salt
14:43 Brew joined #salt
14:49 tiwula joined #salt
14:50 Sam____ joined #salt
14:52 jhauser_ joined #salt
14:56 PatrolDoom joined #salt
15:01 nickabbey joined #salt
15:03 Cottser joined #salt
15:07 mpanetta joined #salt
15:10 rdas joined #salt
15:14 sarcasticadmin joined #salt
15:14 netcho joined #salt
15:17 nickabbey joined #salt
15:19 nickabbey joined #salt
15:20 sh123124213 joined #salt
15:21 sh123124213 joined #salt
15:21 dyasny joined #salt
15:23 Sam____ @channel : I am new to salt stack and using it from last 2 days. Any salt reference document for Windows. Specific to executing power-shell script to salt minion
15:24 Dev0n hey, is there anything special I would need to do to change the minion id other than on the minion?
15:25 Dev0n will the master recognise the new minion id for an existing minion?
15:26 AndreasLutro no, the minion's key is tied to its id, so if the id changes, you have to re-accept the key
15:26 alev joined #salt
15:26 doubletwist I know it's not actually using it, but is it normal for salt-minion to show a large amount of memory for VIRT in top? RES is only 50MB but VIRT is showing > 500MB
15:28 Dev0n gotcha, thanks AndreasLutro
15:28 viq IIRC if I point top.sls or try to include a file that doesn't exist, salt will throw errors. How can I load a file (pillar in this case) if it exists, but ignore it if it doesn't?
15:29 AndreasLutro viq: '*': [ my_sls, ignore_missing: true ]
15:30 viq AndreasLutro: oooh, nice, thank you!
15:30 * viq goes to find it in docs
15:30 AndreasLutro might not be there tbh
15:31 viq ungh, how fun
15:32 * viq checks source
15:32 viq And now to try it
15:34 mpanetta joined #salt
15:36 toastedpenguin joined #salt
15:40 dyasny joined #salt
15:40 drawsmcgraw1 joined #salt
15:41 viq AndreasLutro: so something like this should work? https://pbot.rmdir.de/bQJAyPnnXmg05iMJMytcRQ
15:42 st8less joined #salt
15:44 netcho joined #salt
15:45 TheoSLC joined #salt
15:48 Neighbour how can I specify which saltenv is to be used for salt-run? For some reason I keep getting data from `salt-run pillar.show_pillar` that is from the prod env
15:50 djgerm1 i am getting the following error today with some new instances in ec2 and I don't understand why?  * ERROR: Failed to run install_ubuntu_stable_deps()!!!
15:50 djgerm1 I can run the script after the fail at it works fine… any clue?
15:51 edrocks joined #salt
15:57 Neighbour `salt-run pillar.show_pillar 'saltenv=dev'` does not seem to work as promised if multiple environments are defined (dev, prod)
16:00 djgerm1 should you be using pillarenv=dev ?
16:00 anotherzero joined #salt
16:01 Neighbour https://docs.saltstack.com/en/latest/ref/runners/all/salt.runners.pillar.html
16:01 Neighbour pillarenv=dev also still shows values from the prod environment
16:01 Neighbour I've even got the salt-master setting:top_file_merging_strategy: 'same'
16:01 Neighbour but that doesn't seem to work either
16:09 tercenya joined #salt
16:10 spirit__ hi
16:11 spirit__ does anybody know whether salt allows to use result of one state (cmd.run) in another ?
16:14 Neighbour yes and no...you can use jinja like "{% set result = salt['cmd.run_all']('command_here', other opts here) %}" and then use the result jinja variable elsewhere in the state
16:15 spirit__ yes, but what if command does not exist on server yet ?
16:15 spirit__ I am writing php.sls - install php + ioncube
16:15 spirit__ before running a highstate thereis no /usr/bin/php on server
16:16 spirit__ but I need it to determine php version to extract corresponding file from ioncube.tar.gz
16:19 _JZ_ joined #salt
16:20 spirit__ so I need somehow to execute php -r 'echo PHP_MAJOR_VERSION.".".PHP_MINOR_VERSION;' in some state and use the result in another
16:20 Inveracity joined #salt
16:20 spirit__ as I see there is no any grains function to save the result as a grain :-(
16:22 spirit__ the only suitable is salt mine, but as for me it is very excessive way
16:22 spirit__ and it shares result between all other minions, I don't want this
16:22 viq spirit__: grains.set ?
16:23 coredumb Hey folks is there a way to blacklist env per nodegroup/group/minion ?
16:23 viq On the other hand, what about pkg.version ?
16:24 spirit__ and how use it ? (grains.set)
16:24 spirit__ as I see it accepts static values only
16:25 raspado joined #salt
16:25 spirit__ pkg.version ? - version is not the only thing I need :-(
16:26 viq How about doing that in two passes? Set a grain with php version when it's installed, extract files only when php grain is set?
16:26 viq What else do you need?
16:27 spirit__ {% set php_modules_dir = salt.cmd.shell("php -r 'echo ini_get(\"extension_dir\");'"                , output_loglevel='quiet') %}
16:27 spirit__ {% set php_zts         = salt.cmd.shell("php -i | sed -r '/^Thread Safety\s/ ! d; s/^.*=>\s+//'"   , output_loglevel='quiet') %}
16:27 stooj joined #salt
16:27 spirit__ I need to know where to put ioncube module file, wheter php support thread safety
16:28 viq How about you set those things, instead of trying to figure out what OS threw at you?
16:29 spirit__ in another SLS I need more complicated thing
16:29 spirit__ {% set logrotate_duplicates = salt.cmd.shell("LANG=C logrotate -d /etc/logrotate.conf |& sed -r '/duplicate log entry/ ! d; s|^.*duplicate log entry for\s+||' | xargs --no-run-if-empty -I %% egrep -l '^%%([[:space:]]|$)' /etc/logrotate.d/*", output_loglevel='quiet') %}
16:30 spirit__ here I can not set value statically
16:31 babilen spirit__: Shouldn't the PHP version you install be determined by the platform you target?
16:31 babilen Or rather: Why don't you tell the minion which version to install and use?
16:32 spirit__ may be, but not always
16:32 babilen Not always?
16:32 spirit__ because I dont know which versions are available in current distro
16:33 babilen How many distributions do you target?
16:33 babilen I mean .. the PHP customers that I have are *very* peculiar about their PHP version and insist on very strict version constraints (must be a PHP thing) and all major distributions ship a clearly defined version
16:33 spirit__ I am trying to create universal scenario
16:34 babilen The orchestrate and write custom grains that contain the information you need
16:34 babilen (as an idea)
16:34 aarontc joined #salt
16:35 spirit__ you mean additional grains module to get that data ?
16:37 spirit__ yes, it is possible, but I tried to know whether more simple way exists
16:42 impi joined #salt
16:44 babilen Well, you need this to happen in two steps so orchestrate is an obvious choice
16:44 babilen How you get the information is up to you, but I figured that a custom grain would come in handy
16:44 babilen Entirely depends on your needs .. if your cmd.shell approach is more appropriate, use that
16:45 babilen Salt is very much about enforcing a specific state, not reacting to a given one ..
16:46 spirit__ how to set custom grain ? manually via /etc/salt/grains ?
16:53 raspado joined #salt
16:54 babilen https://docs.saltstack.com/en/latest/topics/grains/#writing-grains
16:54 babilen spirit__: ^
16:56 DammitJim joined #salt
16:56 spirit__ ok, thanks ! reading...
17:06 edrocks joined #salt
17:07 stooj joined #salt
17:08 [CEH] joined #salt
17:09 onlyanegg joined #salt
17:16 Trauma joined #salt
17:17 hop left #salt
17:26 stooj joined #salt
17:27 tercenya joined #salt
17:27 nickabbey joined #salt
17:28 DanyC joined #salt
17:33 hasues joined #salt
17:33 hasues left #salt
17:38 bowhunter joined #salt
17:42 DanyC joined #salt
17:44 ivanjaros joined #salt
17:50 stooj joined #salt
17:50 XenophonF joined #salt
17:51 amy joined #salt
17:54 pipps joined #salt
18:01 nickabbey joined #salt
18:07 Edgan joined #salt
18:13 abednarik joined #salt
18:17 MTecknology Why is cachedout no longer on IRC?! :'(
18:23 DanyC joined #salt
18:23 whiteinge You broke him! Are you happy now?!   ;-)
18:24 keldwud joined #salt
18:24 Bryson joined #salt
18:25 MTecknology whiteinge: I'd cry myself to sleep if I didn't have to work.
18:25 whiteinge FWIW, I do that same thing whenever cachedout is not around.
18:26 nidr0x joined #salt
18:29 stooj joined #salt
18:31 nickabbey joined #salt
18:34 tercenya joined #salt
18:36 DanyC left #salt
18:36 s_kunk joined #salt
18:39 catpig joined #salt
18:39 edrocks joined #salt
18:45 ChubYann joined #salt
18:51 stooj joined #salt
18:54 Praematura joined #salt
18:58 pipps joined #salt
19:12 cebreidian joined #salt
19:15 Rumbles joined #salt
19:28 sjorge joined #salt
19:34 ekristen joined #salt
19:35 gtmanfred i wish that I wasn't so busy so I could be on irc more
19:36 gtmanfred that issue queue just never stops
19:37 jholtom :(
19:37 DanyC joined #salt
19:37 whytewolf salt seems to have gotten to the level they need a code monkey for ticket issues alone
19:37 gtmanfred we have two
19:37 gtmanfred we switch off on odd or even days now
19:38 nZac joined #salt
19:38 gtmanfred for 2 months me and ch3ll were doing it 4 hours a day, but it is a hard contact switch to do in the middle of the day that I now do odd days and she does even
19:39 whytewolf oh wow. pure ticket work can burn someone out :( i know it killed me when i used to do php development [and is one of the reasons i left development alltogether insearch of ops]
19:40 honestly at my work (sysadmin for a computer science department) everyone is on ticket duty for one day out of the week - works great since there's five of us
19:41 Edgan whytewolf: yeah, I hate tickets. It is why I stay away from Jira as best as I can.
19:41 Mattch joined #salt
19:41 Edgan whytewolf: I found a well managed kanban board to be much more pleasant.
19:41 Sketch is there any way i can use include more than once in a state?
19:41 honestly (also there's a first-level support/helpdesk in front of us, we're second-level)
19:42 honestly Sketch, what do you mean by that? include takes a list of states to include
19:42 Sketch right.  i want to include one, then do some stuff, then include another one
19:42 whytewolf Edgan: typically for my personal stuff i use jira iwth the agile setup with kanban boards.
19:43 Edgan It is really bad when the other side who opened the ticket doesn't care. They say something like my computer doesn't work. You reply with questions. You get nothing, and then they complain you did nothing.
19:43 Sketch unless i can assign priority to individual included states
19:43 honestly Sketch: you need to manage state dependencies using 'require'/'onlyif' etc.
19:43 Sketch i want to do something like..  include: - .mysql ... msyql_database.present ... include: - .app_using_database
19:44 Sketch aha, that could work
19:44 Sketch so my database_present could require the included database installation state
19:45 whytewolf which is the proper way to work with it. include isn't like a php style include where it dumps the code from the included file at the location the include is at
19:45 Sketch though my second include was a formula i was hoping not to have to modify more than necessary, i guess i can add a require to it
19:45 ecdhe joined #salt
19:45 ecdhe joined #salt
19:45 Edgan 95% of the time I manage dependencies by order alone. Managing by requires is just extra work, learned this all too well from puppet. Requires end up reordering, and you will have tested something and then the graph will change, and something will break.
19:45 Sketch whytewolf: yeah, i'm aware...though it usually seems to put stuff in the order it's written, i know it's not always the case.
19:45 honestly edgan, salt doesn't guarantee states being executed in order of declaration
19:45 honestly or does it now?
19:46 Sketch nope, it doesn't
19:46 gtmanfred Sketch: there is an open feature request for that https://github.com/saltstack/salt/issues/14899
19:46 saltstackbot [#14899][OPEN] State inclusion is limited | Currently, in a state I can include other modules:...
19:46 Edgan honestly: it does until you still don't use requires
19:46 Edgan honestly: I meant until you use requires
19:47 Sketch Edgan: without requires, do includes get inserted inline where you put them?
19:47 gtmanfred Sketch: the only way to do it now, is include it at the top, and use require_in, with the posibility of using sls: to specify the entire sls that is required_in
19:47 gtmanfred that might work
19:47 Edgan Sketch: yes, the only exception I have found is adding grains in a state doesn't work as expected
19:48 DanyC joined #salt
19:48 Sketch ah.  i have states which have include and add grains, that must be why i have seen that it's not always exactly in the order.
19:49 nickabbey joined #salt
19:49 Edgan Sketch: order of precedence when using grains from yml, grains from states, and customer grains are a mess
19:50 Edgan Sketch: I stick to customer grains and built-in grains
19:50 bowhunter joined #salt
19:50 Sketch what defines a customer grain?
19:50 amy joined #salt
19:50 Edgan Sketch: python grain, like built-in, stored in _grains
19:51 honestly is it a known issue that (custom?) grains aren't accessible inside file templates (i.e. the source of a file.managed state with jinja) with salt-ssh? (Cc: AndreasLutro )
19:51 Edgan honestly: works for me
19:51 honestly with salt-ssh?
19:51 Edgan honestly: yes, use salt-ssh heavily
19:51 honestly wow, that means there's three salt-ssh users in here now :P
19:52 honestly hmmm
19:52 honestly so I specify custom grains in the roster file
19:52 Edgan honestly: ok, that is another form of grain, which I use too, but only in a very limited way
19:52 honestly and {{ salt['grains.get']('foo:bar') }} works fine in a state file
19:53 honestly but not in a file template
19:53 Edgan I use
19:53 Edgan honestly: file template = jinja template?
19:53 honestly yes?
19:53 honestly file.managed with template:jinja
19:54 pipps joined #salt
19:54 denys joined #salt
19:55 sh123124213 joined #salt
19:56 Edgan honestly: plume_web/files/etc/nginx/sites-enabled/plume-web.conf:  server_name {{ grains['fqdn'] }};  is one example
19:56 honestly hm
19:56 cyborg-one joined #salt
19:56 honestly well, that's not a custom grain, is it?
19:57 Edgan honestly: that one isn't but this one is
19:57 Edgan honestly: web/files/etc/nginx/sites-enabled/web.conf:  server_name www-{{ grain.env }}.{{ web.nginx.public.domain }} www.{{ web.nginx.public.domain }};
19:58 honestly where do "grain" / "web" come from?
19:58 Edgan {%- import '_grains/map.jinja' as grain -%}
19:58 Edgan {% set env = salt['grains.get']('ops:env') %}
19:59 MTecknology If anyone wants to read a funny transcript.. https://gist.github.com/MTecknology/063cab5751735e5c7da696e6cf81329f
19:59 honestly Edgan: huh.
19:59 Edgan honestly: _grains/map.jinja includes the set env line, which then is imported at the top of the template as grain. Which then makes it grain.env
19:59 DanyC joined #salt
20:00 honestly Edgan: yeah, I figured
20:00 honestly how does salt load _grains/map.jinja?
20:00 Edgan honestly: and web is deployed via salt-ssh
20:01 Edgan honestly: for salt-ssh you have to
20:01 Edgan extra_filerefs:
20:01 Edgan - salt://_grains/map.jinja
20:01 Edgan in Saltfile for salt-ssh
20:01 honestly interesting
20:01 Edgan to make it copy map.jinja files across to the remote system
20:03 Edgan honestly: I use salt-ssh for jenkins deploys of "our" code. It is also uses for "their" code for "our" code dependencies in development.
20:03 honestly sounds like map.jinja is rendered by a different module than the template file (web.conf)
20:04 Sketch MTecknology: funny
20:04 Edgan honestly: I mostly use custom(python) grains to slice and dice the hostname for matching. env, cluster, region, provider, etc.
20:05 honestly python grains?
20:05 Edgan honestly: The other main use is for what I call breadcrumbs. They look for a file to exist. If it does exist, they are true. If not, they are false. I use this method to run things once as part of a state.
20:06 MTecknology Sketch: worth mentioning.. that was email (the admin was the guy that replaced me at a place in south dakota)
20:06 Edgan honestly: https://paste.fedoraproject.org/536539/85374770/
20:07 honestly Edgan: and where do you put and how do you include that file?
20:07 Sketch MTecknology: i wondered if it was a phone call from a probably-indian consultant with poor english...
20:07 Edgan honestly: The use for this jenkins_setup grain is to put a out of the box config.xml for jenkins master. Since you end up editing the config.xml via the web ui.
20:07 Edgan honestly: _grains in the formulas/states git repo and they are auto included
20:08 honestly okay
20:08 honestly interesting, thanks!
20:08 cebreidian joined #salt
20:11 Edgan honestly: salt-ssh is the least developed/ignored part of salt. It's compatible with salt master mode is only about 90-95%. The biggest issue I just found is salt master mode allows binary pillar data to be rendered, but salt-ssh transports pillar data via json, and so doesn't.
20:12 abednarik joined #salt
20:13 Edgan honestly: extra_filerefs is another key difference. I came up with a clever away around it, but ended up being way more work than just including all map.jinjas with extra_filerefs
20:15 mpanetta joined #salt
20:22 jrklein joined #salt
20:25 MTecknology Sketch: no.. that was in-house, someone that grew up in the midwest
20:25 MTecknology Sketch: email transcript btw
20:27 honestly Edgan: I'm well aware of salt-ssh's state :)
20:28 Edgan honestly: I get the impression I am the biggest user of it.
20:29 Edgan honestly: I use it to be able to test salt master code, deploys, and reuse the same salt master code between deploys and master mode.
20:29 honestly 20:51:58 < honestly> wow, that means there's three salt-ssh users in here now :P
20:29 Edgan honestly: I don't like running salt master in a vagrant.
20:29 honestly AndreasLutro is the third one
20:29 pipps joined #salt
20:30 honestly well at my workplace it's policy to avoid agent daemons if at all possible
20:30 nZac joined #salt
20:30 Edgan honestly: For deploys, say a web app needs rabbitmq. Rabbitmq gets deployed with the web app on the same instance, from a jenkins job. But later in production rabbitmq is it's own instance and is managed via salt master mode.
20:31 honestly you mean master/minion mode?
20:31 honestly or standalone master?
20:31 Edgan honestly: I have never understood policies like that. Yes, master/minon mode scales way better than salt-ssh.
20:31 honestly scales?
20:32 honestly no, one minion instance per host is terrible. so much ram wasted and a process running that doesn't do anything 99.9% of the time
20:33 Edgan honestly: salt-ssh is slow. It is even slower when you work around the caching issues by turning off caching. It is good for things that are deployed, but managing 100+ instances of "their" code systems, salt-ssh doesn't work out well.
20:33 MTecknology honestly: if salt is only busy 0.1% of the time, you're severely under-utilizing salt
20:33 honestly if you need to deploy to 100 nodes at once, sure
20:33 MTecknology also, salt-ssh has lots of overhead that adds up when you hit scale
20:33 honestly MTecknology: salt is config management. your config shouldn't be changing constantly.
20:33 Edgan honestly: you wanted the runs to auto run, not be triggered by a person. Master mode also lets you collect reports from the minions and have a dashboard of the overall status.
20:33 Eugene My 2c: RAM is cheap, time is not
20:33 whytewolf i use it for more then config management
20:34 MTecknology honestly: config management was just one of the shiny things that salt did well
20:34 DanyC joined #salt
20:35 MTecknology It wasn't started to be a config management utility and it's far from just that
20:35 Edgan Part of the power of salt is that is more than configuration management(like Puppet).
20:35 MTecknology system deployment, automation, orchestration, management, insights, reporting, etc.
20:35 Edgan It can also do orchestration, which Puppet left to Mcollective.
20:35 honestly if you need that, sure
20:35 honestly it's not what I use it for
20:35 MTecknology maybe what you really want is ansible?
20:35 MTecknology that's kinda what it sounds like
20:36 Edgan honestly: I also use salt-ssh over master mode for deployments to solve the problem of controlling package versions well for deployment.
20:36 honestly MTecknology: ansible is terrible though :)
20:36 Edgan honestly: With salt-ssh, I can make sure I only get the version of a package I expect
20:36 honestly I use salt-ssh as a better ansible, totally
20:36 Eugene My 2c: the proper way to maintain package versions is to use your own package repos and control what goes in
20:37 MTecknology In my opinion, the basic salt recipe is "pkg.installed, service.running: -watch:-file:foo, file.managed
20:37 MTecknology "
20:37 Edgan honestly: with master mode, I put it in a apt repository, and then I could install just that version, but if someone runs apt-get dist-upgrade and there is a higher version, it gets changed and breaks configuration management
20:37 MTecknology only 1/3rd of that is "config management"
20:37 Edgan Eugene: What if I need five different version on different clusters? Five different apt repositories?
20:39 Rumbles joined #salt
20:39 MTecknology Edgan: I feel like I was just discussing that last night :P
20:39 Eugene Yup, each environment should have a mirror, or at least a subfolder to hunt in
20:39 Edgan honestly: ansible is better compared to just salt-ssh. It is far less buggy. Ansible and Salt code are relatively alike. Both are yaml+jinja, generally. I think Salt is better than Ansible, but only because it has master mode. Salt-ssh is tolerable.
20:40 honestly ansible's configuration language just isn't expressive enough
20:40 CEH joined #salt
20:40 MTecknology Edgan: I thought ansible ~= salt-ssh, but ansible > salt-ssh, so salt-ssh == good only if salt == in-use
20:40 honestly salt also has better concepts for modularisation and separation of concerns
20:40 Edgan MTecknology: basically
20:41 drel joined #salt
20:41 drel hi everyone :)
20:41 MTecknology honestly: you lose about 95% of what salt can do by using salt-ssh, it's usually used as a bridge to pain-in-the-ass devices
20:41 MTecknology (or to bootstrap)
20:41 Edgan Eugene: I find that way more work than just having jenkins pull artifacts from Artifactory, and using salt-ssh to deploy them.
20:42 drel someone care to give me a tip on a beacon/reactor issue I am having ?
20:42 honestly MTecknology: I'm not interested in those 95%
20:43 honestly I used salt in master/minion mode for three years in a different project
20:43 honestly it was okay
20:43 Eugene I did say it was just my 2c ;-). This is how I've always seen it done in large-scale ENTERPRISE deployments. The tooling to rsync from the upstream repos & run it through your QA environment has to be hand-rolled I do admit, but it beats a production outage
20:43 honestly and I used it for orchestration then
20:43 pipps joined #salt
20:44 MTecknology honestly: you can't orchestrate with salt-ssh
20:44 honestly like I said, I used it in master/minion mode
20:44 Edgan MTecknology: I think the thing Salt suffers Massively from, is advanced books/case studies. There are lots of big companies doing crazy things with it, but in the dark and no one else knows.
20:45 Edgan MTecknology: I was teaching co-workers about Salt and found all the books I found find completely lacking on the advanced side.
20:45 MTecknology Edgan: I've been wanting to do a salt talk and my target keeps shifting, but I've noticed a common target... "Salt: You deployed it, now let's do it again... but better." <-- what I think the title of my next talk will be.
20:45 Sketch i find the salt documentation is good for at least 90% of stuff
20:46 whytewolf the salt documentation only gets you the basics.... to get the omg wtf that is awesome stuff you need to be creative with what those functions and modules used together can do
20:46 Edgan Sketch: documentation is good to ok, but documentation doesn't give you ideas of advanced ways to combine ten different pieces into something fancy
20:47 Edgan Sketch: more tutorials explaining how to do advanced things, and Why you would do it that way
20:47 Sketch yeah, that is true
20:47 whytewolf such as things like hubblestack
20:47 Sketch a lot of my learning with salt has just been realizing i could do certain things
20:47 Sketch the basics docs are good, but they're pretty basic
20:47 Sketch -s
20:48 MTecknology I find salt is too free-form for the novice that dives in. It's too flexible and lets you build poor logic too easily... however, I'm happy that it /does/ offer that flexibility, but I feel like people need a better "this looks cool, but lemme esplain you why it be no good" review... I'm hoping to manage that
20:48 MTecknology like sticking environment logic into states
20:49 Edgan MTecknology: One big example of where all the books lack, no mention of map.jinjas.
20:49 bowhunter joined #salt
20:49 MTecknology I've actually *never* come across an appropriate use of jinja maps in real life. I can come up with them in theory, but I've never seen them used well in the real world.
20:50 MTecknology 99% of the time, what they did should have been in salt
20:50 drel Edgan: I am with you on that ... took me 2 hours to figure out how to set a default value
20:50 MTecknology lemme rephrase.. s/99% of the time/every time I've ever seen it/
20:50 Sketch really one of my biggest complaints about salt is inconsistent naming in states
20:50 MTecknology Sketch: how so?L
20:51 amy joined #salt
20:51 Sketch file.missing, pkg.removed, cron.absent, etc...
20:51 Sketch how many different ways can we say something is not there? ;)
20:51 MTecknology do you want pkg.missing?
20:51 MTecknology I agree, file.absent makes more sense
20:51 MTecknology pkg.absent, does not
20:52 Sketch i think it would make more sense if there was one way to say "make sure this doesn't exist"
20:52 whytewolf file.missing != file.absent
20:52 whytewolf file.absent removes the file. file.missing checks if the file is missing
20:52 Sketch oh yeah, file has both
20:52 MTecknology I also forgot that too as well
20:52 MTecknology GAH! I need a name for my new laptop
20:53 Sketch i often have to refer to the docs just to know what command to use in cases like that
20:53 Eugene Steve is a good name
20:53 pipps joined #salt
20:54 whytewolf how about __init__ thats a good computer name
20:54 drel guys can I fire an inotify event from the master's minion process and catch it as normal ?
20:54 MTecknology panther = primary laptop (named after a tattoo and my favorite animal), cross = netbook-style tiny-laptop (after my other tattoo), and now I have a powerhorse of a mother effer ready to beat the piss outta workload... need a name.
20:54 whytewolf drel: watch the master event bus to see if the event is getting there
20:55 Edgan MTecknology: There are two main uses for map.jinjas. One is to give you case statements for different distribution/releases. That is the basic version.
20:55 whytewolf MTecknology: Mammoth?
20:55 drel whytewolf: it gets there and looks fine ... however the reactor does not catch the tag
20:55 MTecknology whytewolf: what's it mean?
20:55 Edgan MTecknology: The advanced version is to give you default configuration values, that can then be overridden by pillars.
20:55 * MTecknology likes meaningful names :S
20:55 whytewolf MTecknology: a big harry animal that stomps on cavemen
20:55 drel event if I put '*' as tag matcher
20:56 MTecknology arctic? because it showed up during the end of a snowstorm?
20:56 stooj joined #salt
20:58 MTecknology awe... now I'm gonna need a way to keep stuff in sync between laptops because my old is going to become a desktop replacement and my new powerhouse is going with me everywhere
20:58 Edgan MTecknology: One advantage of defaults in map.jinjas, is then you can restrict who has access to pillars much more easily. Anyone can write salt code and include their defaults. Then other people can understand that code by just looking at the formula. Pillars can be left as overrides and secrets.
20:58 cmarzullo ^^ that's my pattern
20:58 MTecknology that sounds like a pattern build around using formulas
20:59 * MTecknology happens to strongly discourage using formulas as anything other than playgrounds and learning.
20:59 Edgan MTecknology: yes, and once you go formulas, you realize how cookie cutter this all is
20:59 Edgan MTecknology: formulas are the only way you get overrides, otherwise you are going to do a ton of duplication
20:59 MTecknology ehm...
21:00 MTecknology I shoot for zero duplication of anything regardless of how many environments/domains/etc. that one master is managing.
21:01 Edgan MTecknology: Without a formula, how do you have two different versions of a nginx state without making it nginx1 and nginx2 for two slightly different use cases? Tons of if statements?
21:01 MTecknology pillar
21:02 Edgan MTecknology: example?
21:02 xbglowx joined #salt
21:02 armyriad joined #salt
21:02 cmarzullo pillar can help with that. but the other point of formulas is to break things down into smaller pieces. If you have a large team you can't have everyone mucking about in a single state repo.
21:02 MTecknology I'll admit, this actually /should/ be done in a jinja map now that I'm thinking about it, but a done within pillar
21:02 cmarzullo it gets ugly.
21:03 edrocks joined #salt
21:03 Edgan MTecknology: I think I have played some of your game
21:03 cmarzullo last place I worked had a giant chef repo with all the cookbooks. versioning was very difficult.
21:03 MTecknology no reason to force just one repo
21:03 Edgan MTecknology: The advantage that map.jinja gives you over pillars is shorter variable names, and also more generic ones.
21:04 pipps joined #salt
21:04 MTecknology again, I think the difference here is using formulas
21:05 stooj joined #salt
21:05 MTecknology I discussed this stuff last night, though.
21:05 MTecknology I don't have the energy to do it again... the timestamps will indicate why :P
21:07 armyriad joined #salt
21:08 armyriad joined #salt
21:11 amy joined #salt
21:13 nickabbey joined #salt
21:20 nicksloan joined #salt
21:26 DanyC joined #salt
21:26 stooj joined #salt
21:29 cacasmacas joined #salt
21:30 nZac joined #salt
21:33 sh123124213 joined #salt
21:36 abednarik joined #salt
21:37 stooj joined #salt
21:49 DanyC joined #salt
21:56 pipps joined #salt
21:57 bowhunter joined #salt
21:59 bltmiller joined #salt
21:59 teclator joined #salt
22:01 bltmiller Is there any concern over upgrading the version of Jinja2 on my Salt installation? trying to use the 'equalto' test which is only available starting in Jinja 2.8
22:04 aawerner joined #salt
22:05 whytewolf bltmiller: i use jinja2 2.8 with salt. works fine
22:06 bltmiller sweet! yeah 2.8 seems to work for me as well
22:06 aawerner joined #salt
22:06 bltmiller only concern was whether I'd be breaking something unknowingly
22:07 pipps joined #salt
22:07 xbglowx joined #salt
22:10 nZac joined #salt
22:10 stooj joined #salt
22:13 jhauser joined #salt
22:14 xbglowx joined #salt
22:34 Tanta joined #salt
22:36 DanyC joined #salt
22:38 hackel joined #salt
22:46 s_kunk joined #salt
22:46 s_kunk joined #salt
22:49 madboxs joined #salt
23:01 pipps joined #salt
23:02 edrocks joined #salt
23:07 Edgan Whats new in 2.8?
23:08 lasseknudsen joined #salt
23:18 cebreidian joined #salt
23:18 xbglowx joined #salt
23:22 toastedpenguin joined #salt
23:26 whytewolf truncate filter, equalto filter, a base parameter added to the int filter. target added to urlize function.
23:27 whytewolf http://jinja.pocoo.org/docs/2.9/changelog/#version-2-8
23:30 Xopher joined #salt
23:31 pipps joined #salt
23:31 xbglowx joined #salt
23:32 jagguli- joined #salt
23:34 xbglowx joined #salt
23:40 pipps joined #salt
23:42 sysadmin75 left #salt
23:46 jeddi joined #salt
23:53 ALLmightySPIFF joined #salt
23:57 nZac joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary