Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2014-03-02

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:06 krow joined #salt
00:16 jankowiak joined #salt
00:18 srage joined #salt
00:29 diegows joined #salt
00:31 Gifflen joined #salt
00:37 rgbkrk joined #salt
00:42 helderco joined #salt
00:44 zirpu joined #salt
00:44 yidhra joined #salt
00:46 yidhra joined #salt
00:51 yidhra joined #salt
00:52 yomilk joined #salt
00:53 Math` joined #salt
01:05 baniir joined #salt
01:09 Math` joined #salt
01:12 srage joined #salt
01:14 taion809 joined #salt
01:19 elfixit joined #salt
01:20 krow joined #salt
01:22 xzarth_ joined #salt
01:26 elfixit joined #salt
01:27 mgw joined #salt
01:41 n8n joined #salt
01:49 helderco joined #salt
01:54 jslatts joined #salt
02:06 srage joined #salt
02:16 fxhp I wish there was a way to return data to the master, and then have the master push to another system
02:16 fxhp instead of minions connecting to the backend datastores directly
02:18 dstanek when i set an order in a state is that only considered within the sls file or just over all?
02:19 fxhp dstanek: I think it is the sls file itself, but I try really hard not to use ordering
02:19 mgw joined #salt
02:19 Math` fxhp: you can piggy-back on the zeromq link and pass an event from the minion to the master, I think
02:19 dstanek i have an iptables.sls that has a few rules i want to run first, then there is an iptables.append in my nginx.sls, and then in iptables there is a order:last that i want to run after everything
02:20 dstanek fxhp: this is the only case that i have tried it (other and require, watch, etc)
02:20 dave_den MTecknology: it would blow up
02:20 fxhp dstanek - I use require statements to deal with dependencies
02:21 fxhp (require and watch statement do work between different sls files)
02:21 Katafalkas joined #salt
02:22 fxhp Math` - do you have more information
02:22 fxhp Math` - I basically was able to get the data I wanted using either execution modules or runners
02:23 fxhp but that puts it to STDOUT as json, ideally once all the data was aggregated on the master, I would like to send it to a data store
02:23 Math` http://docs.saltstack.com/topics/event/index.html
02:23 Math` you may have to write some code on the master though
02:23 fxhp aye, ok
02:24 Math` but you can get away with <50 lines of python in my opinion
02:26 fxhp Math - I think this is what I needed
02:27 fxhp Either way, I'm going to dig into this because it seems pretty powerful
02:27 Math` fxhp: did you see the reactor system? http://docs.saltstack.com/topics/reactor/
02:27 Math` unless that is what you attempted to paste
02:34 fxhp Math` yeah this won't work the the reactor system
02:34 Math` why
02:34 goodwill well it work till meltdown and ensuing Chernobyl :)
02:34 fxhp I'm not reacting to anything
02:34 goodwill just saying
02:34 goodwill :-P
02:34 Math` the point is, you fire an event from the minion and you react to that event
02:35 Math` it avoids the writing code part :)
02:36 faldridge joined #salt
02:44 fxhp Math`: I think it might be more straight forward to write the code in this case
02:44 fxhp Math`: thank you for the pointers
02:44 Math` np
02:47 dstanek fxhp: looks like i'll just keep the list of ports in pillar data like i have been doing - i was hoping that just by including an sls i could augment the rule i use
02:50 yomilk joined #salt
02:53 goodwill fxhp: not you, salt ...
02:53 goodwill fxhp: it is the one that has the reactor
02:57 DaveQB joined #salt
02:57 zloidemon joined #salt
03:00 srage joined #salt
03:03 fxhp dstanek - sorry I would need to see the files you are talking about to better understand the issue you are facing
03:04 fxhp dstanek - if you have the energy to pastebin the sls and pillars that are related, I will take a stab at it
03:04 fxhp goodwill : P
03:04 dstanek fxhp: in my top file i include nginx.sls - as a side effect of doing that i want port 80 in the iptables
03:05 dstanek fxhp: i would probably come up with a contrived example
03:05 fxhp Math` - I was able to fire events in both directions using salt-call
03:05 fxhp Math` - while listening to the event bus with the code snippet iterator provided in a '3rd party' python script
03:06 Math` cool
03:06 fxhp Math` - but I cannot get the code example to work
03:07 rgbkrk joined #salt
03:08 Math` which examples specifically?
03:09 fxhp http://docs.saltstack.com/topics/event/index.html
03:10 fxhp #firing-events-from-code
03:10 Math` a good way is to turn on logging, let me give you a snippet
03:10 fxhp I tried both examples (the first example fails with an error message as expected)
03:11 Math` import salt.log and salt.log.setup_console_logger('all')
03:11 Math` that should make the console a lot more chattier
03:11 fxhp Also when using salt-call, the program blocks until it gets a True from the master
03:11 fxhp Math` - ok will do
03:17 baniir joined #salt
03:18 fxhp Math` - I put the logging statements in each script, all it displays is the PUB PULL sockets
03:18 Math` do you see the event being fired?
03:21 fxhp Math` - nope
03:21 Math` can you show me what you see?
03:22 fxhp sure let me prepare a pastebin
03:22 rostam joined #salt
03:24 fxhp Math` - http://pad.yohdah.com/212/8ea137b0-1f54-462f-88a6-a5c17fb02389
03:24 Math` where are you running this? master or minion
03:25 fxhp That was run on a minion
03:25 Math` /var/run/salt/minion/master_event_pub.ipc exists ?
03:25 fxhp and I have another script running on the master which subscribes to that 'mytag'
03:25 fxhp nope that doesn't exist
03:26 Math` zeromq's behavior is to block if a socket is not existing so that messages can flow when the server starts
03:26 Math` there you go
03:26 Math` find /var/run/salt/minion -name "*.ipc"
03:30 fxhp well hmm
03:30 fxhp that connection is generated by the event object
03:32 fxhp so either the example is wrong, or there is a defect somewhere
03:32 fxhp I'm too new to know which
03:32 fxhp (new to this part of the systme)
03:34 favadi joined #salt
03:34 ravibhure joined #salt
03:34 ravibhure left #salt
03:37 Math` fxhp: so you see other stuff in /var/run/salt/minion but not that socket?
03:37 fxhp yes
03:37 fxhp Thinking I'm calling the event constructor wrong
03:37 fxhp __init__(self, node, sock_dir=None, **kwargs)
03:37 fxhp node?
03:38 fxhp I have 'master'
03:38 Math` you said it was working with salt-call
03:38 pdayton joined #salt
03:38 fxhp yes it was
03:38 fxhp (it is working with salt-call)
03:38 Math` self is implicit in python... 'master' is the node
03:38 Math` salt-call -l debug .....
03:38 Math` and watch for which IPC socket it uses
03:38 * fxhp nods
03:38 fxhp k
03:38 pydanny joined #salt
03:39 Math` (self is the equivalent of the this pointer, instance method always define it as their first argument)
03:39 tmwsiy__ joined #salt
03:39 fxhp Math`: I understand the consept of self, : )
03:39 Math` great :)
03:40 krow joined #salt
03:43 fxhp Math` - nothing interesting in debug when running salt-call (salt-call still works)
03:43 fxhp guess I could poke into the code that salt-call is running and see how they are doing ot
03:43 Math` strace it
03:47 fxhp strace is 3.8M
03:47 fxhp haha
03:47 * fxhp falls out of chair
03:47 Math` grep for .ipc
03:47 Math` Im guessing
03:48 fxhp pattern not found
03:50 Math` code says epub_uri = 'ipc://{0}'.format(os.path.join(self.opts['sock_dir'], 'master_event_pub.ipc'), so for sure it's the right file name
03:50 Math` sudo fgrep sock_dir /etc/salt/minion
03:50 Math` fxhp: ^ that should tell you where it is
03:51 fxhp the host running the minion should have 'master' as the node and the path to the minion sock_dir
03:51 fxhp correct?
03:51 Math` yes
03:51 fxhp Math` - could you try to reproduce this on your setup?
03:54 fxhp all the the code for this test is in that document
03:54 fxhp everything works except emitting the event via code from the minion
03:54 srage joined #salt
03:56 Math` fxhp: it seems that the socket is only bound from the master
03:56 Math` can't you send an event from the sls file?
03:57 * Math` wonders if the documentation is user-editable
03:58 fxhp Math i have not tested from an sls file, but that seems to be counter to what the doc says
03:58 fxhp plus sls files are compiled on the minions and run
03:58 Math` well, event is a module so you can call it from anywhere
03:59 Math` but they will run from salt-minion and have a preconfigured channel for them so they can talk to the master
04:00 Math` and therefore won't need that socket
04:00 fxhp But the whole point of this document and excersize is to show that a "3rd party" application on both minion host and master host can communicate over the Salt event bus (encrypted ZMQ protocol)
04:01 Math` sure, if you want to go that route, there is a "client" you can use from python that will talk to the minion
04:01 Math` hold please
04:01 anuvrat joined #salt
04:03 fxhp I shouldn't need to use a client to talk to the minion
04:03 fxhp """
04:03 Math` hmm LocalClient is only usable from the master
04:03 fxhp This allows 3rd party applications to harness the power of the Salt event bus programmatically, without having to make other calls to Salt.
04:04 fxhp A 3rd party process can listen to the event bus on the master and another 3rd party process can fire events to the process on the master, which Salt will happily pass along."""
04:04 Math` Im looking at minion.py, and it seems to use minion_event_{0}_pub.ipc, where {0} is the id_hash
04:05 fxhp ok
04:05 Math` by client I meant the client class in salt.client.*, it's what you use to make salt-calls to minions from code on the master
04:06 Math` ok so now we have to figure out where to pass the right socket path to SaltEvent
04:07 Math` if node == 'master':
04:07 Math` seems like node == master is the wrong setting after all
04:10 Math` fxhp: I checked the code through, the node var is only used to check if it's == "master", or else the value is unimportant
04:11 Math` the doc is on git, I'll shoot a pull request later on
04:11 Math` bbl
04:12 rojem joined #salt
04:13 fxhp The hash that it generates when I put something else into node does not match the hash that exists on the filesystem
04:13 fxhp there is def a defect lingering
04:15 fxhp It goes into the 2nd else block if node is not 'master' or 'ipc_mode' != 'tcp'
04:15 fxhp which is where it is suppose to figure out the hash so it can find the minion socket
04:16 fxhp but in my case, the hash is invalid
04:16 Math` the hash comes from the id
04:16 fxhp id_hash = hashlib.md5(kwargs.get('id', '')).hexdigest()
04:16 fxhp yes ..
04:19 fxhp and I set the id in my '3rd party' script
04:19 fxhp and it doesn't generate the same hash
04:19 fxhp so something else is happening
04:20 kermit joined #salt
04:24 fxhp Well I got it to create the correct hash
04:24 fxhp but it still doesn't work
04:24 fxhp has nobody tried to use this before?
04:24 fxhp event = salt.utils.event.SaltEvent('sagat.foxhop.net',sock_dir=sock_dir,id='sagat.foxhop.net')
04:24 fxhp I needed to pass a kwargs for id
04:26 Math` what's in the logs now?
04:26 fxhp nothign
04:27 fxhp Math` - I would hold off on changing the docs until we understand how this is inteded to be used and the correct API to interact with it
04:27 fxhp because as it stands its kind of foobar'd
04:27 Math` indeed
04:28 Math` can you check if the minion has that socket opened?
04:28 fxhp How do I do that?
04:29 Math` lsof /path/to/socket
04:29 Math` and/or lsof -p $(pidof salt-minion)
04:31 fxhp yes salt-minion has both pub and pull open
04:31 fxhp same pid
04:31 Math` that's going to be verbose, but hopefully it'll get more info... try python -m trace --trace ./your-script.py
04:32 Math` it'll echo every line of python that's executed
04:32 Math` the last ones will be where it's blocking
04:32 n8n joined #salt
04:41 fxhp Well the blocking isn't happening anymore
04:41 krow joined #salt
04:41 fxhp or rather was never happeneing, I'm just not seeing the event pop out the other side
04:41 fxhp that said, I have a scary feeling this might have something to do with tagas
04:41 fxhp tags*
04:42 Math` looking further, in the event module, in the fire_master function, I see this
04:42 Math` return salt.utils.event.MinionEvent(**__opts__).fire_event( {'data': data, 'tag': tag, 'events': None, 'pretag': None}, "fire_master")
04:44 Math` the "fire_master" ends up in the "tag" param to fire_event
04:45 Math` which ends up as the beginning of the string that's pushed, which probably ends up as matching package.startswith('fire_master'):
04:45 Math` fxhp: and I just read your scary feeling msg haha
04:48 srage joined #salt
04:49 Math` so the tag passed on the command line (when using salt-call) is fed into the object, the other tag is defined to "fire_master". it seems you can also do "module_refresh" and "pillar_refresh" which are handled directly by the minion
04:49 Math` which is what salt-util uses
04:49 krow joined #salt
04:50 fxhp So they are manipulating the tag if the event is created with the fire_master ?
04:50 Math` no it's just a different tag
04:50 Math` events have a tag, and the event's payload too
04:50 Math` because in this case the minion is relaying an event to the master
04:50 fxhp yeah and I'm setting them the same on both sides
04:51 fxhp so it must be mutated somewhere
04:51 Math` instead of 'tag' use 'fire_master'
04:51 fxhp is that an argument or a string?
04:51 fxhp fire_master
04:52 Math` string
04:52 Math` no kwargs here
04:55 fxhp I'm lost a bit
04:56 pdayton joined #salt
04:56 fxhp I see that in my app I'm trying to use an object of the class SaltEvent, and using the method fire_event
04:56 fxhp and that the salt-call uses a different file_master
04:57 fxhp fire_master function
04:57 Math` fxhp: if I trust what I read, replace your fire_event by: event.fire_event({'data': data, 'tag': "THE_REAL_EVENT_TAG", 'events': None, 'pretag': None}, "fire_master")
04:57 krow joined #salt
04:58 Math` that's basically what fire_master does
04:59 Math` except it uses MinionEvent instead of SaltEvent, which is the same as setting the node var to "minion" but it has no other repercussions since that var is ignored
04:59 Math` if it's != "master" that is
05:01 fxhp Ok well that is crazy and needs to be documented to fixed
05:01 Math` in recap, minions have their own event bus, so you must send an event asking the minion to relay the event to the master
05:01 fxhp let me give it a try
05:01 fxhp gotcha
05:02 Math` probably should take this discussion on the mailing list, but IMO adding a function to event to support fire_master directly would make a lot of sense
05:02 Math` but I haven't contributed to salt yet so I don't know how others will feel about that change
05:02 fxhp Math`: check out the wording in the SaltEvent, the use pull and push and pub and seem to confuse the terms in the docstrings
05:05 Math` hmm
05:05 Math` in connect_pull ?
05:05 Math` indeed it's messed up
05:06 schristensen joined #salt
05:07 Math` so PUB->SUB takes in messages from the pub side and broadcasts them to the subscribers
05:07 Math` PUSH->PULL takes in messages from may pushers, and interleaves them for a single PULL socket
05:09 Math` (ref http://learning-0mq-with-pyzmq.readthedocs.org/en/latest/pyzmq/patterns/pushpull.html and http://learning-0mq-with-pyzmq.readthedocs.org/en/latest/pyzmq/patterns/pubsub.html)
05:09 fxhp I manipulated the payload like you suggested and it still didn't work ...
05:09 fxhp Although I think you are on the right track
05:12 Math` the minion binds epub_sock as PUB and epull_sock as PULL, so the filenames are relative to the minion
05:12 Math` (thinking out loud)
05:13 Math` so connect_pull() connects to the socket named pull as push, and then sets  self.push which is used throughout the file to send msgs
05:13 Math` so that's fine
05:13 Math` connect_pub() connects to the pub socket, as SUB and sets self.sub
05:14 Math` that checks out
05:14 Math` fxhp: can you run the master with full debug? and see if you see the event? same with the minion
05:17 Math` looking at master.py / _minion_event to see where it could fail
05:17 Math` fxhp: hmm in my previous line data must be the payload, can you update the pastebin with the latest code?
05:18 fxhp sure
05:19 fxhp http://pad.yohdah.com/214/4c1c6ee3-6018-44df-a39c-b39fb9246393
05:23 fxhp # One of 'garbage', 'trace', 'debug', info', 'warning', 'error', 'critical'.
05:24 Math` you can use 'all' too
05:26 fxhp i set both to debug
05:27 fxhp nothing in logs when I run the script
05:27 fxhp will try all
05:30 Math` and on the minion?
05:30 Math` log.debug("Forwarding master event tag={tag}".format(tag=data['tag']))
05:30 Math` you should see that at least
05:32 yomilk joined #salt
05:32 fxhp Math`: nothing in either log related to the event being fired
05:33 Math` then it's not hitting the minion properly
05:34 Math` you see any Handling event  ?
05:34 Math` this is right after epull_sock.recv() you have to see it
05:35 fxhp python -m trace --trace test_event.py
05:35 fxhp madness
05:36 fxhp no I don't see anything in logs
05:38 fxhp I think I'm going to give up for tonight
05:38 fxhp I learned a bunch, I think this part of salt needs a bit of help (code and docs)
05:39 kaictl joined #salt
05:39 Math` fxhp: http://pad.yohdah.com/216/c1e269a2-0d9c-4a3b-8114-beaf7740459c
05:40 Math` this is an attempt to load the opts from the config file (like it normally does)
05:40 Math` that's a part of salt I'll have to work with soon so I'm happy I had a grasp at that code
05:41 Math` argh, forgot  import salt.syspaths as syspaths
05:42 srage joined #salt
05:44 jdenning joined #salt
05:44 fxhp testing
05:47 fxhp Math`: your code produced the invalid hash
05:47 fxhp basically the hash of ''
05:49 Math` means opts['id'] isn't set
05:49 Math` I thought it'd be in the config
05:49 Math` but add in opts['id'] = '....'
05:49 Math` ** wille expand a dictionary in to kwargs
05:50 Math` will*
05:51 fxhp id isn't set on my hosts
05:52 fxhp it is determined by FQDN
05:52 snuffeluffegus joined #salt
05:56 fxhp Math`: also the sock_dir is invalid
05:57 fxhp sock_dir should be '/var/run/salt/minion' but when we use the programatic way from the minion conf file, it returns '/var/run/salt/master'
05:57 fxhp which is a bug in itself
06:28 Math` joined #salt
06:29 krow joined #salt
06:31 ravibhure joined #salt
06:31 ravibhure left #salt
06:33 ndrei joined #salt
06:37 srage joined #salt
06:38 baniir joined #salt
06:40 Katafalkas joined #salt
06:48 n8n joined #salt
06:57 Katafalkas joined #salt
06:57 krow joined #salt
07:02 faldridge joined #salt
07:07 jslatts joined #salt
07:26 younqcass joined #salt
07:28 zain_ joined #salt
07:31 srage joined #salt
07:35 zooz joined #salt
07:37 jslatts joined #salt
07:40 ahammond joined #salt
07:42 jacksontj_ joined #salt
08:03 Katafalkas joined #salt
08:04 faldridge joined #salt
08:04 jeffro anyone got a formula for creating init.d services
08:25 srage joined #salt
08:27 brian joined #salt
08:28 brian hey guys
08:28 brian can someone review my paste? I'm trying to add the "watch" and "reload: True" to this state, not sure what i'm doing wrong (I didn't write the initial state)
08:28 brian http://pastebin.com/a2js2xDJ
08:30 yomilk joined #salt
08:37 jeffro_ joined #salt
08:38 jslatts joined #salt
08:40 tmwsiy_ joined #salt
08:44 jslatts joined #salt
08:45 jeffro joined #salt
08:47 yomilk joined #salt
08:49 harobed joined #salt
08:50 ravibhure joined #salt
08:59 jeddi brian: don't you need a service definition to reload?
09:00 jeddi i may need to go and re-read up on watch :)
09:00 Teknix joined #salt
09:02 brian yes i figured it out, thanks!
09:04 brian wow, i love salt stack
09:05 brian I think the learning curve for puppet would of been a lot higher, I'm almost done with my basic config
09:05 ravibhure1 joined #salt
09:05 ravibhure1 left #salt
09:05 brian Now i just need to figure out how to use this to auto-bootstrap my cloud servers
09:05 ambientsound indeed, getting started with salt is easy :)
09:06 jeddi brian: good to hear.  i noted in the backscroll several threats of storming off to the puppet camp about 24 hours ago :)
09:06 jeddi like every powerful tool, it's not something you should expect to master in the first sitting.
09:07 brian I'm not trying to master it
09:07 brian I'm just trying to do enough to finish this project
09:07 brian But once you understand how it works, it's pretty easy to figure things out
09:07 brian The issue was due to a version mismatch, the software doesn't check to make sure the versions are compatible!
09:07 brian So I had no clue it wasn't working due to that
09:13 jeddi yeah, recent versions seem to be getting pricklier with regards version alignment.  that's going to bite me big time soon, i expect.
09:17 zooz joined #salt
09:19 srage joined #salt
09:19 tmwsiy__ joined #salt
09:25 anuvrat joined #salt
09:34 analogbyte joined #salt
09:34 brian I need to know the best way to add a dynamically spinned up cloud server to the master
09:35 brian The instance is created independent of the master
09:37 faldridge joined #salt
09:37 brian Should I just make the minion SSH into the master and execute a command to accept it's key?
09:39 johtso joined #salt
09:53 harobed joined #salt
09:54 harobed joined #salt
09:59 yomilk joined #salt
10:07 Pate_ joined #salt
10:08 harobed_ joined #salt
10:08 tomasfejfar joined #salt
10:09 pdayton joined #salt
10:13 srage joined #salt
10:28 ajw0100 joined #salt
10:30 brian left #salt
10:33 joehh arnoldB: still working on them
10:33 joehh unstable uploaded to new and to debian.saltstack.com
10:34 joehh backports coming soon
10:34 tomasfejfar joined #salt
10:42 akitada joined #salt
10:42 ml_1 joined #salt
10:45 ndrei joined #salt
10:45 jslatts joined #salt
10:56 yomilk_ joined #salt
10:57 glen_ joined #salt
11:07 srage joined #salt
11:22 generj joined #salt
11:24 generj hello everybody
11:24 generj I have a weird problem: My connection between the minion and master times out
11:24 DaveQB joined #salt
11:24 jeddi generj: tried changing the timeout ? ;)
11:25 generj jeddi: I'd rather it didn't timeout :P
11:25 jeddi generj: what's your timeout set to currently?
11:25 jeddi is this on initial runs (large, full configs), or every time, or infrequently?
11:25 generj this is a stock install on ubuntu 12.04
11:26 generj and its everytime
11:26 generj after a few minutes
11:26 generj do I look at the timeout in the minion config or master or both? :)
11:27 tmwsiy_ joined #salt
11:27 generj jeddi: and thank you for your help by the way, it has been driving me bonkers
11:28 jeddi generj: yeah - master .. mine's set to 20s , which seems to catch most of my (old) issues.
11:28 jeddi generj:  you can also do (from memory) --show-timeout on the commandline for salt while running states.
11:29 jeddi if it's 'few minutes' then you may have ... suboptimal configurations that it's trying to parse / process.   did it happen this way from the beginning?  of course it may just be that you have a cruddy network transit path :)
11:29 generj jeddi: its a stock install, I think its a cruddy network path (azure, ha)
11:29 generj jeddi: what's the setting called exactly?
11:30 jeddi hmm.   can't comment on azure, unfortunately.
11:30 generj lets just say its slow
11:30 jeddi in /etc/salt/master the setting is   'timeout: 20'  (in my case)
11:30 jeddi adjust to flavour.
11:31 generj and restarting master...
11:31 jeddi there's some things you can do to work around / work out network performance problems.  try running specific states rathr than a highstate, to determine which one's causing grief (I don't know if we have better profiling tools than that at the moment - someone else can commento n that)
11:31 jeddi update to the latest versions of minion & master, that'll probably help too.
11:31 generj jeddi: that makes sense, thanks, however this is just a stock install and a simple sudo salt '*' test.echo
11:32 jeddi some iterations (jinja) are more expensive than they first seem -- f.e. if you're pushing packages out, don't iterate around the entire block, but use the 'pkgs' feature and iterate them in a single lump.   i found a big improvement over that - it's the difference between doing 50 'apt-get install's and one apt-get install with 50 package names.
11:32 jeddi wow.   if yo'ure timing out on that kind of thing i'd suggest you have a network problem.
11:33 jeddi does test.ping timeout or is that immediate?
11:34 generj jeddi: test.ping isn't timing out now that I set it to 25
11:35 generj jeddi: as long as 'nc -v -z  master.host 4505' and 4506 can be gotten at successfully from the minions that's all it takes, right?
11:36 generj hopefully with a sky high time out and modest requests it will keep working now
11:36 generj jeddi: is there a proper way to manually deal with a timeout short of restarting the minion?
11:38 jeddi yup - minion needs to be able to see master, but not the other way around, yes.
11:38 generj joined #salt
11:39 generj jeddi: thanks
11:40 generj jeddi: although maybe there needs to be some sort of out of band communication channel or message bus that keeps retrying - because it seems bad to me that responses can get dropped
11:40 generj bad network connectivity can happen after all
11:41 tmwsiy__ joined #salt
11:41 johtso joined #salt
11:42 jeddi generj:  ah - you're back :)
11:42 jeddi generj: i don't know if there's a proper way of doing it .. i tend to restart things whenever they go a bit funny, though obviously that isn't a policy that scales.
11:47 jslatts joined #salt
11:55 younqcass joined #salt
11:59 generj so… it occurs to me, is there a good way to bootstrap salt?
11:59 generj instead of manually apt-getting salt for every server I want as a minion..
12:02 srage joined #salt
12:02 younqcass_ joined #salt
12:04 generj joined #salt
12:05 linjan_ joined #salt
12:08 jeddi generj: depends on your environment.  if you're using aws/ec2 for example you may generate your own ami with requisite packages and config files.
12:08 jeddi i don't have a good answer.  i don't do mass deploys that i need to automate on such a scale.
12:13 svx joined #salt
12:19 yomilk joined #salt
12:28 harobed joined #salt
12:33 harobed joined #salt
12:34 harobed joined #salt
12:46 elfixit joined #salt
12:47 jslatts joined #salt
12:56 srage joined #salt
12:57 yomilk joined #salt
12:59 tharkun joined #salt
13:08 taion809 joined #salt
13:09 ndrei joined #salt
13:17 Katafalkas joined #salt
13:22 taion809 joined #salt
13:31 RMA_ joined #salt
13:31 RMA_ Hi All
13:31 GoKage joined #salt
13:32 RMA_ How to resolve multiple libs error in redhat using salt. like when we get new machine we never going to worry about what version of pkg is installed already. Just we need to downgrade or upgrade a pkg based on our salt config. If a package requires downgrade whihc needs to be resolved while calling salt state
13:33 RMA_ How to do that
13:40 faldridge joined #salt
13:48 jslatts joined #salt
13:50 srage joined #salt
13:50 martoss joined #salt
13:56 Teknix joined #salt
13:57 generj jeddi: are you around? the timeout fix didn't solve things
13:57 generj stock ubuntu 12.04 install, fresh salt master and minion
13:57 generj it works at first and then the connection gets dropped
13:58 generj if I restart the minion the commands (simple echos/pings) start flowing right away again
13:58 generj and stop again soon after
13:58 generj I put the minion under supervisord and that didn't solve it either
13:58 generj anybody know what these can possibly be?
14:08 bhosmer joined #salt
14:08 ndrei joined #salt
14:09 symroe joined #salt
14:30 jeddi generj: I'm sort of around.  :)    timeout bumping fixed it in my case, so i didn't explore beyond that.    what version(s) are you on?
14:30 jeddi supervisord?  (just reading up on it now).   long live systemd in ubuntu :)
14:31 jeddi i'd be checking logs - bumping up to info or verbose or whichever, on master and minion, and tail -f 'ing them both in separate terminals, and watching as they lose contact.
14:32 elfixit joined #salt
14:32 mgw joined #salt
14:35 generj jeddi: it seems like a known problem
14:35 generj https://github.com/saltstack/salt/issues/4199
14:36 generj it has to do with keepalives
14:36 generj and the hacky solution is to do a cron job that sends out pings to every minion from the master every 60 minute
14:40 sgflt joined #salt
14:40 jslatts joined #salt
14:44 srage joined #salt
14:47 xmltok joined #salt
14:54 baniir joined #salt
14:55 tomasfejfar joined #salt
14:59 diegows joined #salt
15:04 bhosmer joined #salt
15:13 jeddi generj: wow.  that's quite an old bug report, though.
15:13 jeddi it's also been closed, though curiously without any comment / commit (other than target 0.15) that i can see.
15:14 fxhp jeffro: a couple weeks ago I would have suggested upstart scripts instead of init.d scripts ... but now that Ubuntu and Debian are switching to systemd in the future, it might be worth while to do that.
15:17 fxhp jeffro: normally we sort upstarts directly on salt file server (normally less then 10 lines with comments) and use jinja substitution with values stored in pillar
15:17 fxhp sort/store
15:18 vejdmn joined #salt
15:21 martoss joined #salt
15:21 martoss left #salt
15:22 rostam joined #salt
15:25 rojem joined #salt
15:28 ndrei joined #salt
15:28 psyl0n_ joined #salt
15:34 harobed joined #salt
15:38 srage joined #salt
15:40 psyl0n_ left #salt
15:53 gnugnu8 joined #salt
16:04 generj jeddi: I think it should be reopened
16:04 dstanek joined #salt
16:04 generj jeddii: a 1 minute cron job from master that pings all the minions makes everything rock solid for me
16:05 jdenning joined #salt
16:17 sroegner_ joined #salt
16:22 farra joined #salt
16:23 CeBe joined #salt
16:32 srage joined #salt
16:39 rojem joined #salt
16:42 GradysGhost joined #salt
16:46 valgrind joined #salt
16:48 Math` joined #salt
16:57 kintel joined #salt
16:59 zooz joined #salt
17:06 Math` joined #salt
17:06 cjbarnes18 joined #salt
17:09 cjbarnes18 Hi all,  I see the debs are released for 2014.1, is backports now needed for wheezy?
17:26 fauxtoniche joined #salt
17:26 srage joined #salt
17:32 johtso joined #salt
17:35 dh joined #salt
17:50 bhosmer joined #salt
17:51 ckao joined #salt
17:55 johtso joined #salt
17:58 Math` joined #salt
18:01 naiyte joined #salt
18:03 ipalreadytaken joined #salt
18:03 gnugnu8 joined #salt
18:08 fllr joined #salt
18:10 vbabiy joined #salt
18:18 pdayton joined #salt
18:18 JonGretar joined #salt
18:20 JonGretar Hi.. Does external_auth.pam not work on SmartOS? I have the master configured correctly I believe but I just get: The specified external authentication system "pam" is not available
18:20 srage joined #salt
18:26 fxhp http://www.foxhop.net/salt-stack-piggy-back-encrypted-zmq-event-bus <- documented my research notes here
18:31 ajw0100 joined #salt
18:35 jeremyfelt joined #salt
18:44 swa_work joined #salt
18:54 ndrei joined #salt
18:57 baniir joined #salt
18:57 Gifflen joined #salt
19:00 yomilk joined #salt
19:01 Math` fxhp: so you got it to work after all
19:01 ndrei joined #salt
19:05 fxhp Math`: nope not really
19:05 fxhp Math`: haha
19:05 fxhp Math`: I got the minion to talk to the minion bus
19:05 fxhp Math`: I got the master to talk to the master bus
19:05 fxhp Math`: But I hav enot gotten the minion to talk to the master bus
19:05 Math` that's the first step
19:05 fxhp (or vice versa)
19:05 Math` in minion.py it decapsulates the event to forward it to the master
19:06 Math` line 485
19:06 Math` salt/minion.py
19:07 fxhp https://github.com/saltstack/salt/blob/develop/salt/minion.py#L485
19:07 Math` 538
19:11 fxhp Math`: I think this might be neat - http://www.foxhop.net/salt-stack-piggy-back-encrypted-zmq-event-bus#ideas
19:11 fxhp I might not be using the proper terms
19:12 Math` if you want to do that from the master, the client will do that for you
19:15 srage joined #salt
19:15 fxhp Send remote execution commands to minions and return data to a special channel?
19:16 Math` I mean get the data back from a python app
19:17 Math` I guess getting it on the event bus directly might be useful for some use of the reactor system
19:18 faldridge joined #salt
19:25 ingwaem joined #salt
19:28 fxhp https://github.com/saltstack/salt/blob/develop/salt/minion.py#L1337 <- fun line number but also the unpacking you are describing
19:29 fllr joined #salt
19:30 Math` fxhp: ah the first line# I gave was in MultiMinion
19:31 * fxhp nods
19:31 fxhp which I don't need to learn about right now
19:31 fxhp heh
19:31 faust joined #salt
19:31 Math` zmq passes around bytes, unpack restores the dictionary structure that was sent
19:31 fxhp I think if the "package" is formatted properly it will forward to master
19:32 fxhp it (being the minion bus)
19:32 Math` the package is the tag you pass to send_event, "fire_master"
19:33 fxhp so 3rd party script publishes to minion bus -> salt-minion daemon sees the properly formatted package and runs _fire_master -> ends up on master bus -> 3rd party script subscribes to the tag
19:33 fxhp I think the package is the whole "event" blog
19:33 fxhp blob*
19:34 mike25ro left #salt
19:34 Math` yep since it's a .startswith()
19:34 fxhp package = self.epull_sock.recv(zmq.NOBLOCK)
19:35 fllr joined #salt
19:35 Math` salt/utiil/event.py:322  event = '{0}{1}{2}'.format(tag, tagend, self.serial.dumps(data))
19:36 zooz joined #salt
19:36 yomilk joined #salt
19:37 Math` where TAGEND = '\n\n'
19:37 zain_ joined #salt
19:39 fxhp Math`: yeah there are two styles of tags old (padded 20 chars) or new (unlimited lenth ending with \n\n)
19:39 ml_1 joined #salt
19:44 tzero joined #salt
19:50 fxhp Math` - so this is my naive understanding so far -
19:50 fxhp http://www.foxhop.net/salt-stack-piggy-back-encrypted-zmq-event-bus#fire-events-between-minion-and-master
19:51 jdenning joined #salt
19:51 fxhp this part of the API isn't documented anywhere
20:09 srage joined #salt
20:17 Math` fxhp: I made a small vagrant setup here to try this out
20:17 Math` the minion has exceptions if 'events' and 'pretag' aren't defined in the payload (KeyError)
20:18 Math` Im getting a _minion_event command on the master by changing the fire_event line to include "fire_master"
20:19 faldridge joined #salt
20:23 fxhp Math`: awesome
20:23 Math` but I can't see it from the program, tracing the master now
20:23 fxhp Math`: what version of the codebase are you using?
20:24 fxhp I'm using 0.17.x (0.17.5)
20:24 Math` salt-master-0.17.5-1.el6.noarch
20:24 Math` the one in centos epel
20:24 fxhp and I was looking at develop version of the code base
20:24 Math` well on that box I edit /usr/lib/python2.6/site-packages/salt/master.py for debugging purposes
20:24 fxhp and if you look at the codebase for 0.17.x it doesn't have a fire_master case in the minion.py
20:24 fxhp ..
20:25 fxhp https://github.com/saltstack/salt/blob/0.17/salt/minion.py#L1174
20:25 fxhp so that could be the issue
20:25 Math` that sure doesn't help
20:26 Math` but my 0.17.5-1 seems to have it
20:26 fxhp hmm
20:26 fxhp which line?
20:26 Math` look in your machine's
20:27 Math` 1331
20:27 Math` in /usr/lib/python2.6/site-packages/salt/minion.py
20:28 martoss joined #salt
20:30 fxhp found it here on Ubuntu /usr/lib/pymodules/python2.7/salt/minion.py
20:31 fxhp That line puts me in an AES method on my version of the file
20:31 Math` look for fire_master
20:31 ndrei joined #salt
20:31 fxhp I find the method _fire_master and a few calls to it
20:31 fxhp but not the string that the case is looking for for forwarding
20:32 fxhp That has to be my issue
20:32 schimmy joined #salt
20:33 Math` it is
20:33 Math` I have another issue, but you have to ugprade
20:35 rgbkrk joined #salt
20:35 * fxhp nods
20:36 fxhp I thought I was up-to-date
20:38 faldridge joined #salt
20:38 beach joined #salt
20:39 fxhp hmm
20:39 fxhp "apt-get remove --purge salt-minion" didn't clean up files
20:39 fxhp hmm
20:40 Math` ah, it seems that zmq doesn't complain if it can't connect to the socket on the listening side
20:40 fxhp Oh yeah salt-minion is a meta package, salt-common is the heavy package
20:40 Math` I had to change line 1226 to make more sense, change self.event.fire_event(load,, tag) to self.event.fire_event(load['data'], tag)
20:40 Math` in master.py
20:40 Math` or else it forwards the encapsulation payload too
20:41 fxhp And now it forwards the to the master ?
20:41 Math` yep
20:41 fxhp erm I mean the gets processed?
20:41 fxhp neat!
20:42 Math` https://www.dropbox.com/s/c0vjhuhl9rndpwr/Screenshot%202014-03-02%2015.42.04.png
20:42 Math` left is minion, right is master
20:43 fxhp on the same box?
20:43 Katafalkas Is there a way in salt-cloud to query for specific information about single instance ? salt-cloud -Q brings info about all running instances. you can add -m for the map. But how do you specify a single instance ?
20:43 Katafalkas can you achieve it without map ?
20:45 Math` fxhp: through ssh
20:46 Math` fxhp: the multi-events method works without tweaking the master at all
20:46 Math` http://pad.yohdah.com/217/e57db07b-4ae1-4733-95fc-f7dd978e1be8
20:47 Math` and payload['id'] is set by the minion automatically
20:49 harobed_ joined #salt
20:49 harobed_ joined #salt
20:50 harobed_ joined #salt
20:51 harobed_ joined #salt
20:52 harobed_ joined #salt
20:53 harobed_ joined #salt
20:53 harobed_ joined #salt
20:54 harobed_ joined #salt
20:55 JonGretar joined #salt
20:56 fxhp Math`: I have the latest 0.17.x package from ubuntu PPA, doesn't have the 'fire_master' changes
20:56 fxhp in minion.py
20:56 fxhp Math`: that paste that you posted, what did you change?
20:57 Math` fire_master at the bottom, 'events': [ data ], 'data': None instead of 'events': None, data: data
20:58 Math` added the tag to data
21:00 fxhp All I see a list of events
21:00 fxhp (in which we are only giving one event)
21:00 Math` yeah, there are two ways to pass in events to the master, through the data and tag keys, or through a list of events
21:00 fxhp neat, still doesn't work over here, because I have old code without 'fire_master' in minion.py event loop
21:00 Math` the former is buggy as it passes the entire encapsulated event from the minion down the event bus
21:01 fxhp which you should submit a patch for if it is still an issue in the latest develop branch*
21:01 fxhp : )
21:01 Math` yep I just forked the repo
21:02 JonGretar Anyone know of a reason to why pam.auth would not load up in SmartOS? https://gist.github.com/JonGretar/9313851
21:03 srage joined #salt
21:03 fxhp JonGretar - I have messed with SmartOS, but not with Salt, sorry
21:04 Math` https://github.com/saltstack/salt/pull/10890
21:05 fxhp Math` - the issue I'm having will be resolved when the PPA for 2014.1.0 is ready
21:09 yomilk joined #salt
21:09 dangra_ joined #salt
21:11 fxhp Math`: last night I was getting invalid data back from configuration
21:11 Math` from configuration?
21:11 fxhp Math`: turns out client_config function != minion_config
21:11 fxhp client_config defaults to "master" settings
21:11 fxhp minion_config defaults to "minion" settings
21:12 Math` it seems that anything client runs on the master
21:12 * fxhp nods
21:14 viod joined #salt
21:18 viod Hi! I'm trying to install a package using salt for the first time, but i can't get it to work. The server and minion are on the same machine, i've set up a pillar named 'packages' which returns the correct package name for the distro, and call it with name: {{ pillar['packages']['emacs'] }} in my state. But I always get this error: "Rendering SLS "base:editors" failed: Jinja variable 'dict object' has no attribute 'packages';". Does anyone know why?
21:21 fxhp viod - you need to target that pillar to the minion
21:22 fxhp you can test what pillars a particular minion has by running: sudo salt-call pillar.data
21:22 fxhp viod: from the minion
21:23 fxhp viod: you target pillars to minion hosts using the top.sls file in the pillar_root
21:23 fxhp viod: it works very similar to targeting states to minions in the top.sls file of the file_root
21:24 viod yep, my top.sls contains:
21:24 viod base:
21:24 viod '*':
21:24 viod - packages
21:24 viod shouldn't it be ok?
21:24 fxhp do you have a packages.sls or packages/init.sls in the pillar root?
21:24 viod a packages.sls
21:24 fxhp yes then that should allow all minions access to the packages pillar
21:25 viod i've also tried doing "salt '*' saltutil.refresh_pillar", but it hasn't changed anything
21:25 fxhp What does the contents of packages.sls look like in the pillar_root?
21:25 cewood joined #salt
21:26 viod packages:
21:26 viod {% if grains['os'] == 'Debian' %}
21:26 viod apache: apache2
21:26 viod emacs: emacs23-nox
21:26 viod {% elif grains['os'] == 'Arch' %]
21:26 viod apache: apache
21:26 viod emacs: emacs
21:26 fxhp do you have an {% endif %} ?
21:26 viod nop
21:27 fxhp You need that for the jinja to be valid
21:27 viod i've added it, but still no change
21:28 fxhp run: salt-call grains.get 'os'
21:28 fxhp does the output of that command match one of the conditionals ?
21:29 viod Debian, so yes
21:29 fxhp hmmm
21:31 fxhp Well, that should work
21:31 fxhp could I see your top.sls and editors.sls file contents from file_root, in a pastebin?
21:32 fxhp viod:
21:32 viod yes, i'm pasting them
21:33 schimmy joined #salt
21:34 viod http://pastebin.aquilenet.fr/?d15b3899600622ca#1cs+90DiB3HfVklae0XGjtRp7f0AvIDyS0/8sWqb9Ec=
21:35 fxhp viod: that all looks correct
21:35 fxhp did you configure a pillar_root in your /etc/salt/master config file?
21:37 viod i put it in /etc/salt/master.d/pillar_roots
21:37 viod pillar_roots:
21:37 viod base:
21:37 viod - /srv/salt/pillar
21:38 fxhp I have no idea what is going on.
21:38 fxhp restart salt-master ?
21:38 viod just in case it matters, this is the command i use to "execute" the states : salt '*' state.highstate
21:39 fxhp yup
21:39 fxhp that is the proper way
21:39 viod no change after restart
21:39 viod (minion and server)
21:39 fxhp viod: I'm at a loss...
21:39 viod so am I ^^
21:40 viod i haven't found anything like that on google, even though i've spent quite a long time at it
21:43 fxhp I feel like it must be a typo or formatting issue
21:43 fxhp because I use what you are doing
21:44 viod i'm gonna retype all my files then, hopefully it will work ^^
21:44 zooz joined #salt
21:45 DaveQB joined #salt
21:47 fxhp viod: try to simplify the problem
21:48 fxhp a pillar with one 'key: value', listed under '*'
21:48 faldridge joined #salt
21:48 fxhp and then try to see if the pillar exists when you run: salt-call pillar.data
21:49 viod ok
21:49 fxhp (the key:value needs to be in a separate file like test.sls)
21:49 fxhp and then - test needs to be listed
21:51 fxhp in top.sls
21:51 viod i've got a test.sls containing "foo: bar"
21:51 ajw0100 joined #salt
21:52 viod so "salt-call pillar.get foo" should return "bar", right?
21:52 fxhp right
21:52 fxhp as long as you listed test in pillar'stop.sls
21:53 fxhp pillar's top.sls under '*'
21:53 viod my pillar.sls :
21:53 viod base:
21:53 viod '*':
21:53 viod - test
21:53 fxhp right
21:53 viod still, it doesn't return anything
21:54 fxhp so something in your setup is broke
21:54 fxhp not the logic in your pillar file_root
21:54 viod is master.d/ read by default, or must it be enabled?
21:55 viod i guess it's read, as it knows where my states are
21:55 fxhp yea...
21:55 fxhp although I think that is the default
21:56 fxhp the /srv/salt area
21:56 viod oh
21:56 fxhp I think it might be bad to embed your pillar into the file_root
21:57 fxhp maybe try /srv/salt for file_root and /srv/pillar for pillar root
21:57 srage joined #salt
21:57 viod ok
21:57 fxhp (I use my home dir )
21:57 fxhp I have a repo for pillar and a repo for states
21:58 fxhp and I check them out in home and use /etc/salt/master to list them
21:58 viod i do have a bar, now \o/
21:58 fxhp I've not tested the drop dir but I assume it works
21:58 fxhp great
21:59 viod so putting pillar in the states dir is not a good idea
21:59 fxhp so it was an issue with embedding pillar_root inside the file_root
21:59 viod it might be worth mentionning it in the docs/tutorial ^^
21:59 * fxhp nods
21:59 viod are you related to the project, or should I send a mail to the mailing?
21:59 fxhp I could make the change (you could too)
22:00 fxhp : )
22:00 viod indeed ^^
22:00 viod i will, then :)
22:00 heewa joined #salt
22:00 viod many thanks for your hel!
22:00 viod p
22:01 fxhp Welcome, please stick around!
22:01 viod I will! i'll probably need more help soon anyway :p
22:01 sroegner_ joined #salt
22:02 fxhp Now that you figured out that bit, you are going to fly, its a huge rush to get automation manifests setup for deployments
22:04 viod indeed, i was getting fed up of installing all packages and cloning/linking my config by hand on every computer i use :D and my own scripts for automation didn't work very well...
22:08 sroegner_ joined #salt
22:09 glen_ joined #salt
22:10 yomilk joined #salt
22:14 Pate_ joined #salt
22:18 rojem joined #salt
22:19 NotreDev joined #salt
22:19 chris____ joined #salt
22:19 chris____ left #salt
22:19 cornmander joined #salt
22:20 forrest joined #salt
22:20 chri6392 joined #salt
22:20 Pate_ joined #salt
22:25 jacksontj_ joined #salt
22:26 jnials joined #salt
22:27 tr_h joined #salt
22:27 danielbachhuber joined #salt
22:28 cornmander joined #salt
22:29 NotreDev joined #salt
22:39 ndrei joined #salt
22:41 fllr joined #salt
22:42 glen_ exit
22:42 jeffro ooo can i pass variables to other state files?
22:42 baniir joined #salt
22:44 faldridge joined #salt
22:46 schimmy joined #salt
22:50 sgflt joined #salt
22:51 srage joined #salt
22:52 Math` joined #salt
22:54 baniir joined #salt
22:55 wmbaum joined #salt
22:55 zain_ joined #salt
23:00 fllr joined #salt
23:00 bhosmer joined #salt
23:02 sgflt joined #salt
23:06 wmbaum I'm having trouble getting windows minions to see anythign in winrepo or salt-winrepo.git.
23:08 wmbaum salt -G 'os:Windows' pkg.list_available returns an empt list per minion
23:15 jeffro joined #salt
23:24 elfixit joined #salt
23:25 viod fxhp: i have pushed a pull request :)
23:26 jslatts joined #salt
23:26 mgw joined #salt
23:27 Math` joined #salt
23:27 bhosmer joined #salt
23:30 tr_h joined #salt
23:31 jalbretsen joined #salt
23:42 jdenning joined #salt
23:44 krow joined #salt
23:45 srage joined #salt
23:48 rgbkrk joined #salt
23:50 chri6392_ joined #salt
23:58 Math` joined #salt
23:59 seksifemale joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary