Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2018-02-07

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:03 pipps joined #salt
00:11 kwork joined #salt
00:18 sami joined #salt
00:18 nledez joined #salt
00:20 sjorge joined #salt
00:24 Guest73 joined #salt
00:25 sjorge joined #salt
00:46 pipps joined #salt
00:47 sjorge joined #salt
00:47 mujx[m] joined #salt
00:51 pipps joined #salt
00:57 pipps joined #salt
01:03 mbologna joined #salt
01:04 wongster80 joined #salt
01:06 Guest73 joined #salt
01:08 K0HAX joined #salt
01:10 pipps99 joined #salt
01:12 freelock joined #salt
01:12 Processus42 joined #salt
01:12 rtr63gdh[m] joined #salt
01:12 Tenyun[m] joined #salt
01:12 kbaikov[m] joined #salt
01:12 benasse joined #salt
01:12 sxar joined #salt
01:12 jerrykan[m] joined #salt
01:12 hackel joined #salt
01:12 toofoo[m] joined #salt
01:12 gomerus[m] joined #salt
01:12 aboe[m] joined #salt
01:12 ThomasJ|m joined #salt
01:12 benjiale[m] joined #salt
01:12 theblazehen joined #salt
01:12 viq[m] joined #salt
01:12 gomerus[m]1 joined #salt
01:12 fujexo[m] joined #salt
01:12 glock69[m] joined #salt
01:12 aviau joined #salt
01:16 jrj_ joined #salt
01:18 shiranaihito joined #salt
01:19 pipps joined #salt
01:21 mikecmpbll joined #salt
01:24 Guest73 joined #salt
01:32 pipps joined #salt
01:49 Guest73 joined #salt
01:52 hunmonk joined #salt
01:53 hunmonk wondering if anyone can shed some light on how to make the npm state play well with an npm version installed via nvm? the problem is that the nvm environment must be loaded into the current shell before it's fully available
01:55 hunmonk from a regular bash shell, there's a script i can run that loads the nvm environment into the shell, but how would i do that via a salt state? i think cmd.run executes in it's own shell, and i need to load the nvm environment into the shell env that's running the highstate, so something like npm.installed can find the npm binary
01:55 hunmonk s/can/can't
01:57 pipps joined #salt
02:02 Guest73 joined #salt
02:10 Guest73 joined #salt
02:17 Guest73 joined #salt
02:23 stanchan joined #salt
02:39 stanchan joined #salt
02:58 ilbot3 joined #salt
02:58 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2016.11.9, 2017.7.3 <+> Support: https://www.saltstack.com/support/ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ <+> See also: #salt-devel, #salt-offtopic, and https://saltstackcommunity.herokuapp.com (for slack) <+> We are volunteers and may not have immediate answers
02:59 exarkun joined #salt
02:59 nledez joined #salt
03:00 nledez joined #salt
03:02 Guest73 joined #salt
03:15 pipps joined #salt
03:18 evle2 joined #salt
03:21 stanchan joined #salt
03:23 lompik joined #salt
03:30 Guest73 joined #salt
03:44 noobiedubie joined #salt
03:49 evle2 joined #salt
03:56 ahrs joined #salt
04:09 lompik joined #salt
04:28 exarkun joined #salt
04:50 lompik joined #salt
05:07 zerocoolback joined #salt
05:26 zerocoolback joined #salt
05:39 gomerus[m]1 joined #salt
05:40 benasse joined #salt
05:46 Larry joined #salt
05:56 mujx[m] joined #salt
06:01 benjiale[m] joined #salt
06:01 Tenyun[m] joined #salt
06:06 exarkun joined #salt
06:14 theblazehen joined #salt
06:16 pualj joined #salt
06:17 Guest73 joined #salt
06:18 aboe[m] joined #salt
06:19 fujexo[m] joined #salt
06:19 rtr63gdh[m] joined #salt
06:19 kbaikov[m] joined #salt
06:20 jerrykan[m] joined #salt
06:22 viq[m] joined #salt
06:22 toofoo[m] joined #salt
06:23 ThomasJ|m joined #salt
06:23 gomerus[m] joined #salt
06:24 glock69[m] joined #salt
06:25 Processus42 joined #salt
06:25 freelock joined #salt
06:25 sxar joined #salt
06:27 hackel joined #salt
06:40 Guest73 joined #salt
06:41 LocaMocha joined #salt
06:44 gnomethrower joined #salt
06:45 wongster80 joined #salt
07:18 aldevar joined #salt
07:18 aldevar left #salt
07:21 aldevar joined #salt
07:23 EthPyth joined #salt
07:31 wongster80 joined #salt
07:35 justanotheruser joined #salt
07:36 justanotheruser joined #salt
07:39 hunmonk joined #salt
07:42 Tucky joined #salt
07:50 Guest73 joined #salt
07:52 exarkun joined #salt
07:58 Ricardo1000 joined #salt
08:00 saltnoob58 joined #salt
08:00 mens joined #salt
08:02 jbulger joined #salt
08:05 scc_ left #salt
08:05 Bryson joined #salt
08:06 scc_ joined #salt
08:06 Hybrid joined #salt
08:07 vb29 joined #salt
08:08 DanyC joined #salt
08:10 DanyC_ joined #salt
08:18 colttt joined #salt
08:21 sri__ joined #salt
08:25 XenophonF joined #salt
08:25 hammer065 joined #salt
08:27 sri__ left #salt
08:29 XenophonF hey babilen i'm ahead of you (time zone-wise) for once!
08:29 XenophonF Wien ist sehr schone!
08:31 saltnoob58 Hi. What's the best way to collect grains and other info from lots of minions in a parallel way using salt-ssh and then give that collection of data to one process to do stuff with?
08:32 sjl_ joined #salt
08:32 saltnoob58 run a jinja for loop and create a massive list of arrays variable in jinja? make one python execution module and use python paralellism to make lots of tiny salt-ssh calls? have orchestrator paralelly call minions and have them write stuff down to a database?
08:37 babilen XenophonF: Sehr gut! Genieß den Kaffee :)
08:39 ct16k joined #salt
08:45 MTecknology use json output and pipe the salt query to a python script that does what you need
08:47 hammer065 joined #salt
08:50 saltnoob58 run a bunch of salt queries in parallel via orchestrator and pipe each one to a python script that somehow waits until all queries have finished sending and then do what I need with the aggregated output/data?
08:51 colttt joined #salt
08:52 MTecknology wha?..
08:52 saltslackbridge <mts-salt> given that you might not know which minions are going to even return, how would you know when they're finished unless salt was able to tell you?
08:53 saltslackbridge <mts-salt> oh i get it. run the salt command and literally pipe the output to a script
08:53 saltnoob58 i dont know? the suggestion was unclear to me and doesnt seem to solve my problem
08:55 saltslackbridge <mts-salt> salt * xxxx --out json | script
08:56 saltslackbridge <mts-salt> based on https://docs.saltstack.com/en/latest/ref/cli/salt.html#output-options then you probably want --static too
08:58 jrenner joined #salt
09:01 MTecknology -s makes sense
09:02 saltnoob58 and this would work for orchestrator.sls that contains several execution modules per target host?
09:03 saltnoob58 or need to make custom execution module that runs several execution modules?
09:03 saltslackbridge <mts-salt> this solves your first request - collect info from minions then give that data to one process
09:04 saltslackbridge <mts-salt> if you want to then run that from orchestrator then i don't know how you'll pipe the results
09:08 MTecknology state.orch can still return json
09:09 saltnoob58 I can't get all the data from just one execution module, unless i define everything like pkg.list for example as custom grains and the query all of the grains and pipe that
09:10 saltslackbridge <mts-salt> so run state.sls against a state that's not in top but has requisites for the other state you want to collect information from...?
09:12 saltslackbridge <mts-salt> or just write a custom module. sync that out to all the minions, and call it
09:18 yuhl joined #salt
09:22 saltnoob58 well that's one way to do things, if i can get one command/execution module to output all of the data i want
09:24 saltnoob58 but shelling out in such a major way, just "feels" something
09:26 MTecknology It sounds like you're trying to really over-complicate something
09:26 mikecmpbll joined #salt
09:27 exarkun joined #salt
09:28 saltnoob58 not really, i'm hoping to do a complicated thing in some simple way without making a new ssh connection several times in every cycle of a for loop
09:28 saltnoob58 and salt doesn't seem to have canonical correct simple ways to do certain things, one might think if you want data from several minions you use the salt mine but this json pipe trick seems better now than somehow preaggregating mine outputs
09:29 Mattch joined #salt
09:30 MTecknology Want to try explaining waht it is you're actually trying to accomplish?
09:31 saltslackbridge <mts-salt> so you want one ssh connection to do all the work. do you have something that can already do that for one minion or do you still need to create that first and worry about how to call it and aggregate that separately
09:32 saltnoob58 collect a bunch of host specific data, think like what's in grains, from a bunch of hosts over ssh. Use data to populate dicts of a certain form ie. create objects. Convert objects and send to ucmdb rest api
09:33 MTecknology so... stick grains into mine, read mine data, done.
09:33 MTecknology easy
09:33 saltslackbridge <ryan.walder> ^
09:34 saltslackbridge <mts-salt> mine can be read by any minion, right? so it depends if there's any secure information in there
09:34 saltslackbridge <ryan.walder> you can use mine to populate pillar
09:34 saltnoob58 if i read mine data via something like mine.get every time i do create an object, like object= {} object['value']= mine.get a thing
09:34 saltslackbridge <ryan.walder> thought it's still avable to the minion i guess
09:34 saltnoob58 it will open a new ssh connection every time i do that
09:34 saltnoob58 and the whole thing will take forever
09:35 * MTecknology blinks
09:35 saltslackbridge <ryan.walder> can you not use the mine to construct the data then just run it once?
09:36 saltnoob58 i'm trying to think of a way to construct the data before running the script that uses it once
09:36 saltslackbridge <ryan.walder> so use the mine to construct a jinja variable then feed that to the command
09:36 MTecknology maybe you should try it before imagining limitations?
09:37 saltnoob58 constructing a giant jinja variable was one of the ways i thought of and wrote in the beginning of my question, but was hoping to avoid it for no real good reason since jinja is a separate confusing thing
09:37 saltslackbridge <mts-salt> how many minions are you controlling via ssh?
09:37 saltnoob58 several thousand targets, not all of them will be actually accessable
09:37 saltnoob58 a discovery scan type of thing
09:38 MTecknology ^ this is why I asked what it is you're actually trying to do (something you didn't answer..)
09:38 saltnoob58 but i did answer what i was actually trying to do
09:38 MTecknology no, you didn't
09:39 saltnoob58 at 11:32
09:39 MTecknology good night!
09:39 saltnoob58 wait, it's 11:32 am local time, i guess everyone has different timestamps
09:40 saltnoob58 time for second coffee
09:40 saltslackbridge <mts-salt> i go back to my previous statement: canyou do what you want, manually, for one minion, using one ssh connection
09:41 saltslackbridge <mts-salt> personally i find working out what it is i'm trying to achieve to be far easier in isolation from the method in which i will end up automating it
09:41 saltslackbridge <mts-salt> put simply: use salt to automate something you've already got
09:42 saltnoob58 for one minion using salt-ssh i can run several execution modules, for example grains.items, use my eye to find the data i want, write it statically into my python thing and create an object out of it and send to ucmdb, yes
09:42 saltslackbridge <mts-salt> so you have a single command that can return the data you want?
09:42 saltnoob58 not a single command
09:42 saltslackbridge <mts-salt> even if it's not in the format you ultimately need
09:42 saltslackbridge <mts-salt> i suggest you start there
09:43 MTecknology ^ +1
09:44 saltnoob58 well the single command would be either writing a custome module that calls several modules, writing an sls that calls several modules, or writing custom grains that call existing other modules and then one command would be grains.items with custom grains
09:44 saltslackbridge <mts-salt> good. you have choices :slightly_smiling_face:
09:44 saltnoob58 and i'm trying to think what this one command should be and how i should be creating it
09:44 saltnoob58 or if my assumed choices are correct and wether im missing some choices
09:45 saltslackbridge <mts-salt> break the problem down and tackle each part in isolation and you'll almost certainly find that you solve your problem along the way without getting stuck on the minutiae
09:47 MTecknology I love how much success with salt depends on breaking each problem into smaller problems that need to be solved as simply as possible.
09:48 saltslackbridge <ryan.walder> same with anything really
09:48 saltslackbridge <mts-salt> i don't have the experience yet to know if my dislike of grains, custom or otherwise, is simply unfounded or not, but i do know that having something that's easy to understand at a glance and thus maintain in the future it significantly more valuable than a technological marvel that no-one, yourself included, will understand at some point in the future
09:48 saltslackbridge <mts-salt> simple is always best
09:49 saltslackbridge <mts-salt> for me, that would be states
09:49 MTecknology there's no reason to dislike grains
09:49 saltnoob58 grains are awesome, they are like a python ohai
09:50 saltnoob58 and if they contained all the things i wanted i could just pipe their json output to python script and just loop through the minions cherry picking which grains i want for what
09:50 saltslackbridge <mts-salt> i think my hesitation comes from the lack of control they provide, as grains are controlled by the minion rather than something like pillar that's controlled by the master. i guess i've yet to come across a problem that benefits from that model
09:50 saltnoob58 but it seems suspicios to stick a different builtin salt module inside a custom grain >D
09:50 MTecknology you're ignoring the original thing I told you which covers what you need
09:51 MTecknology you're clearly the type of person that prefers a complicated solution over something simple and repeatable...
09:51 saltslackbridge <mts-salt> if you don't like using grains to solve your problem then don't use them
09:53 saltslackbridge <mts-salt> thinking about it, this is the first time i'd imagine that a custom formula might be the best solution
09:53 saltslackbridge <mts-salt> (having never used formulas so far, btw)
09:54 MTecknology no formula needed... it's infinitely more simple than than
09:54 MTecknology that*
09:54 saltnoob58 isnt a formula just any sls files and custom_* that you write to achieve a thing?
09:55 heyimawesome joined #salt
09:55 saltslackbridge <mts-salt> has anyone heard the teddy bear story?
09:55 saltslackbridge <mts-salt> (it's relevant)
09:56 saltnoob58_ joined #salt
09:57 oida joined #salt
09:58 saltslackbridge <mts-salt> a professor has open hours where students can discuss problems, but before he'll see them he asks them to explain their problem to a teddy bear that's sitting on a chair outside his office. the students were found to be able to solve their own problems due to having to explain it out loud.
09:58 saltslackbridge <mts-salt> i'm sure you can draw your own moral from this story.
09:59 MTecknology rubber ducky
09:59 hunmonk joined #salt
10:04 mikecmpb_ joined #salt
10:18 colttt joined #salt
10:19 Hybrid joined #salt
10:29 tys_ joined #salt
10:35 Edgan joined #salt
10:46 tys__ joined #salt
10:49 mattfoxxx joined #salt
10:54 tys101010 joined #salt
10:56 mavhq joined #salt
10:56 tys101010 joined #salt
10:59 tys101010 joined #salt
11:11 EthPyth joined #salt
11:18 EthPyth joined #salt
11:39 zerocoolback joined #salt
11:40 notCalle joined #salt
11:41 zerocoolback joined #salt
11:43 mikecmpbll joined #salt
11:52 jrj_ joined #salt
12:08 EvaSDK joined #salt
12:15 colttt joined #salt
12:21 EthPyth joined #salt
12:31 zerocoolback joined #salt
12:38 tiwula joined #salt
12:44 GrisKo joined #salt
12:49 exarkun joined #salt
12:55 EthPyth joined #salt
12:57 tys101010 joined #salt
12:59 sjorge joined #salt
13:02 EthPyth joined #salt
13:05 Guest73 joined #salt
13:08 XenophonF you're the one
13:15 KevinAn2757 joined #salt
13:19 tys101010_ joined #salt
13:20 exarkun https://gist.github.com/exarkun/3b20795b071bbe3af6d7f786f2972760 - Why does the first fail with "State 'mine.send' was not found in SLS 'introducer'" while the second works?  Also, what mine key does the second record the result at?  And can I specify that key somehow?
13:26 Nahual joined #salt
13:27 babilen There is no "mine.send" function in any state module. You typically don't do this directly, but configure mine functions / mine function aliases
13:28 babilen https://docs.saltstack.com/en/latest/topics/mine/ is probably worth a look if you haven't read/found that already
13:30 GnuLxUsr joined #salt
13:31 exarkun Yea... I read that (a few times).  I guess `module.run` does something other than call a function from a state module?
13:31 babilen No, that's exactly what it does
13:32 babilen It's a "state module for the poor" as it lacks logic to determine if it should run or not
13:32 babilen The issue is rather than you wouldn't necessarily use the mine in this way
13:33 exarkun So ... I don't understand why one form works and the other doesn't
13:33 exarkun That may be one issue but it's not the only one :)
13:35 babilen One tries to call the (not existing) "send" function in the "mine" state module (also doesn't exist)
13:36 babilen The other calls execution modules via the "run" function in the "modules" *state" module
13:36 Ricardo1000 joined #salt
13:40 exarkun So "mine" is an "execution" module and not a "state" module and the syntax on line 2 of the gist is for using state modules, not execution modules
13:42 babilen aye
13:42 babilen In salt you almost always have the dichotomy of state and execution modules
13:43 exarkun Okay, maybe I kinda get that.
13:43 babilen Execution modules provide the "actions" that can be taken by state modules (or interactively from the user) and state modules add the logic that determines which (if any) actions have to be executed in order to achieve a certain .. well .. state
13:44 exarkun The idea behind the module.run version is that the contents of introducer.url get changed by the state that's applied immediately before this one.  So ... using module.run here gets the value updated in mine more quickly than it would with time-based polling as done by mine_functions?
13:44 babilen So everything you run with "salt 'foo' bar.baz" is an execution module, while state modules are executed by calling the "state.apply" execution module function (cf. https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.state.html)
13:46 nafg joined #salt
13:49 exarkun What about retrieving the mine value?  Or assigning a custom identifier for it?  It seems like the 2nd example makes the value available as `file.read`?  Which seems like a rather ambiguous and conflict-prone name.
13:50 exarkun Is that what aliases are for?
13:51 babilen Yes, that is what you use mine function aliases for
13:51 babilen When retrieving you pass the minion target expression and the mine function alias to identify your information
13:52 GrisKo joined #salt
13:52 haam3r_ joined #salt
13:55 cyteen joined #salt
13:57 Larry joined #salt
14:02 gh34 joined #salt
14:05 gh34 joined #salt
14:07 haam3r_ joined #salt
14:14 Guest73 joined #salt
14:17 edrocks joined #salt
14:34 saltnoob58_ is there any way to make salt-ssh NOT ask to deploy key to minions where it gets permission denied? just ignore them and not write any prompts at all?
14:45 racooper joined #salt
14:55 glass-wire joined #salt
14:57 hunmonk joined #salt
15:09 glass-wire joined #salt
15:13 KevinAn2757 joined #salt
15:15 notorious joined #salt
15:20 rjc joined #salt
15:20 rjc hi all
15:20 rjc salt beginner here
15:30 cgiroua joined #salt
15:37 viq rjc: welcome to salt :) You have any questions, or just reading and playing with things for now?
15:38 rjc viq: I though I had a question but now I'm trying to formulate a different one ;)
15:38 viq :D
15:38 rjc ok so here it goes: I unserstand architectures in pkgrepo.managed as it does it's job
15:39 rjc I fail to see the reason for dist and comps since I still have to put them in the repo name :/
15:40 saltslackbridge <ryan.walder> they're optional, so just don't include them ;)
15:41 saltslackbridge <ryan.walder> that said i think they get added on the end if they're not specified after the url, it has been a whilesince I last touched that though so it may have changed since
15:42 BitBandit joined #salt
15:44 _JZ_ joined #salt
15:46 kewball joined #salt
15:48 notorious joined #salt
16:04 Naresh joined #salt
16:07 Hybrid joined #salt
16:13 Sacro joined #salt
16:14 Sacro Afternoon all, trying to use salt-ssh to check ssh authkeys, it needs the user passing, anyway to have that automatically filled with the current ssh user?
16:20 saltslackbridge <gtmanfred> you could use sdb:// and the environment to pull the $USER variable from the bash shell
16:20 saltslackbridge <gtmanfred> https://docs.saltstack.com/en/latest/ref/sdb/all/salt.sdb.env.html
16:21 Sacro That is via a state rather than the command line I assume?
16:22 saltslackbridge <gtmanfred> oh yeah if you are on the commandline, you should just be able to pass $USER directly to the argument
16:22 edrocks joined #salt
16:23 Sacro salt-ssh minion ssh.check_key_file "$USER" "salt://pubkeys/salt-ssh.rsa.pub"
16:23 Sacro Returns nothing
16:23 saltslackbridge <gtmanfred> echo $USER and make sure it is actually set? it should be set on all login shells in bash
16:24 saltslackbridge <gtmanfred> and will be your current use on the machine you are running salt-ssh from
16:24 saltslackbridge <gtmanfred> not on the other machine
16:24 saltslackbridge <gtmanfred> that is going to be more difficult
16:24 Sacro Actually, thinking about it it might be that the salt:// isn't getting sent
16:24 Sacro That's a thing with salt-ssh isn't it
16:25 Sacro Yeah, '$USER' works, "$USER" sends root
16:25 saltslackbridge <gtmanfred> well, it would be if ssh.check_key_file actually pulls from the fileserver or not
16:26 saltslackbridge <gtmanfred> it looks like it does, so salt:// /should/ work
16:28 Sacro Ah, getting somewhere, looks like check_key has key, enc, comment rather than enc, key, comment though ><
16:29 Sacro Wonder if a PR to accept things either way would go down well
16:30 schemanic joined #salt
16:32 ct16k joined #salt
16:36 dendazen joined #salt
16:39 viq gtmanfred: how about renaming the bridge to something shorter? It feels like half the line is taken just to inform me that it's someone on slack writing ;) Maybe something like "slackbot" or "on_slack" ?
16:40 Sacro Why not just ditch Freenode and have people IRC directly to slack
16:40 user-and-abuser joined #salt
16:41 saltslackbridge <gtmanfred> sacro if you just pass the key= method, you can have it in whatever order you want
16:42 viq Sacro: https://xkcd.com/1782/
16:42 jkerr joined #salt
16:42 Sacro I think I'm that guy
16:42 Sacro New job, get slack creds, set up irsis
16:42 Sacro *irssi
16:47 Hybrid joined #salt
16:48 viq I'm quite happy with wee-slack actually
16:55 babilen viq: That's great! Thanks for mentioning it
16:55 babilen But I agree .. Saltstack should just ask people to connect to Slack if that's what they want to use
16:56 aldevar left #salt
16:58 viq https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.virt.html#salt.modules.virt.reset - would the argument to it be "vm=XXX" ? Trying to fit it into https://docs.saltstack.com/en/latest/topics/reactor/
16:58 Bryson joined #salt
17:01 babilen vm_ surely
17:01 viq "vm_=XXX" ?
17:02 babilen Not sure why it's not "vm", but yeah
17:02 babilen Might also be able to pass it as positional argument
17:02 saltslackbridge <gtmanfred> because python specifies that all variables need to be at least 3 characters long
17:02 babilen Ah, yeah
17:03 viq Hah, ok.
17:03 viq babilen: not sure how to pass positional arguments to reactors
17:03 babilen Could have just used 'virtualmachine' or something similarly obvious
17:03 gtmanfred yeah, that decision was made before my time
17:03 gtmanfred i don't like it either
17:03 babilen viq: It was more of a "this would be possible" - It's been a long day :)
17:04 viq babilen: yeah, "you probably could", "I'm too stupid to figure out how yet" ;)
17:04 gtmanfred ugh, i am going to have to go find the slack bridge user password and auth to it aren't i
17:05 * viq is setting up a reactor to automatically restart a VM that's constantly hanging when prometheus notices it and hits webhook on salt :)
17:06 babilen I shall call it the "slap hammer"
17:07 onslack joined #salt
17:08 onslack <gtmanfred> better?
17:08 viq babilen: to the tune of https://www.youtube.com/watch?v=hZPECFQ4NhE
17:08 viq gtmanfred: <3
17:13 user-and-abuser joined #salt
17:27 ange joined #salt
17:29 vexati0n so i have to git pillars defined under ext_pillar - both of them have exactly the same config in /etc/salt/master, and both of them have exactly the same directory layout. but one of them works, and the other one gives me 'specified sls is not available on the salt master'
17:29 vexati0n whyyyy
17:30 aldevar joined #salt
17:32 losh joined #salt
17:32 vexati0n is there a way to list all the available pillar files on the master, like you can do with fileserver.file_list for the fileserver?
17:37 DoomPatrol vexati0n: hrm, "saltutil.*"  has something but i can't remember what
17:37 DoomPatrol i had a issue trying to figure out why a weird sls was being rendered and figured out i had a branch in master gitfs (forgot that salt will munge the branches)
17:39 onslack <gtmanfred> i do not believe there is, because the minion should not be able to see any of those files, and I don’t think the master actually has a way to query files on the ext_pillar, since those behave differently than the filesystem
17:40 bbbryson joined #salt
17:42 vexati0n so how do i troubleshoot this issue where two identically laid out repos fail to behave similarly?
17:42 Edgan DoomPatrol: I explicitly tell which branch to use. It is most likely to screw with you over the top.sls.
17:43 DoomPatrol Edgan: yeah i've learned to just not branch in master
17:43 Edgan vexati0n: Are they actually identical or just very very close?
17:43 DoomPatrol vexati0n: salt-call -l debug state.apply $blah (maybe it will demonstrate what's occurring)
17:44 vexati0n no they're exactly the same. they're both very simple. the lay out is /name/init.sls in both cases
17:44 gtmanfred probably not, because the master generates the pillars
17:44 Edgan vexati0n: A diff -uNr one two   would be a quick way to find the differences
17:44 vexati0n they do different things
17:44 vexati0n there are different values, but they're in the same place
17:44 Edgan vexati0n: same pillar name, or just alike in style?
17:44 vexati0n repo is: /pillar/thing/init.sls in both cases. "thing" is a different word, but that hardly matters
17:45 Edgan vexati0n: ok, so top.sls defined such that both are included in the right circumstance?
17:45 vexati0n yes
17:45 Edgan vexati0n: I would double check that
17:45 Edgan vexati0n: That is the most likely problem
17:45 vexati0n ffs
17:46 Edgan vexati0n: Doesn't matter what is in the git repo or on the filesystem if it isn't included
17:46 vexati0n yes, obviously that is correct, because otherwise the minion wouldn't complain about "thing2" not being available on the amster
17:46 Edgan vexati0n: So if the pillar is that, are you saying it has the wrong value?
17:46 Edgan is there
17:47 vexati0n no, the pillar is referenced by top.sls, but the minion can't find it despite the fact that its ext_pillar config matches and the repo's layout matches a different pillar configured exactly the same way, which the minion CAN find.
17:47 nebuchadnezzar joined #salt
17:47 vexati0n thing1 = everthing is good; thing2 = can't find it. both repos are /pillar/thing1/init.sls (ok), and /pillar/thing2/init.sls (can't find it)
17:48 Edgan vexati0n: ok, so you have(making up names) foo and bar, and the minion can see foo, but not bar? Give us examples of the top.sls and pillars. Also what ext_pillar method are you using?
17:49 aldevar joined #salt
17:52 vexati0n https://pastebin.com/nykAKiYD
17:55 Edgan vexati0n: ok, so you are just using gitfs
17:55 vexati0n well these are pillars, but yes
17:55 vexati0n i'm also using gitfs on the fileserver side, and there, the files from the 'bar' repo are available without any problems
17:56 Edgan vexati0n: does the contents of the bar init.sls start with bar:. You can name the files and include whatever you want, and then call the values in the actual yaml something different, like blaz:
17:57 pipps joined #salt
17:57 vexati0n what would the contents of init.sls have to do with it? they are just pillar values that should be included in the minion's pillar data
17:57 vexati0n if there is an init.sls and it's valid yaml, it shouldn't matter what they actual words are
17:57 Edgan vexati0n: Let me give you a example
17:57 vexati0n the contents of the 'foo' init.sls also do not say "foo"
17:58 pipps99 joined #salt
17:59 Edgan vexati0n: https://paste.fedoraproject.org/paste/1m8A8dQZP1nPZiLNkbPVEQ
17:59 Edgan vexati0n: both good and bad will appear in pillars, but bad will give you blaz when you expect bar.
18:00 vexati0n i think you are confused.
18:00 vexati0n the problem is not when i try to retrieve the data from the pillar
18:00 vexati0n the problem is that the pillar file *can't be found in the first place*
18:01 pipps joined #salt
18:02 Edgan vexati0n: You are right that probably isn't your problem, but I am trying to check everything, because it is easy to screw up.
18:03 Edgan vexati0n: what provider are you using for gitfs?
18:03 vexati0n gitlab for both repos
18:04 Edgan vexati0n: no, I mean the python module that manages gitfs
18:04 vexati0n pygit2
18:04 vexati0n which works fine for literally every other gitfs and git_pillar repo in the config (there are more than a few)
18:05 vexati0n but this one particular repo refuses to appear in the pillar data. i assume i'm doing something wrong, but i can't find it.
18:05 Edgan vexati0n: develop for foo vs master for bar, are you sure the branch names are correct?
18:05 vexati0n yes i'm sure, but i also branched the repository and moved the config to develop on bar just to make sure, and it didn't change the result.
18:05 pppingme joined #salt
18:06 Guest73 joined #salt
18:06 jkerr joined #salt
18:06 Edgan vexati0n: tried  salt-call event.fire_master update salt/fileserver/gitfs/update  ?
18:07 vexati0n well, i use salt-run fileserver.update, but that doesn't matter becasue gitfs isn't having the problem, pillar is
18:07 swills joined #salt
18:07 swills joined #salt
18:08 vexati0n either way, yes, it's been run multiple times
18:08 Edgan vexati0n: Are both repos authenticated with the same ssh key, or no authentication?
18:08 vexati0n same ssh key
18:09 Edgan vexati0n: have you manually tried git cloning both with that key, and made sure that was the only key avaliable?
18:09 vexati0n yes dude all the level 1 helpdesk stuff is done
18:09 Edgan vexati0n: If you turn up the log level on the master I would think it would have entries for the pillar git downloads
18:10 Edgan vexati0n: You admit you don't know what it is, so it could be anything, and they are tons of details
18:11 Edgan vexati0n: gitfs stuff is especially opaque, so generally the easiest thing is double check everything
18:11 swills joined #salt
18:11 vexati0n i get that. still. if it was just a typo or an auth key thing, i would have found it
18:11 vexati0n the logs don't show anything helpful for git pillar
18:12 Edgan vexati0n: what log level?
18:12 vexati0n just that the minion asked for it, and then 'oops can't find it'
18:12 vexati0n debug/trace
18:12 Edgan vexati0n: What about when you restart the salt master? It should sync the git repos, and you might see something then. When dealing with the minion it is just going to use it's cache
18:13 vexati0n yes it shows gitfs logs, and like i said, gitfs *works*
18:13 vexati0n gitfs isn't the problem. git_pillar is the problem
18:13 vexati0n two different thigns
18:13 Edgan vexati0n: The on a high level basically do the same thing.
18:14 vexati0n yes but they don't log to the same places. apparently git_pillar doesn't log anything anywhere
18:14 vexati0n they're handled by different routines in salt, too
18:15 vexati0n the only log related to it is error 25230 "specified sls is not available" which i already knew
18:15 Guest73 joined #salt
18:15 vexati0n it's fine, i'll just host the pillar locally instead of using git pillar for it.
18:15 edrocks joined #salt
18:15 Edgan vexati0n: Random thing I just noticed. Your example master_config doesn't have colons on the end of the git urls, but my previously working git pillar example does
18:16 vexati0n yeah i left those off when transcribing, my mistake
18:16 vexati0n they're there in the actual config file
18:16 Edgan vexati0n: ok
18:17 Edgan vexati0n: version of salt?
18:19 vexati0n 2017.7.2
18:22 edrocks joined #salt
18:23 Edgan vexati0n: can you find cached copies of the pillar files on the master's filesystem?
18:24 Edgan vexati0n: if so, I would compare them to the git repo and see if you can learn anything
18:25 Edgan vexati0n: Did you add foo and bar together, or did you add bar after foo, more recently?
18:25 vexati0n bar is more recent
18:25 Edgan vexati0n: Are you sure the salt master got restarted properly on the configuration change?
18:26 vexati0n yes
18:26 Edgan vexati0n: what about the cache files?
18:26 vexati0n the cache files are there, everything matches the repo
18:26 vexati0n everything is in the same layout between both.
18:27 vexati0n it's fine tho, i just gave up on hosting pillar in git. i'll keep it local on the master until whoever writes the git_pillar stuff decides to log something so i can understand it better
18:27 vexati0n gitfs still works as advertised and that's more important for me
18:28 Edgan vexati0n: I just looked back at my working config, and I only had one repo. So maybe it doesn't support mixing repos properly
18:28 vexati0n according to the docs, that's a primary function
18:28 Edgan vexati0n: Bugs never happen in salt :)
18:29 vexati0n i doubt i'd be the only person to find that one
18:29 Edgan vexati0n: I was just searching github
18:29 Edgan vexati0n: and found your ticket
18:29 MTecknology what's the tl;dr version of the problem?
18:31 vexati0n i have 2 git repositories, both with the same folder structure, both defined the same way in /etc/salt/master under ext_pillar, but only one of them works. the other one gives me 'specified sls is not available on the salt master'
18:31 vexati0n they're both also defined under gitfs, and there, both of them work fine
18:31 vexati0n but only one's pillar data is available
18:32 Edgan vexati0n: This is interesting, but probably doesn't apply, https://github.com/saltstack/salt/issues/45422
18:32 MTecknology did you check teh master log to make sure it's pulling from both repos without error?
18:32 MTecknology the*
18:32 Edgan vexati0n: oh, I have an idea
18:32 vexati0n yes, it pulls from both of them without a problem. or at least, it does for gitfs, git_pillar doesn't really log anything
18:33 Edgan vexati0n: which repo is the top.sls in?
18:33 vexati0n top.sls is local to the master because salt loses its mind over top files
18:33 vexati0n so it's safer just to keep it in one place
18:33 onslack <gtmanfred> oh you can’t do that for pillar
18:33 onslack <gtmanfred> because the ext_pillar is completely seperate
18:33 onslack <gtmanfred> it has to be in the external pillar git repo
18:34 Edgan vexati0n: so you are actually mixing pillar and ext_pillar
18:34 onslack <gtmanfred> it operates differently than gitfs
18:34 vexati0n meh
18:34 vexati0n see and i knew that from the last time i complained about this
18:34 gtmanfred yeah, the prbolem is that gitfs and git_pillar operate differently on that part
18:34 gtmanfred mostly because the pillar system is just way different
18:34 gtmanfred for gitfs, salt can just pull files down from the different fileserver_backends, using salt://
18:35 gtmanfred ext pillars are all completely seperate and have no crossover unfortunately
18:35 gtmanfred so top.sls in pillar_roots knows nothing about git_pillar, and a top.sls in git_pillar knows nothing about pillar roots or any other ext git_pillar
18:35 Edgan vexati0n: I was going to say it if was in foo, and not bar, it might be doing a weird I only see pillars in my repo of the repo that defined the top, and the solution might have been to define it in both.
18:35 gtmanfred there is no way to cross call them
18:36 Edgan gtmanfred: Is there a merging strategy option for git pillars?
18:36 gtmanfred no
18:36 gtmanfred well
18:36 gtmanfred it merges all the pillars together
18:36 gtmanfred but when top references them, it does not do any top.sls merging
18:36 Edgan gtmanfred: it merges at the yaml level?
18:36 gtmanfred no
18:36 gtmanfred it merges at the python dictionary level
18:37 gtmanfred because the ext_pillar returns a dictionary
18:37 gtmanfred and then that is merged into the pillar dictionary
18:37 gtmanfred if you have ext_pillar_First: True, then all ext pillars are rendered first, and then the pillar_roots gets merged in
18:37 gtmanfred but by default it does the other way
18:37 Edgan gtmanfred: I think vexati0n needs to know how that works intimately to figure out how to make his situtation work
18:38 Guest73 joined #salt
18:38 vexati0n no, i get it now. there has to be a top.sls in each -git group
18:38 vexati0n under ext_pillar
18:38 gtmanfred yeah
18:38 vexati0n the master's local top.sls can't reference git pillars
18:38 babilen gtmanfred: Thanks for looking into https://github.com/saltstack/salt/issues/43340 -- pity that it gets reintroduced every other quarter. How will it be fixed this time?
18:38 * babilen had really hoped to be able to deploy .3, but I guess I have to wait for .4 now
18:38 gtmanfred it should def be fixed and i will fight someone if try try to remove it again
18:39 vexati0n they keep removing the fix because it interferes with something else, i thought
18:39 gtmanfred babilen: because of the way that the services update, you have to put the KillMOde file down first, before it can be fixed.
18:39 Edgan babilen: build your own packages :)
18:39 gtmanfred so you can either update to 2017.7.3 now, and it will update cleanly in 2017.7.4, or you can put the KillMode.conf file down, and it will upgrade cleanly for any version
18:39 babilen vexati0n: The problem is rather that people see something in salt-master's unit file and blindly copy it to salt-minion
18:41 babilen https://github.com/saltstack/salt/pull/36806 triggered the last round
18:42 gtmanfred there are so many of these things, i can't keep track, but i will fight someone if they try to change it again, because process is correct
18:42 gtmanfred all it does is cause problems for me on twitter and when triaging issue
18:42 babilen Thank you
18:42 babilen I'll weep if I run into this again
18:42 gtmanfred heh
18:43 gtmanfred i wish i could figure out how to fix it in upstart and sysvinit, but those only have like 2 more years, so hopefully i won't have to
18:43 zer0def joined #salt
18:43 Edgan babilen: what distro/release are you using?
18:44 babilen We are primarily on Debian .. mostly stretch, but some jessie and wheezy boxes
18:45 Edgan babilen: not that familiar with Debian, stretch is systemd or upstart?
18:45 gtmanfred 8 was systemd
18:45 babilen Edgan: I collected links to discussions of this in https://github.com/saltstack/salt/issues/43340#issuecomment-329760470
18:45 gtmanfred i think 7 was too
18:46 babilen wheezy/7 is sysvinit
18:46 gtmanfred oh right
18:46 gtmanfred but you could put systemd on it can't you?
18:46 MTecknology Debian 9 was the transition to systemd
18:47 gtmanfred no it wasn't, debian 8 has it too
18:47 babilen gtmanfred: Absolutely .. not that you'd really want to use the version of systemd in wheezy
18:47 MTecknology it was available, but it wasn't the default
18:47 gtmanfred babilen: heh, yeah fair
18:47 gtmanfred https://wiki.debian.org/systemd
18:47 gtmanfred systemd is a system and service manager for Linux. It is the default init system for Debian since DebianJessie. Systemd is compatible with SysV and LSB init scripts. It can work as a drop-in replacement for sysvinit. Systemd
18:47 gtmanfred jessie is debian 8
18:48 pipps joined #salt
18:48 babilen Yeah, definitely jessie
18:48 MTecknology Have we seriously gone through a full release cycle since that debate?
18:48 gtmanfred yes
18:49 babilen I vividly remember the transition and flamewars in #debian and #debian-next :)
18:49 * MTecknology feels old
18:49 gtmanfred heh yeah
18:49 gtmanfred i was just starting my first real job when that argument happened
18:49 lxsameer joined #salt
18:49 gtmanfred that was a good time
18:51 MTecknology OH!! I remember the release!!
18:51 pipps99 joined #salt
18:52 MTecknology I was balls deep in gitea packaging and hadn't taken the time to breath in a few months.
18:52 MTecknology what a horrible/terrible init system :(
18:54 gtmanfred better than runit
18:54 gtmanfred or as my favorite name for runit, ruinit
18:57 MTecknology runit isn't really an init system, is it?
18:57 gtmanfred yeah it is
18:57 Edgan systemd has had it's growing pains, but I like it better than sysvinit and upstart. I love it over the mess that is supervisord, monit, pick your random tool.
18:57 gtmanfred i used to use it back when arch switched to systemd
18:57 gtmanfred and i was still running in an openvz container
18:58 gtmanfred sysvinit is awful, pidof was a terrible idea
18:58 gtmanfred and disabling services in upstart was a nightmare
18:58 gtmanfred why do i have to touch a file to delete it, just give me a utility
18:58 gtmanfred MTecknology: https://wiki.archlinux.org/index.php/runit
18:58 MTecknology I don't have an issue with sysV
18:58 gtmanfred `It includes runit-init, which can replace sysv's init as pid1`
18:58 babilen The scripts are horrible and buggy
18:59 babilen (sysvinit)
18:59 gtmanfred yeah
18:59 MTecknology If you can't copy/paste decent scripts, ya..
18:59 Edgan bash scripts that are all written differently and are hundreds of lines of code vs a clear .service file.
18:59 Edgan babilen: yes
18:59 gtmanfred just look at the salt sysvinit script
18:59 gtmanfred the things people do in there are terrible
19:00 Edgan gtmanfred: yeah, pretty much any non-distro software company giving you an init script is bad
19:00 gtmanfred https://github.com/saltstack/salt/blob/2017.7/pkg/rpm/salt-minion#L150
19:00 babilen I'm also fond of MySQL/MariaDB scripts
19:00 MTecknology I won't argue sysV wasn't old and needed to go... but I do argue that sysD is a terrible replacement option.
19:01 Edgan MTecknology: Are we just talking the init part of systemd, or the whole ball of wax?
19:01 MTecknology the whole ball of wax is part of what bugs me.
19:02 MTecknology It could potentially be a decent init system (it's not), but they spend all their time half-implementing garbage replacements of standardized/effective tools
19:02 Edgan MTecknology: I agree with you there. On just the init part, Lennart needs to be beaten for some of the issues/bugs along the way, but I still think it is better.
19:03 Edgan On the whole ball, they are just in a massive lets rewrite because we can, and NIH. But this is a huge problem with code these days. No one wants to touch anything of size that someone else wrote.
19:04 MTecknology good? leave it to the people that bother learning how things work?
19:05 gtmanfred all i want is for systemd to absorb wayland, so that it can be a default everywhere too
19:06 MTecknology didn't wayland die?
19:06 gtmanfred /s incase that wasn't obvious
19:06 gtmanfred it did not
19:07 gtmanfred gnome and kde both support wayland
19:07 gtmanfred and there are people only running in wayland now on distros like arch and fedora
19:09 Jsmith_ joined #salt
19:10 gtmanfred ok, time for me to start mocking data for unit tests
19:11 MTecknology is there any point to doing grains.get('id') vs. just grains.id?
19:13 jsmith0012 joined #salt
19:13 onslack <gtmanfred> not if you aren’t setting defaults
19:13 pipps joined #salt
19:18 pualj joined #salt
19:19 ymasson joined #salt
19:20 pipps99 joined #salt
19:21 swa_work joined #salt
19:25 jsmith0012 HI, can someone help with a question on folder states?  is there a way to set the sticky bit via state?
19:26 MTecknology file mode
19:26 lordcirth_work jsmith0012, the -mode argument to file.directory will set them
19:28 jsmith0012 ahh i think i see what is confusing me.  i dont use octal very often.  so - mode 1660 for example should work?
19:29 lordcirth_work You normally want directories to have the executable bits set to allow traversal
19:29 DammitJim joined #salt
19:30 lordcirth_work So - mode: 1770
19:30 MTecknology You should learn to use only octal! :D
19:31 jsmith0012 yea i now but it so easy to g+s -R on a dir and be done :)
19:32 whytewolf actually ... that whole g+s and the other none octal commands confuse the hell out of me
19:32 whytewolf esp when i use masks
19:33 lordcirth_work Yeah, the letters are maybe easier but diffs instead of absolute set is confusing
19:34 jsmith0012 i am somewhat new to linux development. but seriously thank you all!
19:34 aldevar joined #salt
19:35 lordcirth_work jsmith0012, this website is very useful: https://linuxjourney.com/
19:37 MTecknology hm.. is it possible to have a directory in a pillar path that's just an underscore?  " - foo._.bar" -> foo/_/bar.sls
19:41 lordcirth_work MTecknology, It ought to be, it's just a string.  Why tho?
19:42 MTecknology testing a theory... I'll share soon.
20:04 cyteen joined #salt
20:04 vexati0n ughh
20:10 MTecknology lordcirth_work: I'm trying to stick my magic-sauce jinja files neatly away in a sub-directory so that the parent directory doesn't confuse anyone with extra sls files.
20:10 lordcirth_work MTecknology, then why not call it 'jinja'?
20:11 MTecknology I thought about _jinja or _macro or _magic or _keepout or _voodo or _v
20:12 bluenemo joined #salt
20:17 lordcirth_work 2017.7.3 has two of my PRs woot
20:19 viq _here_be_dragons_ye_who_enter_abandon_all_hope
20:21 MTecknology I have a commit message that made it to release notes about cthulu taking mercy on my soul.
20:22 lordcirth_work I remember seeing that mentioned here, lol
20:23 Hybrid joined #salt
20:26 cyteen joined #salt
20:26 viq whee, my reactor to restart a hanging machine works! \o/
20:26 peters-tx Yay, happy to see 2017.7.3
20:37 MTecknology lordcirth_work: s,::,/, - https://gist.github.com/MTecknology/082df07516ebc691722bd701da8a314f
20:40 pipps joined #salt
20:41 lordcirth_work MTecknology, personally I'd give it a more descriptive name
20:42 MTecknology _sauce?
20:45 xet7_ joined #salt
20:46 xet7_ joined #salt
20:48 pipps joined #salt
20:50 MTecknology lordcirth_work: _init seems logical
20:51 MTecknology still less pretty in top.sls
20:54 user-and-abuser joined #salt
20:58 Guest73 joined #salt
21:01 el_wood_le joined #salt
21:02 MTecknology lordcirth_work: I added two files for what this ends up looking like - I kinda like it. :)   https://gist.github.com/MTecknology/082df07516ebc691722bd701da8a314f#file-1_console
21:06 sjorge joined #salt
21:09 Guest73 joined #salt
21:12 pipps joined #salt
21:14 pipps99 joined #salt
21:18 Guest73 joined #salt
21:23 aldevar left #salt
21:31 ct16k joined #salt
21:38 systemexit joined #salt
21:41 MTecknology lordcirth_work: btw- I'm replacing a super-duper massive and monolithic file of *ALL* data available to *ALL* minions and a custom module used to parse that pile and pick out data relevant to the node.
21:48 hemebond MTecknology: Is that something you took over or something you created?
21:48 MTecknology $client
21:49 hemebond s2bu
21:49 hemebond Good luck :-)
21:50 MTecknology $client_boss finally took off so now I'm free to make useful changes ... I already wedged this test in place to see if it'd work.
22:01 shakalaka joined #salt
22:03 user-and-abuser joined #salt
22:12 schasi joined #salt
22:14 schasi I currently have salt py27-salt_2017.7.2_1 installed on FreeBSD 11.1. For some reason, I can start neither minion nor master, they both hang when trying to start them and consume 100% cpu. Anyone any idea why that could be the case?
22:17 exarkun How long did you wait?
22:23 pipps joined #salt
22:27 MTecknology check strace?
22:30 hunmonk joined #salt
22:31 hunmonk is there a way, from a state file, to run a shell command in the currently running shell of a running highstate? IIRC, cmd.run runs in a separate shell, which won't work for my situation
22:32 schasi I waited for 30 mins approximately
22:32 schasi MTecknology: I shall check strace, I guess
22:32 wwalker is there a way to run a single state on all the boxes that would run it if highstated?  I have about 30 states that need to be run only on the machines that top.sls would run them for, and we can't safely run highstate on a lot of them (I know...we're working on that problem too....)
22:32 exarkun hunmonk: What makes you think the highstate is running a shell?
22:33 MTecknology schasi: could also check -l debug
22:34 hunmonk exarkun: i can ask a better question. I need npm.installed to find the npm binary installed via nvm. nvm uses a shell command to active it in the current shell
22:34 hunmonk exarkun: so how do i replicate that in a highstate?
22:36 exarkun hunmonk: You could cmd.run both the nvm activation and an npm command.  Then you are controlling one shell that has all of the other programs you need run in one shell run in it.  I don't know if that's the best solution, or even a good one.  I've been using salt for about a week.
22:36 hunmonk exarkun: what i can tell you is that if nvm is activated in a shell which i then use to run a highstate, npm.installed works. but this fails if i'm installing npm from another state first in the same highstate
22:36 hunmonk oh
22:37 theologian joined #salt
22:37 hunmonk exarkun: manually running those via cmd.run kind of defeats the purpose of the state commands
22:37 hunmonk exarkun: i only use that approach as a last resort
22:38 exarkun hunmonk: yes, a bit.  When you say "run a highstate", you mean you run the `salt target state.apply` command?  And the nvm environment variables set in the shell of that command make a difference to how npm.installed behaves?
22:39 hunmonk exarkun: yes, the npm module probably does some kind of detection to find the npm binary
22:39 hunmonk exarkun: which it can find if nvm has loaded itself in already
22:39 user-and-abuser joined #salt
22:40 hunmonk exarkun: but when i'm also installing nvm then npm via the same set of states, she no worky
22:40 exarkun I am very surprised by that behavior.  It sounds absolutely terrible.
22:40 theologian hello, I'm looking for a good "best practice" for file structure for environments and one offs. https://docs.saltstack.com/en/latest/topics/best_practices.html
22:41 exarkun hunmonk: I guess I don't have any better answers for you, anyhow.  I haven't used the npm module so I don't know if it has any specific tricks for dealing with this.
22:41 hunmonk exarkun: i really don't know for sure how the npm module works to find the binary, i haven't dug into it
22:41 hunmonk exarkun: could be a simple as using $PATH. which the nvm loading command takes care of fixing…
22:41 schemanic joined #salt
22:42 exarkun It strikes me as wrong that modules would alter their behavior based on the environment of the `salt` caller like this.
22:42 exarkun It seems to me that modules should have well-defined behavior that I can count on being the same however and wherever I deploy them.
22:42 hunmonk exarkun: well the npm module does its work via the npm binary
22:42 hunmonk it has to be able to find it
22:43 exarkun Otherwise, it's not a repeatable system.  And if it's not repeatable, I might as well just manually type out the commands each time.
22:44 hunmonk exarkun: if i installed npm via a package manager, then it would probably work. but that gives me an ancient version of node.js
22:44 exarkun hunmonk: I'd _hope_ that there would be some way to tell it what to use in the state module.  Then the desired behavior is encoded and the state is actually fully defined and you can have repeatable operations.
22:45 hunmonk exarkun: it would be acceptable if the npm module allowed specifying a path to the binary you wanted to use, but i didn't see that option in the doc
22:46 pipps99 joined #salt
22:47 hunmonk exarkun: http://ix.io/FrD
22:47 hunmonk there ya go
22:49 exarkun Terrible
22:49 * exarkun subtracts a point from salt
22:50 exarkun hunmonk: I guess you could submit a patch adding the option to find npm with nvm or with a hard-coded path instead
22:52 hunmonk exarkun: salt.utils.which seems to be a standard method for binary discover in salt
22:52 hunmonk s/discover/discovery
22:53 exarkun Yea that makes me even sadder
22:53 exarkun Probably worth subtracting a few points
22:53 hunmonk i'm not really sure of a better way to do it though
22:53 exarkun Surely that code must run on the minion and not the master, though?
22:53 hunmonk a lot of state modules rely on system binaries to do their work
22:54 hunmonk i would think it has to run on the minion
22:54 hunmonk that's where the work is being done
22:54 exarkun Sure.  All that's needed to fix the problem, though, is for those system binaries to be properly specified instead of discovered from incidental external state.
22:54 exarkun Yea, that's what would make sense to me.  But then, why would the PATH of the `salt` command make a difference?
22:55 hunmonk exarkun: i'm running it directly on the minion at the moment
22:55 exarkun But still, why wouldn't it be the PATH of `salt-minion` that matters?
22:56 exarkun Is the `salt` program itself doing all the work, not giving the minion a command?
22:56 whytewolf hunmonk: salt-call?
22:56 exarkun If this is only a problem for direct application of commands on minions without the agent ... then maybe it's not as eggregious.
22:56 hunmonk exarkun: no, i think it's the PATH of the current shell that salt/salt-call is executed from
22:56 hunmonk whytewolf: yes, salt-call
22:56 whytewolf salt-call runs the command with out touching the running daemon
22:56 hunmonk exarkun: sorry, that wouldn't apply to salt, just salt-call
22:57 exarkun okay...
22:57 exarkun still kind of bad, because you think you're applying the same configuration but you're actually not
22:57 hunmonk whytewolf: i don't know if that matters. what's at issue here is how salt.utils.which works
22:57 exarkun but not as bad, because at least if you just use the agent you shouldn't run into this kind of thin
22:58 exarkun g
22:58 whytewolf well, salt-call would also have the enviroment of the user running the command instead of the enviroment the daemon has
22:58 oida joined #salt
22:59 hunmonk whytewolf: at the end of the day, i need *whatever* environment salt is running under to be able to load the nvm environment in
22:59 hunmonk otherwise npm.installed can't find the binary
22:59 exarkun But it doesn't need to be able to find it with no explicit direction
22:59 exarkun It could very easily require that you tell it which npm you want it to use.
23:00 exarkun Instead, it guesses from unstable hints and produces non-repeatable behavior.
23:00 hunmonk ah, it looks like i can maybe use the env setting of npm.installed to adjust the path? "A list of environment variables to be set prior to execution. The format is the same as the https://docs.saltstack.com/en/latest/ref/states/all/salt.states.cmd.html#salt.states.cmd.run. state function."
23:02 hunmonk that still feels pretty clunky though
23:05 exarkun yea.  presumably you would like something like `nvm: ...` (I don't know how nvm works, I don't know what argument you would pass it, but probably something like the args you passed on the cli)
23:05 exarkun or some nested states, where you have an `nvm.something` state that you configure and can also tell to do some npm stuff with that configuration
23:06 hunmonk i give up for now. sometimes a break results in an inspiration :)
23:09 onslack joined #salt
23:27 masber joined #salt
23:29 pipps joined #salt
23:30 schasi I did a strace (or rather a "truss") on my salt-minion running on FreeBSD: For some reasons it stats "/etc/nsswitch" all the time. I have no idea why :)
23:33 _xor joined #salt
23:33 schasi "/etc/nsswitch.conf" I guess
23:34 pipps joined #salt
23:35 Edgan schasi: don't see it in the code, only one comment reference
23:35 Edgan schasi: Could be some module it depends on
23:37 fxhp joined #salt
23:37 user-and-abuser joined #salt
23:37 schasi Edgan: I think it references something that references something that calls it
23:38 schasi Apparently it's very common to stat /etc/nsswitch.conf. I don't understand why it does it so often though.
23:39 Eugene nsswith.conf is used by libc for figuring out all of hte nsswitch things: most commonly UIDs & GIDs
23:40 Eugene http://www.gnu.org/software/libc/manual/html_node/NSS-Basics.html#NSS-Basics
23:40 Eugene Its kind of a core piece of UNIX system configuration in networked environments, and totally unsurprising to see in strace
23:41 exarkun Yea, re-reading nsswitch.conf isn't much of an issue.  If you don't see it doing much else, you might want the equivalent of ltrace instead.
23:41 exarkun ie, it's spending time in library code, not making syscalls.
23:44 schasi The point is that it reads it like 100 times per second
23:45 schasi VERY very often. And that is probably the reason why salt-minion hovers around 100% cpu usage
23:45 exarkun 100 isn't that many
23:45 Eugene Seconds are really long
23:45 exarkun It's checking it because of some _other_ operation.  It's probably the other operation, or the fact that it's stuck in a loop doing it, that is causing the problem.
23:46 exarkun Like, it's stuck in a connect() loop that's constantly failing fast and the real problem is something to do with the network config (just an example of a possibility, of course I have no idea what's really broken)
23:47 exarkun (actually I know it's not connect() because you would have seen that in truss output (aiui))
23:47 schasi exarkun: I agree. So I guess I'll have to find a FreeBSD ltrace alternative
23:47 exarkun you could also look at the python stack w/ gdb and the python gdb extension, probably
23:48 exarkun that might be the most informative
23:48 whytewolf how about running the application with -l all
23:48 whytewolf see where salt stops responding
23:49 schasi Huh. I've only run it with "-l debug" so far. Is "-l all" more verbose even?
23:49 whytewolf all is as verbose as it gets.
23:50 whytewolf it includes trace, and garbage.
23:50 whytewolf https://docs.saltstack.com/en/latest/ref/configuration/logging/
23:54 yidhra joined #salt
23:56 schasi Thanks for all the ideas. I actually retired for the night, but know where to pick up tomorrow :)
23:56 bowhunter joined #salt
23:59 yidhra joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary