Perl 6 - the future is here, just unevenly distributed

IRC log for #salt, 2018-05-11

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:00 onslack joined #salt
00:01 Edgan hemebond: ah, logins not spam.
00:01 Edgan hemebond: So you really want to rewrite fail2ban based on centralized logs
00:01 Edgan s/rewrite/replace/g
00:02 hemebond I'm going to tell fail2ban, across all public servers, to ban IPs based on the centralised logs.
00:03 Edgan hemebond: You can feed it a file during runtime, or have to restart it?
00:03 hemebond fail2ban? You can update at runtime.
00:03 Edgan You mean on start?
00:03 Edgan or during runtime?
00:04 hemebond During runtime
00:04 Edgan You want to keep using it for the auto-expire?
00:05 hemebond Yeah, I'll still have then auto-expire.
00:05 * hemebond will brb
00:06 Edgan ok, so replace the majority of it's functionality, and leave it for auto-expire. Has no one written another implementation of fail2ban that is closer to what you are trying to do?
00:06 hemebond I could probably do it manually using iptables.
00:08 Edgan hemebond: yeah, that came to mind, but then you have to write the auto-expire. I would google/github search and see if someone has already done something like this
00:08 hemebond but then wouldn't have the expiry
00:08 hemebond I don't think I'm replacing that much functionality.
00:08 hemebond I'm still going to be calling fail2ban with the IP.
00:09 hemebond All I'm not using is the fail2ban regex of logs.
00:16 hrumph_ hi
00:16 hrumph_ it seems that pillar.get won't work in the jinja of a pillar definition
00:16 hrumph_ probably makes sense
00:16 Eugene My 2c replacement of fail2ban: iptables -A INPUT -m tcp -p tcp --dport 22 -m hashlimit --hashlimit-name ssh --hashlimit-upto 5/minute --hashlimit-mode srcip --hashlimit-srcmask 24 -j ACCEPT
00:16 hrumph_ sorry thinking out loud
00:17 Eugene Or, just don't allow SSH from 0.0.0.0/0 ;-)
00:17 hrumph_ it's true that the pillar data will all be reset at that point?
00:26 _xor joined #salt
00:36 hemebond hrumph_: Correct, do not try and use pillar.get within a pillar.
00:37 hemebond No wait, the pillar will not be reset.
00:37 hemebond But don't try to reference pillars within pillars; you will get strange results.
00:38 hemebond Eugene: iptables has an expiry mechanism?
00:39 Eugene hashlimit does
00:39 hemebond oh it's just a limit
00:39 Eugene Most spambots go away pretty quickly if you don't ACK their SYN ;-)
00:41 Eugene If you want to "ban" IPs, the correct way to do it is to monitor netflows & report them to the Abuse contact for their IP space. However, most Chinese & Russian ISPs care about abuse, so you're wasting time fighting the problem at all. Also, fail2ban doesnt speak IPv6 last i looked.
00:43 Eugene don't care*
00:46 zerocoolback joined #salt
00:47 Heartsbane joined #salt
00:48 iggy sshguard supports ipv6... not sure how well you can signal it from a remote process (I think it's pretty logfile centric)
01:00 tom[] joined #salt
01:03 hemebond joined #salt
01:09 dendazen joined #salt
01:15 Corey joined #salt
01:28 rideh joined #salt
01:41 hemebond left #salt
01:57 ilbot3 joined #salt
01:57 Topic for #salt is now Welcome to #salt! <+> Latest Versions: 2017.7.5, 2018.3.0 <+> Support: https://www.saltstack.com/support/ <+> Logs: http://irclog.perlgeek.de/salt/ <+> Paste: https://gist.github.com/ <+> See also: #salt-devel, #salt-offtopic, and https://saltstackcommunity.herokuapp.com (for slack) <+> We are volunteers and may not have immediate answers
02:06 shiranaihito joined #salt
02:15 hrumph_ how can i target based on the a pillar entry being other than None?
02:26 MTecknology because options?
02:26 MTecknology https://docs.saltstack.com/en/latest/topics/targeting/
02:28 zerocoolback joined #salt
02:37 hrumph_ well i'm not sure if it should be pillar_path:* or what
02:38 hrumph_ i'm trying that this second
03:01 XenophonF OK so PKCS#12 files are binary, and even though that's getting read into the Pillar data just fine, file.managed chokes on it with a UnicodeDecodeError
03:01 XenophonF so... now what
03:01 XenophonF I could base64-encode the PKCS#12 file, then use certutil to base64-decode it, I guess.
03:05 Psi-Jack I'm trying to follow this example, https://docs.saltstack.com/en/latest/ref/states/all/salt.states.x509.html, but it is failing to create the CA certificate itself with an exception traceback.
03:06 Psi-Jack https://paste.linux-help.org/view/f742d22a is the exception.
03:07 hemebond Psi-Jack: Did you copy the example verbatim?
03:08 Psi-Jack Other than making a state directory, internal-ca, and putting ca.sls, cert.sls, etc in there, mostly, changing the certificate parts up to match what I'm making.
03:09 Psi-Jack Also seems I missed keyUsage. heh
03:15 Psi-Jack https://paste.linux-help.org/view/43da4b97 this is what I have for the ca.sls
03:20 Psi-Jack Going to try splitting up the ca.key anc ca.crt, which worked partially.
03:20 Psi-Jack I just put the wrong require on the ca.crt, heh. Now it's working.
03:21 Psi-Jack So, either the documentation is wrong... Or this is a bug, per issue 39608
03:25 Psi-Jack So, now, just trying to get the cert.sls to work from that example. That's a different issue, and its likely because I didn't put ca.sls and cert.sls in the base, but put it in a subdir, internal-ca.
03:25 Psi-Jack Rendering SLS 'base:internal-ca.cert' failed: Jinja variable 'dict object' has no attribute 'ca'
03:27 XenophonF maybe I'd be better of deploying OpenSSL or a bit of PowerShell/.NET code to generate the PKCS#12 files client-side
03:27 XenophonF s/better of/better off/
03:29 hemebond XenophonF: Spoke to someone last night about certs in Windows domains and he said it'd take about 15-20 mins to setup a CA and push it out. Of course, that assumes a you want a self-signed root cert.
03:33 hemebond Psi-Jack: Your paste doesn't have internal-ca.cert in it.
03:37 Psi-Jack Heh, I just got it working.
03:37 Psi-Jack The 'ca' just needed to be changed to 'salt.home.ld' in both parts. I understand it now. :)
03:38 Psi-Jack Took a little bit to understand how mine stuff works, which I'd never worked with before.
03:38 XenophonF I've run Windows Certificate Services before.  I'd really rather not maintain an internal CA.
03:38 * Psi-Jack cringes at the mention of Windows.
03:39 XenophonF I have the certs from Let's Encrypt sitting on my Salt Master.  If I can get the PKCS#12 files onto the Windows clients, I can load them using certutil.exe easily enough.
03:40 XenophonF Salt is, astonishingly, not 8-bit-clean.
03:41 XenophonF or at least contents_pillar isn't
03:43 Psi-Jack Heh, now to figure out how to get subjectAltName working. :)
03:44 XenophonF wow that x509 state is something
03:44 Psi-Jack Indeed.
03:45 Psi-Jack I was close to working with cfssl, but if salt can do all this...
03:46 Psi-Jack I needed a decent internal CA for use with Vault that doesn't rely on external 3rd parties like Let's Encrypt.
03:48 Psi-Jack And because I want to make proper use of consul's methods. active.vault.service.dc1.consul, so internal-CA.
03:51 Psi-Jack Now, since using gitfs, is there a way to quickly get the salt-master to refresh the git cache?
03:52 Psi-Jack ahh, salt-run fileserver.update
03:52 cgiroua joined #salt
03:54 XenophonF you got it
03:57 Psi-Jack All I got left now is a certificate to be issued. heh
04:00 XenophonF OK different approach might work - using a Jinja template instead of contents_pillar
04:00 XenophonF hemebond: here's the abomination ;) https://github.com/irtnog/salt-states/tree/development/wincert
04:01 XenophonF fingers crossed: setting test=False ;)
04:03 XenophonF nope - the file's still getting corrupted somewhere, so let's run it through base64
04:07 Psi-Jack https://paste.linux-help.org/view/ec3c6a67  -- This is where I am stuck. A big exception stack on the certificate generation/signing.
04:07 Psi-Jack I included the hv.sls (and it's included init.sls)
04:07 _xor joined #salt
04:14 dxiri joined #salt
04:14 hemebond XenophonF: Yikes.
04:14 zerocoolback joined #salt
04:16 XenophonF Psi-Jack: the state docs skip the private key step
04:16 XenophonF see the last example labeled /srv/salt/www.sls
04:16 Deliant joined #salt
04:17 Psi-Jack Hmm, okay.
04:17 XenophonF see the managed_private_key flag?
04:17 Psi-Jack Yes. That had a totally different error. heh
04:19 esharpmajor joined #salt
04:19 XenophonF LOL
04:19 XenophonF sorry
04:21 zerocoolback joined #salt
04:23 Psi-Jack I'll have a paste in a sec, if this fails again.
04:24 Psi-Jack https://paste.linux-help.org/view/efac290c
04:32 Psi-Jack Ahh, managed_private_key had a list.
04:33 Psi-Jack YES :D
04:34 Psi-Jack Now, subjectAltNames..
04:39 Psi-Jack Suwheeeeet.
04:39 Psi-Jack Yep.. This.. Rocks.
04:52 hiroshi joined #salt
05:23 sauvin joined #salt
05:29 Eugene joined #salt
05:31 orichards joined #salt
05:40 c4rc4s joined #salt
05:46 noobiedubie joined #salt
06:04 Deliant joined #salt
06:32 _jurgen_ joined #salt
06:39 briner_ joined #salt
06:40 DanyC joined #salt
06:42 CrummyGummy_ joined #salt
06:42 v12aml joined #salt
06:53 zerocoolback joined #salt
06:58 DanyC joined #salt
07:00 Hybrid joined #salt
07:01 DanyC joined #salt
07:03 awerner joined #salt
07:04 Xenophon1 joined #salt
07:06 JacobsLadd3r joined #salt
07:10 awerner_ joined #salt
07:11 kiorky joined #salt
07:13 tom[] joined #salt
07:24 jrenner joined #salt
07:25 tom[] joined #salt
07:27 orichards joined #salt
07:30 Pjusur joined #salt
07:35 xet7 joined #salt
07:51 mikecmpbll joined #salt
07:59 Waples_ joined #salt
08:04 snath joined #salt
08:12 xet7 joined #salt
08:13 Ricardo1000 joined #salt
08:15 oida joined #salt
08:28 briner_ joined #salt
08:32 awerner_ joined #salt
08:33 briner joined #salt
08:34 awerner__ joined #salt
08:37 briner joined #salt
08:38 arlyon joined #salt
08:41 uncool_ left #salt
08:41 uncool joined #salt
08:46 DanyC joined #salt
08:50 wryfi joined #salt
09:03 __peke__ joined #salt
09:18 xet7 joined #salt
09:23 dankolbrs joined #salt
09:24 McNinja joined #salt
09:24 __peke__ joined #salt
09:32 evle2 joined #salt
09:39 colegatron joined #salt
09:39 inad922 joined #salt
09:51 xet7 joined #salt
10:07 gmoro joined #salt
10:58 sjohnsen joined #salt
10:58 sjorge joined #salt
11:26 noobiedubie joined #salt
11:26 gmoro joined #salt
11:27 briner joined #salt
11:34 sjorge joined #salt
11:36 dendazen joined #salt
11:36 inad922 joined #salt
11:49 justan0theruser joined #salt
11:50 arlyon joined #salt
11:53 stooj joined #salt
12:07 zerocoolback joined #salt
12:33 awerner joined #salt
12:37 _xor joined #salt
12:39 Nahual joined #salt
12:49 briner joined #salt
12:54 motherfsck joined #salt
12:57 inad922 joined #salt
13:03 cgiroua joined #salt
13:06 oida joined #salt
13:08 miruoy_ joined #salt
13:08 garphy`aw joined #salt
13:12 briner_ joined #salt
13:13 XenophonF joined #salt
13:23 edrocks joined #salt
13:26 garphy`aw joined #salt
13:29 englishm_work joined #salt
13:34 nixjdm joined #salt
13:49 ddg joined #salt
13:54 dxiri joined #salt
13:55 dxiri joined #salt
13:56 crux-capacitor Can anyone give an example of how to use the validate function for a beacon?
14:02 mchlumsky joined #salt
14:12 noobiedubie joined #salt
14:17 sjorge joined #salt
14:18 toddejohnson joined #salt
14:18 noobiedubie joined #salt
14:24 spiette joined #salt
14:39 schemanic joined #salt
14:43 edrocks joined #salt
15:20 avgtechie joined #salt
15:20 megamaced joined #salt
15:34 Heartsbane joined #salt
15:34 Heartsbane joined #salt
15:37 Pjusur joined #salt
15:39 dxiri joined #salt
15:39 doubletwist Ok I think I'm calling some pillar data wrong here but not sure how: https://paste.fedoraproject.org/paste/JzgX0Xt3A3IWIRXW30C2yw
15:48 doubletwist Well, I've figure out the set enabled needs to use authconfig:pam_access_enabled
15:49 arlyon joined #salt
15:49 doubletwist that does start it processing the rest of the template rather than just going with the not-enabled default
15:49 doubletwist And I'm guessing the same for set pam_access to use 'authconfig:pam_access', but it's not reading the allowed_users.items or denied_users.items
15:51 Sacro joined #salt
15:54 onslack <msmith> you want `pam_access.allowed_users` rather than using a non-existing get()
16:01 doubletwist would {%- set allowed_users = pillar.get('authconfig:pam_access:allowed_users', {}) %}  also work?
16:02 miruoy joined #salt
16:03 tiwula joined #salt
16:05 doubletwist if I use      {%- set allowed_users = pam_access.allowed_users %}
16:06 doubletwist I get an error "Unable to manage file: Jinja variable 'dict object' has no attribute 'allowed_users'
16:06 stooj joined #salt
16:07 briner joined #salt
16:09 babilen doubletwist: I'd recommend to use salt['pillar.get'](...) throughout which would make line 23 salt['pillar.get']('authconfig:pam_access:allowed_users', []) for example
16:09 doubletwist I'll try it
16:09 babilen err .. use {} in lieu of [] as default
16:10 babilen (or make that a list in the pillar)
16:11 arlyon joined #salt
16:11 doubletwist so you mean this [for line 23]
16:11 doubletwist 23   {%- for user,locations in pillar.get('authconfig:pam_access:allowed_users', {}) %}
16:11 babilen Line 23 is {%- set allowed_users = pam_access.get('allowed_users', {}) %}
16:11 arlyon joined #salt
16:12 babilen All I mean is: Use salt['pillar.get'] for pillar access
16:12 doubletwist er right
16:12 babilen So I specifically do *not* mean pillar.get(....)
16:12 Deliant joined #salt
16:15 doubletwist ah-hah
16:15 doubletwist ok yeah set line 22 to use salt['pillar.get'] and it works
16:16 doubletwist Thank you!
16:16 Trauma joined #salt
16:16 doubletwist er also lines 23/24 to just get direct from the pillar
16:16 doubletwist er that's nit right either. The point is it works now
16:17 babilen salt['pillar.get'] uses a special execution function that can handle nested data structures (unlike Python's braindead default .get())
16:17 doubletwist removed line 22, and used salt['pillar.get'] on lines 23/24
16:26 arlyon joined #salt
16:49 relidy I feel like this is probably a FAQ, but can I have salt ignore something that looks like Jinja (Go templating) in a file that's being parsed as Jinja (I do need it parsed)?
16:52 babilen {% raw %} ... {% endraw %} might come in handy (content in between those tags will not be templated)
16:52 relidy babilen: Thanks, I'll see if that will do the trick.
16:52 babilen Or do you mean that the file shouldn't be templated as jinja at all?
16:53 relidy You got it right the first time. I do need it templated, but need it to ignore some stuff that looks like jinja that isn't.
16:53 babilen The above is what you're looking for in that case
16:54 relidy Indeed it is, thank you.
16:54 Psi-Jack Hehe. Now that i have salt managing my internal certificate authority, i need to get let's encrypt working via salt with the acme v2 wildcard stuff.
16:58 Trauma joined #salt
17:00 ExtraCrispy joined #salt
17:02 h202 joined #salt
17:04 h202 how can i target based on a pillar entry having a value other than none?
17:08 briner joined #salt
17:09 zerocoolback joined #salt
17:15 h202 in the scheduling system what does "splaying the time" mean?
17:18 Edgan h202: It means "randomizing" the timing to prevent a thundering herd of processes all starting at the exact same moment.
17:18 h202 got it
17:22 h202 ok not really so it says splaying between 0 and ten when you splay?
17:22 h202 0 and ten seconds
17:23 h202 so if i schedule for 3:30am, with splay: 10 does that mean the job will go off between 3:30:0 and 3:30:10?
17:23 briner joined #salt
17:24 h202 or does it mean that it will be randomized five seconds on either side?
17:24 h202 like 3:29:55 to 3:30:05
17:26 Edgan h202: url?
17:27 h202 ed on a pillar entry having a value other than none?
17:27 h202 * briner has quit (Quit: briner)
17:27 h202 oops
17:27 h202 https://docs.saltstack.com/en/latest/ref/states/all/salt.states.schedule.html
17:29 Edgan h202: you could call it spread, or you could call it "randomizing" but it is over a time window. You are going to do it repeatedly, so it can be randomized over all time. There does have to be a time window.
17:30 Edgan I mean so it can't be randomized over all time
17:31 h202 yes but is the windows centered on the time you give or does it start at the time you give?
17:32 Edgan it starts at the time, but then spreads it over the whole time window. If want more than 10 seconds, set it higher.
17:32 h202 ok it *starts* at the time
17:32 h202 that what i wanted to establish
17:34 Edgan h202: Looking at the documentation again it looks like this implementation will do it in a slot style. So lets say you have 10 minions, and you want to do them over an hour(the frequency).
17:35 Edgan h202: It is saying it will start with the first one and randomize the time it is run between 12:00:00 and 12:00:10. Then do the next one between 12:00:10 and 12:00:20.
17:36 Edgan h202: It is both spreading and randomizing
17:36 Edgan h202: But ultimately it is spreading them out over time in a overlapping way, being that most Salt runs take 30+ seconds.
17:37 Edgan h202: I prefer the batch method. If you set the batch to eight, do eight, and when one finishes, pop another one on the stack.
17:38 Edgan h202: Then the Salt master is never overloaded, and you get a nice orderly/efficient run of all the minions.
17:39 chutzpah joined #salt
17:40 Edgan h202: Splaying also is basically pre-calculating the spread over the window. Which if you number of minions outnumber the splay, then it would seem like either some will be dropped or you are going to have conflict between runs.
17:41 Edgan h202: and the risk of running out of time from too many minions is greater, because it is waiting to run the next one a random amount of time, instead of after the last one finished.
17:43 h202 how can it know how to spread? Isn't the scheduling determined on the minion?
17:45 h202 it doesn't sound plausible
17:46 h202 wait i think you'r probably right
17:48 JacobsLadd3r joined #salt
17:50 Edgan h202: scheduling is done on the master
17:50 h202 yes
18:07 h202 how can i schedule to run on some random day?
18:08 h202 at a specific time
18:09 h202 i guess i can hash the minion id
18:09 h202 and do a mod
18:12 Edgan h202: Look at the doc url you gave me. It shows two styles. Cron and when.
18:12 MTecknology I remember thinking modulus division was the stupidest thing we were ever going to learn in school. It's become some of the most advanced math I use.
18:14 Edgan MTecknology: yeah, I use it for spreading instances across AZs
18:14 Edgan MTecknology: But it is heavily used in encryption
18:14 MTecknology My brain doesn't comprehend encryption...
18:15 h202 Edgan, i can say when but i don't see how i can randomise the day
18:15 h202 maybe with cron i can
18:15 Edgan h202: Why do you really need random across days?
18:15 MTecknology Encryption: Magicness moves magic bits in magic ways so that magic can re-move them magically into the places they magically began. ;)
18:18 Psi-Jack heh
18:20 Edgan MTecknology: Encryption in the base case, of ciphers, aren't that hard to understand. They them take it to the Nth degree with complicated math. But even the fundamental concept of public_key encryption isn't that complex. Big primes multipled together. So it is easy to go one way and not the other, unless you already know the values. Why primes, because then you can guarantee there is only one key.
18:31 edrocks joined #salt
18:31 h202 Edgan my thinking is that salt will only clobber 1 in 7 work stations if it only runs a highstate on 1/7th every day
18:35 Edgan h202: yes
18:35 Edgan h202: I run everything every half hour
18:39 h202 salt has root/system privileges and one mistake could cause a mess
18:39 h202 no way i want to run a highstate on every machine on the same day
18:40 ymasson joined #salt
18:41 Edgan h202: This is why you have test environments that you test you code with first.
18:41 Edgan h202: Software as a service companies tend to have something like dev, qa, preprod, and prod environments.
18:41 Edgan h202: So you go through three rounds of testing before it gets to prod, in theory.
18:42 Edgan h202: It is about automation, and quick turn around.
18:42 NEOhidra joined #salt
18:43 Edgan h202: If I only run Salt once a week, how is it any better than just doing it myself manually?
18:43 h202 h202: i wouldn't want to install certificates manually on all of the machines for example
18:43 h202 i am  looking into windows updates too
18:43 h202 especially with things like windows updates i wouldn't even be confident after three rounds of testing
18:44 Edgan h202: I am not a fan, but I am currently working on letsencrypt in Salt. Adding support for wildcards for internal web services.
18:45 Edgan h202: That is more Microsoft's fault. In the Linux world 99% of the time we trust our updates break stuff.
18:45 Edgan don't break stuff
18:47 Edgan h202: Also using instances in AWS, our hardware is virtualized, and then acts a lot more standardized. No driver issues.
18:47 briner joined #salt
18:47 Edgan h202: and we don't have to run the hosts, so those drivers aren't our problem
19:00 snath joined #salt
19:16 TheCzar_ joined #salt
19:31 DanyC_ joined #salt
19:36 DanyC joined #salt
19:37 edrocks joined #salt
20:19 DanyC joined #salt
20:19 stooj joined #salt
20:20 mikecmpbll joined #salt
21:09 dendazen joined #salt
21:29 dendazen joined #salt
21:47 Edgan gtmanfred: ETA on 2018.3.1?
22:02 snath joined #salt
22:05 gyles19 joined #salt
22:49 anon joined #salt
22:55 briner joined #salt
23:04 zulutango joined #salt
23:19 dendazen joined #salt
23:45 exarkun joined #salt

| Channels | #salt index | Today | | Search | Google Search | Plain-Text | summary