Perl 6 - the future is here, just unevenly distributed

IRC log for #crimsonfu, 2016-02-23

crimsonfu - sysadmins who code

| Channels | #crimsonfu index | Today | | Search | Google Search | Plain-Text | summary

All times shown according to UTC.

Time Nick Message
00:07 searchbot joined #crimsonfu
00:09 pdurbin joined #crimsonfu
01:37 pdurbin huh, my digital ocean VM that hosts searchbot (and other stuff) ran out of memory: http://i.imgur.com/Znfldk5.png
01:38 pdurbin nothing really jumps out at me from my munin graphs: http://i.imgur.com/DAow03L.jpg
01:52 bene which one is the memory graph?
02:12 pdurbin bene: sorry, here's the data: http://munin.greptilian.com/greptilian.com/server2.greptilian.com/index.html
02:32 bene do you not have any swap configured on that box?
02:34 bene python meetup is tomorrow: web app testing with selenium and some other python tools
02:41 bear :) - I just gave a talk about some of that at DeveloperWeek
02:42 bear (web app testing with python flask)
03:24 pdurbin bene: the digital ocean image didn't come with any swap. first outage since I created it (or at least started monitoring it) in November: http://munin.greptilian.com/greptilian.com/server2.greptilian.com/memory.html
07:21 Azgarech joined #crimsonfu
10:48 Azgarech joined #crimsonfu
13:58 bene i think no swap is poor practice
13:58 bene and gives you no leeway when there's memory pressure
14:00 bene huh, you can tune the OOM killer to select/deselect certain processes: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html
14:00 bene but you have to do it by PID
14:00 pdurbin are you saying digital ocean slipped me a defective image? it was the default (ubuntu)
14:00 bene i'm saying they sent you the cheapest thing
14:01 bene and it's up to you to manage it :-)
14:01 bene "work within the constraints" or some such
14:01 bene easy enough to create a 1-2 GB swap file on the filesystem and use it
14:02 pdurbin well, let's see how often it crashes :)
14:02 bene let the kernel do its job and swap out things that aren't being used so you can use the RAM to get shit done
14:02 bene there's that too :-)
14:02 pdurbin I'm fine with a crash every four months. :)
14:02 bene that's 4 nines at least
14:03 pdurbin heh
14:03 pdurbin the crisis of low expectations
14:11 bene any unix system where you care about the availability, i think you should have a few GB of swap and then alert when it actually starts to fill up
14:11 bene i suppose that's an approach on the pets side of the pets-v-cattle divide
14:12 bene if it was cattle you'd just set the OOM killer to kernel panic on OOM and have your stack start another one :P
14:17 dotplus "4 nines" depends on how it takes you to notice/recover:)
14:19 pdurbin dotplus: well, I run my IRC client (weechat) from that VM so I tend to notice fairly quickly :)
14:20 dotplus yeah, that's one approach to monitoring: constant manual vigilance
14:26 bene as opposed to constant automated vigilance?
14:29 dotplus I was being flippant. But actually, in the real world, both approaches are valid. The automated tools are more reliable and the humans are better able to deal with "new" (events, responses). So in those circumstances where the "best possible" is worth paying for, you see both approaches at the same time.
14:30 bene i was being flippant too :-)
14:30 dotplus E_MORNING
14:30 dotplus E_GERMS
14:31 bene i still like swap as a canary for various things
14:32 bene and i think systems run better with some swap available :-)
14:32 pdurbin sounds like the default digital ocean image should have some swap
14:32 bene still a lotta snowflake servers in this world
14:32 pdurbin https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04
14:33 bene they sell you an image with "20 GB storage" e.g.
14:33 bene if it came with 2 GB of swap by default, some people would complain about 10% of their storage being underutilized
14:33 bene it's super easy to add swap after the fact
14:34 pdurbin "As has been noted elsewhere, DigitalOcean servers are by default configured without any sort of swap" http://serverfault.com/questions/578896/is-there-any-reason-to-not-have-a-swap-file-on-centos-digitalocean
14:34 bene so this was probably just a design decision that dovetailed with their marketing/etc
14:34 dotplus -e 's/^.&,//' -e 's/10%/anything'
14:35 dotplus oh I give up. I'm going readonly until I wake up.
14:36 bene heh
14:36 pdurbin http://serverfault.com/questions/218750/why-dont-ec2-ubuntu-images-have-swap
14:36 bene yeah, i read that too
14:37 pdurbin "This is by design. Swap is turned off by default on EC2 EBS-backed instances, to avoid unpredictable costs. If you have a memory-hungry app that goes rogue (say, on a tiny or small instance), it can generate quite a large amount of I/O requests on your EBS volume."
14:39 bene that makes sense on the AWS side, since there's no persistent storage on the EC2 AMIs
15:06 searchbot` joined #crimsonfu
18:23 Azgarech joined #crimsonfu
19:59 Azgarech joined #crimsonfu
20:52 Azgarech joined #crimsonfu
23:01 mhayden joined #crimsonfu
23:01 dotplus joined #crimsonfu
23:01 bear joined #crimsonfu
23:01 skay joined #crimsonfu
23:01 searchbot` joined #crimsonfu
23:01 Azgarech joined #crimsonfu
23:01 westmaas joined #crimsonfu
23:01 mezzer joined #crimsonfu
23:01 Whoop joined #crimsonfu
23:01 prologic joined #crimsonfu
23:01 arcanine joined #crimsonfu
23:01 sivoais joined #crimsonfu
23:03 chasmo77 joined #crimsonfu
23:03 JoeJulian joined #crimsonfu
23:03 semiosis joined #crimsonfu
23:03 hydrajump joined #crimsonfu
23:03 mag009_ joined #crimsonfu
23:06 mag009 joined #crimsonfu

| Channels | #crimsonfu index | Today | | Search | Google Search | Plain-Text | summary

crimsonfu - sysadmins who code