Server Post Mortem 7-Jun-2016

[Null]

Prologue:
The Kiwi Farms uses two servers to manage load. These are called the Pawns. Our second pawn runs FreeBSD.

Timeline:
1. (Months ago) I install Nginx on our new pawn running FreeBSD. I configure it correctly, but save a change that never gets applied because it doesn't work on FreeBSD Nginx.
2. I go to sleep this morning.
3. Our host reboots the machine that our second Pawn is on.
4. When the machine loads, it attempts to load the erroneous configuration file. It can't apply it and it doesn't have a preloaded configuration to ignore it for like it did months ago.
5. Pawn 2 goes into a permanent failure state and Pawn 1 remains up. 50% of incoming connections are dropped at a coinflip.

Solution:
1. Temporarily route all traffic through Pawn 1.
2. Fix Pawn 2.
3. Have you fucking lazy kikes call my fucking phone number like I've been telling you to do if the server has a problem so you don't need to endure 6 goddamn hours of intermittent errors.


PHONE
+1 (757) 932-5494

I usually don't answer but I do receive voice mail and text messages.

 

[Yog-Sothoth]

We aren't dead~

 

[A Hot Potato]

We've let you down grand master Null.

I must now commit sudoku.

 

[Wildchild]

I don't have your phone number. Can you give it to me so I can give you phone calls daily and wish you a good day?

 

[Cthulu]

3. Have you fucking lazy kikes call my fucking phone number like I've been telling you to do if the server has a problem so you don't need to endure 6 goddamn hours of intermittent errors.

I don't have you phone number. Blame the hotpocket cucks.

 

[GS 281]

2. I go to sleep this morning.

mornings are for wakies not sleepies

 

[repentance]

Sounds like a job for Based Blue FreeBSDGirl.

 

[0xDEADBEEF]

The answer:

use Arch for EVERYTHING

 

[crunchysalty]

We aren't dead~

Dude I'm a sys admin, (tho I haven't played with bsd for over a year.) if you want, ms me your phone #, and I'll call or txt you if something's going on. I'm on MST.

 

[chimpburgers]

Yeah I don't have your number either. Just Skype. Fug.

 

[Rou]

Oh wow, I didn't know your phone number was public.

 

[Cthulu]

Oh wow, I didn't know your phone number was public.

Doesn't matter if I call I yell .

 

[Null]

The second pawn is running. We're good.

 

[alex_theman]

Is that number a BBS? Should I call it up with my VIC-Modem?

 

[Cthulu]

You could direct us at the hotpocket cucks responsible for calling you so we can mock them.

 

[OtterParty]

Let's call him and shout julay then dox his mom

 

[Jaimas]

We've let you down grand master Null.

I must now commit sudoku.

I bring terrible news - @A Hot Potato was not kidding. We broke into his office and we were too late:

He is survived by his proxy accounts and a collection of Warhammer Fantasy miniatures.

 

[AN/ALR56]

I think Sammy will announce your phone number on his blog.

 

[ChuckSlaughter]

Prologue:
The Kiwi Farms uses two servers to manage load. These are called the Pawns. Our second pawn runs FreeBSD.

Timeline:
1. (Months ago) I install Nginx on our new pawn running FreeBSD. I configure it correctly, but save a change that never gets applied because it doesn't work on FreeBSD Nginx.
2. I go to sleep this morning.
3. Our host reboots the machine that our second Pawn is on.
4. When the machine loads, it attempts to load the erroneous configuration file. It can't apply it and it doesn't have a preloaded configuration to ignore it for like it did months ago.
5. Pawn 2 goes into a permanent failure state and Pawn 1 remains up. 50% of incoming connections are dropped at a coinflip.

Solution:
1. Temporarily route all traffic through Pawn 1.
2. Fix Pawn 2.
3. Have you fucking lazy kikes call my fucking phone number like I've been telling you to do if the server has a problem so you don't need to endure 6 goddamn hours of intermittent errors.


PHONE
+1 (757) 932-5494

I usually don't answer but I do receive voice mail and text messages.

I'll call you anytime there is a problem. My name is JULLLAAAAAAAYYYYYY.

 

[AnOminous]

The answer:

use Arch for EVERYTHING

Install Gentoo!