teh TODDtastic webLog-a-majig

Last night the power went out from approximately 1:30:44 EST until about 5:45 AM. I had to shut down the server since it was due to thunderstorms and I had no idea how long it would take them to repair the lines. Plus, I needed to save the battery power in case it would become useful for other purposes. I did the same with my desktop, but I kept the network UPS running until its batteries depleated. I also have my cell phone charging off of the network UPS, and I had to charge my phone for the next day.

Then I went to school and I didn’t get my server up and running until about 6:30pm. I know, I know. Unacceptable… but it actually spent more time up than it did during my RAID-5 issues of last week.

Also, I would like to use this opportunity to state that the connection this server runs on is now fatter. Previously we had a 7Mbps/512Kbps, now we have 22Mbps/768Kbps. When we upgraded we thought it was supposed to be 2Mbps up… but clearly Time Warner likes to practice false advertising and they refuse to provide that (even though their infrastructure here is definitely capable of providing it, a la download often reaches 30Mbps). Perhaps in the future they will provide such speed. Either way, we have 50% more upload bandwidth, which should help with handling the high volume of traffic my site gets.

This is a log of the events that happened this week. I am posting this mostly to explain what happened, how I resolved it, and also to state that it will probably happen again in the future. This post may be of great assistance to you if you came across it via google. I know I spent a lot of time dealing with these problems, and lost 200+GB of data (although nothing that was of crucial importance).

If nothing else, the problems I had were not excellently documented on the internet, so hopefully this will serve as such documentation for someone. When searching for the specific errors in dmesg, I found people just saying to ignore them as they were frivolous errors. I did just that until, of course, it turned into a big problem.

Read the rest of this entry »