Last night, at about 4:30pm (EST), thousands of servers went offline due to Rackspace’s Dallas data center having a major power management issue. The issue is not 100% clear yet, you can read what Rackspace had to say at their blog.
When my sites and servers went down, I first went to Rackspace to login to my account. But I couldn’t access the site. So I then tried calling, but I was getting a busy signal, due to the volume of people calling. I then tried Twitter and found @rackspace there. There were no updates yet, but I followed @rackspace and waited.
I then saw the first tweet that read “We are having an issue that is affecting part of our DFW data center. No details yet. Will update as we get more information.” More followed. Twitter Search was swarming with mentions of [rackspace], so it was good to know I was in good company. I noticed people began to say there sites were coming back up, which meant mine was coming up soon also. I assume they began flipping on the power to the servers slowly until all came on.
The most incredible part was that @Rackspace doubled their twitter followers. Before the outage, they had about 1,300 followers and now they have over 2,300. Like I tweeted yesterday, “Expensive way to get @rackspace more followers ;-).” That tweet was when Rackspace had about 1,300 followers.
In any event, I always find it interesting how my customers react to outages. We haven’t had an outage in a really long time. But when this happened, some emailed, some IMed, some called. Some of the ones that called were not happy. I wasn’t happy either, but I told them what I knew and sometimes that is not good enough, sometimes it is.
The outage, I believe, lasted about 40 minutes or so. Was it the end of the world? No, but it wasn’t something that should be accepted. Will I switch from RackSpace? Probably not. They have been the most stable and most responsive company in this space I have ever dealt with and trust me, I have been with many and I spend a fortune on this stuff.