Well, that took more than 140 characters. Following sporadic service for much of Thursday afternoon ET, Twitter published a late blog entry apologizing for the downtime and explaining what happened. As Twitter engineering vice president Mazen Rawashdeh explained in the post:
At approximately 9:00am PDT, we discovered that Twitter was inaccessible for all web users, and mobile clients were not showing new Tweets. We immediately began to investigate the issue and found that there was a cascading bug in one of our infrastructure components. This wasn’t due to a hack or our new office or Euro 2012 or GIF avatars, as some have speculated today. A “cascading bug” is a bug with an effect that isn’t confined to a particular software element, but rather its effect “cascades” into other elements as well. One of the characteristics of such a bug is that it can have a significant impact on all users, worldwide, which was the case today. As soon as we discovered it, we took corrective actions, which included rolling back to a previous stable version of Twitter.
We began recovery at around 10:10am PDT, dropped again around 10:40am PDT, and then began full recovery at 11:08am PDT. We are currently conducting a comprehensive review to ensure that we can avoid this chain of events in the future.
Rawashdeh noted that Twitter’s service track record is actually pretty stellar: 99.96 percent over the past six months, but also noted “how critical Twitter has become for you — for many of us” and as such offered on behalf of the company “most sincere apologies.”
