I’ve spoken at length with the two men closest to Sen. Joe Lieberman’s ((D/I?)-CT) re-election Web site, joe2006.com, to understand at length what happened to the site yesterday morning. Their versions appear to differ, although it’s not immediately clear why. Sam Hubbell, proprietor of myhostcamp.com, which hosted the site, is more involved in the health of the server than Dan Geary, who designed the site and interfaces with the campaign.
Geary runs a small web consulting shop — not much bigger than himself — in Nevada, and sometimes uses Hubbell for design work, he told me when we spoke yesterday evening. For his part, Hubbell — whom I spoke with this afternoon — told me that myhostcamp.com consists of himself, a co-owner, and fewer than 10 servers located at a facility in Texas. Support, he said, is mostly handled by the Texas facility, Server Matrix.
So, guys, what happened?
On Monday morning, Dan told me, “It was as if suddenly all these people showed up to hit the video files. . . but it was everywhere, emails, FTP access.”
(For non-techies, FTP is how site managers upload, download, move and erase files on their server.) Hundreds and hundreds of emails to nonexistent “joe2006.com” addresses were pouring in, he said. “They all did go down,” Geary said, referring to the other sites sharing space on joe2006.com’s server. “When we took Joe2006 off, they all went back up again.”
Hubbell, however, told me this afternoon the attack affected only joe2006.com’s Web site and email. “FTP was fine,” he said. And the other sites? “The server lagged a little bit.” Otherwise, Hubbell said, they were only interrupted because he had to keep restarting the server.
Their first action, according to Geary, was to “suspend [the] domain. [Then] we tried putting up a single blank white page,” but it was immediately bombarded with traffic. “So at that point, we were like, ‘Oh my God!’ We dropped the whole thing — suspended the site, pulled the site files down, and pulled the account down.”
Hubbell recalls differently. “We put a hold on the account,” he said, but did not delete it. “We stripped out various modules and components in the content management system. . . additional questionaire forms, photo galleries, videos,” to see if that would help. “[But] there was something else going on, and that’s when we began to investigate more.”
The site uses a software package called Joomla to manage its content, according to both men. Hubbell insists his company kept the servers up-to-date with all security upgrades and patches. Right now, he theorizes that an as-yet-unreported flaw in Joomla was exploited by a hacker to bring the site down.
“It was potentially various components and modules, we haven’t figured out which one,” Hubbell said. “That’s kind of the guess. . . . The security patches were so fresh that. . . there might have been an additional undocumented loophole that someone got through.”
A hacked module — a form, Hubbell theorized — was generating thousands of emails to joe2006.com addresses. Even after removing various functions from the site, the problems persisted, Hubbell told me. “There were multiple spam attacks,” he recalled. “It seemed like it was internally spamming itself, and there was also potentially an outside source that was hitting it.”
“There’s. . . some investigation going on to as to seeing where the [outside] spamming came from,” Hubbell said. “That’s offsite, more on where the Lieberman committee is at.”
Do you mean that the Lieberman campaign is investigating the spamming itself? “Yes,” Hubbell replied.
Does any of this ring true? Does it make sense? And what do the emerging details of the web site’s less-than-stellar hosting tell us? I’ll have more on that tomorrow.
