In it, but not of it. TPM DC

Specter: A Conservative Railing Against Ultra-Conservatives

He later added:

And for the people who are Republicans, to sit by and allow them to continue to dominate the party after they beat Chafee, cost us Republican control of the Senate, and cost us 34 federal judges, there oughtta be a rebellion. There oughtta be an uprising.

So Specter's problem wasn't that he's not a conservative, but that he's not conservative enough for a shrinking GOP. And if he's not a complete right-winger, his only real choice is to be a Democrat.

And that's where politics seems to be right now: The right-wing nuts versus everyone else. Specter is on the more conservative end of the "everyone else" category, but being a Democrat now offers him more chance for political success (and sane, rational discourse) than the other option.