The Ryan budget calls for the repeal of Obamacare but assumes the deficit reduction from the law's substantial Medicare provider cuts and new taxes on high-income people and businesses.
Gingrey was one of just 10 House Republicans to vote against the Ryan budget -- he voted for it last year even though it also included the new revenues from Obamacare tax hikes.
"As we mark the third anniversary of the health care law," he said, "small business owners, families and patients are beginning to feel its impact."
Gingrey isn't the only Republican congressman to make that observation.
"We're not done. We must repeal Obamacare -- including the associated taxes, which the Ryan budget leaves intact by assuming the enactment of tax reform later on," Rep. Paul Broun (R-GA), a 2014 Senate candidate, wrote Tuesday in a New York Times op-ed.
Broun also voted against the Ryan budget Thursday.