"There are other options. And this was what so angered for those of us who have followed this for years," he said on ABC's "This Week." "The administration has this theory that you're either with them or you're for thermonuclear war and there's not in between. That's just wrong."
"And so the reason they avoided Congress, this isn't about we didn't get invited to the party, so we shouldn't have our feelings hurt. It is because we can empower all of the people -- diplomats -- who disagreed with this decision, intelligence folks who disagreed with this decision, military folks," he continued.
Rogers said that the prisoner exchange will put Americans abroad at greater risk.
"This is a huge regional and geopolitical problem for the United States moving forward. Hostages are now currency in this war on terror. That's always dangerous for both diplomats, aide workers, soldiers on the battlefield," he said.