But an official with the USDA told TPMmuckraker that the White House wasn't involved, and that the decision was Vilsack's alone.
The White House has also denied any involvement, but says it supports Vilsack's decision.
When the department heard about the video yesterday afternoon, they immediately put Sherrod on administrative leave, according to the official. Then, officials called Vilsack, who asked if Sherrod was willing to resign. According to the official, Sherrod said yes.
The official said Sherrod had a chance to explain herself. But her explanation -- that her story is "about getting beyond the issue of race" -- had little effect.
"The comments create a situation that make it really hard for her to do her job," said the official, explaining as Vilsack did in an earlier statement that the agency is struggling to overcome a perception of racism. And it doesn't matter that Sherrod apparently ended up helping the white farmer. "The issue is the comments that she made."
Vilsack, speaking to CNN, said he was concerned about future allegations of racism against Sherrod.
"When you're the state rural development director your principle job is to try to develop job growth in the state of Georgia," he said.
"Unfortunately, the statements and the context of the statements created a circumstance where in the future if people were not satisfied with the decisions that the rural development director made, they could attribute the decision to a wide variety of reasons that weren't necessarily related to the job," he said.