The official said it was difficult to verify the accounts Bergdahl has given since his release a week ago.
Bergdahl, now 28, was captured in June 2009 after he disappeared from his infantry unit. He was held for nearly five years by Taliban militants.
The New York Times reported Sunday that military doctors at the Landstuhl Regional Medical Center say that while Bergdahl is physically able to travel he's not yet emotionally prepared to be reunited with his family. He has not yet spoken to his family.
Bergdahl was returned to the U.S. military in exchange for the release of five Taliban militants from the U.S. military prison at Guantanamo Bay, Cuba.
The deal, which the Obama White House brokered without consulting Congress, ignited a political firestorm that shows no signs of abating.
Lawmakers, both Republicans and Democrats, who initially praised Bergdahl's release, quickly backed off amid questions about whether he was a deserter who walked away from his post and an outcry over the exchange. Some of Bergdahl's fellow soldiers maintain that Americans died during efforts to find and save him.
On Wednesday, Bergdahl's hometown of Hailey, Idaho, abruptly canceled plans for a welcome-home celebration, citing security concerns. And on Saturday the FBI said Bergdahl's family had received threats that are being investigated by federal, state and local authorities.
The U.S. official told the AP that Bergdahl's parents were being harassed and threatened, including death threats.
Copyright 2014 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.