President Hamid Karzai's office said Fahim, who held the rank of field marshal, died from an illness in Kabul. The exact cause of death was not immediately known. Fahim had survived several assassination attempts, most recently in 2009 in northern Afghanistan.
His longtime friend and Afghanistan's ambassador to Spain, Masood Khalili, said Fahim "was not feeling good. He had diabetes. He had had two heart operations and three times he had gone to Germany for check-ups." Khalili was badly wounded in the same suicide bombing that killed Massoud.
Fahim served as defense minister in Karzai's first administration and most recently was the first of two vice presidents. But he was best remembered as a former warlord who fought against the Soviets when they occupied the country and for taking part in the bitter internecine fighting that marked the early 1990s. He went on to battle alongside Massoud against the Taliban.
In a televised address to the nation, Karzai called Fahim his close friend and brother.
"No one can replace him. It is a loss for all of us," Karzai said. "Fahim was part of every historic decision made for the future of Afghanistan."
The government also called for a three-day mourning period to begin Monday.
Fahim "started his fight for the liberation of Afghanistan," when he was barely out of his teens, Khalili said in a telephone interview from Spain.
"He was one of the heroes of Afghanistan. He was the one who stood alongside Massoud. He never accepted the Taliban, their ideas, their government. He was always rejecting al-Qaida as terrorists," Khalili said.
The Pashtun-dominated Taliban seized Kabul in 1996 and ruled from the capital until they were ousted five years later by the U.S.-led coalition and its Afghan allies in the Northern Alliance, made mostly of minorities including ethnic Tajiks, Uzbeks and Hazaras.
Fahim was widely accused of marginalizing all Pashtuns, particularly in the security services, during his tenure as defense minister in the first years after the Taliban's collapse. He was bitterly criticized for alleged past atrocities, such as killing civilians by rocketing residential areas and booby-trapping homes, his heavy handedness and allegations of corruption.
Human Rights Watch accused Fahim, as well as several other prominent warlords allied with the U.S.-led coalition, of war crimes when they last ruled in Afghanistan in the early 1990s, before the Taliban took over.
He was removed from his post as defense minister in 2004.
"He kept quiet when he was removed as defense minister. He wanted the country to move toward democracy," Khalili said.
As Afghanistan headed into its first presidential elections in 2004, Fahim distanced himself from Karzai and threw his support behind his fellow Tajik, Yunus Qanooni. Eventually the two men reconciled and Karzai chose Fahim as his first vice president in the 2009 presidential elections, putting him first in line to fill in for the Afghan leader during absences from the country.
"I was just writing in my diary my thoughts. He was a good man. I have good memories of my friend," Khalili said. "He wasn't just a fighter. He had a kind, soft heart for culture, for poetry. There was a milder side to him. It was not just always that he was thinking with the gun. He also thought of poor Afghans."
Khalili recalled last year's ceremony to mark the anniversary of Massoud's death.
"I turned to Fahim and I said: 'Next year Fahim, either you or I will not be here in this world. Only God will be here,'" he said.
Kathy Gannon is AP special regional correspondent for Afghanistan and Pakistan and can be followed on Twitter at www.twitter.com/kathygannon.
Associated Press writer Kim Gamel contributed to this report.
Copyright 2014 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.