Rajoy did not say when the handover would happen because the government must now craft a law creating a legal mechanism for the abdication and for 46-year-old Crown Prince Felipe's assumption of power.
Juan Carlos was expected to address the nation later Monday, the royal palace said in a tweet. The palace also tweeted a photo of the king shaking Rajoy's hand and presenting him with a letter announcing the abdication.
Juan Carlos, 76, has been on the throne since 1975 and was a hero to many for shepherding Spain's democratic and economic transformation, but has had repeated health problems in recent years.
But his longstanding popularity took a big blow following royal scandals, including an elephant-shooting trip he took in the middle of Spain's financial crisis during which he broke his right hip and had to be flown from Botswana back to Spain for medical treatment aboard a private jet.
His image was also tarnished by the investigation of his son-in law, who is suspected of embezzling large amounts in public contracts.
Felipe would presumably take the title Felipe IV. He has a law degree from Madrid's Autonomous University and obtained a masters in international relations from Georgetown University in the United States.
Felipe is married to Princess Letizia, a former television journalist, and they have two daughters.
Like his father, Felipe has traveled the globe trying to maintain Spain's influence especially in former Latin American colonies, while seeking to promote the nation's international business interests.
King Juan Carlos came to power in 1975, two days after the death of longtime dictator Francisco Franco. He endeared himself to many Spaniards in large part by putting down an attempted military coup in 1981 when he was a young and largely untested head of state.
As Spain's new democracy matured over the years, the king played a largely figurehead role, traveling the globe as an ambassador for the country, and was a stabilizing force in a country with restive, independence-minded regions such as the Basque region and Catalonia.
"He has been a tireless defender of our interests," Rajoy said.
Juan Carlos has melded the trappings of royalty with down-to-earth, regular-guy charm. The king is an avid sports fan and after the Madrid terror bombings of March 11, 2004, showed he could grieve like anyone else.
At an emotional state funeral for the 191 people killed in the train bombings by Islamic militants, King Juan Carlos and Queen Sofia slowly went row-by-row through Madrid's Almudena Cathedral, clasping the hands of sobbing mourners or kissing them on the cheek.
But his patient work nearly came undone during the financial crisis, with people questioning after the elephant-hunting trip whether a hereditary monarchy was needed.
The World Wildlife Fund's branch in Spain ousted Juan Carlos as its honorary president — a title he'd held since 1968 — after deciding the hunt was incompatible with its goal of conserving endangered species. Juan Carlos took the unprecedented step of apologizing to Spaniards for his actions.
He recently said that he wanted to be remembered as "the king who has united all Spaniards."
Copyright 2014 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.