The somewhat surprising 219-189 vote came as the House debated a bill funding the Justice Department's budget.
The amendment by conservative GOP Rep. Dana Rohrabacher of California — the first state to legalize medical marijuana — came as almost half the states have legalized marijuana for medical uses, such as improving the appetites of cancer patients undergoing chemotherapy.
"Public opinion is shifting," Rohrabacher said, noting a recent Pew Research Center that found 61 percent of Republicans support medical marijuana. The numbers are higher for independents and Democrats.
"Despite this overwhelming shift of public opinion, the federal government continues its hard line of oppression against medical marijuana," he said.
Oregon Democrat Earl Blumenauer told opponents that "this train has already left the station."
Opponents said that marijuana is regulated too loosely by the states and harms the brain.
Rep. Andy Harris, R-Md., cited a recent Drug Enforcement Administration study that said that many in the medical marijuana movement are using it as "a means to an end," meaning legalization for recreational use.
"Congress is officially pulling out of the war on medical marijuana patients and providers," said Dan Riffle, director of federal policies for the Marijuana Policy Project.
The measure now heads to the Democratic Senate.