The marchers, starting at the intersection where Garner was first confronted, walked behind a banner that said: "We Will Not Go Back, March for Justice."
Police estimated that 2,500 people had taken to the streets.
James O'Neill, chief of patrol with the NYPD, credited the march organizers with helping to keep things orderly. "Everything is good," he said.
Earlier, Sharpton urged about 100 marchers gathered at a Staten Island church to remain nonviolent or go home.
He also repeated his call for a federal takeover of the criminal probe into the death of the 43-year-old Garner, an asthmatic father of six who was placed in a chokehold after police officers stopped him for selling loose cigarettes.
Activists have urged that criminal charges be brought against the officers involved.
Many in the crowd carried signs. Some said: "Police the NYPD" or "RIP Eric Garner." But the most popular signs were "Hands Up, Don't Shoot," which emerged during protests in Missouri over the police killing of Michael Brown, and "I can't breathe," Garner's last words.
Garner's widow, Esaw, urged the crowd to march in peace toward justice.
She said she is too afraid to let her sons go outside and asked those at the rally to "get justice" for her husband.
The marchers walked alongside dozens of police officers in parade gear, including polo shirts and pants. There were also officers in formal blue uniforms, but none had riot gear.
The rally proceeded past the office of Staten Island District Attorney Daniel Donovan, who this week sent the case to a grand jury.
Sharpton has repeatedly called Garner's death — and the shooting death of the 18-year-old Brown by police in Ferguson, Missouri — a "defining moment" for policing nationwide. Members of both Garner's and Brown's family were invited to the march.
Garner, an asthmatic father of six, was unarmed when he was stopped July 17. Garner was placed in a chokehold — an illegal police tactic — and could be heard screaming "I can't breathe!" as he was forced to the ground.
Soon after, he was declared dead. The city medical examiner ruled the death a homicide and two NYPD officers have been reassigned during the investigation.
So far, the U.S. Justice Department has signaled it likely will wait for the local probe to conclude before deciding whether to launch a formal civil rights investigation.
Saturday's half-mile-long route wound through a heavily minority neighborhood, one of several in the nation's largest city where residents have said they feel unfairly targeted by police for suspicion of crime and enforcement of low-level offenses.
Associated Press writer Eileen AJ Connelly contributed to this report.
Copyright 2014 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.