The victims included 10 men, seven women, a boy and a girl. The children, both 10 years old, were grazed and in good condition.
Mayor Mitch Landrieu urged witnesses to come forward with information during a news conference Sunday night at a hospital where gunshot victims were taken.
"These kinds of incidents will not go unanswered. Somebody knows something. The way to stop this violence is for you all to help," he said.
Mary Beth Romig, a spokeswoman for the FBI in New Orleans, said federal investigators have no indication that the shooting was an act of terrorism.
"It's strictly an act of street violence in New Orleans," she said.
As many as 400 people joined in the second-line procession that stretched for about 3 blocks, though only half that many were in the immediate vicinity of the shooting, said Police Superintendent Ronal Serpas. Officers were interspersed with the marchers, which is routine for such events.
Police saw three suspects running from the scene in the city's 7th Ward neighborhood. No arrests had been made as of early evening.
Outside the hospital on Sunday night, Leonard Temple teared up as he talked about a friend of his who was in surgery after being shot three times during the parade. Temple was told the man was hit while trying to push his own daughter out of the way.
"People were just hanging out. We were just chilling. And this happened. Bad things always happen to good people," said Temple, who was at the parade but didn't see the shootings.
In the late afternoon, the scene was taped off and police had placed bullet casing markers in at least 10 spots.
Second-line parades are loose processions in which people dance down the street, often following behind a brass band. They can be impromptu or planned and are sometimes described as moving block parties.
A social club called The Original Big 7 organized Sunday's event. The group was founded in 1996 at the Saint Bernard housing projects, according to its MySpace page.
The neighborhood where the shooting happened was a mix of low-income and middle-class row houses, some boarded up. As of last year, the neighborhood's population was about 60 percent of its pre-Hurricane Katrina level.
The shootings took place about 1.5 miles from the heart of the French Quarter and near the Treme neighborhood, which has been the centerpiece for the HBO TV series "Treme."
Sunday's violence comes at a time when the city is struggling to pay for tens of millions of dollars required under a federal consent decree to reform the police department and the city jail.
Shootings at parades and neighborhood celebrations have become more common in recent years as the city has struggled with street crime. Police say gang turf wars often are the root cause.
Police vowed to make swift arrests. Serpas said it wasn't clear if particular people in the second line were targeted, or if the shots were fired in a random fashion.
"We'll get them. We have good resources in this neighborhood," Serpas said.
Contributing to this report were Associated Press writers Michael Kunzelman and Kevin McGill in New Orleans and AP Radio reporter Jackie Quinn in Washington.