"We will put all effort, we will put all resources and we will put all of our energy into finding this individual who committed this crime tonight," he said.
The FBI will aid the investigation, Mullen told a news conference that was attended by FBI Special Agent in Charge David A. Thomas.
Charleston Mayor Joseph P. Riley called the shooting "the most unspeakable and heartbreaking tragedy."
"The only reason that someone could walk into a church and shoot people praying is out of hate," Riley said. "It is the most dastardly act that one could possibly imagine, and we will bring that person to justice. ... This is one hateful person."
State House Minority leader Todd Rutherford told The Associated Press that the church's pastor, state Sen. Clementa Pinckney, was among those killed.
Pinckney 41, was a married father of two who was elected to the state house at age 23, making him the youngest member of the House at the time.
"He never had anything bad to say about anybody, even when I thought he should," Rutherford, D-Columbia, said. "He was always out doing work either for his parishioners or his constituents. He touched everybody."
The attack came two months after the fatal shooting of an unarmed black man, Walter Scott, by a white police officer in neighboring North Charleston that sparked major protests and highlighted racial tensions in the area. The officer has been charged with murder, and the shooting prompted South Carolina lawmakers to push through a bill helping all police agencies in the state get body cameras. Pinckney was a sponsor of that bill.
In a statement, Gov. Nikki Haley asked South Carolinians to pray for the victims and their families and decried violence at religious institutions.
"We'll never understand what motivates anyone to enter one of our places of worship and take the life of another," Haley said.
Soon after Wednesday night's shooting, a group of pastors huddled together praying in a circle across the street.
Community organizer Christopher Cason said he felt certain the shootings were racially motivated.
"I am very tired of people telling me that I don't have the right to be angry," Cason said. "I am very angry right now."
Even before Scott's shooting in April, Cason said he had been part of a group meeting with police and local leaders to try to shore up relations.
The Emmanuel AME church is a historic African-American church that traces its roots to 1816, when several churches split from Charleston's Methodist Episcopal church.
One of its founders, Denmark Vesey, tried to organize a slave revolt in 1822. He was caught, and white landowners had his church burned in revenge. Parishioners worshipped underground until after the Civil War.
Copyright 2015 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.