Police and ambulances swarmed to the scene near the city's rail station, blocked off a nearby road, and secured the Islamic Center.
On Twitter, the Zurich police department said: "Large operation after shooting at mosque in the Eisgasse. Several injured. Perpetrator on the run." It said the situation was under control and the wounded had been taken to hospital.
There were conflicting news reports about whether the shooting had taken place inside a prayer room itself, or outside.
A Zurich police official, speaking on condition of anonymity because he wasn't authorized to speak to reporters, said the shooting occurred near the intersection of Militaerstrasse and Eisgasse streets.
Police told German-language daily Blick that three people had been injured.
Attacks by gunmen are rare in Switzerland. The country was shaken in 2013 by at least two multiple-fatality shootings.
The country has a long-standing tradition for men to keep their military rifles after completing compulsory military service — which partly explains a high rate of gun ownership in the country of about 8.2 million people.
Jamey Keaten contributed from Geneva.
Copyright 2016 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.