According to the TPC, half are people who simply didn't make enough money to pay income tax. The rest all have some special tax benefit that leaves them with no tax liability. 44% are people who have tax benefits because they're elderly. Mainly that's the non-taxability of most Social Security benefits. Another 30.4% are Credits for Children and the Working Poor. That's mainly what I referred to earlier: the Earned Income Tax Credit and Child Tax Rebates. The remaining 25.6% runs the gamut from itemized deductions, education credits, special rates for capital gains and a lot else. See page two of the document for the pie chart with the details.
Needless to say, this is "income tax", which is far from the only federal tax. Virtually everyone who gets a pay check pays the payroll tax. And when you combine the amount you pay and your employer pays on your behalf its about 13% from the first dollar on up.