David Einhorn, manager of the Greenlight Capital hedge fund, said Wednesday that Ballmer's "continued presence is the biggest overhang on Microsoft's stock," and that Ballmer should give someone else a chance.
Microsoft was once the largest U.S. company by market capitalization, but is now not even the largest tech company, having been surpassed by its fierce rival Apple.
Einhorn made the remark at the Ira Sohn Investment Research Conference in New York on Wednesday. Among other things, he's known for his call about Lehman Brothers' financial problems before its collapse.
The article noted that Einhorn was simply verbalizing a lot of people's sentiments.
Microsoft's record on innovation of late has been mixed.
Like a lot of large companies, it often tries to buy its way into a market.
For example, in 2008, it bought Danger, the start-up that created cool mobile device operating systems for products like the Sidekick.
That ultimately led to the disastrous release of Microsoft's Kin phone, which met with a critical reception and then was shortly thereafter killed.
Most recently, it bought Skype for $8.5 billion in cash.
While many people have perceived it to have fallen behind in the mobile market, Microsoft has seen a lot of success with Kinect, a super-sophisticated 3D imaging device that enables people to play video games without any physical controls on the Xbox 360.