My single biggest problem with the PS3? The loss of backwards compatibility. It's a gaping hole in the system's feature set. Then again, it may also be one that's on the way back.
Some background: when the PS3 first launched, it was backwards-compatible, meaning you could play PlayStation 2 games on your new PlayStation 3. The 20GB and 60GB units released in North America and Japan featured hardware emulation (they literally had a PS2 chip inside), while those released in PAL territories featured software emulation (similar to how the 360 handles original Xbox games).
Later, though, this feature was removed. Anyone buying a 40GB, a later model of the 80GB or the 120GB PS3 can't play a single PS2 game on them. It was a stupid, stupid move on the part of Sony.
A patent discovered by Siliconera, however, suggests that Sony might be re-thinking this stance. Filed in December 2008, it's basically a patent for a method that would allow the PS3's Cell chip to translate code from the PS2's Emotion Engine. Not half-assed software emulation (which in previous PS3 models couldn't run some games), full, total replication of the functionality of the Emotion Engine.
Which means, theoretically at least, you could play any PS2 game on any PS3, regardless of the model or year of release.
Whether this would allow you to play actual PS2 discs, or would just be the advance party for the sale of PS2 downloads on the PlayStation Store is unclear. We'd like the former, but with Sony being a business and all, would expect the latter.
Sony Patents Emotion Engine Emulation Technology For Cell Processors [Siliconera]