If you've never seen a supercomputer in the flesh, the name is a little misleading. They don't look as clean as it sounds. More often they're banks and banks of individual units networked together to form a single hivemind, a gigantic mess of cables, stacks and fans.
To give you an appreciation of how much work goes into putting one together, here's a timelapse video of one being put together at my alma mater, the Australian National University. Its full name is pretty great: the official title is the "Fujitsu Primergy cluster high-performance supercomputer".
Construction of the NCI supercomputer - timelapse video [YouTube, via RiotACT]