First revealed earlier in the year, Valve today released Steam Music Player for all users of its desktop client.
All you need to do is point the client towards your music collection and you're off. Turn it on and you'll get a little floating media player (which can be activated and controlled in-game), which is able to play albums, mixes and playlists.
It's not a huge deal for a lot of games, considering the original score is so important to their pace and tone, but for long-running games with lots of downtime (like Civilization) it's a god-send.
Oh, and if you're stuck for something to listen to, we've got you covered.
Steam Music Player [Valve]