I think it goes without saying that I love music in video games. But not all games require a soundtrack, and in fact, some games are better off without one.
Music can be a distraction, an unnecessary layer of sound that pulls attention away from a game rather than enhancing it. Sometimes, there's nothing to do but turn the music off.
The first big-budget game that made me turn off the music was Fallout: New Vegas. I played for a handful of hours and found myself feeling smothered by the endless mournful strings and guitars that played as I wandered the wastes. Every time the music would stop, I'd hear the wind in my ears, the chirping of insects. The desolation around me felt palpable; it was a breath of fresh post-apocalypse.
And then, the music would start up again. Endless syrupy strings and mournful guitars.
So, I turned it off. I never play that game with the music on anymore (though I do still listen to the excellent in-game radio stations). In fact, when I go back to Fallout 3, I do the same thing.
I think it's something about open spaces, at least for me. I adore Bill Elm and Woody Jackson's music for Red Dead Redemption, but sometimes I just don't want to hear music in that game. I want to wander the prairie, me and my horse, and take in the fantastic sound design, which I've long thought to be some of the best of all time. The sound design is almost a musical score of its own; and while its interplay with the dynamic music in the game is organic and never feels crowded, sometimes it's cool to just listen to the audio on its own.
(Seriously, I recommend doing that. Even if you haven't played RDR in a while. Boot it up, turn off the music, and put on headphones. Go sit on the prairie, close your eyes, and listen. Notice every sound that comes and goes. It's really cool.)
Sometimes I'd turn off the music in Bioshock and Bioshock 2, as well. As amazing as the music in those games was, there were times where exploring Rapture while taking in the ear-bustingly incredible sound design was enough.
Is it a sin to turn off the music in a game? No. I remember a while back, boss-man Stephen Totilo wrote a thoughtful editorial about how he finally decided to turn off some games' music.