It depends on the type of game for me. In rpgs (both jrpgs and western) it's a combination of interesting npcs and hidden areas/quests. Anything that makes it seem like there's more going on in the world than just what's happening to the main characters. Final Fantasy used to be good at that, especially during the PSX era. I used to explore every inch of each town talking to everyone to make sure I didn't miss anything funny or interesting. Same with Morrowind and Oblivion.
For open world games like GTA I like to see things happening that have nothing to do with me. Police chases I didn't start, conversations between npcs, stuff like that. Life just going on in the world whether the player does anything or not.
I'd have to think about it more for other genres, those are just the big two for me when it comes to interesting game worlds.
About Speak Up on Kotaku: Our readers have a lot to say, and sometimes what they have to say has nothing to do with the stories we run. That's why we have a forum on Kotaku called Speak Up. That's the place to post anecdotes, photos, game tips and hints, and anything you want to share with Kotaku at large. Every weekday we'll pull one of the best Speak Up posts we can find and highlight it here.