The first game that comes to mind is the 1996 sequel to Chrono Trigger, Radical Dreamers. Radical Dreamers has far less in the way of graphics than most visual novels (mainly due to the differences between what a PC and the Satellaview Super Famicon add-on can handle). While characters and backgrounds appear in every scene of most visual novels, in Radical Dreamers the screen is often black, save for the text. And yet compared to visual novels like Katawa Shoujo, it has a ton of choices due to the "chose your own adventure" battle system for the random encounters.
Probably my first introduction to a game with very little in the way of gameplay was back in the mid-nineties when interactive movies were the new big thing. The first thing I did when I got a computer with both a sound card and a CD drive was go out and buy Star Trek: Borg. Like most of these interactive movies, gameplay was relegated to a few puzzles on top of the occasional narrative choice. And on the correct path, Star Trek: Borg can be beaten in a mere 20 choices. However, this was one of the few games I've ever played that rewarded you for losing—as dying got you a series of amazingly hilarious John de Lancie "Q" scenes.
Katawa Shoujo, Radical Dreamers, and Star Trek: Borg are all games I have greatly enjoyed. And even though any actual input on my part was minimal, I really felt like I had a major impact on the story each time. When it comes down to it, the only thing needed to make either a movie or novel into a game is a single interactive element—a single point where you, a person outside the game's creation, can alter how it plays out. And while gameplay—even a single button click—is necessary for something to be a game, games like those above prove that story can do the heavy lifting and carry a game as well.