I'd suggest unchecking "Use iTunes as the default player for audio files" under Preferences -> Advanced tab.

Then your voice mail wav's/mp3's won't be added to your Library when you play them.

However, it will be a little more trouble to add new files to iTunes through the drag and drop.