I always feel like visitors to Japan are asking the wrong questions. Instead of asking what's already in numerous guidebooks, it's probably better to ask: what words do I need to get around?
By my count, there are five basic words that will help make your trip to Japan smoother. Yep, five.
• Doko: "Where". "Doko" (pronounced "doh-koh") can help you find things. You can ask where the station is, where the toilet is, or where a shop is. For example, if you don't know where Tokyo's geek district Akihabara is, you could ask, "Akihabara doko?" ("Where's Akihabara station?"). If you wanted to know where the toilet is, you could say, "Toire doko?" ("Toy-ray doko?" or "Where's the toilet?"). Or if you wanted to know where retro game shop Super Potato was, you could ask, "Super Potato doko?" ("Where's Super Potato?").
Yes, in school, students learn the longer "Super Potato wa doko desu ka?" ("Where is Super Potato?"), but Japanese people do say things like, "Super Potato doko?" It's short and sweet. Remember it!
• Sumimasen: "Excuse me/Sorry". If you are taking the subway in Japan or navigating crowds of people, learn "sumimasen" (pronounced "sue-me-mah-sen"). You will use it a lot, if you bump into someone. It's just good manners to excuse yourself. Be polite.
• Wakarimasen: "I don't understand". If someone starts talking to you in Japanese and you don't understand, you can reply with a "Wakarimasen" (pronounced "Wah-ka-ri-mah-sen"). You can even add a "Sumimasen" after uttering it to apologize! Once again, being polite in Japanese is key. And after saying "Wakarimasen", if you begin speaking English slowly, whoever you are speaking with, will understand that, yes, there is a language barrier and hopefully find an English speaker to help you. Hopefully.
This list, of course, does not apply to people who already speak the lingo. Those folks will have a much smoother time because if they get into a pickle, they can express themselves in a way locals will understand. What's more, they can talk to regular folks in the language everyone speaks.
Japanese people do study English in junior high and high school. Since reading is the focus, most people in Japan do not feel comfortable speaking English. Listening is also difficult for many.
If you do not speak Japanese, I would be hesitant to start with remembering things like numbers. Shop clerks will usually show you the amount you owe, whether that's pointing to the register or even showing you a calculator. Also, since the majority of things have price tags, there might be less of a point asking the price.
The caveat is that in some restaurants, prices are written in kanji characters instead of numeric numerals. If that happens, you can point to the characters and simply say, "Wakarimasen". Or if you are feeling adventurous and have a good handle on the numbers, ask, "Kore ikura?" ("How much is this?").
The word "arigatou" (thanks) is very famous in the English speaking world, so it wasn't included in this list. Use it if people help you along during your travels.
And if you do speak Japanese and see visitors from other countries who seem to be having a tough time communicating, do the nice thing, and see if you can help them out of whatever jam they are in!
Safe travels and enjoy your time in Japan.