I’ve found that you can typically get around most countries just fine by knowing only a few words and phrases in the local language. That’s not to say that you’ll be free of problems and misunderstandings, but knowing at least a little bit of the language the locals grow up speaking is a great diplomatic move, if nothing else. It shows at least that you don’t disrespect their culture even if you’re not a connoisseur.

Thank you. You’ll use this one a lot. If you don’t, you’re not really traveling. This is the basis of politeness and appreciation. Knowing this one in the local language is and should be expected.
Where is…? This one is pretty critical for getting around. People will usually point and gesture so you don’t have to understand whatever they’re saying. You can use this when you know the name of the place you’re trying to go, when you’ve got a map, when you’re trying to get back to your hotel and, most importantly, to find the toilet. Toilet is nearly a universal word.
Hello. Some cultures have a formal and informal greeting. It’d be nice to know both, but if you can’t go for the formal one.
I’m sorry. Know this one for when you bump into someone or do something rude, etc. The person already probably thinks you’re a jerk, don’t insult him further by hurling foreign tongues his way.
Yes and No. I lumped these together as one because they go together. Sometimes these can be difficult. For example, in many Asian languages, there is not really a way to say these things. You’ll often use yes or no when someone who doesn’t speak much English asks you something in your native tongue. They’re making an effort, you can too. Also, sometimes the situation will tell you what someone is asking you, for example, if someone bends to pour tea and says something you don’t understand, they’re probably asking if you want some.

If you go beyond just those basics and learn some other pleasantries, you’ll get a lot more smiles. There are times when you’ll be sharing a few minutes with a local and may want to speak with them or you can see that they want to speak with you.

I like… …oranges. …your country. …this city. …blondes. …beer. The possibilities are endless.
I am from… …America. …the United States. …Georgia. …Atlanta. You’ll probably use this one a lot.
What is your name?/My name is… This one is pretty common as well. I lumped them together because usually you hear one and then the other. And you use one and then the other.
What do you do?/I am a… This is the same concept – small talk that people use. It’s more common to talk about your profession in the US than anywhere else but it can still be very useful as people try to figure out who you are.
How old are you?/I am… This one can come in handy too, as you try to chat up your new acquaintance.

Now for a bonus, the one phrase you shouldn’t use in the local language, even if you know it:
Do you speak English? There is a misconception that you need to learn this in the local language, if only to be polite. It seems counter intuitive until you consider that you’d only use that phrase if you know almost none of the local language and are speaking to a total stranger.

If the stranger speaks English, he’ll understand you anyway. If he doesn’t, you’ve gained nothing by asking in his language. Actually you’ve lost something. The stranger has no idea that you only know those four words and will probably assume that you know more of his language than he of yours. So it could silence someone who may otherwise engage you in a broken conversation – not the best of worlds, but better than you had before.

In fact, it seems that the only people who use this phrase the most are Americans. Most other English speakers – even if non-native – just launch into English and hope they’re understood. It seems rude, but it’s probably the best way to get your question answered fastest. If someone knows the answer, they’ll tell you. If they don’t, they won’t.