It's mostly safe, yes. I had a friend send me a small gift as a test, before I posted my wishlist because I had the exact same concern. He was able to see the first/last name I used on Amazon (which is my "stage" name, not my legal name) as well as my city and state. No street address.
However, a PO Box is a small investment for that extra layer of privacy, just in case!
