I have just been reading about an English man called William Adams who drifted onto a Japanese beach in 1600 and stayed and made himself into a samurai! I had no idea foreigners even lived in Japan in the 1600s, let alone made it to the ranks of the rich.
Now I am no historian, and I was reading this in Japanese, so I have no idea how accurate this is, but from what I gather, he got himself friendly with a guy called Toyotomi, who must be the first known gaijin whore. Basically, he worked out that this guy had lots of weapons on his boat, and a nice trade link to Europe, so he put him on the pay role and set up in business. He later went on to get himself a wife and two children and when given the chance at the age of 49 to go back to England, he stayed in Japan.
He is said to be the first Englishman to land on Japanese shores, so is like some sort of forefather to us. If you care to find out more Wikipedia has a very thorough entry on him.
This I believe is a picture of the man in question, but one can never be too sure.