The company said the deal would not affect the position of Yahoo Inc as its strategic partner.
Yahoo Japan is 35%-owned by Yahoo Inc, but its majority shareholder is mobile phone and internet service provider Softbank, which has a 40% stake.
Yahoo Japan is the most popular search engine website in Japan.
It has about a 57% share of the market, while Google's share is just over 30%. Microsoft has a share of about 3%.
The deal will see Yahoo Japan use online advertising and distributions systems developed by Google.
The announcement was made alongside Yahoo Japan's first-quarter results.
The company reported a net profit of 21.7bn yen ($248m; £160m) for the three months to June, up 13% from the 19.2bn yen it made in the same period a year ago.