He is the highest-ranking Israeli official to have laid a wreath at the war memorial.
Mr Ayalon said before the trip that it was “aimed at further enhancing Israel's relationship with Japan.”
He added: "We have many areas of mutual interest and are both ancient civilisations with shared values."
Mr Ayalon’s itinerary while in Japan includes meetings with Yutaka Banno, Japan’s Foreign Secretary on the peace process and nuclear concerns, as well as a discussion with Japanese businessmen on trade ties.
He is also set to visit an art exhibition hosted by Prince Tomohito.