“The decision to move on from the Highlanders was the toughest... I have ever had to make in my coaching career,” Joseph said in a statement on New Zealand Rugby's allblacks.com website.
“For me, the challenge of the next few years with Japan is what I need to keep moving forward.”
Japan had been in the market for a new head coach after Eddie Jones, who guided them to an unprecedented three wins at last year's Rugby World Cup, left the post.
The Australian joined South Africa's Stormers before being appointed England coach through to the next World Cup in 2019, which will be hosted by Japan.
Joseph, the 46-year-old former All Blacks loose forward, knows all about the sport in Japan, having played club rugby there for five years. He also represented the Brave Blossoms at the 1999 World Cup.
“They are the hosts of the World Cup in 2019 and to be involved in the team and host nation is probably the biggest carrot,” he told reporters in Dunedin.
Japan host Scotland in two tests on June 18 and 25 and despite Super Rugby taking a four-week break to allow for the international window, Joseph will not be joining up with his new side.
“My answer to that is that I am committed right to the end with the Highlanders, so I didn't want and couldn't be involved with it,” he said of the Scotland fixtures.
“If I was going to do it justice and show the integrity that I am passionate about with the Highlanders, then I couldn't be involved with another team. So that's the way it is.”
Joseph earned 20 All Black test caps from 1992-95 and said he still retained ambition to return and coach New Zealand one day.
The Highlanders were disappointed to lose the coach who took them to a first Super Rugby title last year.
“Obviously, we have mixed emotions about Jamie moving on,” Highlanders chief executive Roger Clark said.
“We would have liked him to sign a new contract with our organisation. However, we understand his reasons for signing with Japan.” – Reuters
Original source: Former All-Black named Japan coach