In a nutshell, once you divorce you dont have any rights to access or custody and in 99% of the cases custody goes to the mother, unless you can prove that she is an unfit mother.
I would not got making any accusations of adultery unless you have proof. that would mean hiring a private investigator I imagine, who would trail her and record her movements, where she goes. Either that you follow her and catch her in the act.
The guy I mentioned was British, Japanese wife, spent 3 years in the courts trying to get a divorce (if she doesn’t want one you have to petition a judge or go through mediation). A judge here will base his judgement on his definition of a marriage between Japanese. The Brit had to educate the judge about what marriage means to a western person.
You might have to see a lawyer and as the page above points out, Japan is not a signatory to the hague convention on child kidnapping, so if your wife takes off with the kids you have no way of getting them back in Japan, as she is the mother. Thats why i would be careful about divorce in japan until you have the kids locked in. In Japan you basically have no legal rights as a parent and as a father. in 99% of cases kids never see the foreign parent again.