My website has lots of links that are linking to it from different websites, but I don't see any visitors from China or Japan.
Would translating my website help to start seeing new visitors from those countries?
Yes, The problem of not much traffic from China and Japan is mainly because of the language barriers.
If you have a localized website, it definitely will help.
But content is always the king.
You will get more traffic from China if more of your website pages could be indexed by Baidu,
Baidu is working quite different from Google.
Are your relevant search terms are generic and popular, that are searched by users in each country in their own language? (such as, "learning French"?) Then translating it and doing some SEO work should definitely help with Baidu, which for these kind of terms shows results in Chinese.
Yes, if you are having seo friendly keywords that are globally searches, then you need to put them in the language of the country as local people know their language only. You can change the language of your website as per country. showing keywords in their language is definitely going to help in regards to SEO. Do not forget to change the Meta tags of the site as they play an important role.