If you're mostly dealing with Latin characters
Only English (or actually Latin characters) is much better for SEO. I have noticed that some of my URLs that contain stuff like ä
lead to Google getting confused because of URLs like /g%C3%A4stebucheintrag
. In the long run this can really add up.
So if you have the choice, stick to plain English characters.
Of course there's also the typing issue, but not many people type addresses these days.
If you're mostly dealing with non-Latin fonts
E.g. Japanese, Hebrew, Chinese, Arabic, go for the URL that people may still be able to type. This will also be better for SEO than transliterated URLs, which is the opposite from the above case.