^(https?://[^/]+/)([^/]+/)+[^/]+(/[^/]+)*/\2
This would seem to be overly complex for this particular task (and. And, like you say, doesn't match the instance when the two "duplicate" directories are next to each other. But neither does it match the first two examples in your question that don't have the language-code prefix in the first path-segment (ie. /abc/def/abc/ and /xxx/yyy/xxx/). In other words, it doesn't match when the first duplicate "directory" appears in the first path-segment.
The following would suffice, to match all 4 examples where a whole path-segment is duplicated anywhere in the URL.
/([^/]+)/(.+/)?\1/
(The / prefix and suffix are part of the regex, they are not delimiters.)
Broken down, the regex matches as follows:
/([^/]+)- This matches a whole path-segment and captures the part between the slash delimiters./(.+/)?- This matches any additional path-segments (or none) that might occur before the "duplicated" path segment.\1- An internal backreference that matches against the first captured group. This "tests" for a duplicate./- A literal slash (since your URLs all appear to end in a trailing slash).
This does assume you don't have legitimate path-segments ("directories") of the same name as the hostname. eg. https://www.example.com/at-en/def/www.example.com/hscxv/ would also be caught by this regex. (Assuming the absolute URL is indeed being tested here.)
As mentioned above, this regex should be sufficient for this specific "debugging" task, but it's not particularly efficient due to the backtracking that can potentially occur.
Although, as @StephenOstermiller mentioned in comments, the incorrect use of relative URL-paths in your HTML anchors could also result in "malformed" links of this nature. However, from your second lot of examples, if that was the case then your links would also seem to be missing the language-code prefix, so simply making them root-relative (by including a slash prefix) would not be sufficient.