No doubt that resource hints like preconnect and prefetch improve performance. The only pitfall could be preconnecting too many domains. Based on an old StackOverflow thread (https://stackoverflow.com/questions/55445293/is-there-a-limit-to-how-many-domains-we-should-dns-prefetch-preconnect-with-ch) preconnecting more than 4-6 domains could consume resources on the client-side and generally not recommended (subject to testing in real-world scenario). But since you are speaking about a single domain, you should be able to preconnect without adverse effects.
A good practice is to issue a preconnect followed by a prefetch as fallback. The thread I had mentioned has the following example
<link rel="preconnect" href="http://example.com">
<link rel="dns-prefetch" href="http://example.com">
Regarding implementation, you should be fine by adding these within head tags.
For preconnect hint specifically, another implementation method (may be more effective) is via a http header. (In this case, I would recommend issuing preconnect via http header and prefetch from within the html document.) The below URL discusses both the implementation methods.
https://web.dev/preconnect-and-dns-prefetch/
cdn.jsdelivr.net