Say I have registered domain
Am I right that only the registrar NS server where I've registered the domain have SOA record describing
my-domain.example domain and therefore is the authoritative server for
Any authoritative DNS server can potentially have a SOA record for
my-domain.com. However, only the authoritative DNS provider is the one that will be serving that SOA record.
The authoritative DNS server is determined by the delegation at the registry (configured from the registrar). When you point the name servers for a domain at the registrar to the specific DNS provider A, then A will become the authoritative DNS provider for the domain.
Clients will connect to one of the authoritative servers for the provider A (as specified by the name server entries) and fetch the relevant DNS records (including the SOA) from that provider.
But nothing prevents you to have the zone configured elsewhere. In fact, if you are switching from provider A to B, you will likely have the zone (and SOA) in both providers. However, only one will be authoritative, the delegated one.
Your registrar keeps an SOA record of your domain. The SOA record has information such as your DNS nameservers. Your DNS nameservers are provided to you by your webhosting company.