I have a VPS in one provider (Provider A), and I have set the email server all works ok. in another provider (Provider B) I have the domain registered. So, let's assume it is mydomain.com
On the provider B: I had created a subdomain called mail.mydomain.com, configured the dns as follow:
HOST Type Result
(empty) A 123.123.123.123
www A 123.123.123.123
mail A 123.123.123.123
Then on mail exchange records I've configured as follow:
HOST Priority Result
(empty) 30 123.123.123.123
On the subdomain mail.mydomain.com, I have configured the dns as follow:
HOST Type Result
(empty) A 123.123.123.123
www A 123.123.123.123
mail.mydomain.com
set to your IP, a MX record onmydomain.com
set tomail.mydomain.com
and you probably also wanna set up two TXT records: DKIM and SPF…mail.mydomain.com
for SMTP, IMAP and POP. Have you actually set up your mail server and opened that ports on a firewall?