For the most part, Google doesn't care about what tags you use in your HTML. (The few exceptions being things like titles, meta tags, and structured data.) You could implement your dialogs with
<dialog> tags or with
<div> tags and it probably won't change your SEO or how Google sees your site.
No matter how they are implemented, dialogs are not usually good for SEO. They usually have little content and they happen in response to events such as a user clicking something. Google isn't going to want to index thin content such as a couple lines of text and a couple buttons from most dialogs.
That usually isn't a problem because if dialogs appear in response to user initiated events such as clicks, Googlebot won't be able to crawl them anyway. Googlebot doesn't simulate clicking on anything in your pages. It simply loads the page and indexes the text that appears on or shortly after document load with no interaction.
If you are assigning URLs to your dialogs, Googlebot is likely to find those URLs and crawl them. You should ensure that Google doesn't try to index the content from those dialogs. You should either block crawling of the dialog URLs using
robots.txt or you should including
noindex meta tags in the dialog page.