I have a WordPress blog and I have no intention to have a featured image or any image in my post content.
Will this affect my SEO?
Note: I understand I won't get search traffic for images.
Whether it affects your SEO or not, it depends on user search queries. If your image alt text and caption have some keywords which are not included in any of your paragraphs, and those caption or alt text gives you 1% traffic to a specific webpage, then obviously removing those images with their caption and alt text detail will give you 1% less traffic on those keywords only (on other keywords nothing will change).
So first analyse your blog post on Google search console (Search analytic tool), and check whether alt text and caption give you any traffic benefits from general search results or not. If it does give you some traffic and you still want to remove those images then you can include those keywords in your other paragraph to save 1% traffic loss.
And, if you're not using alt text and caption
<figcaption> tag at all, then it's totally fine to remove those images, because as you said you're not concerned about image search engine.
By the way if you read old lazy seo from someone who said "by adding images on your blog post will boost your search position", then it's not totally right. Google always analyses user queries first, and if your blog post image is not related to it, then it will add zero impact in SEO.
But yes, adding images on your blog post is plus point when you share your blog post on social media sites because it displays rich snippet/preview, and it is good for CTR.