To prevent your PDF file (or any non HTML file) from being listed in search results, the only way is to use the HTTP
X-Robots-Tag response header, e.g.:
You can do this by adding the following snippet to the site's root .htaccess file or httpd.conf file:
<Files ~ "\.pdf$">
Header set X-Robots-Tag "noindex, nofollow"
Note that for the above to work, you must be able to modify the HTTP headers of the file in question. Thus you may not be able to do this, for example, on GitHub Pages.
Also note that robots.txt does not prevent your page from being listed in search results.
What it does is stop the bot from crawling your page, but if a third party links to your PDF file from their website, your page will still be listed.
If you stop the bot from crawling your page using robots.txt, it will not have the chance to see the
X-Robots-Tag: noindex response tag. Therefore, never ever ever disallow a page in robots.txt if you employ the
X-Robots-Tag header. More info can be found on Google Developers: Robots Meta Tag.