The original specification is that robots "should," read should, read the robots.txt from the top to the bottom.
User-agent: *
Allow: /
[...]
Disallow: /
Should allow all pages on the site to be indexed. Robots.txt is not a site map. The disallow comes after the Allow and top to bottom the root plus all pages under the root are allowed.
/index.html is allowed
/directory/ is allowed
/index.html?key=value is allowed.
Case 1
Allowing https://my_website.com/webapp/page.jsp
Also allows https://my_website.com/webapp/page.jsp?query_string
This is correct. if you want to disallow query strings? disallow them before the allows in the robots.txt file.
User-agent: *
Disallow: /webapp/page.jsp?*
Allow: /webapp/page.jsp
Disallow: /
or disallow all query strings using a wildcard.
User-agent: *
Disallow: /*?*
Allow: /webapp/page.jsp
Disallow: /
Second, you should no that bots have nothing more than a moral requirement to honor the robots.txt files if they have read it and know a page should not be crawled.
It does not block bots from crawling pages disallowed in robots.txt file. If a link points to a page that the owner asked the bot not to crawl the bot may crawl the page to see what is there for the purpose of ranking the page, checking for bad things, or for whatever reason they desire.
But they really are not trying to find content you don't want indexed. There are trillions of pages on the internet. Google the biggest index only has 400 billion indexed.
If you need to disallow file types, like doc you can do that as well at the top of the robots.txt file
Disallow: /*.pdf
But changing the robots.txt files is not going to generate a request to have a file that is already in the search engine indexed removed. It will just no longer be updated.
To trigger a removal for a pdf file you would need to be through webmaster tools or by using a HTTP header, which can be done with .htacess. and do not block the file from being updated in robots.txt or the bot would never get the header telling it to not index the content.
<files *.pdf>
Header set X-Robots-Tag "noindex"
</files>
If content is on the site that is not to be public
Content on the site that should not be made public needs to be protected by password or other options provided by the server. Best to have such content in a sibling to the www directory so it does not have a URL to begin with.
But to block a directory with .htaccess
<If "%{REQUEST_URI} =~ m#^/site/includes($|/)#">
Require all denied
</If>
which would allow applications like PHP or Jakarta to read and serve the content but prevent the content from being accessed directly.