I have one folder named Frontend and under this folder I have subfolder names.
Like:
Images
CSS
JS
Font
I want to disallow below folder:
/forntend/css/
/forntend/js/
/forntend/font/
but I don't want to disallow /forntend/images/ or /forntend/
If I am adding this text to robots.txt. Is it correct or not?
Disallow: /forntend/css/,
Disallow:/forntend/js/,
Disallow: /forntend/font/
Disallow: /forntend/css/, Disallow:/forntend/js/, Disallow: /forntend/font/
There should be no comma (,) at the end of the path argument. (This isn't an array of elements.) The EOL separates each directive. The directive is prefix-matching, so any trailing , (comma) will match a literal comma.
It's not required, but you should leave a space after the directive name for readability and consistency. ie. Disallow: /forntend/js/, not Disallow:/forntend/js/.
You obviously need a User-agent: directive at the start of this group, otherwise these directives will be ignored.
The path is also case-sensitive, so /forntend/css/ is not going to match /Forntend/css/mystyles.css.
It is against Google's recommendations to block CSS and JavaScript (and probably Font) files that affect page rendering as it also prevents Google from rendering (and consequently indexing) the page properly. As noted in the Google help docs:
if the absence of these resources make the page harder for Google's crawler to understand the page, you should not block them, or else Google won't do a good job of analyzing pages that depend on those resources.
So, the above should be written as:
User-agent: *
Disallow: /forntend/css/
Disallow: /forntend/js/
Disallow: /forntend/font/
This blocks all robots from crawling all URLs that start /forntend/css/, /forntend/js/ or /forntend/font/ and allows everything else. "Allowing" is the default behaviour.
Note that this prevents "crawling", it doesn't necessarily prevent "indexing" if the URL is linked to from other sites. As noted in the Google help docs:
A robotted page can still be indexed if linked to from from other sites
Test with Google's robots.txt tester tool in GSC.
Further reference:
