I have a domain like
example.com under which everything is blocked and links to an SPA.
example.com, and we've disabled everything like so:
User-agent: * Disallow: /
Now this SPA uses parameters to fully control content on the site, such as for example:
Now there are many
test_project values we use, and 100% of the time the content being displayed from these projects should never be crawled. So current rule works.
However we do get the request now that for example
test_project_a specifically needs to have all its pages crawled, meaning we need some kind of exception to the
Disallow rule whenever specifically the value of
project is equal to
test_project_a, but still apply to any other URL where this parameter's value is different.
I have seen on Google's documentation that there are some ways to manipulate parameters mostly to "block" some specific parameters but it doesn't appear to be based on the exact value, only the parameter name, for example it will check
project= but not
test_project_a, at least as far as I understand.
Does anyone know any way to achieve this?