I have a domain like example.com under which everything is blocked and links to an SPA.
Like example.com, and we've disabled everything like so:
User-agent: *
Disallow: /
Now this SPA uses parameters to fully control content on the site, such as for example:
example.com/#/home?project=test_project
Now there are many test_project values we use, and 100% of the time the content being displayed from these projects should never be crawled. So current rule works.
However we do get the request now that for example test_project_a specifically needs to have all its pages crawled, meaning we need some kind of exception to the Disallow rule whenever specifically the value of project is equal to test_project_a, but still apply to any other URL where this parameter's value is different.
I have seen on Google's documentation that there are some ways to manipulate parameters mostly to "block" some specific parameters but it doesn't appear to be based on the exact value, only the parameter name, for example it will check project= but not test_project_a, at least as far as I understand.
Does anyone know any way to achieve this?
#from your URLs? Here is Google's guide to fixing SEO for an SPA that includes that suggestion: codelabs.developers.google.com/codelabs/…#is called the "fragment identifier." It typically instructs browsers to scroll to a specific place in a document. Any "parameters" used in it aren't really parameters. There is no formal structure for it. When you use a#right after the domain name, there is only one document to download from your site which can really throw crawlers for a loop because they have to "crawl" virtual pages that look like they don't exist.