9
votes
Accepted
Can Googlebot discover URLs that are not in an anchor?
Yes they do crawl everything that looks like a URL. I have made a test (in German, but red="not crawled" and green="crawled"): http://www.sirpauls.com/welchen-links-folgt-google-ein-experiment/
...
- 503
6
votes
Google Search Console reports blocked resources from DoubleClick while rendering my pages, where do these calls come from?
Doubleclick is subsidiary of Google, which develops and provides Internet ad serving services. Using Adsense with dynamic allocation and targetting will automaticlly receive more relevant adds from ...
- 33k
3
votes
Will Google remove links restricted by the robots.txt from the soft 404 list?
I wouldn't worry too much because 404 errors are not penalized. It basically represents files that have been temporarily hidden. If removed, it will clear itself from the search. I usually suggest not ...
- 2,400
3
votes
Accepted
Google Search Console: 404 errors on existing pages
If Google is reporting these pages as a 404 (and this has been confirmed with the "Fetch as Google" tool in Google Search Console), yet they appear to be perfectly accessible and return a 200 OK when ...
- 4,297
3
votes
What is the solution for "Submitted URL not selected as canonical" in Google Search Console beta?
First you have to find the duplicates. It would be nice if Google told you in Search Console which URL it thinks is duplicate, but they don't divulge this information as far as I've been able to ...
- 99.2k
3
votes
What is the solution for "Submitted URL not selected as canonical" in Google Search Console beta?
You explicitely requested an indexing of some urls, through sitemap or webmaster tools, which are duplicates without canonical. The question is rather why do you want duplicated urls are explicitely ...
- 10.1k
3
votes
Accepted
Google Search Console still reporting 5xx errors even when my server has been running flawlessly for 7 days
The screenshots indicate you are looking at the http and non www version of your site.
The home page inspection report indicates that the home page for that site (http without www) is redirecting to ...
- 2,931
3
votes
Accepted
Why is Google making up pages to crawl on my site?
Googlebot never makes up URLs. It crawls URLs because it finds links to them. If Google is crawling a URL, it must be linked from somewhere on the web.
In this case, one bad link into a page with ...
- 99.2k
3
votes
About sudden drop in crawl request of our site
It is important to firstly check somethings before any judgement on crawl requests.
1- Dont judge it too quickly and it may be a sudden drop for only one day. If it continues to be stable there that ...
- 363
3
votes
Accepted
Google Search Console reports "Page with redirect" as "errors", are they?
The Page Indexing report doesn't say these are errors, but just lets you know the reason Google aren't indexing links it has found, so that you can review if there is an actual problem.
Their ...
- 2,843
2
votes
Google reports a sudden increase in 404 errors from crawling pages
This could happen in the case you have a lot of URL parameters which usually result in a lot of pages being created -Googlebot automatically ensures that not all of these are indexed. However, when ...
- 290
2
votes
Bing webmaster indicating crawl errors on http to https 301 redirect
...my question is why is it looking at http pages when I have https pages in sitemap...
This is the key.
The answer is simple. Sitemaps are not the authority by which search engines submits pages to ...
- 32.9k
2
votes
Accepted
Google notified me that I have a sudden spike in 404's, but they are pages that don't exist
Check the search console - it will show which pages are linking to these 404 pages.
If these links are all external than there is probably nothing to worry about - just hacked sites linking to your ...
- 366
2
votes
Can 301 redirects to homepage solve 404 errors in Google Search Console?
Will redirecting to homepage be a good strategy than displaying a 404?
Probably not. Mass redirections to the homepage are likely to be seen as a soft-404s by the search engines (ie. Google) anyway. ...
- 43k
2
votes
Why is Google crawling non-existant URLs?
Googlebot crawls any URL that it finds:
Links on your own and third party websites
Text on the page that looks like a URL
JavaScript strings that look like they might be URLs
Check your own site to ...
- 99.2k
2
votes
Getting usability issues in Google Search Console for the responses of API calls
API endpoints are usually not useful to have in Google Search Results, so the best thing to do would be to block them using robots.txt as you suggested.
These API endpoints are not supposed to be ...
- 16.3k
2
votes
Google Search Console Warning: " Indexed, though blocked by robots.txt " (BUG)
This exact bug is being reported by others in the Google webmaster forum. It may be an actual bug & you might want to report it there.
- 61
2
votes
How to inspect and fix Bing's "Crawl errors"
Uncheck the options aside from the errors, and then click the SEO Reports section to see the errors. They should be listed individually, and a lot of them, (missing alt tags, multiple h1, description ...
- 915
2
votes
Accepted
Google Crawl 404 Issue for different links
The crawl repost has nothing to do with performance. This report displays, which assets the Googlebot got, and what are the results of the crawling procedure.
There are only few variants, how ...
- 10.1k
2
votes
Accepted
My website 301 response crawl requests has risen 10 times
I will assume you made no significant changes to your website recently that would explain this increase.
What is/are the cause/s?
Most likely, Google is recrawling old URLs that it had previously ...
- 1,263
1
vote
How to fix 302 errors in my Google Search Console?
The root of your problem is probably a link on every page using the relative URL "Search.php". That link will cause those many URLs with Search.php added to the end of them.
Your system seems to try ...
- 2,931
1
vote
How to fix 302 errors in my Google Search Console?
Those 302 indicate that your pages are redirected. When you switch from HTTP to HTTPS, that is what you want. You want your site from HTTP to HTTPS you want visitors and search engine bots to get ...
- 99.2k
1
vote
404 Crawl Error to almost all pdf files
To correct these crawl errors and redirect to remove the part after and including the semicolon (;) then you can do something like the following near the top of your .htaccess file:
RewriteRule ^(wp-...
- 43k
1
vote
Old category pages with URL parameters
To remove the URLs with error messages from Google search, I would log into Google Search Console for your site and use the URL parameters tool. Add the parameter error_message. It is an active ...
- 99.2k
1
vote
After changing WordPress themes, search console reports 404s for existing URLs that can be rendered as well as many images
There could be a plethora of reasons that this is occurring.
The most likely reason is probably that your webhosting provider isn't giving you 100% reliable service. This can happen if you're on ...
- 4,813
1
vote
Google Search Console is reporting an hAtom error for every page on my site
Hatom is apart of Hentry which is a microformat for structured data and by default comes enabled in WordPress. It is a common occurrence that WP themes especially those that are 'free' have one or ...
- 33k
1
vote
What is the solution for "Submitted URL not selected as canonical" in Google Search Console beta?
The reason for this error is that Google is ignoring your AMP page. Simply because you have a responsive / mobile version that AMP page is canonicalised to.
If you have a responsive / mobile version ...
- 71
1
vote
Accepted
How do I do an htaccess rewrite on an internal URL without Google crawl giving me a soft 404 error?
I can't see how your original internal rewrite would have anything to do with the soft-404 reported by Google. Google is not aware of any rewrite - this is entirely hidden from users, bots and Google.
...
- 43k
1
vote
Outdated 404 errors that are linked to from the pages themselves
I just want to make sure this doesn't end up counting as an internal 404 error that could impact my search rankings? Is it okay to leave it alone?
As closetnoc pointed out, remove public-facing links ...
- 13.6k
1
vote
Search Console: Server Connectivity Error - Connect Timeout
Added your website to a monitoring service. Then you will know if the problem is related to your hosting provider.
Only top scored, non community-wiki answers of a minimum length are eligible