Timeline for Why doesn't Google follow the links on my site while crawling?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 10, 2023 at 18:00 | history | tweeted | twitter.com/StackWebmasters/status/1612871929861382144 | ||
| Aug 26, 2022 at 9:23 | comment | added | 144226734 | Thanks for all of your inputs. | |
| Aug 26, 2022 at 9:23 | comment | added | 144226734 | @RohitGupta But it did crawl it again, but ignoring all of its links. I can manually request it again. It doesn't help me understand that blackbox that is the google bot, though. But I guess nobody really knows what it does exactly. :/ | |
| Aug 26, 2022 at 2:21 | comment | added | Wayne Smith | @RohitGupta Google likes what others like. Before being able to submit one's site to google we had to wait for google for find a signal that it liked the page. IE a link from another site or sometimes a few. At present Google does not seem to be picking up much new stuff ... if history repeats it will do a purge to make room. | |
| Aug 26, 2022 at 1:50 | comment | added | Wayne Smith | I would get links from other sites. For example you could put the link in your network profile across StackExchange. Googlebot should find that and consider the page more interesting ... post on twitter, youtube, (googlebot likes those), facebook, (bing likes facebook) ... etc, and et la. Google is about to do another major update ... May bring in a lot of new fresh content ... The update is called helpful content and may cause a lot of people to react. It may clean out a lot of old information that google deems as more of less the same as 1000 other sites. | |
| Aug 26, 2022 at 0:46 | comment | added | Rohit Gupta | I got the impression that if in the first pass for that link it didn't like the page, it would never crawl it again. Have you tried telling Google to reindex it ? | |
| Aug 25, 2022 at 16:23 | comment | added | 144226734 | @StephenOstermiller Thank you =) I have checked the logs but nothing particularly alarming. Some calls from several crawlers, but errors are seemingly only from blackhats trying weird URLs. All other calls that I guess was Google, are the same ones listed inside Google Search Console. (Main page + some JS + some images) | |
| Aug 25, 2022 at 16:20 | history | edited | John Conde♦ | CC BY-SA 4.0 |
deleted 118 characters in body
|
| Aug 25, 2022 at 16:19 | comment | added | Stephen Ostermiller♦ | I like your site, BTW. I have my own sudoku software and site (qqwing). Your solver rates puzzles with more granularity than mine and looks like it tests for a couple techniques that mine does not. | |
| Aug 25, 2022 at 16:17 | comment | added | Stephen Ostermiller♦ | Yes, it should crawl all the linked pages. I assume you are looking in your server's log files to know what Googlebot is accessing? | |
| Aug 25, 2022 at 16:15 | comment | added | 144226734 | @stephen It was first uploaded ca. 5 weeks ago. I immediately asked Google to crawl via Search Console, which was then done on July 27th. Google revisited/recrawled the site since then three or four times, but it just keeps recrawling the same URLs (main page, and some resources). I would think, that once Google crawls a page even once, it should at least now know the subpages' URLs, even though they themselves would be indexed way later. Isn't that correct? | |
| Aug 25, 2022 at 16:05 | comment | added | Stephen Ostermiller♦ | How old is the site? You say "in the past couple weeks" which makes me suspect that it just hasn't been around very long. | |
| S Aug 25, 2022 at 15:37 | review | First questions | |||
| Aug 25, 2022 at 16:21 | |||||
| S Aug 25, 2022 at 15:37 | history | asked | 144226734 | CC BY-SA 4.0 |