2

I frequently submit URLs using the Submit URL method here:
https://www.google.com/webmasters/tools/submit-url

I have noticed that when I submit URLs that need to be re-crawled, they do not get indexed. I frequently submit URLs that I do not own here, so would submitting URLs that I do not own effect the ability to submit URLs that I do own?

Improve this question
3
  • 1
    Quoted from that page. We don't add all submitted URLs to our index, and we can't make predictions or guarantees about when or if submitted URLs will appear in our index. This is probably not the best place for this. For example, Search Console allows for the same thing for verified sites. Otherwise, no SE is real time and will index pages according to it's own schedule. If you are trying to hurry Google up, that is a bad idea. Fresh sites will get visited quickly. I have had new pages show up in less than 20 minutes without doing anything at all. Strive for freshness.
    – closetnoc
    Apr 15, 2018 at 20:15
  • No... but there's a virtual reward for wasting your time. Google often ignores that submission. To speed up and have better index times you need to improve your authority.
    – Simon Hayter
    Apr 15, 2018 at 22:41
  • I had the same issue, I am pretty sure google penalize for frequently using fetch as google for re-indexing. Now my concern is how long would it take to release the flag google has put on my site.
    – Perthian
    Oct 19, 2018 at 3:38

3 Answers 3

Reset to default
5

Unless you are effectively spamming Google with 1000's of submissions then there would be no reason to receive some kind of penalty that would effect your own site submissions.

However, for your own sites you should really be using the "Fetch as Google" tool within GSC - which is built for this purpose and allows you to more closely monitor your index status. The "Fetch as Google" tool within GSC allows you to "Fetch" or "Fetch and Render" your URLs (to see exactly what Google sees) and then allows you to submit the URL (and optionally all sub-URLs) to Google's index via the "Request indexing" option.

I have noticed that when I submit urls that need to be re-crawled, they do not get indexed.

As @closetnoc has already pointed out in comments, there are no guarantees that submitting a URL for indexing will result in that URL being indexed and returned in the SERPs. Crawling and indexing are two separate things.

Improve this answer
1

I do think there is likely some kind of algorithmic scoring regarding the submitting URLs feature. If a user were to submit thousands of URLs this would be too much and those submissions probably wouldn't be treated the same way as someone who submits just a few URLs through that service. -That's my guess.

I think it's best not to abuse the submit to Google tool too much. It's useful for having Google discover a site or a page that it hasn't before, but beyond that probably not so much. Once Google discovers a site or a page it will decide if it wants to crawl it, and then internal links pointing to your other pages is a much more efficient way to have Google crawl and index your site.

As others have mentioned, there's also no guarantee that Google will crawl or index pages just because they've been submitted through the Submit to Google tool. Google chooses which pages and sites to index based on many factors.

Improve this answer
1
  • I think it's best not to abuse the submit to Google tool too much. It is like poking a bear to me...
    – closetnoc
    Apr 16, 2018 at 2:42
1

A personal anecdote.
A few months ago, I noticed a huge drop (30%) in my site traffic. The traffic remained low from that point on. Thinking about what might have caused this drop, I looked at my site's git logs, sitemaps, etc. Then I noticed that the traffic saw that sudden dip right around 36 hours after I had run "fetch as google" on a URL many times.

36 hours earlier:
I was used to running "fetch as google" religiously after every article I published. I would then check google to ensure the article is indexed. But that one day, Google just wouldn't index my URL. Also, there was no news of any algorithm change by Google (per Mozcast, SEMrush etc.). Assuming it was a glitch, I ran fetch as google a few more times. Still, no luck.

Wondering if Google crawler Gods thought my article is not unique or valuable enough, I edited the article to make it lengthy and "more unique." Then I did fetch as google a couple more times. Still, Google just wouldn't index the page.

My stubbornness didn't get my anything, and I began to think by running fetch as google too many times, my site got flagged or something. I still have to check about this with search engine experts like Rand Fishkin(MOZ), Neil Patel etc. The cynicism that followed could be entirely in my head only, but ever since that point, I have been very careful about using fetch as google.

I am a changed man.

Improve this answer
4
  • 1
    The problem I have with this is that for Google to actively penalise accidental/innocent overuse of a tool that every webmaster is positively encouraged to use makes no sense. And to penalise "silently" in such a drastic fashion really makes no sense at all. The Google docs do state that "you have a daily limit of 10 fetches", however, in my experience this just seems to prompt a human check (and possibly a notice to "wait a few minutes" if you are hitting it hard) - but after this check you can continue to fetch URLs.
    – MrWhite
    Apr 16, 2018 at 0:11
  • (Aside: "Fetch as Google" alone doesn't submit the URL for indexing, you must also hit the "Request indexing" button.)
    – MrWhite
    Apr 16, 2018 at 0:11
  • Setting aside GSC, submitting g a URL can be abused so of course there is an algorithm to combat it. I would not expect a penalty for GSC, but for other submit forms this could be a possibility.
    – closetnoc
    Apr 16, 2018 at 2:44
  • @MrWhite : That's the thing I don't like about google. They've become so big that every business is dependent on them, but they are not concerned about explaining the reason behind their decision. The whole Search engine industry is based on guesswork, except for some grand announcement by them that contains the same generic message of improving the end-user experience
    – deltavin
    Apr 16, 2018 at 12:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.