I've used Google's "submit URL" feature in hopes of getting certain URLs indexed (or updated in the index), but I'm curious if submitting the URLs you want to be indexed via the xml sitemap is sufficient? I see the "submit URL" feature as a next step if a URL on my xml sitemap is still not being indexed -- is this correct? Or do they have two totally separate functions?
-
I don't think either is likely to get a page indexed. Google chooses not to index lots of pages that have been submitted.– Stephen Ostermiller ♦Commented Jun 6, 2018 at 23:15
-
@StephenOstermiller Google's definitely going to do what it wants in that regard, but I'm just wondering if the methods differ in intent at all - do you know? Was one created for one purpose and the other created for a different purpose?– KjenkinsssCommented Jun 7, 2018 at 0:16
2 Answers
There is generally no reason to submit URLs to Google at all. Googlebot is perfectly capable of finding your pages by following links from other pages. Google will also come back and re-crawl pages at regular intervals, so when you make changes, all you need to do is wait.
Sitemaps are widely mis-understood. It is a common belief that they help get pages crawled and ranked better. If Google can't find a page through links then it is possible that including a URL in the sitemap will get Google to crawl and index it. However, without any links pointing to a page, it is unlikely to rank for anything with any competition. Sitemaps just don't help with rankings at all. Most sites don't need sitemaps. At best, they give you more stats about your pages in Google Search Console. See The Sitemap Paradox.
Google's submit URL feature is similarly useless. The only time I can imagine that it should be used is if you create a single page that isn't linked from anywhere and don't have it in a sitemap. If you have that page in a sitemap, there is no need to submit it separately.
Neither sitemaps nor submitting URLs will encourage Google to index or re-crawl pages. Those features are mostly for new pages. Even though sitemaps have "priority" and "lastmod" fields, Google says that it ignores them because webmasters don't usually fill them in accurately. See Sitemap update lastmod and ping. what is the advantage?
If you want Googlebot to come back and re-crawl a page right away, the only tool that Google gives you is the Fetch as Google tool. Using that you can get Googlebot to immediately crawl the page and update its content in the Google index. However, if a page isn't indexed, it is unlikely to get Google to start indexing the page. That tool is limited to 10 fetches per day, so it won't work if you update a large number of pages. For more info see Need Google to recrawl the pages.
If you have a page that isn't being indexed, that is usually because it isn't linked well, it doesn't cover topics for which people are searching, or because it is duplicate of something else. See Why aren't search engines indexing my content? If you want to get more pages indexed you need to make you content unique, cover topics for which users search, and get more inbound links to your website. It is almost never an issue of getting Googlebot to come take another look.
-
1Just to back up @Stephen Ostermiller's quote, "without any links pointing to a page, it is unlikely to rank" ... I once had an internal link, on my Homepage, pointing to a Category page. Unfortunately, it contained a typo. As such, search engines saw it as a Broken Link. Despite the target page, being correctly listed, in the Sitemap, search engines had not indexed the page. As soon as I corrected the typo, search bots were able to successfully crawl to the Category page, via the Homepage link. As such, this resulted in the Category page being indexed by search engines.– CraigCommented Jun 8, 2018 at 0:10
-
@Craig that is super interesting! Thanks for including that example to illustrate how sitemaps might not be enough to get a page indexed. Commented Jun 9, 2018 at 5:49
-
I think you are making many assumptions here that don't really hold in many different cases; for instance: with large, eCommerce websites, a sitemap is practically a requirement to get all your content crawled and indexed (and yes, just being crawled doesn't mean it will get indexed, but if you Google can't find it they won't crawl it and for large eCommece stores, a sitemap is by far the best choice for this). Commented Feb 10, 2020 at 13:04
-
@AndreGuelmann Even for big ecommerce sites, it is far better to link your products together than to use sitemaps. That is a big part of the reason that so many sites have lists such as "related products", "commonly bought together", and "you might also like". That is why this site has the "Related" question list in the right column. Commented Feb 10, 2020 at 16:38
-
@AndreGuelmann I have a lot more detail in some other answers: What is the consensus on handling pagination in 2019 on big result sets? and Impact of blocking site search from googlebot, but listing content pages in sitemap.xml and Google isn't finding content in paginated lists Commented Feb 10, 2020 at 16:39
To answer your question, submitting a URL in GSC is for changes made on that URL. Submitting it adds that URL to the queue to be crawled in a timely manner. If the URL is in the sitemap and a change is made, the crawler will pick up on it on it's own time.
submit URL: Hey, I made a change, can you come check this XML Sitemap: Hey, these URL's exist on my site. Here they are for reference.