0

I have created a web site using asp.net mvc-5 . and I have hosted it to one of the hosting provider and I register the web site with Google app.

Now after 1.5 months of publishing the web site online it starts to be returned by search engines mainly Google and Bing. But the search result will not list my web site links (such as contact, our project, our team, etc). now I read that I need to create a site map for my web site as follow:-

1.I need to create a robots.txt inside the root of my asp.net mvc and I need to specify the location of the site map as follow:-

User-agent: *
Disallow:

Sitemap: http://mywebsite.com/sitemap

2.Then I need to create an xml which list my main links.

But I have the following 2 questions regarding creating and submitting a site map:-

  1. If I do the above steps , then will search engines be able to understand the site map automatically, or I will have to submit the site map to Google and to Bing ?

  2. Is there any change that my site map will be returned in the search result without creating a sitemap.xml . I mean can search engine automatically extract my site urls and create a site map ?

Thanks

  • 1
    I did not understand question 2. Sorry. Search engines do not create sitemaps, but will look for updates to your sitemap from time to time. You have to create the sitemap. This is easy to do with code or you can find a tool that works for you. I recommend writing code (personally). But I have also used a tool to get started. I do not have a recommendation for you though... it has just been so long ago. – closetnoc Jan 17 '16 at 3:10
  • @closetnoc ok so I have to create a sitemap.xml and then inside the robots.txt I need to provide the sitemap url is this correct ? so my question now if I have to subsite (register) the sitemap.xml url to Google and bing .. or they will automatically reads the robots.txt and find the site map url ?????? – John John Jan 17 '16 at 3:16
  • 1
    Search engines do look for sitemaps on their own - at least Bing does fairly aggressively. Google will pick it up eventually. I recommend creating an account on Google at least to help guide you with analytics. Bing may be good too. In Google Search Console, you can submit the sitemap and see metrics as they audit the sitemap against the number of pages they have indexed. These numbers shift around a bit from time to time. If they do not match, that is not always an indication of a problem. Do not freak-out. Often it is okay and just part of the process. – closetnoc Jan 17 '16 at 3:27
1

The search engines should find the sitemap as you have it. However, I would create accounts for Google and Bing anyway and submit them through their tools.

Having said that, there may be no reason to do this. SEOs like to advise creating sitemaps without context. It is often an unnecessary step. Sitemaps are only required for extremely large sites or sites with content behind a paywall or login. If this is not you, then do not bother.

Sitemaps are generally ignored if Google can crawl your site the traditional way. What Google will do is compare the entries of your sitemap to what it can find on it's own to make sure that your site is properly crawled.

If you are thinking that sitemaps are required for site links, you would be incorrect. One has nothing to do with the other. Google does not guarantee site links for any site and who gets site links changes often. It is possible to gain site links to only lose them again later. One requirement is traffic and popularity. If your site is new, then site links are not automatic and you will have to be patient.

  • thanks for your reply. but if it is not that important to submite a site map as you mentioned... then why Google provide facilities to submit site maps and to build and test the robots.txt ? now my website have been published 4 months ago and still when I search for it using its url I only get the home page link , without any sub links ,, so I though that building a site map and submit it to Google and Bing will help to get my site map links when users search for our site .. is this correct ? – John John Jan 17 '16 at 3:12
  • 1
    Just because Google gives you a way to submit a sitemap does not mean it is a necessary thing. It certainly does not hurt. It is just more work and most of the time unnecessary work. Site links are not guaranteed at all. It is not automatic. They only appear when a site becomes popular within search for branded searches. 4 months often will not be enough. There is also the markup to help enable this: schema.org/SiteNavigationElement Make sure you have this markup at least. – closetnoc Jan 17 '16 at 3:22
  • thanks for your valuable replies... but I am a bit confused now. Question 1) is there a relation between submitting a site map and having my site map links appear in the search result ? Question2) will Google and Bing automatically reads the robots.txt and find the site map url ,,or I have to subsite the sitemap.xml url to the search engine (maybe this is not required but will make the process to show the site map links faster is this correct ?) – John John Jan 17 '16 at 3:27
  • 1
    Okay. I want to gently correct something. You are talking about Site Links not Site Map Links. The Site Links have nothing to do with a sitemap. It never huts to have a sitemap. It is just that SEOs give the wrong impression about how important they are. You have to remember that most SEOs are not technical people and parrot the same advice because someone else said it. Try the markup in the schema.org link I provided. You will see Site Links as your site grows. It just may take a while - do not be surprised by this. Search engines are very slow - remember that the Internet is HUGE! – closetnoc Jan 17 '16 at 3:33
  • 1
    Two things. You should always have a robots.txt file even if it is empty. Search engines prefer it that way. The advantage to having a sitemap is that you can definitively tell search engines about your pages and they can tell you how many of your pages they know about. It is just a bit of small information either way. If a sitemap is something that is easy to do, then I say, Why not? If it is difficult to do, then I say wait until it gets easier to do. ;-) – closetnoc Jan 17 '16 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.