0

I have been searching for an answer to this for a while now, but haven't posted a topic since I am under the assumption that the answer will be pretty straight forward and that I will slap myself in the face when I finally discover the answer.

I am attempting to index all of the webpages located on the sitemap here:

https://www.leagueofsmurfs.com/sitemap.xml

Unfortunately, when I run a filtered search on Google using the following directive, the results do not display all of the webpages referenced in my sitemap.

site:https://www.leagueofsmurfs.com/

My robots.txt file can be found here:

https://www.leagueofsmurfs.com/robots.txt

What am I doing wrong? It seems that Google does not process any of the webpages with a subdirectory.

  • 1
    This answer will explain how Google uses sitemaps: webmasters.stackexchange.com/questions/74393/… It may have nothing to do with anything you have done. As well, it just may simply be that your sub-directories have not been indexed yet because it is a number of links from the home page and Google just has not gotten that far yet. It takes time- trust me on that. It can take months and most sites are not fully indexed. – closetnoc May 3 '15 at 21:27
1

You should define your sitemap contents as an XML file. and that's what the first line in this code does:

    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">
    <url>
    <loc>
    https://www.leagueofsmurfs.com/
    </loc>
    <changefreq>
    monthly
    </changefreq>
    <lastmod>
    2015-05-01
    </lastmod>
    <priority>
    1.00
    </priority>
    </url>
    <url>
    <loc>
    https://www.leagueofsmurfs.com/products/summoner-leveling.php
    </loc>
    <changefreq>
    monthly
    </changefreq>
    <lastmod>
    2015-05-01
    </lastmod>
    <priority>
    0.80
    </priority>
    </url>
    <url>
    <loc>
    https://www.leagueofsmurfs.com/products/level-30s.php
    </loc>
    <changefreq>
    monthly
    </changefreq>
    <lastmod>
    2015-05-01
    </lastmod>
    <priority>
    0.80
    </priority>
    </url>
    <url>
    <loc>
    https://www.leagueofsmurfs.com/products/influence.php
    </loc>
    <changefreq>
    monthly
    </changefreq>
    <lastmod>
    2015-05-01
    </lastmod>
    <priority>
    0.80
    </priority>
    </url>
    <url>
    <loc>
    https://www.leagueofsmurfs.com/terms.php
    </loc>
    <changefreq>
    monthly
    </changefreq>
    <lastmod>
    2015-05-01
    </lastmod>
    <priority>
    0.70
    </priority>
    </url>
    <!-- Privacy Policy -->
    <url>
    <loc>
    https://www.leagueofsmurfs.com/privacy.php
    </loc>
    <changefreq>
    monthly
    </changefreq>
    <lastmod>
    2015-05-01
    </lastmod>
    <priority>
    0.70
    </priority>
    </url>
    <url>
    <loc>
    https://www.leagueofsmurfs.com/refund.php
    </loc>
    <changefreq>
    monthly
    </changefreq>
    <lastmod>
    2015-05-01
    </lastmod>
    <priority>
    0.70
    </priority>
    </url>
    </urlset>

Based on your sitemap, you are only interested in indexing certain pages, not entire subfolders, and its best not to index URLs pointing to only subfolders unless you plan to have an open directory of downloadable files to the world. It's also a security benefit to prevent directory listings from appearing on your site.

So far, your security is decent since I received no content from visiting:

https://www.leagueofsmurfs.com/products/

Once you fix your XML sitemap, go to google webmaster tools and access leagueofsmurfs.com domain (and verify it if you haven't already done so) then choose the option to submit a sitemap and google will tell you if there are issues. If not, then wait at least half a day to give google a chance to process your sitemap because google likes to spend time at things.

I do apologize for my lack of indentation and lack of commenting in my code but its better to make the code as compact as possible to save on bandwidth, as search engines have a limited crawling budget for every website.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.