I have a webpage. It has a form and the "results" section is blank. When the user searches for items, and a list that pops up, he/she chooses one option from list and then the corresponding results are displayed in results section.

I once decided to log every ip,url of person with time that visits my page. One ip was, and on doing google search I came to know it is ip of google bot.

link for whatmyipaddress google bot

Now when I searched for the links that this ip visited, it was like this:


then afterwards it incremented in steps of 1, like 400,401..

But people search for strings and not numbers. And because googlebot searches for numbers like this, I think the corresponding content is never displayed and so my page content is never indexed, even though it has rich content.

So I want to ask you is that in order to show google bot all the content that the webpage has, should I list all the results in index page and ask users to enter string to filter results?

1 Answer 1


You should create a site map that describes all the content of your website.

  • I tried to use automated site map generator, but it searches for links in same way, like search?id=1,2,3,4,5...so on. Should I create site map manually?
    – S P
    Commented Nov 20, 2012 at 12:11
  • google bot and other bots can try whatever they want. You need to run over you db and generate all possible urls.
    – YardenST
    Commented Nov 20, 2012 at 12:24
  • all possible urls? Meaning all possible keywords? Like for keyword "jaguar", we have "j","ja","jag"..like this?Is there is a limit to all possible urls?
    – S P
    Commented Nov 20, 2012 at 12:26
  • there is a limit of 50,000 urls or 10 MB per sitemap, but you can have as many sitemaps as you want. Also register to Google Webmaster Tools, there you can tell google about your sitemaps and other usefull tools
    – YardenST
    Commented Nov 20, 2012 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.