1

So I have created a video website which has thousands of videos and thousands of videos get added to it on a daily basis. Here is my problem :-

I have created a website which basically loads the skeleton in html and puts all the content through javascript and Ajax. The problem is search engines aren't going anywhere except for the home page.

Is there a way say in robots.txt where i give a link to a single html which has links to all these videos ?

I agree my site is not accessible for a non-javascript user but stats show that this ratio is very low ( > 0.2 % ).

Is there a way I can still keep the complete AJAX website and still get each individual videos listed on google ?

Improve this question

2 Answers 2

Reset to default
1

You should add sitemap.xml to your site which gets updated with each new video added. Refer google's documentation for sitemaps.
About Sitemap

Improve this answer
3
  • +1 You're presumably using some form of CMS to handle all this content. You need to work out a way for it to produce a sitemap file for search engines.
    – Su'
    Nov 21, 2011 at 10:12
  • Ah, that should work. Thanks. Also does the sitemap.xml size have a problem ? We are expected to have 3-4 million videos by end of next year.
    – MarutiB
    Nov 21, 2011 at 10:15
  • If it is that large amount of videos, you need not add all of them to the sitemap. Just add the recent few thousand videos. Once they get indexed there is no need to keep them in sitemap.xml. This is how even the stackoverflow maintain it's sitemap.xml file for fresh questions.
    – Jigar Tank
    Nov 21, 2011 at 10:55
1

Google suggests This solution for indexing ajax sites which use 'hash bang' urls (I don't know if yours does).

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.