I'm trying to crawl a Salesforce-generated site. This site appears to be fully generated by Javascript (if you load the page with JS disabled only a white screen shows up). For instance, Salesforce's own KB behaves this way.

Some of the pages come back as rendered and crawled, most come back as just blank. I'm unsure of if there's a way to get Screaming Frog to wait until page content loads on each article.

I currently have rendering set to "Javascript" which allows me to view some pages but not all.

  • 3
    You might need to increase the AJAX timeout (default is 5 seconds), which is how many seconds before a snapshot of the page is taken. More at screamingfrog.co.uk/crawl-javascript-seo.
    – Nick
    May 16 '18 at 14:55
  • @Nick Sounds like an answer! Cheers!!
    – closetnoc
    May 16 '18 at 16:07
  • That is the risk you take rendering the page using JS. You can expect to have issues periodically using this method. IP is not a guaranteed delivery protocol. Placing all the burden on the client end means increased likely hood of failure. Just a warning. Cheers!!
    – closetnoc
    May 16 '18 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.