3

To speed up my website (or at least the time to first byte (TTFB)), I'd like to only send the layout of the page and then get the page content with an asynchronous request. This way, the TTFB will be really fast, because this will be short, lightweight and cached content. For the content to be indexed, I will analyze the User-Agent to know if I have to send the content the old-fashion way.

Is this a safe way to do so, or will I be blacklisted by Google (because of the different content)?

Thank in advance !

PS.: I know about the Google AJAX craw method (https://developers.google.com/webmasters/ajax-crawling/), but this is not really what I want.

Improve this question
4
  • 1
    Using User-Agent to determine content is a very borderline to blackhat method of doing things, your best of making sure that your Ajax executes gracefully. While page speed is important there are many other ways of improving the speed of your site. I have a few sites under 1sec without any JavaScript optimization by using good cloud hosting and using PNG sprites.
    – Simon Hayter
    Mar 12, 2013 at 13:37
  • I'm already using the Google CDN, but the slower part is the server computings. I'm wondering why this is very borderline to blackhat method, because this is not with bad intentions.
    – Sébastien
    Mar 12, 2013 at 13:43
  • What is the current number (in milliseconds) that does concern you when you request your biggest page?
    – initall
    Mar 12, 2013 at 13:59
  • This is generally between 500 and 6000 ms. But sometimes the page content is very important and I don't use cache so much.
    – Sébastien
    Mar 12, 2013 at 14:06

3 Answers 3

Reset to default
3

Unfortunately, if Google knew intentions SEO would be much easier :) What you are attempting to do, regardless of intention, is the same as someone with a bad intention would do, at least according to Google.

I would do exactly the opposite. Send the content with minimal layout the usual synchronous way and load media asynchronously. You may notice that Bing does that. You can search before the images arrive.

You may also delay-load non-core content which is not needed by the Google crawler. You have to decide based on your site but things like areas with random news or renderings from another part of the site, etc.

Improve this answer
3
  • Ok, thank you for the explanations. Your first proposal is interesting, but only for websites that use much media. What takes much time for me is the server computations. The second proposal seems for me the best solution in addition with a better cache system.
    – Sébastien
    Mar 12, 2013 at 14:02
  • I have to criticize google, there is no reason why they wouldn't be able to simple allow a meta field in the header for an alternative url with static content served sense they can't crawl asynchronous webpages .. I mean this is 2016, they are keeping back new technology in a fashion I'm pretty sure is illegal (or at least should be). People who are trying to manipulate the bot by shadowing it can still do so as it is today by exploiting the lack of asynchronous support.
    – superhero
    Sep 8, 2016 at 10:40
  • also, related: centrical.com/test/… proves that google has some sort of understanding of asynchronous content...
    – superhero
    Sep 8, 2016 at 10:51
1

The most important thing for search engines is your content, so don't make it complicated to get.

The developer extension of browser will give you so much hints about what really slows down things I would start there. Take a look, for example, at this screenshot from the Audit tab of the Chrome Developer Tools:

enter image description here

And these are just some hints you get for free. You can also dive into what time it takes for rendering the content, the memory usage, latency and more (One of many introductions to the DevTools on YouTube).

Caching your computed content in the usual sense ("creating a static html file") will almost always be the #1 effective starting point. From there you can gzip content (via your webserver or proxy), compress your JS and CSS files, remove not needed webfont styles (e.g. extra-bold 800 if you don't use it), load static files from a different (and cookieless) domain and much much more.

Improve this answer
4
  • I know about the DevTools, but the main time consuming part is the server computation. As you said, I think I'll have to improve the usage of cache. I'm already using a subdomain for static content, but it does not seems to be helpfull for cookie, is it normal ?
    – Sébastien
    Mar 12, 2013 at 19:18
  • The idea behind the different domain is that cookies won't (and don't need to) be sent to it, saving some bytes and time. When computation of the content is heavy think about this: Whenever your content is ready to go live, pre-create the cache, e.g. by triggering an HTML request yourself (via AJAX or a wget-call or smth. else). So even the first "human" request from a visitor will then receive the already computated cached file.
    – initall
    Mar 13, 2013 at 7:36
  • I understand about the reason to use an other domain for static content, but I'm using a subdomain and the DevTools said me that it's still sending cookies, is it normal? This is of course a possible to call the pages before users to create the cache, but I have around 2 million pages, so it would take too much time.
    – Sébastien
    Mar 13, 2013 at 13:44
  • 1
    Cookie/subdomain: You can change this behaviour by setting the domain value of the cookie appropriately. 2M Pages: Ok, without further details, I would let it cache-on-demand. This way you don't waste time and space for not-requested pages.
    – initall
    Mar 13, 2013 at 16:26
0

There is an easier way to do it. You can cache your HTML as static object and send it from CDN, thus bypassing the rendering process. The idea is similar. This way you can send your HTML as quickly as possible and kick off parsing to improve SEO (and UX).

Note that browsers will - by default - start parsing as soon as HTML arrives so you don't need to split your content and layout, they will do it themselves: rendering the page over time, as more and more object arrive.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.