Reputation
5,831
Top tag
Next privilege 10,000 Rep.
Access moderator tools
Badges
10 30
Newest
 Guru
Impact
~197k people reached

Jun
28
answered Server log what does “ho=<something>” mean?
Jun
28
comment Most invariant URI to refer to a HTML5 element from w3.org?
You've obviously spent some thought on this, but for the rest of us, it might help if you told us a little bit more about what you'd need such a URI for. Knowing more about your motivation might help in judging the relative merits of different options, or it might even suggest some completely different solution.
Jun
27
comment How do you tell search engines not to index this page just yet, but maybe in the future?
...that is, unless there's something meaningful that users could do even with a "blank" page, such as supplying information for it. (Even then, though, it may be better not to link to the blank page directly, but to some different URL that indicates the (possible) absence of data. Note that you'll need to deal with the edge case where a user follows such a link after the page has recently been created; see for example how Wikipedia does it.)
Jun
26
comment How do you tell search engines not to index this page just yet, but maybe in the future?
@Binarysurf: According to the OP, they're using a custom script to serve dynamic content for "virtual pages" constructed out of information stored in a database (just like Wikipedia, SE and most major websites do nowadays). This does mean that "regular web server behavior" doesn't apply -- as far as the web server (Apache, IIS, nginx, etc.) is concerned, the content-generating script exists and runs, so it defaults to a "200 OK" response, unless the script tells it otherwise. I'm just suggesting that the OP should make their script send a 404 response, if there's no actual content to show.
Jun
26
answered How do you tell search engines not to index this page just yet, but maybe in the future?
Jun
25
comment 403 Error Fetching Image with JS
+1, this appears to be the correct explanation in this case. Further testing shows that the server accepts an empty referrer (-e ""), as a browser would send if you typed the image URL into the address bar, but not a referral from another site. (Also, despite what I suggested in earlier comments, the Accept header does not seem to make any difference for this site.)
Jun
25
answered What is Google's policy about separate content on same URLs in mobile and desktop versions?
Apr
25
comment Serve a different robots.txt file for every site hosted in the same directory
@EdgarQuintero: An internal rewrite, as implemented by the rewrite rules I show above, happens entirely within the webserver. A crawler requesting the URL path /robots.txt has no way of even knowing whether the content it receives comes from a file named robots.txt (as usual) or from a file named robots_ar.txt (to which the request was rewritten) or even from a script named robots.php (or even whatever.php).
Apr
25
comment Serve a different robots.txt file for every site hosted in the same directory
@EdgarQuintero: Why on earth could it not be?
Apr
25
answered Serve a different robots.txt file for every site hosted in the same directory
Apr
18
revised Too many duplicate image alt attributes. What to do?
added 52 characters in body
Apr
18
answered Too many duplicate image alt attributes. What to do?
Mar
18
comment Combine user-agents in robots.txt
The original robots.txt spec agrees: "If more than one User-agent field is present the record describes an identical access policy for more than one robot."
Mar
18
answered Are these backlinks hurting my site
Mar
5
answered Deleted page still in Google results
Feb
23
comment Submit example.com/#!/page URLs to Google
...and you're just toggling the visibility of the <li>s, rather than actually loading the content via AJAX? In that case, you could just add some code to your index.php so that, if it receives an _escaped_fragment_ parameter, it styles the matching <li> with display:block and all the others with display:none (or omits them entirely). Oh, and you probably should a) change your <title> to describe the page being shown, and b) use the <meta name="fragment" content="!"> trick for the main page.
Feb
23
comment Submit example.com/#!/page URLs to Google
"There is no way to directly submit this to Google" is kind of misleading, since, in fact, the instructions you've linked to say you should directly submit such URLs to Google in your sitemap. You just needs some extra server-side support to let Googlebot fetch them.
Feb
23
answered Submit example.com/#!/page URLs to Google
Feb
9
revised Mass deletion of spam revisions in Mediawiki
note that attachLatest.php needs the --fix parameter, per http://webmasters.stackexchange.com/review/suggested-edits/16970
Jan
27
comment Does the 'Server' header serve any purpose?
A 100-byte header, like the one quoted in the question, might well cause the combined length of the response headers and content to spill over into an extra packet. Besides, packet-counting tells the whole story only for purely latency-limited connections. In (at least partially) bandwidth-limited situations (such as slow mobile connections, or large data centers with a huge request volume), the total amount of bytes transmitted starts to matter too.