5,272 reputation
729
bio website vyznev.net
location Helsinki, Finland
age
visits member for 2 years, 11 months
seen Dec 15 at 23:08

I'm a PhD student in biomathematics, working on stochastic individual-based models of evolution in spatially structured populations. My other interests include cryptography, programming games and puzzles, photography and graphic design.

I'm the main author and maintainer of the Stack Overflow Unofficial Patch (SOUP), a user script for browsers with GreaseMonkey-compatible user script support (Firefox, Chrome, Opera, possibly Safari) that fixes or works around a number of outstanding issues with the Stack Exchange user interface.

I tend to answer a lot more questions than I ask. Some answers I'm rather proud of:

CC-Zero Please consider any (original) code I post to Stack Overflow and other Stack Exchange sites to be released under CC-Zero unless stated otherwise. You may do whatever you want with it and don't have to credit me in any way, although of course that would be nice.


Dec
3
comment How do I find when an URL was first indexed by Google?
There may not be any answer, other that "you can't" or "nobody knows how". But you're right, let me edit that into my answer.
Dec
3
comment How do I find when an URL was first indexed by Google?
@Stephen: That may not always be the case; see the examples in my new answer below.
Dec
3
comment How do I find when an URL was first indexed by Google?
@Stephen & Zistoloen: That is not generally the date when the page was first indexed by Google. Based on some testing, the date shown via this method appears to be either pulled from the content of the page (if Google thinks it sees something that looks like a "published on" or "last modified on" date) or, if no such date is found in the content, based on the date when Google last observed a (substantial?) change to the page. Of course, if the page was never changed after its first publication, this might happen to be the same as its publication date, but there's no guarantee of that.
Dec
3
comment How do I find when an URL was first indexed by Google?
It's quite possible that the answer is "no". Google may not even store this information (since there's no particular reason why they'd have to), or, even if they do, they may not expose it to third parties.
Oct
28
comment Does Google ignore robots.txt
Agreed. Except for the misleading first sentence pointed out by @unor, this would be an excellent answer.
Aug
29
comment Remove extension from URL using a rewrite without resulting in a redirect loop
+1, this may be cleaner than my solution.
Aug
24
comment Apache Rewrite Rule non-www to www + http to https + remove trailing slash
Re: the bounty, is there something specific you'd like more "credible and/or official sources" for?
Aug
21
comment 301 redirect to 404 page or set status code to 404 and stay on page?
The use of 401 for authentication methods other than HTTP Basic or Digest auth (or other RFC2617-compatible auth schemes) has been discussed here before; my opinion at the time, which I still stand by, is that it may work in practice, but it's not really valid according to the HTTP spec, and that in any case, 403 or even 404 would be preferable.
Aug
21
comment 301 redirect to 404 page or set status code to 404 and stay on page?
@WPRookie82: As far as anyone except you and your webserver is concerned, there's no difference between a non-existent page and an existing page that returns a 404 response.
Aug
19
comment Index page content identical to page 1 of a gallery-type website
@WordPressDeveloper: Which way would you point it? If you make the permalink URLs canonical, users will never see the index URL in SERPs. If you make the index URL canonical, you risk having old content dropped from Google when the index page changes, because Google still thinks the old pages are just copies of the canonical index page. Anyway, none of the sites I checked with this kind of a pagination scheme (mostly blogs and webcomics) used rel=canonical, and they all seem to do just fine on Google.
Aug
9
comment Apache Rewrite Rule non-www to www + http to https + remove trailing slash
There's no point in removing trailing slashes from HTTP(S) URLs with an empty path. The URLs http://example.com and http://example.com/ are canonically equivalent per RFC 3986, and it's up to your browser which one it chooses to show in the address bar.
Jul
17
comment Will canonical tag referencing unsecure HTTP URLs on a HTTPS page cause the unsecure message?
Here's a live example. (Note that HTTPS support on SE is still experimental, and in particular, some pages may still have insecure images. However, user profile pages shouldn't have any, at least if the user in question is using Gravatar for their user icon.)
Jun
28
comment Most invariant URI to refer to a HTML5 element from w3.org?
You've obviously spent some thought on this, but for the rest of us, it might help if you told us a little bit more about what you'd need such a URI for. Knowing more about your motivation might help in judging the relative merits of different options, or it might even suggest some completely different solution.
Jun
27
comment How do you tell search engines not to index this page just yet, but maybe in the future?
...that is, unless there's something meaningful that users could do even with a "blank" page, such as supplying information for it. (Even then, though, it may be better not to link to the blank page directly, but to some different URL that indicates the (possible) absence of data. Note that you'll need to deal with the edge case where a user follows such a link after the page has recently been created; see for example how Wikipedia does it.)
Jun
26
comment How do you tell search engines not to index this page just yet, but maybe in the future?
@Binarysurf: According to the OP, they're using a custom script to serve dynamic content for "virtual pages" constructed out of information stored in a database (just like Wikipedia, SE and most major websites do nowadays). This does mean that "regular web server behavior" doesn't apply -- as far as the web server (Apache, IIS, nginx, etc.) is concerned, the content-generating script exists and runs, so it defaults to a "200 OK" response, unless the script tells it otherwise. I'm just suggesting that the OP should make their script send a 404 response, if there's no actual content to show.
Jun
25
comment 403 Error Fetching Image with JS
+1, this appears to be the correct explanation in this case. Further testing shows that the server accepts an empty referrer (-e ""), as a browser would send if you typed the image URL into the address bar, but not a referral from another site. (Also, despite what I suggested in earlier comments, the Accept header does not seem to make any difference for this site.)
Apr
25
comment Serve a different robots.txt file for every site hosted in the same directory
@EdgarQuintero: An internal rewrite, as implemented by the rewrite rules I show above, happens entirely within the webserver. A crawler requesting the URL path /robots.txt has no way of even knowing whether the content it receives comes from a file named robots.txt (as usual) or from a file named robots_ar.txt (to which the request was rewritten) or even from a script named robots.php (or even whatever.php).
Apr
25
comment Serve a different robots.txt file for every site hosted in the same directory
@EdgarQuintero: Why on earth could it not be?
Mar
18
comment Combine user-agents in robots.txt
The original robots.txt spec agrees: "If more than one User-agent field is present the record describes an identical access policy for more than one robot."
Feb
23
comment Submit example.com/#!/page URLs to Google
...and you're just toggling the visibility of the <li>s, rather than actually loading the content via AJAX? In that case, you could just add some code to your index.php so that, if it receives an _escaped_fragment_ parameter, it styles the matching <li> with display:block and all the others with display:none (or omits them entirely). Oh, and you probably should a) change your <title> to describe the page being shown, and b) use the <meta name="fragment" content="!"> trick for the main page.