I refer to this webpage: https://developers.google.com/search/docs/advanced/robots/robots_txt

Could someone elaborate on the difference between these two:


allow: /page
disallow: /*.htm

Applicable rule: disallow: /*.htm, because it matches more characters in the URL, so it's more specific.


allow: /page
disallow: /*.ph

Applicable rule: allow: /page, because in case of matching rules, Google uses the least restrictive rule.

They look rather similar but one disallows while the other allows.

1 Answer 1


I'm answering this myself.

If both allow and disallow match, then compare the length of the paths. The longer one wins. If both are of the same length, then the least restrictive (allow) wins.

  • While counting the lengths of the paths, should we include the * (wildcard)? (for example, should the length of /*.htm be 6?)
    – Kannan
    Commented Jul 9, 2021 at 2:48
  • 1
    @Kannan yes, count every character as if it's a non special character. In your case, 6 is correct.
    – annette
    Commented Jul 9, 2021 at 4:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.