# To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-Agent: * Disallow: /domains/ Disallow: /domain_actions/ Disallow: /licensees/ Disallow: /billing_addresses/ Disallow: /remarks/ Disallow: /orders/ Disallow: /checkout/