# www.robotstxt.org/ # Allow crawling of all content User-agent: * Disallow: /regions/ Disallow: /update/ Disallow: /directions/ Disallow: /add-review/ Disallow: /pages/ Disallow: /new/