# ------------------------------------------------------------------------------ # Robots Exclusion File (for help see: http://www.robotstxt.org/wc/exclusion.html) # # A sub-domain is considered as an entirely different site and should be treated as such by placing the robots.txt file in the root of the subdomain. # # An asterisk * can be used to identify collections of entries (i.e. /projects/client*.htm). # Multiple *'s may be included in any line. # # A dollar sign $ can be used to control access to a specific directory. # For example, Disallow: /project/text$ will disable access to the file /projects/text but will still allow access to /projects/text.doc and /projects/text/home.htm # The entry must exactly match for this to be effective. # ------------------------------------------------------------------------------ User-agent: * Disallow: /images/ Disallow: /scripts/ Disallow: /cgi-bin/ User-agent: Googlebot Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.png$ Disallow: /*.psd$ Disallow: /*.tif$ Disallow: /*.doc$ Disallow: /*.ai$ Disallow: /*.zip$ User-agent: Googlebot-Image Disallow: / User-agent: 008 Disallow: /