User-Agent: * Crawl-Delay: 10 Disallow: Disallow: /documents Disallow: /images Disallow: /Fonts Disallow: /css Disallow: /Users Disallow: /manage Disallow: /js Disallow: /sites # Blocks user "track" pages Disallow: /*/track$ # Blocks common URL parameters created by the Views module on tables Disallow: /*sort= Disallow: /*size= User-agent: * Crawl-Delay: 10 Disallow: /aggregator Disallow: /tracker Disallow: /comment/reply Disallow: /node/add Disallow: /search/ Disallow: /book/print Disallow: /logout Disallow: /user/register Disallow: /user/password Disallow: /user/login # Paths (no clean URLs) User-agent: * Crawl-Delay: 10 Disallow: /?q=aggregator Disallow: /?q=tracker Disallow: /?q=comment/reply Disallow: /?q=node/add Disallow: /?q=user/register Disallow: /?q=user/password Disallow: /?q=user/login Disallow: /?q=search/ Disallow: /?q=book/print User-agent: * Crawl-Delay: 10 Disallow: /admin Disallow: /?flush Disallow: /myotherprivatedirectory Allow: / User-Agent: Googlebot-Image Crawl-Delay: 10 Disallow: /admin Disallow: /?flush Disallow: /myotherprivatedirectory Disallow: /assets Allow: /