# All robots will spider the domain User-agent: * Disallow: /_cms/ Disallow: /_code/ Disallow: /_content/ Disallow: /_search/ Disallow: /_style/ Disallow: /assets/ Disallow: /css/ Disallow: /images/ Disallow: /img/ Disallow: /randomhome/ Disallow: /randomprogramme/ Disallow: /randomsite/