# robots.txt for http://wol.ra.phy.cam.ac.uk # welcome, come on in! # reason for disallowing stats: we don't want search engines # to discover URLS which aren't linked to anywhere but people have # nonetheless accessed (e.g. http://coll.ra.phy.cam.ac.uk URLS) User-agent: * Disallow: /wwwwais/ Disallow: /stats/ Disallow: /coll-stats/ Disallow: /wol-stats/ Disallow: /mackay/codes/EN/ Disallow: /djw30/dasher/files/ Disallow: /stats.html Disallow: /cgi-bin/