User-agent: * Disallow: /admin.php Disallow: /admin/ Disallow: /downloads/ Disallow: /HTML/ Disallow: /cgi-bin/ Disallow: /pub/ Allow: /