# /robots.txt file for http://webcrawler.com/ User-agent: * Disallow: /home.html Disallow: /add.htm Disallow: /blank.html Disallow: /exit.htm Disallow: /vari_main.html Disallow: /clubromanceusa/ Disallow: /Backup Files/ Disallow: /calculators/ Disallow: /Counter/ Disallow: /elena/ Disallow: /holidays/ Disallow: /logs/ Disallow: /male/ Disallow: /temp/ Disallow: /Copy of index Disallow: /count Disallow: /counter Disallow: /home-original Disallow: /index-for local Drive Disallow: /index-old Disallow: /sendemail Disallow: /sendemail_auto Disallow: /sendemail_prop Disallow: /clubromanceusa/ Disallow: /flcondoinsurance/ Disallow: /flhomeinsurances/ Disallow: /floridafloodInsurances/ Disallow: /flauto-insurance/ Disallow: /flhealthquote/ Disallow: /davidflrealtor/ Disallow: /elena/ Disallow: /avantgarde/ Disallow: /privata/ Disallow: /friends/ Allow: /