# Robots.txt file from http://www.searchengineworld.com # # All robots will spider the domain User-agent: * Disallow: /cgi-bin/ Disallow: /images/ Disallow: /Webgate3.5/admin/ Disallow: /Webgate3.5/lib/ Disallow: /PHP/ Disallow: /lib/ Disallow: /modlogan/ Disallow: /webalizer/ Disallow: /wis.co.il/ Disallow: /cp/ Disallow: /awstats/ Disallow: /Webguardian/ Disallow: /webgatedns.com/ Disallow: /phpwebftp/ Disallow: /phpAdsNew-2.0.5/ Disallow: /PaymentGateway/ Disallow: /Webgate3.5/support/filebin/ Disallow: /Webgate3.5/support/admin/ Disallow: /Webgate3.5/support/install/ Disallow: /Webgate3.5/support/config/ Disallow: /Webgate3.5/WebgateLiveOperator/ Disallow: /linkmachine/ Disallow: /hsphereorder/ Disallow: /golden-lotus.co.il/ Disallow: /idevaffiliate/ Disallow: /index-style/