How to block rogue crawlers and bad bots?
Navigate to ‘Origin Shield’ > 'Bot management'. On the 'Application layer' box, look for 'Block rogue crawlers and bad bots'.
Toggle the switch to turn on/off this security feature.
Rogue crawlers/bad bots list
You can click the ‘View’ (eye) icon to see the full rogue crawlers/bad bots list.
Here's what will be blocked once this feature is activated:
; widows 8484 boston project a href= adsarobot advanced email extractor agdm79@mail.ru amiga-aweb/3.4 athens atomic_email_hunter atspider attache autoemailspider backdoor black hole botversion butch__2.1.1 bwh3_user_agent cheesebot cherrypicker china local browse 2. chinaclaw compatible ; msie compatible ;. compatible- concealed defense contactbot/ contentsmartz converacrawler copyguard copyrightcheck core-project/1 crescent internet toolpak datacha0s digimarc webreader digout4uagent dts agent ecollector educate search vxb eirgrabber email email extractor emailcollect emailharvest emailmagnet emailreaper emailsiphon emailwolf eo browse extractorpro fantombrowser fastlwspider faxobot fiddler floodgate foobar/ franklin locator full web bot gameBoy powered by nintendo gecko/2525 grub crawler grub-client hanzoweb hhjhj@yahoo hl_ftien_spider indy library injection internet exploiter six internet ninja internet-exprorer isc systems irc search 2.1 jakarta kenjin spider larbin@unspecified mailto:craftbot@yahoo.com microsoft internet explorer/5.0 microsoft url microsoft url control missigua morfeus fucking scanner morzilla via mosiac 1 mozilla/3.mozilla/2.01 murzillo compatible nameofagent nessus neuralbot/0.2 newt activeX; win32 nikto no browser nokia-waptoolkit packrat panscient.com pcbrowser pe 1.4 pleasecrawl/1. pmafind poe-component-client production bot program shareware 1.0. prowebwalker psurf psycheclone rsync s.t.a.l.k.e.r. safexplorer tl searchbot admin@google security scan shai sitesnagger stress test surveybot takeout teleport pro telesoft this is an exploit toata dragostea mea pentru diavola turing machine under the rainbow 2. user-Agent user-agent: vadixbot voideye w3mir web by mail web downloader webaltbot webbandit webemailextrac webemailextract webmole webroot webster pro webvulnscan wells search ii wep Search 00 windows-update-agent wisebot wisenutbot wordpress hash grabber wordpress/4.01 www.weblogs.com xmlrpc exploit zeus