AhrefsBot
MJ12bot
Yandex
YandexBot
UbiCrawler
DOC
Zao
sitecheck.internetseer.com
Zealbot
MSIECrawler
SiteSnagger
WebStripper
WebCopier
Fetch
Offline Explorer
Teleport
TeleportPro
WebZIP
linko
HTTrack
Microsoft.URL.Control
Xenu
larbin
libwww
ZyBORG
Download Ninja
rogerbot
spbot
Sosospider
BacklinkCrawler
ia_archiver
Ezooms
Gigabot
findlinks
SurveyBot
SEOENGBot
BPImageWalker
bdbrandprotect
LinkWalker
Updownerbot
HTTrack
SemrushBot
SemrushBot-SA