Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 35 additions & 33 deletions src/Resources/lists/bad-user-agents.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
libredtail-http
opendirme-credhunt
1h4x.com
1h4x\.com
360Spider
404checker
404enemy
Expand Down Expand Up @@ -138,7 +136,7 @@ Dotbot
Download Wonder
Dragonfly
Drip
ECCP/1.0
ECCP
EMail Siphon
EMail Wolf
EasyDL
Expand All @@ -160,7 +158,7 @@ FHscan
FacebookBot
FemtosearchBot
Fimap
Firefox/7.0
Firefox/7\.0
FlashGet
Flunky
Foobot
Expand All @@ -175,6 +173,7 @@ GT::WWW
GalaxyBot
GeedoProductSearch
Genieo
GenomeCrawlerd
GermCrawler
GetRight
GetWeb
Expand Down Expand Up @@ -470,7 +469,6 @@ SpankBot
Spanner
Spbot
Spider_Bot
Spider_Bot/3.0
Spinn3r
SplitSignalBot
SputnikBot
Expand Down Expand Up @@ -501,6 +499,7 @@ TheNomad
The Intraformant
Thumbor
TightTwatBot
Timpibot
TinyTestBot
Titan
Toata
Expand Down Expand Up @@ -541,7 +540,6 @@ WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers
Wallpapers/3.0
WallpapersHD
WeSEE
WebAuto
Expand Down Expand Up @@ -579,7 +577,7 @@ Webster
Whack
Whacker
Whatweb
Who.is Bot
Who\.is Bot
Widow
WinHTTrack
WiseGuys Robot
Expand All @@ -603,48 +601,49 @@ ZoominfoBot
ZumBot
ZyBorg
adscanner
allenai.org
allenai\.org
anthropic-ai
archive.org_bot
aria2
arquivo-web-crawler
arquivo.pt
arquivo\.pt
autoemailspider
awario.com
awario\.com
backlink-check
cah.io.community
check1.exe
cah\.io\.community
check1\.exe
clark-crawler
coccocbot
cognitiveseo
cohere-ai
com.plumanalytics
crawl.sogou.com
crawler.feedback
crawl\.sogou.com
crawler4j
dataforseo.com
crawler\.feedback
dataforseo\.com
dataforseobot
dataprovider
demandbase-bot
domainsproject.org
domainsproject\.org
eCatch
evc-batch
everyfeed-spider
facebookscraper
fasthttp
git
gopher
heritrix
imagesift.com
imagesift\.com
instabid
internetVista monitor
ips-agent
isitwp.com
isitwp\.com
iubenda-radar
l9scan
leakix
libredtail-http
linkdexbot
linkfluence
litellm-recon
lwp-request
lwp-trivial
magpie-crawler
Expand All @@ -655,43 +654,46 @@ netEstate NE Crawler
oBot
omgili
openai
openai.com
openai\.com
opendirme-credhunt
page scorer
pcBrowser
plumanalytics
polaris version
probe-image-size
ripz
s1z.ru
satoristudio.net
s1z\.ru
satoristudio\.net
scalaj-http
scan.lol
scan\.lol
seobility
seocompany.store
seoscanners
seostar
serpstatbot
sexsearcher
sitechecker.pro
sitechecker\.pro
siteripz
smart-seo-tools.sbs
smart-seo-tools\.sbs
sogouspider
sp_auditbot
spy4x-domain-scanner
spyfu
sysscan
tAkeOut
terrabot-owned-you
trafilatura
trendiction.com
trendiction.de
ubermetrics-technologies.com
voyagerx.com
trendiction\.com
trendiction\.de
ubermetrics-technologies\.com
voyagerx\.com
webgains-bot
webmeup-crawler
webpros.com
webpros\.com
webprosbot
wp2shell
x09Mozilla
x22Mozilla
xpymep1.exe
zauba.io
xpymep1\.exe
zauba\.io
zgrab