Original research · Edition archive

AI Bot Blocking Index — June 2026 Edition (Q2)

Name: AI Bot Blocking Index: B2B SaaS robots.txt scan (390 sites, June 2026)
Creator: AI-Advisors
Published: 2026-06-27
License: https://creativecommons.org/licenses/by/4.0/

This is the permanent archive of the inaugural June 2026 scan. Its numbers will never change, so it is safe to cite. The canonical index page always carries the latest edition, full methodology, and FAQ.

390 B2B SaaS websites, 17 AI crawlers, scanned 2026-06-27. The sample spans 153 recognizable established brands and 237 active growth-stage B2B SaaS companies from the Y Combinator directory. Of the 362 sites with a readable robots.txt, 3 (0.8%) block an AI search crawler, 1.7% block GPTBot, and 48.2% of all 390 sites are served behind Cloudflare's edge.

Block rate by crawler

Crawler	User-agent	Operator	Type	Blocked	Rate
ChatGPT Search	OAI-SearchBot	OpenAI	search	1	0.3%
Claude Search	Claude-SearchBot	Anthropic	search	1	0.3%
Perplexity Search	PerplexityBot	Perplexity	search	1	0.3%
Apple AI Search	Applebot	Apple	search	1	0.3%
DuckAssist	DuckAssistBot	DuckDuckGo	search	1	0.3%
ChatGPT (user-initiated)	ChatGPT-User	OpenAI	user	1	0.3%
Claude (user-initiated)	Claude-User	Anthropic	user	0	0.0%
Perplexity (user-initiated)	Perplexity-User	Perplexity	user	0	0.0%
OAI-AdsBot (ChatGPT Ads validation)	OAI-AdsBot	OpenAI	ads	0	0.0%
GPTBot (OpenAI training)	GPTBot	OpenAI	training	6	1.7%
ClaudeBot (Anthropic training)	ClaudeBot	Anthropic	training	2	0.6%
Google-Extended (training)	Google-Extended	Google	training	4	1.1%
Meta AI (training)	Meta-ExternalAgent	Meta	training	5	1.4%
Bytespider (ByteDance training)	Bytespider	ByteDance	training	7	1.9%
CCBot (Common Crawl)	CCBot	Common Crawl	training	8	2.2%
Cohere AI (training)	cohere-ai	Cohere	training	3	0.8%
Applebot-Extended (training)	Applebot-Extended	Apple	training	3	0.8%

The full dataset (390 sites)

Every scanned site, tagged by cohort, with the per-crawler matrix downloadable as JSON or CSV under CC BY 4.0.

Download JSON Download CSV


15five.com	Established	200	Yes	No	No	0
1password.com	Established	200	Yes	No	No	0
abacus.com	YC	200	No	No	No	0
abstra.io	YC	200	No	No	No	0
accessowl.io	YC	200	No	No	No	0
activecampaign.com	Established	200	Yes	No	No	0
advantageclub.co	YC	200	No	No	No	0
aha.io	Established	200	No	No	No	0
ahazou.com	YC	200	Yes	No	No	0
ahrefs.com	Established	200	Yes	No	No	0
airtable.com	Established	200	No	No	No	0
alex.com	YC	200	Yes	Yes	No	0
algolia.com	YC	200	Yes	No	No	0
alguna.com	YC	200	Yes	No	No	0
ambition.com	YC	200	Yes	Yes	No	0
amplitude.com	Established	200	No	Yes	No	0
animaapp.com	YC	200	No	No	No	0
apollo.io	Established	200	Yes	Yes	No	0
apptimize.com	YC	200	No	No	No	0
appx.co.in	YC	200	Yes	No	No	0
arengu.com	YC	unreachable	Yes	No	No	0
arintra.com	YC	200	Yes	Yes	No	0
asana.com	Established	200	No	No	No	0
ascen.com	YC	200	Yes	No	No	0
aspireapp.com	YC	200	Yes	No	No	0
atlas.so	YC	unreachable	No	No	No	0
atlassian.com	Established	200	No	No	No	0
auth0.com	Established	200	Yes	No	No	0
bamboohr.com	Established	200	Yes	Yes	No	0
basecamp.com	Established	200	Yes	No	No	0
benchling.com	YC	unreachable	No	No	No	0
bik.ai	YC	200	No	No	No	0
bill.com	Established	200	Yes	No	No	0
bitrise.io	YC	200	Yes	No	No	0
bonusly.com	Established	200	Yes	No	No	0
boostly.com	YC	200	No	No	No	0
borong.com	YC	200	No	No	No	0
box.com	Established	200	Yes	No	No	0
braze.com	Established	200	Yes	No	No	0
brex.com	Established	200	No	No	No	0
brightedge.com	Established	200	No	No	No	0
briohr.com	YC	200	No	No	No	0
buffer.com	Established	200	Yes	No	No	0
buildwithfern.com	YC	200	No	No	No	0
bujeti.com	YC	200	No	No	No	0
calendly.com	Established	200	Yes	Yes	No	1
calltree.ai	YC	unreachable	No	No	No	0
cashflowportal.com	YC	200	Yes	No	No	0
cekura.ai	YC	200	No	No	No	0
chargebee.com	Established	200	No	Yes	No	0
chipax.com	YC	200	No	No	No	0
churnzero.com	Established	200	No	No	No	0
cinder.ai	YC	200	No	No	No	0
circleci.com	Established	200	No	Yes	No	0
clari.com	Established	200	Yes	No	No	0
clearscope.io	Established	200	No	No	No	0
clickup.com	Established	200	No	No	No	0
close.com	Established	200	Yes	No	No	0
cloudchipr.com	YC	200	Yes	No	No	0
cloudflare.com	Established	200	Yes	Yes	No	0
clueso.io	YC	200	No	No	No	0
cococart.co	YC	200	No	No	No	0
coda.io	Established	200	No	No	No	0
command.ai	YC	200	No	No	No	0
communityphone.org	YC	200	No	No	No	0
complif.com	YC	200	Yes	No	No	0
compup.io	YC	200	No	No	No	0
conductor.com	Established	200	Yes	No	No	0
confidentlims.com	YC	200	Yes	Yes	No	0
confidotech.com	YC	200	Yes	No	No	0
confluent.io	Established	200	No	Yes	No	0
contentful.com	Established	200	No	No	No	0
copper.com	Established	200	No	No	No	0
copy.ai	Established	200	Yes	No	No	0
cortex.io	YC	200	No	No	No	0
coval.dev	YC	200	No	No	No	0
crowdstrike.com	Established	200	Yes	No	No	0
cultureamp.com	Established	200	No	No	No	0
curacel.co	YC	200	Yes	No	No	0
customer.io	Established	200	No	No	No	0
customily.com	YC	200	Yes	Yes	No	0
databricks.com	Established	200	Yes	No	No	0
datadoghq.com	Established	200	No	No	No	0
datamechanics.co	YC	unreachable	Yes	No	No	0
datlo.com	YC	200	No	No	No	0
dealls.com	YC	200	Yes	No	No	0
deel.com	Established	200	Yes	Yes	No	0
demodesk.com	YC	200	Yes	Yes	No	0
devcycle.com	YC	200	Yes	No	No	0
digitalocean.com	Established	200	Yes	No	No	0
dimorder.com	YC	200	No	No	No	0
disclosures.io	YC	200	Yes	No	No	0
distro.app	YC	200	No	No	No	0
docker.com	Established	200	No	No	No	0
docusign.com	Established	200	No	No	No	0
dojah.io	YC	200	Yes	No	No	0
doublehq.com	YC	200	Yes	No	No	0
draftwise.com	YC	200	Yes	No	No	0
drift.com	Established	unreachable	Yes	No	No	0
dropbox.com	Established	200	No	No	No	0
dyspatch.io	YC	200	Yes	No	No	0
edenworkplace.com	YC	200	Yes	No	No	0
elyos.ai	YC	200	Yes	No	No	0
embeddables.com	YC	200	No	No	No	0
enableus.com	YC	unreachable	No	No	No	0
epsilon3.io	YC	200	No	Yes	No	0
expensify.com	Established	200	Yes	No	No	0
expent.ai	YC	200	Yes	No	No	0
fathom.ai	YC	200	Yes	No	No	0
fibery.io	Established	200	No	No	No	0
figma.com	Established	200	No	Yes	Yes	8
finmark.com	YC	200	Yes	No	No	0
firstbase.io	YC	200	Yes	No	No	0
fivetran.com	Established	200	No	No	No	0
fondo.com	YC	200	Yes	No	No	0
frase.io	Established	200	No	Yes	No	0
freshdesk.com	Established	200	No	No	No	0
freshpaint.io	YC	200	Yes	No	No	0
freshworks.com	Established	200	Yes	No	No	0
front.com	Established	200	Yes	No	No	0
fullstory.com	Established	200	No	No	No	0
gainsight.com	Established	200	Yes	Yes	No	0
gaspos.co	YC	unreachable	No	No	No	0
gem.com	YC	200	No	No	No	0
getagency.com	YC	200	Yes	Yes	No	0
getdbt.com	Established	200	No	No	No	0
getguru.com	Established	200	Yes	No	No	0
getofficely.com	YC	200	Yes	No	No	0
getontop.com	YC	200	Yes	No	No	0
getshogun.com	YC	200	No	No	No	0
getstrada.com	YC	200	No	No	No	0
getswipe.in	YC	200	Yes	Yes	No	1
getthematic.com	YC	200	No	No	No	0
gimbooks.com	YC	unreachable	No	No	No	0
github.com	Established	200	No	No	No	0
gitlab.com	Established	200	Yes	No	No	0
gitprime.com	YC	unreachable	No	No	No	0
givechariot.com	YC	200	No	No	No	0
gladly.com	Established	200	No	Yes	No	0
glowing.io	YC	200	Yes	No	No	0
go.givecampus.com	YC	200	Yes	No	No	0
gocopia.com	YC	200	No	No	No	0
gong.io	Established	200	Yes	No	No	0
goodkind.com	YC	200	No	No	No	0
gotrebol.com	YC	200	Yes	Yes	No	0
govpredict.com	YC	200	Yes	No	No	0
grammarly.com	Established	200	No	No	No	0
greatquestion.co	YC	200	Yes	No	No	0
greenhouse.io	Established	200	Yes	No	No	0
guesty.com	YC	200	No	No	No	0
guruhotel.com	YC	200	No	Yes	No	0
gusto.com	Established	unreachable	Yes	No	No	0
haddock.app	YC	200	Yes	No	No	0
hadrius.com	YC	200	Yes	No	No	0
hashicorp.com	Established	unreachable	No	No	No	0
heap.io	Established	200	No	No	No	0
helloconduit.com	YC	200	Yes	No	No	0
helpscout.com	Established	200	No	No	No	0
herondata.io	YC	200	Yes	No	No	0
hireart.com	YC	200	Yes	No	No	0
hockeystack.com	YC	200	Yes	No	No	0
hona.com	YC	200	Yes	No	No	0
hootsuite.com	Established	200	No	No	No	0
hotglue.com	YC	200	No	No	No	0
hotjar.com	Established	200	No	No	No	0
hubspot.com	Established	200	Yes	No	No	0
hudu.com	YC	200	No	No	No	0
humaans.io	YC	200	No	No	No	0
humand.co	YC	200	No	Yes	No	0
inaccord.com	YC	200	Yes	No	No	0
inevent.com	YC	200	No	No	No	0
infisical.com	YC	200	No	No	No	0
instrumentl.com	YC	200	Yes	Yes	No	0
intercom.com	Established	200	No	No	No	0
inventive.ai	YC	200	Yes	No	No	0
itsrever.com	YC	200	Yes	No	No	0
jasper.ai	Established	200	Yes	Yes	No	0
jfrog.com	Established	200	No	No	No	0
jobvite.com	Established	200	No	No	No	0
joinarc.com	YC	200	No	No	No	0
joindeed.com	YC	200	Yes	No	No	0
journey.io	YC	200	No	No	No	0
jutsoo.com	YC	200	Yes	No	No	0
kapital.mx	YC	200	No	No	No	0
klaviyo.com	Established	200	Yes	Yes	No	0
kular.ai	YC	200	Yes	Yes	No	0
kustomer.com	Established	200	No	No	No	0
lapzo.com	YC	200	Yes	No	No	0
lattice.com	Established	200	Yes	No	No	0
launchdarkly.com	Established	200	No	No	No	0
lever.co	Established	200	No	Yes	No	0
linear.app	Established	200	Yes	No	No	0
linkana.com	YC	200	Yes	Yes	No	0
loom.com	Established	200	No	Yes	No	1
luciq.ai	YC	200	Yes	No	No	0
luminai.com	YC	unreachable	No	No	No	0
mailchimp.com	Established	200	No	No	No	0
mailwarm.com	YC	200	No	No	No	0
make.com	Established	200	Yes	No	No	0
malga.io	YC	200	No	No	No	0
mantys.io	YC	200	Yes	No	No	0
marketmuse.com	Established	200	No	No	No	0
marketo.com	Established	200	No	No	No	0
marqvision.com	YC	200	Yes	No	No	0
medmehealth.com	YC	200	Yes	No	No	0
merlinai.co	YC	200	No	No	No	0
mesh.ai	YC	200	Yes	No	No	0
methodfi.com	YC	200	No	No	No	0
middesk.com	YC	unreachable	Yes	No	No	0
miliopay.com	YC	unreachable	No	No	No	0
miro.com	Established	200	No	No	No	0
mixpanel.com	Established	200	No	No	No	0
mixrank.com	YC	200	No	No	No	0
modelml.com	YC	200	No	No	No	0
momomedical.com	YC	unreachable	No	No	No	0
monday.com	Established	200	Yes	Yes	No	0
mongodb.com	Established	200	No	No	No	0
moz.com	Established	200	Yes	Yes	No	0
mozartdata.com	YC	200	Yes	No	No	0
mutiny.com	Established	200	No	Yes	No	3
mutinyhq.com	YC	200	No	No	No	0
myworkpay.com	YC	200	Yes	No	No	0
nanonets.com	YC	200	No	No	No	0
natero.com	YC	unreachable	No	No	No	0
navattic.com	YC	200	No	No	No	0
netlify.com	Established	200	No	Yes	No	0
netsuite.com	Established	200	No	No	No	0
newrelic.com	Established	200	No	No	No	0
notion.so	Established	200	Yes	No	No	0
novahq.com	YC	200	Yes	No	No	0
okta.com	Established	200	Yes	No	No	0
okteto.com	YC	200	No	No	No	0
olaclick.com	YC	200	No	Yes	No	0
omnistrate.com	YC	200	Yes	No	No	0
oneleet.com	YC	200	No	No	No	0
oneschema.co	YC	200	Yes	No	No	0
onesignal.com	YC	200	Yes	No	No	0
onswitchboard.com	YC	200	Yes	No	No	0
optimizely.com	Established	200	No	No	No	0
otterly.ai	Established	unreachable	No	No	No	0
outreach.io	Established	200	Yes	No	No	0
outset.ai	YC	200	No	No	No	0
paddle.com	Established	200	No	No	No	0
pagerduty.com	Established	200	Yes	No	No	0
palettehq.com	YC	200	Yes	No	No	0
paloaltonetworks.com	Established	200	No	No	No	0
panoramaed.com	YC	200	Yes	Yes	Yes	6
parrotsoftware.io	YC	200	Yes	Yes	No	0
partnerstack.com	YC	200	Yes	No	No	0
peakflo.co	YC	200	No	No	No	0
peec.ai	Established	200	No	No	No	0
pendo.io	Established	200	No	No	No	0
peoplebox.ai	YC	200	No	No	No	0
persistiq.com	YC	200	No	No	No	0
pipedrive.com	Established	200	Yes	No	No	0
plaid.com	Established	200	No	No	No	0
plane.com	YC	200	No	No	No	0
plerk.io	YC	unreachable	No	No	No	0
podium.com	YC	200	No	No	No	0
pointone.com	YC	200	No	No	No	0
polotab.com	YC	200	No	Yes	No	0
popl.co	YC	200	Yes	No	No	0
porter.run	YC	200	Yes	No	No	0
posthog.com	Established	200	No	No	No	0
postman.com	Established	200	Yes	No	No	0
preki.com	YC	unreachable	No	No	No	0
productboard.com	Established	200	No	No	No	0
properfinance.io	YC	unreachable	No	No	No	0
pump.co	YC	200	No	No	No	0
qualtrics.com	Established	200	No	No	No	0
quanwellbeing.com	YC	unreachable	No	No	No	0
quartzy.com	YC	200	Yes	No	No	0
quotabook.com	YC	200	No	No	No	0
rallyuxr.com	YC	200	Yes	No	No	0
ramp.com	Established	200	Yes	No	No	0
rapid7.com	Established	200	Yes	No	No	0
rdash.io	YC	200	No	No	No	0
re2.ai	YC	200	No	No	No	0
ready.net	YC	200	No	No	No	0
rebill.com	YC	200	Yes	No	No	0
recurly.com	Established	200	No	No	No	0
rehook.ai	YC	unreachable	No	No	No	0
revl.com	YC	200	No	Yes	No	0
ridecell.com	YC	unreachable	Yes	No	No	0
rippling.com	Established	200	Yes	No	No	0
rootly.com	YC	unreachable	Yes	No	No	0
runway.team	YC	200	Yes	No	No	0
salarybox.in	YC	200	No	Yes	No	0
salesforce.com	Established	200	No	No	No	0
salesloft.com	Established	200	Yes	No	No	0
salespatriot.com	YC	200	No	No	No	0
salt.pe	YC	200	No	No	No	0
salvy.com.br	YC	200	No	No	No	0
santehq.com	YC	200	Yes	No	No	0
scispot.io	YC	200	Yes	Yes	No	0
secoda.co	YC	200	Yes	Yes	No	1
segment.com	Established	200	No	No	No	0
semrush.com	Established	200	No	No	No	0
sendbird.com	YC	200	No	Yes	No	3
sentinelone.com	Established	200	Yes	No	No	0
sentry.io	Established	200	No	No	No	0
seoclarity.net	Established	200	Yes	No	No	0
shortcut.com	Established	200	Yes	No	No	0
sigmamind.ai	YC	200	Yes	No	No	0
signoz.io	YC	200	No	No	No	0
slack.com	Established	200	No	No	No	0
smartrecruiters.com	Established	200	Yes	No	No	0
smartsheet.com	Established	200	No	No	No	0
snapdocs.com	YC	200	Yes	No	No	0
snowflake.com	Established	200	Yes	No	No	0
snyk.io	Established	200	No	No	No	0
spadeworks.co	YC	200	No	No	No	0
spott.io	YC	200	Yes	Yes	No	0
sproutsocial.com	Established	200	No	No	No	0
stafbook.com	YC	200	Yes	No	No	0
stilt.com	YC	200	Yes	No	No	0
storylane.io	YC	200	Yes	Yes	No	0
stream.claims	YC	200	Yes	No	No	0
stripe.com	Established	200	No	No	No	0
submittable.com	YC	200	Yes	Yes	No	0
suger.io	YC	200	Yes	No	No	0
sully.ai	YC	200	No	No	No	0
superb-ai.com	YC	200	No	No	No	0
superorder.com	YC	200	Yes	No	No	0
supertokens.com	YC	200	Yes	No	No	0
surferseo.com	Established	200	Yes	No	No	0
surveymonkey.com	Established	200	No	No	No	0
swif.ai	YC	200	Yes	No	No	0
swiftsku.com	YC	200	Yes	No	No	0
swiftype.com	YC	200	No	No	No	0
syncly.app	YC	200	No	No	No	0
sytex.io	YC	200	No	No	No	0
tableau.com	Established	200	No	Yes	No	1
tailor.tech	YC	200	No	No	No	0
talentropy.ai	YC	unreachable	No	No	No	0
tamarind.bio	YC	200	No	No	No	0
tenable.com	Established	200	Yes	No	No	0
tesorio.com	YC	200	No	No	No	0
torch.io	YC	200	No	No	No	0
tractian.com	YC	200	Yes	No	No	0
trello.com	Established	200	No	No	No	0
truewind.ai	YC	200	No	No	No	0
trycoast.com	YC	200	Yes	No	No	0
tryprofound.com	Established	200	Yes	No	No	0
tryscribe.com	YC	200	No	No	No	0
tryterra.co	YC	200	No	No	No	0
twenty.com	YC	200	Yes	No	No	0
twilio.com	Established	200	No	No	No	0
tyltgo.com	YC	unreachable	No	No	No	0
typeform.com	Established	200	Yes	No	No	0
ubits.com	YC	200	Yes	No	No	0
unbounce.com	Established	200	Yes	No	No	0
unravelcarbon.com	YC	200	Yes	No	No	0
unsaddl.com	YC	200	Yes	No	No	0
upkeep.com	YC	200	Yes	No	No	0
uplane.com	YC	200	No	No	No	0
useagave.com	YC	200	Yes	No	No	0
usecaribou.com	YC	200	No	No	No	0
usefini.com	YC	200	No	No	No	0
usehatch.ai	YC	200	Yes	No	No	0
usekaya.com	YC	200	No	No	No	0
usemotion.com	YC	200	Yes	No	No	0
usenash.com	YC	200	Yes	Yes	No	7
userpilot.com	Established	200	Yes	No	No	0
vector.co	YC	200	Yes	No	No	0
vercel.com	Established	200	No	No	No	0
verihubs.com	YC	200	Yes	No	No	0
versori.com	YC	200	Yes	No	No	0
vitalize.care	YC	200	No	No	No	0
vobi.com.br	YC	200	Yes	No	No	0
volopay.com	YC	200	No	Yes	No	8
walkme.com	Established	200	No	No	No	0
waterplan.com	YC	200	No	No	No	0
webflow.com	Established	200	Yes	No	No	0
withblaze.app	YC	unreachable	No	No	No	0
wondercraft.ai	YC	200	No	No	No	0
workday.com	Established	200	No	No	No	0
wrike.com	Established	200	Yes	No	No	0
writesonic.com	Established	200	Yes	No	No	0
xero.com	Established	200	No	No	No	0
ycombinator.com	YC	200	Yes	No	No	0
zapier.com	Established	200	No	Yes	No	0
zeal.com	YC	200	Yes	Yes	No	0
zendesk.com	Established	200	Yes	No	No	0
zeorouteplanner.com	YC	200	No	Yes	No	0
zoho.com	Established	200	No	No	No	0
zoom.us	Established	200	Yes	No	No	0
zoominfo.com	Established	200	Yes	Yes	Yes	4
zscaler.com	Established	200	Yes	No	No	0
zuddl.com	YC	200	Yes	No	No	0

AI-Advisors AI Bot Blocking Index, June 2026 (Q2) edition. 390 B2B SaaS sites, scanned 2026-06-27. Free to cite and reuse with attribution (CC BY 4.0).

Cite this edition

For the methodology, cohort definitions, FAQ, and the always-current numbers, use the canonical AI Bot Blocking Index. To cite this frozen edition:

AI-Advisors (2026). AI Bot Blocking Index: how 390 B2B SaaS sites treat AI crawlers in robots.txt (June 2026 edition). https://www.ai-advisors.ai/research/ai-bot-blocking-index/2026-q2