User-agent: *
Content-Signal: search=yes,ai-input=yes,ai-train=yes
Allow: /

Sitemap: https://nielsenai.ch/sitemap-index.xml

# ---------------------------------------------------------------------------
# AI crawler position — decided 2026-08-17, deliberately permissive.
#
# The Content-Signal line above is an affirmative statement, not silence:
#   search=yes     index this and return it in search results
#   ai-input=yes   use it for RAG / grounding / generative answers
#   ai-train=yes   use it to train and fine-tune models
#
# Rationale: this site is public marketing copy. Nothing on it gains value from
# models not having seen it. Being findable by AI is the point.
#
# Note this is Nielsen AI's position for THIS DOMAIN only. It says nothing about
# client systems or client data, where the opposite rule applies.
#
# HISTORY: until 2026-08-17 Cloudflare's managed-robots.txt feature PREPENDED a
# block ahead of this file that published `ai-train=no` and `Disallow: /` for
# ClaudeBot, GPTBot, Google-Extended, CCBot, Amazonbot, Applebot-Extended,
# Bytespider, meta-externalagent and CloudflareBrowserRenderingCrawler. It was a
# Cloudflare default, never a decision. Turned off via:
#   Security Settings -> Bot traffic -> Block AI training bots -> Do not block
#   Security Settings -> Bot traffic -> Manage your robots.txt -> Disable
# If AI crawlers ever appear blocked again, check the LIVE file before this one:
#   curl https://nielsenai.ch/robots.txt
#
# Which crawler governs what — training and citation are different questions:
#   OAI-SearchBot     ChatGPT search citations
#   GPTBot            OpenAI model training
#   ChatGPT-User      ChatGPT user-triggered page fetch
#   Claude-SearchBot  Claude search index
#   Claude-User       Claude user-triggered page fetch
#   ClaudeBot         Anthropic model training
#   Googlebot         Google Search, AI Overviews AND AI Mode
#   Google-Extended   Gemini app training/grounding — NOT AI Overviews
#
# Blocking a training crawler does not stop citation, and allowing a search
# crawler does not get you into the model's memory. Do not add a Disallow here
# without reading plans/2026-08-17-ai-search-visibility-fix.md first.
# ---------------------------------------------------------------------------