GPTBot and OpenAI Crawler IP Ranges: How to Verify, Allow and Rate-Limit Them

OpenAI publishes the IP ranges for GPTBot, OAI-SearchBot and ChatGPT-User as JSON files at openai.com/gptbot.json, openai.com/searchbot.json and openai.com/chatgpt-user.json. Matching a request's IP against the right file is the only documented way to tell a real OpenAI crawler from a fake one. This guide gives you the files, a log-checking script, nginx and Cloudflare rules, and a rate-limiting setup for when a bot hits too hard.

Published: 2026-10-23 · 6 min read · Satelo

The official IP lists

OpenAI's Overview of OpenAI crawlers links one JSON file per bot. Each bot has a different job, so keep the lists separate rather than merging them into one "OpenAI" allowlist.

The prefix counts are what we saw when we opened the files on the publication date; all entries were IPv4 at the time. Each file has a creationTime field and a prefixes array of objects with an ipv4Prefix key, for example {"ipv4Prefix": "104.210.140.128/28"}. The lists change: on our check, chatgpt-user.json had been regenerated three days earlier. Never hard-code a copy you downloaded once.

What OpenAI does not offer, as far as its documentation shows: reverse DNS hostnames for verification. Some crawlers support a reverse-then-forward DNS check (Common Crawl, for instance, documents one for CCBot), but OpenAI's crawler page only gives the JSON lists. Verify by IP range.

Why the user agent alone proves nothing

Anyone can send User-Agent: GPTBot. Scrapers do it precisely because many sites let known bots through. That is the usual explanation behind two complaints you will find on forums:

• "GPTBot ignores robots.txt." OpenAI says its robots.txt settings control GPTBot and OAI-SearchBot. If you see "GPTBot" crawling disallowed paths, check the IP first. Requests from outside gptbot.json are not OpenAI.
• "GPTBot is DDoSing my site." Same first step. A flood under OpenAI's name from random hosting IPs is someone else. If the IPs are genuine, see the rate-limiting section below.

One legitimate exception to remember: OpenAI says ChatGPT-User visits are triggered by users, and robots.txt rules "may not apply" to them. A ChatGPT-User request to a disallowed page from an IP in chatgpt-user.json is expected behavior, not a violation. OpenAI also notes that when it fetches robots.txt it may add a robots.txt marker to the user agent, which helps you spot those requests in logs.

Check your logs: a 20-line script

This Python script (standard library only) downloads the current lists and labels every OpenAI-claiming request in a log as REAL or FAKE. It assumes the common log format, where the client IP is the first field.

import ipaddress, json, sys, urllib.request

BOTS = {"OAI-SearchBot": "searchbot", "GPTBot": "gptbot", "ChatGPT-User": "chatgpt-user"}
nets = {}
for ua, name in BOTS.items():
    with urllib.request.urlopen("https://openai.com/" + name + ".json") as r:
        data = json.load(r)
    nets[ua] = [ipaddress.ip_network(p.get("ipv4Prefix") or p.get("ipv6Prefix"))
                for p in data["prefixes"]]

for line in open(sys.argv[1], errors="ignore"):
    for ua in BOTS:
        if ua in line:
            ip = ipaddress.ip_address(line.split()[0])
            real = any(ip in net for net in nets[ua])
            print("REAL" if real else "FAKE", ua, ip)
            break

Run it with python3 verify_openai_bots.py access.log | sort | uniq -c | sort -rn to get counts per IP. We tested it on a sample log before publishing. Two adjustments you may need:

• Behind a CDN or proxy, the first field may be the proxy's IP. Log the real client IP (for example the header your CDN sets) or configure nginx's realip module, otherwise every request looks fake.
• Each bot is checked against its own list. A GPTBot user agent from a searchbot.json IP is still flagged. That is deliberate: the combination does not match OpenAI's documentation.

Allow or block real OpenAI bots in nginx

nginx's geo module sets a variable from the client IP. Generate an include file from the JSON (here with jq):

curl -s https://openai.com/searchbot.json \
  | jq -r '.prefixes[] | (.ipv4Prefix // .ipv6Prefix) + " 1;"' \
  > /etc/nginx/openai-searchbot.conf

Then reject requests that claim to be OAI-SearchBot but come from elsewhere:

geo $openai_search_ip {
    default 0;
    include /etc/nginx/openai-searchbot.conf;
}
map $http_user_agent $claims_searchbot {
    default 0;
    ~OAI-SearchBot 1;
}
map "$claims_searchbot$openai_search_ip" $fake_searchbot {
    default 0;
    "10" 1;
}
server {
    if ($fake_searchbot) { return 403; }
    ...
}

Repeat the pattern for GPTBot and ChatGPT-User with their own files. Put the download in a daily cron job followed by nginx -t && nginx -s reload, so new ranges are picked up and a broken download never takes the server down. On Apache, the equivalent building block is Require ip, described in the access control guide.

Allow or block in Cloudflare

Cloudflare's custom lists let you store IP ranges and reference them in rule expressions with ip.src in $list_name.

1. Create one IP list per bot, for example openai_searchbot and openai_gptbot, and import the prefixes from each JSON file (the lists accept CSV upload or the Lists API).
2. Add a WAF custom rule that blocks impostors: (http.user_agent contains "OAI-SearchBot" and not ip.src in $openai_searchbot), action Block. Repeat for GPTBot.
3. Check Security Events for requests from the real lists that are being challenged or blocked by other settings. A challenge page served to OAI-SearchBot keeps you out of ChatGPT search just as surely as a robots.txt rule.
4. Refresh the lists on a schedule through the API, for the same reason as the nginx cron job.

If you simply want to keep OpenAI's training crawler out, a robots.txt rule is the cleaner tool; firewall rules are for impostors and for load. Our guide to blocking AI crawlers in robots.txt has copy-paste rules for each bot.

Too many requests? Rate-limit instead of blocking

Blocking OAI-SearchBot outright removes you from ChatGPT search answers. If genuine OpenAI traffic is straining a small server, slow it down instead.

Crawl-delay is not a reliable lever. OpenAI's crawler documentation does not mention it, so whether its bots honor it is undocumented. The robots.txt standard, RFC 9309, does not define Crawl-delay at all, and Google's robots.txt documentation lists it among unsupported fields. Some crawlers do honor it (Common Crawl's FAQ says CCBot does), but you cannot count on it for OpenAI.

Server-side rate limiting works for any client. nginx's limit_req module limits requests per key, and its documentation notes that requests with an empty key are not counted. That lets you throttle only AI bots:

map $http_user_agent $ai_bot_key {
    default "";
    ~GPTBot $binary_remote_addr;
    ~OAI-SearchBot $binary_remote_addr;
}
limit_req_zone $ai_bot_key zone=aibots:10m rate=2r/s;

location / {
    limit_req zone=aibots burst=20 nodelay;
    limit_req_status 429;
}

The rate and burst values are illustrative; set them from your own logs and server capacity. Cloudflare offers the same idea as rate limiting rules. How OpenAI's crawlers react to a 429 response is not documented, so watch your logs for a week after the change. Also check the cheap fixes first: caching static pages and fixing redirect chains reduces load from every crawler at once.

A decision table for admins

What to do next

Run the script on last week's log. If most "GPTBot" traffic turns out to be fake, an impostor rule will cut more load than any robots.txt change. If it is real, decide what you want from each bot, using our guide to getting indexed by ChatGPT for the search side.

To see what your setup currently tells each AI crawler, use Satelo's free AI crawler checker: it reads your robots.txt and requests the page as each bot, so you can catch a rule or firewall setting that locks out OAI-SearchBot by accident. For the broader picture of crawlability and speed, the free site check takes about 30 seconds.

FAQ

What is the GPTBot IP range?

OpenAI publishes GPTBot's current IP ranges at openai.com/gptbot.json. The list changes over time, so fetch it regularly instead of copying ranges from a blog post. OAI-SearchBot and ChatGPT-User have their own files: searchbot.json and chatgpt-user.json.

How do I verify that GPTBot is really from OpenAI?

Check that the request's IP address falls inside a prefix listed in openai.com/gptbot.json. OpenAI's documentation does not provide reverse DNS hostnames for verification, so the IP list is the method to use.

Does GPTBot respect robots.txt?

OpenAI says robots.txt controls GPTBot and OAI-SearchBot independently. If a bot calling itself GPTBot ignores your rules, check its IP; requests from outside gptbot.json are not OpenAI. ChatGPT-User is different: OpenAI says robots.txt may not apply because users trigger it.

Does GPTBot support Crawl-delay?

OpenAI's crawler documentation does not mention Crawl-delay, and the robots.txt standard (RFC 9309) does not define it. Use server-side rate limiting with HTTP 429 if you need to reduce load.

Should I allowlist OpenAI's IP ranges in my firewall?

If you want to appear in ChatGPT search, yes for OAI-SearchBot: OpenAI says sites must allow traffic from its published search bot IPs to be eligible. Pair the IP list with the user agent so impostors stay blocked.

How often do OpenAI's IP lists change?

OpenAI does not publish an update schedule. Each file has a creationTime field; when we checked on October 23, 2026, chatgpt-user.json had been regenerated three days earlier. Refresh your copies automatically, for example daily.

Read next

Can ChatGPT Read JavaScript Sites? How to Test What OpenAI's Crawlers SeeCan ChatGPT crawl JavaScript? What OpenAI documents, what independent crawler data shows, a curl test you can run today, and fixes for client-side sites.

ChatGPT Ads: How They Work, What They Don't Touch, and Whether a Small Site Should CareHow ChatGPT ads work as of October 2026: who sees them, how they're bought and ranked, whether they affect answers, and what small sites should do instead.

ChatGPT Atlas Is Retired: What It Means for Site Owners and What to Do InsteadChatGPT Atlas is retired. What that means for SEO: what still decides ChatGPT visibility, how ChatGPT's cloud browser reaches your site, and what to check now.

ChatGPT SEO Services: An Honest Buyer's Guide (What to Pay For and What to Do Yourself)A buyer's guide to ChatGPT SEO services: what agencies actually deliver, what no one can promise, red flags, questions to ask and what you can do yourself.