The Bot Evasion Atlas: How Modern Bots Hide — and How They Get Caught

1. The Naive Bot Is Solved. The Stealth Bot Isn't.

Most bot traffic is not trying very hard. A curl script announces itself in its user agent. A vanilla headless browser leaves navigator.webdriver set to true. A scraper running from a hosting provider arrives from an ASN no consumer ever uses. By ClickStream's internal engineering estimate, straightforward heuristics against this naive tier catch over 90% of bot traffic by volume.

The tier that matters commercially is the one that remains: stealth browsers. These are real browser builds — patched Firefox distributions, Chromium with stealth plugin stacks, patched WebDriver builds, commercial "anti-detect" browsers, and hybrid tools that mix a scripted HTTP client with a real browser for JavaScript-bearing pages. ClickStream's internal Bot Evasion Atlas, the living engineering reference this whitepaper is drawn from, tracks three populations that rely on them:

• Scraper farms — content theft, price scraping, and aggregator feeds. The cost is bandwidth, content leakage, and SEO dilution.
• Ad-fraud rings — impression and click fraud against CPM and CPC placements. The cost is invalid traffic and distorted analytics.
• Credential-stuffing rigs — leaked credentials replayed against login endpoints. The cost is account takeover and compliance exposure.

These tools share a defining property: they are not trying to be invisible. They are trying to be ambiguous — to keep every individual signal plausible enough that no detector crosses its confidence threshold. A detection system built on any single signal loses that game by design. The architecture below is built on the premise that the win condition is not one perfect signal but the reconciliation of many imperfect ones.

2. A Taxonomy Built for Decisions: 11 Categories + Human

"Bot or not" is the wrong question. A search engine indexing your pricing page, an AI assistant reading your docs on a user's behalf, an uptime monitor pinging your homepage, and a stealth scraper harvesting your catalog are all "bots" — and each one warrants a different response. ClickStream classifies every visitor into one of 12 lanes: human, plus 11 bot categories.

This is the same taxonomy published in ClickStream's customer documentation and rendered as per-category rollups in the Traffic Quality dashboard — the marketing version and the shipped version are the same list. The design intent is that each lane maps to a decision you would actually make, rather than collapsing everything into a single "invalid traffic" bucket.

3. The Registry: 158 Named Bots, 38 of Them AI Agents

Most declared bots are honest about who they are, and naming them precisely is more useful than lumping them together. ClickStream maintains a registry of 158 named bots across 8 categories, matched by user-agent signature. When a registry entry matches, the bot's name and category attach to every event that visitor produces:

The remaining three taxonomy lanes — stealth_bot, kiosk, and unknown_bot — are assigned by behavioral and heuristic paths rather than user-agent matching, because the traffic in those lanes by definition doesn't announce itself.

3.1 The AI-Agent Roster

The fastest-growing registry category is AI agents: 38 named entries, including GPTBot, ChatGPT-User, and OAI-SearchBot; ClaudeBot, Claude-User, and Claude-SearchBot; PerplexityBot and Perplexity-User; Google-Extended, Google-CloudVertexBot, and Google-NotebookLM; plus CCBot, Bytespider, Amazonbot, Meta-ExternalAgent, Applebot-Extended, DuckAssistBot, and others. The distinction between a training crawler (Google-Extended, CCBot) and a user-initiated assistant fetch (ChatGPT-User, Claude-User) matters — one is building a corpus, the other is a person asking a question through a machine — and named identity is what lets you treat them differently.

ClickStream detects and labels these agents; what to do with that knowledge stays in your hands. The documented pattern is to use the classification in your own site or edge code — for example, serving structured, machine-readable content when the visitor is an answer engine — so you decide what AI systems see. The dashboard closes the loop with answer-engine coverage reporting: which of your most-visited human pages no AI or search crawler has seen yet.

4. Two-Layer Detection: botScore Meets humanConfidence

A single request tells you about the network and the claimed identity. A session tells you about behavior. ClickStream scores both, separately, then reconciles them with one canonical expression used everywhere a verdict appears.

4.1 Layer One: The Per-Request botScore

Every incoming event gets a composite botScore from 0 to 100, assembled at the edge from network-level and declaration-level evidence:

The header-inconsistency check deserves a note because it catches an entire class of low-effort spoofing with no client-side code at all: a user agent claiming to be Chrome that arrives without the Accept-Language or Sec-CH-UA headers real Chrome always sends is flagged as a UA-spoofing script. The tool patched the JavaScript layer and forgot the HTTP layer.

4.2 Layer Two: The Per-Session humanConfidence

In parallel, each session accumulates a behavioral humanConfidence score from 0 to 100, computed from the interaction stream: clicks, form interactions, mouse-dynamics quality, hover events, text selection, clipboard use, scroll regression, timing regularity, pageview velocity, and passive-display heuristics. Humans are messy — they hesitate, re-read, select text, scroll back up. Automation is clean. The behavioral layer measures the mess.

4.3 One Canonical Verdict

The two layers are reconciled by a single shipped expression — the same one used at storage-write time, in live sessions, and in the dashboard:

The logic reads the way you'd want it to: a strong network signal is sufficient on its own, and a moderate network signal only convicts when the behavioral evidence agrees. A visitor with a middling botScore who then fills out a form, hesitates over pricing, and selects text stays human. In the dashboard, this collapses to a three-state verdict — Real person, Needs a check, or Automated (AI / bot / tool) — with a single 0–100 Human score per visitor: the behavioral humanConfidence when the session produced enough interaction data, or 100 − botScore as the estimate when it didn't.

5. The Stealth Playbook: What Modern Bots Fake

To understand the counter-signals, start with what stealth tooling can credibly falsify. The current generation — patched-Firefox stealth builds, Chromium stealth-plugin stacks, patched drivers, and commercial anti-detect browsers — reliably fakes, per the atlas:

• navigator.webdriver — set to false by every serious stealth tool.
• Plugin and language lists — padded to match a typical consumer Chrome profile.
• Canvas fingerprints — per-call randomized noise, so the same draw hashes differently every session.
• WebGL vendor and renderer — spoofed to realistic consumer GPU strings.
• Screen dimensions and pixel ratio — drawn from weighted lists of common consumer displays.
• Audio-context fingerprints — randomized noise in the rendered output.
• WebRTC local addresses — masked so the LAN address never leaks.
• hardwareConcurrency and deviceMemory — set to plausible consumer values.

The practical consequence: any detector that leans on one of these fields as a sole signal is already obsolete. Each is a soft signal at best once stealth tooling is in play — which is why they matter in combination rather than in isolation, as the next section shows.

6. The Counter-Signals: What They Can't Fake

Stealth tools operate at the JavaScript layer. The signals that survive are the ones anchored below it, beside it, or across time.

6.1 Cross-Signal Inconsistency

Each faked field is individually plausible; the combination is where the persona falls apart. Impossibility patterns cataloged in the atlas from observed stealth traffic include: a user agent claiming iPhone with a 1,600-pixel-wide screen (real iPhones don't exceed roughly 430 CSS pixels); navigator.platform reporting Linux while the WebGL renderer reports an Apple M1 GPU; a device timezone of Asia/Manila on an IP that geolocates to Frankfurt; 16 reported CPU cores alongside a quarter-gigabyte of reported memory; and JavaScript languages of ['en-US'] under an Accept-Language header of ru-RU. Every input for these checks already rides on the event envelope, which makes fingerprint-consistency scoring the highest-density counter-signal per unit of collection cost.

6.2 Mouse Entropy

Human pointer movement has a texture that is surprisingly hard to synthesize: irregular micro-jitter, hesitation, curved approaches, overshoot. Automation fails at both extremes. Scripts that click via automation coordinates without moving the pointer produce teleport clicks — near-zero movement entropy. Tools that bolt on a "human-like" cursor library generate bezier curves that are too smooth and too consistent, while naive noise injectors produce uniform randomness no human hand can. ClickStream's stealth scoring evaluates mouse dynamics alongside fingerprint consistency, catching both the tools that ship no mouse engine at all and the ones that fake it too well.

6.3 TLS Fingerprints (JA3 / JA4)

A TLS ClientHello exposes cipher ordering, extension ordering, and signature algorithms — a fingerprint of the network stack itself. Real Chrome's TLS fingerprint changes only when the Chrome binary updates. A stealth setup routing requests through a Python HTTP library or a patched driver produces a fingerprint that is reliably not-Chrome, no matter what the user agent claims. This is the counter-signal that survives client-side patching entirely: faking it means rewriting the TLS stack itself, an order of magnitude more work than the JavaScript patches stealth tools actually ship. TLS fingerprint checks feed ClickStream's stealth detection alongside the behavioral and consistency signals.

6.4 The "Neither" Band as a Signal

Cloudflare Bot Management scores every request, and stealth tools explicitly engineer to land in the ambiguous middle band — not clearly bot, not clearly human. ClickStream treats that ambiguity itself as evidence: a mid-band network score combined with any other stealth indicator multiplies rather than dilutes suspicion.

6.5 The stealth_bot Lane

These counter-signals combine into a stealthScore from 0 to 100. Sessions crossing stealthScore ≥ 60 are promoted into the dedicated stealth_bot lane — so billing, analytics storage, live sessions, and the Signals SDK all agree on the same non-human split. Promotion is classification, not punishment: stealth-flagged sessions are excluded from human metrics and surfaced for review, but ClickStream never blocks or challenges the traffic. Gating is your decision to make in your own stack, with better evidence than you had before. The atlas also catalogs session-layer impossibilities — the same visitor ID appearing across dozens of IPs on multiple continents within minutes — as the next investment layer beyond per-session scoring.

7. The Crawlers That Never Run JavaScript

There is a blind spot built into most analytics: the pixel is JavaScript, and the most important non-human visitors never execute JavaScript. Search crawlers and AI answer engines overwhelmingly fetch raw HTML and move on. A JS-only pixel doesn't undercount them — it counts zero of them.

Because ClickStream's collection runs on the customer's own tracking subdomain at the edge, it offers edge capture for exactly this traffic: requests are observed and classified at the HTTP layer, before any script does or doesn't run. That is what makes the AI-agent registry useful in practice — GPTBot is not going to run your analytics bundle, but it will make an HTTP request, and that request is attributable and classifiable.

The dashboard turns this into a working GEO/AEO tool: answer-engine coverage reporting compares AI-agent and search-crawler activity against your human traffic and lists the top human-visited pages that no AI or search crawler has seen — the pages invisible to the systems increasingly answering your prospects' questions.

8. The Kiosk Lane: Traffic That Is Neither Bot Nor Human

Some traffic is automated but entirely legitimate, and classifying it either way corrupts a metric. Consider a hypothetical operations team that keeps a status dashboard open on a wall-mounted TV, auto-refreshing every minute. Call it human, and you've invented a tireless visitor with thousands of pageviews and zero interactions, wrecking engagement and conversion baselines. Call it a bot, and your bot rate looks alarming while describing your own signage.

ClickStream gives this traffic its own lane: kiosk. Passive-display heuristics — sustained pageview patterns with the interaction profile of an unattended screen — route digital signage, TV dashboards, and auto-refresh tickers into a category that pollutes neither the human nor the bot side of the ledger. It is a small feature with an outsized effect on metric integrity, and a useful test of any vendor's taxonomy: ask where the lobby TV goes.

9. When We're Wrong: The Operator Override

Every classifier has a false-positive rate, and a serious detection product should answer "what happens when you're wrong?" with a mechanism, not a reassurance. ClickStream's answer is the operator override: from the dashboard, you can mark any visitor as human. The override is stored for 90 days, beats the known-bot cache, and suppresses stealth promotion for that visitor — one click, effective immediately. The same lane works in the other direction: you can force a visitor to be classified as a bot, and every subsequent event carries is_bot = 1.

This is also why the pipeline is deliberately conservative about consequences. Suspicious sessions are bucketed for review — Needs a check — rather than silently discarded, and ClickStream never blocks, challenges, or rate-limits anyone's traffic. Classification you can inspect and overrule is worth more than enforcement you can't.

10. Bot Signals in Your Own Code

Every verdict described above is exposed to your application through the Signals SDK (@clickstreamhq/signals, currently a 0.1.0-alpha developer preview — APIs may change before general availability). The visitor object carries the bot verdict, the named identity when the registry matched, and the session's behavioral class:

A hypothetical use: an edge middleware that checks visitor.bot.category === 'ai_agent' and swaps in a structured, fact-dense variant of the page for answer engines while humans get the full experience. ClickStream supplies the classification; the serving logic is yours.

Two guarantees back these signals. First, bot-quarantined identity: flagged visitors never merge into your identity graph, so a scraper can never become a "Person" in your CRM sync. Second, quota fairness: billing counts human pageviews, while bot and AI traffic is metered on the separate Signals Coverage allowance — 5× the pageview allowance at every tier — so a crawler storm classifies without consuming your plan.

11. Honest Scope: Heuristics, Not a Trained Model

A candid statement of method, because bot-detection marketing is full of unfalsifiable AI claims: ClickStream's bot detection is not a machine-learning model trained on customer traffic. It is a layered heuristic system — Cloudflare Bot Management signals weighted into the score, the named-bot registry, ASN and header-consistency checks, and behavioral scoring computed at the edge — reconciled by the composite expression in section 4. ClickStream does not train AI models on customer data, full stop.

We consider this framing a feature rather than a confession. Every verdict decomposes into inspectable parts: which signals fired, what the network layer said, what the behavior showed. When the atlas identifies a new evasion technique, the response is a reviewable rule change, not an opaque retraining run. Stealth scoring runs inside the same edge scoring pass as the rest of ClickStream's 26 behavioral models, a pipeline held to a CI-enforced benchmark of p95 under 3 ms per event — detection depth without a latency tax. And the boundaries are explicit: stealth detection is a cat-and-mouse game with no finish line. The atlas exists precisely because the evasion surface moves, and a detection vendor that claims a solved problem is describing a product that has stopped being maintained.

12. Conclusion

Modern bots don't hide by being invisible; they hide by being ambiguous. The answer is not a single clever signal but an architecture: a taxonomy with enough lanes to match reality, a registry that names what can be named, two scoring layers that check the network's story against the session's behavior, counter-signals anchored below the JavaScript layer where stealth tools can't reach, edge capture for the crawlers that never run a script, and an override for the day the classifier is wrong.

That is what the Bot Evasion Atlas documents, and what ships in ClickStream today: 158 named bots, 38 named AI agents, 11 categories plus human, one composite verdict — attached to every visitor, exposed to your code, and honest about how it was reached.