Scraper and automation detection

The three shapes of automation

An HTTP client wearing a user agent. Python requests, Go net/http, curl. No JavaScript at all. Cheap, high volume, and the easiest to catch, because the TLS and HTTP construction has nothing to do with a browser.

An impersonating client. curl-impersonate, utls and the current scraping stacks ship a Chrome-shaped ClientHello on purpose, so a JA3 blocklist does not see them. Still no real browser above it.

A driven browser. Playwright or Puppeteer against real Chromium, usually with a stealth plugin patching the obvious tells, often on a residential proxy. This is the expensive tier and the one that gets through most systems.

The page-level checks, and their ceiling

Run them, because they are cheap and they still catch the long tail. navigator.webdriver. ChromeDriver's cdc_ properties. Playwright and Selenium bindings. A user agent that still says HeadlessChrome, which we treat as absolute, because a browser truthfully announcing that it is headless is not a person.

Do not build on them. Chrome's rewritten headless mode is the same binary with the same rendering pipeline, so most of the classic differences are gone, and the ones that remain are one line each to patch. A headed Chrome driven by Playwright on a rented VPS was never headless anyway.

What a patch cannot hide

A stealth plugin rewrites the page. Rewriting leaves marks, and the marks are structural rather than statistical.

• A native navigator getter throws when called with a plain object as the receiver, because the engine checks the receiver type. A JavaScript replacement usually does not.
• Patching a getter changes its property descriptor, and patching toString to hide that changes toString.
• A rebound WebGL method has a different name or length, and a context claiming WebGL 2 without the required methods is contradicting itself.
• navigator.plugins is a collection whose entries cross-reference MIME types. Faked as an array of literals, it contradicts its own lookup methods.
• Init scripts run in the main world. Getting the same patch into every worker and every frame with the same values is hard, so realms disagree: on the timezone, on a stable navigator value, or on the printed length of an engine builtin.

When one of these fires we mark the checks that depended on that function as abstaining. A rewritten reflection primitive means we can no longer trust what it told us, and pretending otherwise would be worse than reporting nothing.

What runs outside the page

The load-bearing part. Our sensor is bare metal, so it reads the connection rather than asking the browser about it.

The TCP SYN options identify the kernel. The TLS ClientHello identifies the library, and Chromium's GREASE values are absent from a Go or Python stack pretending to be Chrome. The HTTP/2 settings and pseudo-header order identify the HTTP implementation. A single Chromium session opening both TCP and QUIC offers the same key exchange groups on both, so a disagreement means the two connections were not made by the same software. A browser advertising HTTP/3 that never sends a QUIC Initial is a stack that cannot do QUIC.

None of this is reachable from JavaScript, so no stealth plugin patches it. Fixing it means changing the machine.

Refusing to be measured is a result

Automation frameworks routinely block the detection script, strip headers, drop the UDP a QUIC probe needs, or never execute the payload. Most products treat that as missing data and allow the session, because a missing signal feels like an unfair reason to block.

We return refused, and refused carries the same weight as a contradiction. Not because non-completion proves malice, but because if not answering is safer than answering, everyone stops answering.

Where we stop

A real Chrome on a real Windows machine, driven by a person or by an operator who paid for the hardware, behind a residential proxy near the timezone it claims, produces an honest transport stack and honest geometry. Every wire check passes, and that is the correct answer.

That is not detection failing. It is the cost floor moving. Requests-with-a-user-agent, impersonating clients and stealth-patched Chromium on a VPS all fail without any behavioural modelling, any machine learning and any history of the visitor. What is left costs real machines on real networks, and most scraping economics do not clear that bar.

We report the physics. What you do about a contradicted session is yours: rate limit it, serve it something cheaper, send it to a ceremony, or let it through and log it. We have no opinion, and the same honest report serves a customer who refuses everything and one who allows VPN users.