Methods and limits

Every number on this site comes from one Caddy access log and one file of dated events. This page states how each figure is made, and where the method stops being trustworthy.

The classification cascade

pipeline/classify.sql assigns one class per request. It tests the user agent against an ordered list of rules and stops at the first match. Order is the method, not a detail. Named actors resolve before generic patterns, so a crawler that calls itself ClaudeBot lands in ai-anthropic even when its user agent also carries a Chrome version. Every page reads the same two DuckDB views, so a class cannot mean two things in two charts.

Version strings do not identify an actor. AionBot wears Chrome/126.0.0.0, Reflectionbot wears Chrome/151.0.0.0, and Amazonbot wears Chrome/119.0.6045.214. The rotator uses the reduced form that real Chrome also emits. Only the behaviour separates them.

Outcomes

aborted means HTTP status 0, the value Caddy writes when abort closes a connection: the server refused without answering. denied means status 403, a refusal that arrived with a response. Both fingerprint tiers answered with 403 until 2026-10-18 and close the connection after, so a blocked fingerprint counts as aborted from that date and the akin field, not the status, identifies it. A request counted as served returned 200.

Change detector

Each class becomes a daily series, zero-filled across the window, so a class that stops still registers. At each day the detector compares the mean of the three days before with the mean of the three days after. It adds 1 to both sides to survive zeros and flags a ratio at or above 2 or at or below 0.5. A window needs at least 20 requests to qualify. Consecutive flags of the same class and direction collapse into one episode.

This is a detector, not an estimator. The window carries no control series, and several rules change within days of each other, so it cannot support an intervention analysis, and a flagged day is not proof that a rule caused the change. The dated events in src/data/events.csv name the rule changes. The detector names the departures.

Behaviour fingerprints

Requests per address, the share of addresses seen once, and the share of addresses seen on three or more days. These columns separate a fleet from a reader. A named crawler hides in a handful of addresses with thousands of requests each. The rotator appears as tens of thousands of addresses at about one request each.

Read the browser class with those columns in view. A real reader returns. A fleet does not.

Request fingerprints

caddy-akin computes a token for each request from the header names it carries and the grammar of their values. ASCII letter runs collapse to a, digit runs to 9, and runs of three or more identical characters shrink to two before the result is hashed, so the value itself never enters the token. Two consequences matter on the rules page. A caller that rotates a user-agent version keeps one token, which is how one row can cover thousands of addresses. And the token is coarse: two clients whose header sets and value shapes agree share a token, so a row can merge clients that the user agent would have separated.

The token describes whatever opened the connection. Behind a load balancer that terminates HTTP, it fingerprints the balancer.

What the site cannot tell you

Not measured