# How bots are handled

> How OakData classifies bot and crawler traffic - tagged at ingest, stored but never billed, and filterable.

Source: https://oakdata.co/docs/concepts/bots

---

Bots hit every public site - search indexers, AI agents, uptime monitors, preview scrapers. OakData's approach is **tag, don't drop**: bot traffic is classified and stored so you can see it, never counts against billable usage, and can be excluded from any view with a single flag.

A visit is scored using client and server signals, then classified as human or bot.

Client and server signals combine at ingest to classify each visit - verified crawlers keep their name.

## Where classification happens

The **server is the source of truth**. At ingest - where the real request headers are available - each request is classified from its user agent, and the rows are marked with `is_bot` and a `bot_name` (e.g. `Googlebot`, `GPTBot`, `ClaudeBot`).

## The $bot hint

Some signals only exist in the browser - `navigator.webdriver`, headless markers, automation brands. The SDK runs a lightweight client-side check and tags suspect events with a `$bot` hint. The server combines that hint with its own classification, so a headless browser spoofing a normal user agent is still caught.

> **Default behaviour: tag, not block** The SDK does *not* drop bot traffic client-side - it tags it and lets the server decide. Crawlers still show up in your dashboard; they're just flagged.

## Stored, but not billed

Bot-flagged events are written to your project so the dashboard can surface crawler activity - useful for confirming an AI agent or search engine is reaching your pages. But OakData only bills **non-bot** events, so crawler floods never inflate your usage or push you toward a plan limit.

## Excluding bots from a view

Most read surfaces exclude bots by default, and the traffic [overview](https://oakdata.co/docs/api/rest) takes an explicit `exclude_bots` flag:

**exclude bots**

```bash
curl "https://oakdata.co/api/v1/overview?range=30d&exclude_bots=true" \
  -H "Authorization: Bearer oak_sec_xxxxxxxxxxxxxxxxxxxxxxxx"
```

Over [MCP](https://oakdata.co/docs/api/mcp), `get_overview` accepts the same `exclude_bots` argument. Every session and event also carries `is_bot` / `bot_name`, so you can split human and bot traffic however you like.