---
title: "Site Auditor"
description: "Crawl a whole site, re-crawl or cancel, estimate cost and read page-level issues."
canonical_url: "https://developers.rankability.com/guides/site-auditor"
section: "Guides"
last_updated: 2026-11-14
---

# Site Auditor

Crawl a whole site, re-crawl or cancel, estimate cost and read page-level issues.

## Site Auditor

The Site Auditor crawls an entire site (not a single page) and reports per-page technical health plus site-wide issues: broken internal/external links, duplicate content, indexability, canonical clusters, and more. It is a pass-through bridge over the same engine that powers the in-app Site Auditor — the agent surface reuses the existing crawl pipeline with no scoring changes.

**Pricing.** A crawl costs **1 credit per 5 pages**, based on the project's `max_pages` (e.g. a 500-page crawl ≈ 100 credits). The credit model is **charge-on-success only**:

- The up-front check on create / crawl is an **affordability gate** (returns `402 insufficient_credits` if the org cannot afford the crawl). It reserves nothing.
- Credits are **charged once** when a crawl completes successfully (status `completed`), based on the pages actually crawled.
- A crawl that is **cancelled** or ends in **error** charges nothing — because nothing was reserved, there is nothing to refund.

This is **identical to the in-app Site Auditor's credit behavior** — the bridge reuses the same crawl launcher, which deducts credits on successful completion based on actual pages crawled. (This differs from the Page Auditor's flat reserve-then-charge model because the two features bill differently: Page Auditor is a flat per-audit charge, Site Auditor is metered per actual pages crawled. The agent surface mirrors each feature's existing billing, not a single shared model.)

Use `GET /site-auditor/credit-estimate?max_pages=N` to preview the cost before starting.

Crawls run asynchronously. Create a project with `POST /site-auditor/projects` (which also starts the first crawl), then poll `GET /site-auditor/projects/:project_id` until `status` is `completed` or `error`. Large crawls can take several minutes.

### Scopes

| Scope | Grants |
|---|---|
| `site-auditor:write` | Create projects + start crawls via `POST /site-auditor/projects` and `POST /site-auditor/projects/:project_id/crawl` |
| `site-auditor:read` | List/read projects and estimate cost via the `GET /site-auditor/*` endpoints |

## Site Auditor — Create project & crawl

`POST /api/agent/v1/site-auditor/projects`

Create a Site Auditor project and immediately start the first full-site crawl. Returns with `status: "crawling"`.

**Required scope:** `site-auditor:write`

### Request body

| Field | Type | Required | Description |
|---|---|---|---|
| `domain` | string | Yes | Domain or root URL to crawl (e.g. `"example.com"`). A leading `http(s)://` and trailing slash are stripped. Max 2000 chars. |
| `name` | string | No | Human-readable project name. Max 255 chars. |
| `client_id` | UUID | No | Optional client to associate the project with. Must belong to your organization. |
| `max_pages` | number | No | Max pages to crawl (default 500, 1–1000). Values outside the shared Site Auditor limit are rejected before work starts. Drives the credit cost. |
| `max_depth` | number | No | Max crawl depth from the root (default 10, 1–50). |
| `check_external_links` | boolean | No | Whether to validate external links (default false). |

### Example request

```bash
curl -X POST https://yourapp.com/api/agent/v1/site-auditor/projects \
  -H "Authorization: Bearer rk_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "name": "Example full-site audit",
    "max_pages": 500
  }'
```

### Response (201 Created)

```json
{
  "project_id": "8c2a...uuid",
  "client_id": null,
  "name": "Example full-site audit",
  "domain": "example.com",
  "status": "crawling",
  "max_pages": 500,
  "max_depth": 10,
  "check_external_links": false,
  "total_pages_found": 0,
  "pages_crawled": 0,
  "pages_analyzed": 0,
  "issues_found": 0,
  "created_at": "2026-06-14T15:00:00.000Z",
  "estimated_credits": 100
}
```

### Errors

| Status | Code | When |
|---|---|---|
| 400 | `invalid_input` | Missing/invalid fields or empty domain |
| 402 | `insufficient_credits` | Org cannot afford the crawl for `max_pages` |
| 404 | `not_found` | `client_id` does not exist or belongs to another organization |

## Site Auditor — Re-crawl project

`POST /api/agent/v1/site-auditor/projects/:project_id/crawl`

Start a fresh crawl on an existing project (reuses the project's stored crawl configuration). Prior crawl data is cleared; user-set page triage is carried over by URL.

**Required scope:** `site-auditor:write`

### Errors

| Status | Code | When |
|---|---|---|
| 402 | `insufficient_credits` | Org cannot afford the crawl |
| 404 | `not_found` | Project does not exist or belongs to another organization |
| 409 | `conflict` | A crawl is already in progress (`status` is `crawling` or `analyzing`) |

## Site Auditor — Cancel crawl

`POST /api/agent/v1/site-auditor/projects/:project_id/cancel`

Abort an in-progress crawl and flip the project `status` to `cancelled`. Only valid while the crawl is running (`status` is `crawling` or `analyzing`). No credits are charged for a cancelled run. Returns the updated project. If the crawl finished between your status check and the cancel call, the request fails with `409 conflict` rather than clobbering the completed project.

**Required scope:** `site-auditor:write`

### Errors

| Status | Code | When |
|---|---|---|
| 400 | `invalid_input` | No crawl in progress to cancel (`status` is not `crawling`/`analyzing`) |
| 404 | `not_found` | Project does not exist or belongs to another organization |
| 409 | `conflict` | The crawl finished before the cancel could be applied |

## Site Auditor — Get project

`GET /api/agent/v1/site-auditor/projects/:project_id`

Fetch a project without forcing its complete inventory into every call. Poll with `?view=status` until terminal. The default `view=summary` returns project, findings totals, and verification totals. Use `view=full` for paginated page and issue records.

**Required scope:** `site-auditor:read`

### Response

```json
{
  "project": {
    "project_id": "8c2a...uuid",
    "client_id": null,
    "name": "Example full-site audit",
    "domain": "example.com",
    "status": "completed",
    "max_pages": 500,
    "total_pages_found": 312,
    "pages_crawled": 312,
    "pages_analyzed": 312,
    "issues_found": 47,
    "broken_internal_links": 3,
    "broken_external_links": 1,
    "duplicate_content_pairs": 2,
    "unique_content_percent": 96,
    "started_at": "2026-06-14T15:00:01.000Z",
    "completed_at": "2026-06-14T15:04:22.000Z",
    "created_at": "2026-06-14T15:00:00.000Z",
    "updated_at": "2026-06-14T15:04:22.000Z"
  },
  "findings_summary": { /* severity and lifecycle totals */ },
  "verification_summary": {
    "unverified": 12,
    "semantics": "unverified means the finding was detected from crawl evidence but has not been rechecked against a later live fetch; it does not mean the initial detector lacked evidence"
  },
  "view": "summary"
}
```

For `view=full`, paginate inventories independently with `page_limit`, `page_offset`, `issue_limit`, and `issue_offset`. Limits default to 5 and max at 100. Nested keys are canonical `snake_case`; use `naming=legacy` only for compatibility with the older mixed-casing payload.

### Status values

| Status | Meaning | Credits status |
|---|---|---|
| `pending` | Created, not yet started | Not charged |
| `crawling` / `analyzing` | Crawl is in flight | Not charged |
| `completed` | Crawl finished successfully | **Charged** (1 credit / 5 pages crawled) |
| `cancelled` | Crawl was cancelled | No charge |
| `error` | Crawl failed | No charge |

### Errors

| Status | Code | When |
|---|---|---|
| 400 | `invalid_input` | Missing or malformed `project_id` |
| 404 | `not_found` | Project does not exist or belongs to another organization |

## Site Auditor — List projects

`GET /api/agent/v1/site-auditor/projects`

List projects for your organization, most recent first.

**Required scope:** `site-auditor:read`

### Query parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `client_id` | UUID | — | Filter by client |
| `status` | enum | — | Filter by `pending`, `crawling`, `analyzing`, `completed`, `error`, or `cancelled` |
| `limit` | number | 50 | Max 100 |
| `offset` | number | 0 | For pagination |

### Response

```json
{
  "projects": [
    {
      "project_id": "8c2a...uuid",
      "domain": "example.com",
      "status": "completed",
      "pages_crawled": 312,
      "issues_found": 47,
      "created_at": "2026-06-14T15:00:00.000Z",
      "completed_at": "2026-06-14T15:04:22.000Z"
    }
  ],
  "pagination": { "limit": 50, "offset": 0, "count": 1, "total": 1, "has_more": false }
}
```

## Site Auditor — Credit estimate

`GET /api/agent/v1/site-auditor/credit-estimate`

Estimate the credit cost of a crawl. Read-only, charges nothing.

**Required scope:** `site-auditor:read`

### Query parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `max_pages` | number | 500 | Pages to estimate for (1 credit / 5 pages) |

### Response

```json
{ "max_pages": 500, "estimated_credits": 100 }
```
