---
name: compliant-browser-scraping
description: "Safety and workflow guidance for authorized public or user-owned browser scraping with SeleniumBase. Use for website scraping runs, selector extraction, pagination, robots.txt checks, challenge detection, and artifact reporting."
---
# compliant-browser-scraping

# Compliant Browser Scraping

This agent is for authorized scraping of public or user-owned websites only.

## Mandatory safety rules

- Do not bypass, solve, defeat, evade, or work around CAPTCHAs, anti-bot protections, login gates, paywalls, robots.txt restrictions, ToS restrictions, IP blocks, access controls, or technical protection measures.
- If a CAPTCHA, anti-bot challenge, login wall, paywall, access denial, or equivalent challenge is detected, stop the run and return a clear `manual_intervention_required` or `blocked` status with warnings.
- Do not use stealth/undetected-browser modes, CAPTCHA solvers, proxy rotation, credential stuffing, cookie injection, or other circumvention techniques.
- Respect robots.txt when requested by the caller.
- Paginate only within the same scheme+host as the starting URL unless the caller starts a new scrape explicitly for another URL.
- Apply polite rate limiting between page visits.

## Selector mapping conventions

The public `selectors` input is a mapping from output field names to CSS selectors. Examples:

```json
{
  "title": "h1",
  "price": ".price",
  "links": "a.result::attr(href)",
  "next": "a[rel='next']::attr(href)"
}
```

Special selector names `next`, `_next`, `__next__`, `pagination_next`, or `next_page` are interpreted as pagination selectors and are not emitted as data fields.

Selector suffixes:

- `::text` extracts visible text (default behavior for element selectors).
- `::html` extracts innerHTML.
- `::attr(name)` extracts an attribute.

When multiple elements match a field, return a list. When one element matches, return a string. When nothing matches, return null.

## Output artifacts

Save machine-readable output under `outputs/seleniumbase-website-scraper/` as JSON or CSV. If screenshots are requested, save PNG screenshots in the same directory. Include warnings and status in the response.