seleniumbase-website-scraper/skills/compliant-browser-scraping/SKILL.md

name, description

name	description
compliant-browser-scraping	Safety and workflow guidance for authorized public or user-owned browser scraping with SeleniumBase. Use for website scraping runs, selector extraction, pagination, robots.txt checks, challenge detection, and artifact reporting.

compliant-browser-scraping

Compliant Browser Scraping

This agent is for authorized scraping of public or user-owned websites only.

Mandatory safety rules

• Do not bypass, solve, defeat, evade, or work around CAPTCHAs, anti-bot protections, login gates, paywalls, robots.txt restrictions, ToS restrictions, IP blocks, access controls, or technical protection measures.
• If a CAPTCHA, anti-bot challenge, login wall, paywall, access denial, or equivalent challenge is detected, stop the run and return a clear manual_intervention_required or blocked status with warnings.
• Do not use stealth/undetected-browser modes, CAPTCHA solvers, proxy rotation, credential stuffing, cookie injection, or other circumvention techniques.
• Respect robots.txt when requested by the caller.
• Paginate only within the same scheme+host as the starting URL unless the caller starts a new scrape explicitly for another URL.
• Apply polite rate limiting between page visits.

Selector mapping conventions

The public selectors input is a mapping from output field names to CSS selectors. Examples:

{
  "title": "h1",
  "price": ".price",
  "links": "a.result::attr(href)",
  "next": "a[rel='next']::attr(href)"
}

Special selector names next, _next, __next__, pagination_next, or next_page are interpreted as pagination selectors and are not emitted as data fields.

Selector suffixes:

• ::text extracts visible text (default behavior for element selectors).
• ::html extracts innerHTML.
• ::attr(name) extracts an attribute.

When multiple elements match a field, return a list. When one element matches, return a string. When nothing matches, return null.

Output artifacts

Save machine-readable output under outputs/seleniumbase-website-scraper/ as JSON or CSV. If screenshots are requested, save PNG screenshots in the same directory. Include warnings and status in the response.