seleniumbase-website-scraper
Browser-to-API discovery agent.
This upgrades the original SeleniumBase website scraper idea into a hosted API discovery workflow:
- opens a site in a browser
- records observable XHR/fetch/document traffic
- captures JSON response bodies when the browser can read them
- filters analytics/static noise
- templates paths and query parameters
- splits GraphQL/JSON-RPC style dispatches
- infers request and response JSON schemas
- emits
openapi.json,openapi.yaml,report.md,index.html,client.mjs,confidence.json, and redacted samples
Run Locally
python -m pip install -r requirements.txt
python -m playwright install chromium
python agent.py https://example.com --wait-seconds 8 --max-clicks 4
Run as A2A
a2a dev
a2a test --invoke --skill discover_api_from_browser --args-json '{"url":"https://example.com","wait_seconds":8,"max_clicks":2}'
Use origins to focus on the API host:
{"url":"https://app.example.com","origins":["api.example.com"],"min_samples":2}
The replay skill accepts JSONL request/response captures and runs the same OpenAPI emitter without opening a browser.