41 lines
1.2 KiB
Markdown
41 lines
1.2 KiB
Markdown
# seleniumbase-website-scraper
|
|
|
|
Browser-to-API discovery agent.
|
|
|
|
This upgrades the original SeleniumBase website scraper idea into a hosted API
|
|
discovery workflow:
|
|
|
|
- opens a site in a browser
|
|
- records observable XHR/fetch/document traffic
|
|
- captures JSON response bodies when the browser can read them
|
|
- filters analytics/static noise
|
|
- templates paths and query parameters
|
|
- splits GraphQL/JSON-RPC style dispatches
|
|
- infers request and response JSON schemas
|
|
- emits `openapi.json`, `openapi.yaml`, `report.md`, `index.html`,
|
|
`client.mjs`, `confidence.json`, and redacted samples
|
|
|
|
## Run Locally
|
|
|
|
```bash
|
|
python -m pip install -r requirements.txt
|
|
python -m playwright install chromium
|
|
python agent.py https://example.com --wait-seconds 8 --max-clicks 4
|
|
```
|
|
|
|
## Run as A2A
|
|
|
|
```bash
|
|
a2a dev
|
|
a2a test --invoke --skill discover_api_from_browser --args-json '{"url":"https://example.com","wait_seconds":8,"max_clicks":2}'
|
|
```
|
|
|
|
Use `origins` to focus on the API host:
|
|
|
|
```json
|
|
{"url":"https://app.example.com","origins":["api.example.com"],"min_samples":2}
|
|
```
|
|
|
|
The replay skill accepts JSONL request/response captures and runs the same
|
|
OpenAPI emitter without opening a browser.
|