bobseares-2/seleniumbase-website-scraper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

write README.md

Browse Source
This commit is contained in:
a2a-cloud
2026-06-07 23:16:11 +00:00
parent bfa86da68e
commit a828111d49
1 changed files with 24 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
README.md
View File

@@ -79,6 +79,30 @@ Pagination is restricted to the same scheme and host as the start URL.
} }
``` ```
## Smoke test payload
Use this payload to verify the scraper against the public demo site `https://quotes.toscrape.com/` while respecting robots.txt. It should return parseable JSON, quote items, a saved JSON file, an HTML capture path, and screenshot paths only when `screenshot` is `true`.
```json
{
"url": "https://quotes.toscrape.com/",
"selectors": {
"quotes": ".quote",
"text": ".text",
"author": ".author",
"tags": ".tag"
},
"max_pages": 1,
"wait_seconds": 1,
"output_format": "json",
"headless": true,
"user_agent": "",
"respect_robots": true,
"rate_limit_seconds": 1,
"screenshot": false
}
```
## Deployment notes ## Deployment notes
The agent declares a larger runtime budget because browser work can be CPU/memory intensive. Actual browser execution is performed through the workspace sandbox helper using the SeleniumBase image, so files written under `/workspace/outputs/seleniumbase-website-scraper/` are durable workspace outputs. The agent declares a larger runtime budget because browser work can be CPU/memory intensive. Actual browser execution is performed through the workspace sandbox helper using the SeleniumBase image, so files written under `/workspace/outputs/seleniumbase-website-scraper/` are durable workspace outputs.