Documentation Index
Fetch the complete documentation index at: https://docs.scrapely.io/llms.txt
Use this file to discover all available pages before exploring further.
The official Python client for Scrapely.io, a powerful API for web scraping and CAPTCHA solving.
Our SDK provides a seamless developer experience with fully typed responses, auto-polling for task completion, and both synchronous and asynchronous support for high-concurrency applications.
Installation
You can install the SDK via pip:
pip install scrapely-python-client
Usage Examples
The SDK provides two clients: Scrapely for standard synchronous scripts, and AsyncScrapely for high-performance, concurrent applications using asyncio.
Synchronous (Sync)
Asynchronous (Async)
from scrapely import Scrapely
# Initialize the client
client = Scrapely(api_key="YOUR_API_KEY")
# 1. Crawl a website
response = client.crawler.crawl(
website_url="https://example.com",
return_page_text=True
)
print(response.result.text)
# 2. Solve reCAPTCHA V3
captcha = client.google.RecaptchaV3(
website_url="https://example.com",
website_key="6LdKlZEpAAAAAAOQjzC2v_d36tWxCl6dWsozdSy9"
)
print(captcha.result.solution)
import asyncio
from scrapely import AsyncScrapely
async def main():
# Initialize the async client
client = AsyncScrapely(api_key="YOUR_API_KEY")
# 1. Crawl a website asynchronously
response = await client.crawler.crawl(
website_url="https://example.com",
screenshot=True
)
print(f"Screenshot URL: {response.result.screenshot}")
# 2. Solve Cloudflare Turnstile asynchronously
captcha = await client.cloudflare.Turnstile(
website_url="https://example.com",
website_key="0x4AAAAAA..."
)
print(f"Token: {captcha.result.solution}")
if __name__ == "__main__":
asyncio.run(main())
Advanced Usage
Automation Instructions
You can pass a list of instructions to interact with the page before scraping. The SDK provides helper classes (SendKeys, Click, Wait, etc.) for type safety and auto-completion.
from scrapely import Scrapely
from scrapely.models.types import SendKeys, Click, Wait
client = Scrapely(api_key="YOUR_API_KEY")
instructions = [
SendKeys(selector="#search", text="scraping"),
Click(selector="#submit-button"),
Wait(timeout=2) # Wait 2 seconds
]
response = client.crawler.crawl(
website_url="https://example.com",
instructions=instructions,
return_page_text=True
)
print(response.result.text)
Typed Response Objects
All API responses are returned as strictly typed Python dataclasses for better IDE support.
CrawlerResponse: Returned by crawler tasks. Contains a result (CrawlerResult) object with fields like html, text, cookies, screenshot, and user_agent.CaptchaResponse: Returned by CAPTCHA tasks. Contains a result (CaptchaResult) object with the solved solution string.
