Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.scrapely.io/llms.txt

Use this file to discover all available pages before exploring further.

Welcome to the Scrapely documentation! Scrapely is a powerful developer API and SDK designed to simplify extracting data from the modern web. We handle the heavy lifting of headless browser management, JavaScript rendering, proxy rotation, and anti-bot bypassing so you can focus entirely on your application’s logic.
If you find Scrapely useful, please consider giving our Python SDK a star on GitHub ⭐!

​
Why Scrapely?

Building a scalable web scraper today requires stitching together multiple complex systems. Scrapely unifies these into a single, easy-to-use API:
  • JS Rendering & Headless Browsers: Automatically render single-page applications (SPAs) and execute JavaScript without maintaining your own browser infrastructure.
  • Anti-Bot Bypass: Scrape protected sites without getting blocked by services like Cloudflare or Datadome.
  • Interactive Browser Actions: Simulate real user behavior by clicking buttons, typing text, scrolling, and waiting for elements to load before extracting data.
  • Full-Page Screenshots: Capture high-quality viewport or full-page screenshots of any URL.
  • CAPTCHA Solving: Built-in support for bypassing and solving complex CAPTCHAs (reCAPTCHA v2/v3 and Cloudflare Turnstile).

​
Getting Started

To get started, you’ll need a Scrapely API key. You can generate one by creating an account at Scrapely.io.

​
1. Installation (Python SDK)

If you are using Python, we highly recommend using our official SDK for the best developer experience: 
pip install scrapely-python-client

​
2. Your First Request

Here is a quick example showing how to fetch the fully-rendered HTML of a web page using Scrapely. 
from scrapely import Scrapely

# Initialize the client with your API key
client = Scrapely(api_key="YOUR_API_KEY")

# Create a crawl task
task = client.crawler.crawl(
    website_url="https://example.com",
    return_page_source=True
)

# Print the fully rendered HTML
print(task.result.html)

​
Explore the Documentation

Ready to dive deeper? Explore our core features below:

Basic Crawling

Learn how to extract fully rendered HTML, text, and metadata.

Browser Instructions

Automate interactions like clicks, typing, and scrolling.

CAPTCHA Solving

Bypass reCAPTCHA v2, v3, and Cloudflare Turnstile.

Screenshots

Capture beautiful viewport or full-page screenshots.