instructions array to automate browser actions before the page is returned. Instructions run in order and each one occupies the thread until it completes or times out.
The total
timeout across all instructions cannot exceed 50 seconds.Available actions
- click
- mouse_click
- send_keys
- scroll_into_view
- wait
Clicks an element using JavaScript. Use this for most buttons and links.
curl -X POST https://api.scrapely.io/v2/tasks/create \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{
"crawler": {
"websiteURL": "https://example.com",
"return_page_source": true,
"instructions": [
{
"action": "click",
"selector": "button[type=submit]",
"timeout": 5
}
]
}
}'
from scrapely import Scrapely
from scrapely.models.types import Click
client = Scrapely(api_key="YOUR_API_KEY")
task = client.crawler.crawl(
website_url="https://example.com",
return_page_source=True,
instructions=[
Click(selector="button[type=submit]", timeout=5)
]
)
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
action | string | Yes | — | Must be "click". |
selector | string | No* | — | CSS selector of the element. |
xpath | string | No* | — | XPath of the element. |
timeout | number | No | 5 | Max wait time in seconds. Max 30. |
index | integer | No | 0 | Element index when using xpath and multiple matches exist. |
You must provide either
selector or xpath, not both.Simulates a physical mouse click. Use this for elements that don’t respond to JavaScript clicks.
curl -X POST https://api.scrapely.io/v2/tasks/create \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{
"crawler": {
"websiteURL": "https://example.com",
"return_page_source": true,
"instructions": [
{
"action": "mouse_click",
"selector": "div.custom-button",
"timeout": 5
}
]
}
}'
from scrapely import Scrapely
from scrapely.models.types import MouseClick
client = Scrapely(api_key="YOUR_API_KEY")
task = client.crawler.crawl(
website_url="https://example.com",
return_page_source=True,
instructions=[
MouseClick(selector="div.custom-button", timeout=5)
]
)
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
action | string | Yes | — | Must be "mouse_click". |
selector | string | No* | — | CSS selector of the element. |
xpath | string | No* | — | XPath of the element. |
timeout | number | No | 5 | Max wait time in seconds. Max 30. |
index | integer | No | 0 | Element index when using xpath and multiple matches exist. |
Types text into an input field.
curl -X POST https://api.scrapely.io/v2/tasks/create \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{
"crawler": {
"websiteURL": "https://example.com",
"return_page_source": true,
"instructions": [
{
"action": "send_keys",
"selector": "input#email",
"text": "user@example.com",
"timeout": 5
}
]
}
}'
from scrapely import Scrapely
from scrapely.models.types import SendKeys
client = Scrapely(api_key="YOUR_API_KEY")
task = client.crawler.crawl(
website_url="https://example.com",
return_page_source=True,
instructions=[
SendKeys(selector="input#email", text="user@example.com", timeout=5)
]
)
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
action | string | Yes | — | Must be "send_keys". |
selector | string | No* | — | CSS selector of the input. |
xpath | string | No* | — | XPath of the input. |
text | string | Yes | — | Text to type. Max 1000 chars. |
timeout | number | No | 5 | Max wait time in seconds. Max 30. |
index | integer | No | 0 | Element index when using xpath. |
Scrolls the page until the element is visible in the viewport.
curl -X POST https://api.scrapely.io/v2/tasks/create \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{
"crawler": {
"websiteURL": "https://example.com",
"return_page_source": true,
"instructions": [
{
"action": "scroll_into_view",
"selector": "div#load-more",
"timeout": 5
}
]
}
}'
from scrapely import Scrapely
from scrapely.models.types import ScrollIntoView
client = Scrapely(api_key="YOUR_API_KEY")
task = client.crawler.crawl(
website_url="https://example.com",
return_page_source=True,
instructions=[
ScrollIntoView(selector="div#load-more", timeout=5)
]
)
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
action | string | Yes | — | Must be "scroll_into_view". |
selector | string | No* | — | CSS selector of the element. |
xpath | string | No* | — | XPath of the element. |
timeout | number | No | 5 | Max wait time in seconds. Max 30. |
index | integer | No | 0 | Element index when using xpath. |
Waits for a specified duration or until an element appears.
curl -X POST https://api.scrapely.io/v2/tasks/create \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{
"crawler": {
"websiteURL": "https://example.com",
"return_page_source": true,
"instructions": [
{
"action": "wait",
"timeout": 3
}
]
}
}'
from scrapely import Scrapely
from scrapely.models.types import Wait
client = Scrapely(api_key="YOUR_API_KEY")
task = client.crawler.crawl(
website_url="https://example.com",
return_page_source=True,
instructions=[
Wait(timeout=3)
]
)
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
action | string | Yes | — | Must be "wait". |
selector | string | No | — | If provided, waits until this element appears. |
xpath | string | No | — | If provided, waits until this XPath element appears. |
timeout | number | No | 5 | Duration to wait in seconds. Max 30. |
index | integer | No | 0 | Element index when using xpath. |
Full example
Login flow using multiple instructions:curl -X POST https://api.scrapely.io/v2/tasks/create \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{
"crawler": {
"websiteURL": "https://example.com/login",
"return_page_source": true,
"instructions": [
{ "action": "send_keys", "selector": "input#email", "text": "user@example.com" },
{ "action": "send_keys", "selector": "input#password", "text": "password123" },
{ "action": "click", "selector": "button[type=submit]" },
{ "action": "wait", "timeout": 3 }
]
}
}'
from scrapely import Scrapely
from scrapely.models.types import SendKeys, Click, Wait
client = Scrapely(api_key="YOUR_API_KEY")
task = client.crawler.crawl(
website_url="https://example.com/login",
return_page_source=True,
instructions=[
SendKeys(selector="input#email", text="user@example.com"),
SendKeys(selector="input#password", text="password123"),
Click(selector="button[type=submit]"),
Wait(timeout=3)
]
)
print(task.result.html)