Scraping API
https://scraping-api.datafuel.ai/api/v1
Help centerOpenAPI spec

Config

Reference data for building requests

GET/config/js-instructions

List the supported js_instructions actions

The canonical list of browser actions accepted in attributes.js_instructions, with their argument shape and an example. Actions with iframe: true also accept the iframe_ prefix (e.g. iframe_click). Static and cacheable for an hour.

Request
cURL
curl https://scraping-api.datafuel.ai/api/v1/config/js-instructions
Response
{
  "instructions": [
    {
      "action": "fill",
      "description": "Type text into an input.",
      "value": "array",
      "args": [
        {
          "name": "selector",
          "type": "string",
          "required": true
        },
        {
          "name": "text",
          "type": "string",
          "required": true
        }
      ],
      "iframe": true,
      "example": {
        "fill": [
          "input[name=q]",
          "laptops"
        ]
      }
    }
  ]
}
GET/config/proxy/locations
Countries, regions and cities available on a proxy plan
Query
proxy_type
string
Proxy plan; your account default when omitted.
BasicPremium
Request
cURL
curl https://scraping-api.datafuel.ai/api/v1/config/proxy/locations \
  --header 'X-API-Key: df_key_your_key_here'
Response
[
  {
    "code": "us",
    "name": "United States",
    "regions": [
      {
        "code": "ny",
        "name": "New York",
        "cities": [
          {
            "code": "newyork",
            "name": "New York"
          }
        ]
      }
    ]
  }
]
{
  "code": "INVALID_ATTRIBUTES",
  "message": "Invalid attributes for selected task type"
}
{
  "code": "UNAUTHORIZED",
  "message": "You are not authorized to perform this action"
}
GET/config/proxy/asn
ASNs available in one country on a proxy plan
Query
proxy_type
string
BasicPremium
countryREQUIRED
string
ISO 3166-1 alpha-2 code; anything else answers 400 INVALID_COUNTRY.
Request
cURL
curl https://scraping-api.datafuel.ai/api/v1/config/proxy/asn \
  --header 'X-API-Key: df_key_your_key_here'
Response
[
  {
    "code": "7922",
    "name": "Comcast Cable"
  }
]
{
  "code": "INVALID_ATTRIBUTES",
  "message": "Invalid attributes for selected task type"
}
{
  "code": "UNAUTHORIZED",
  "message": "You are not authorized to perform this action"
}
POST/filter/check
Detect the anti-bot protection of URLs
Probes each URL and reports the protection in front of it. Invalid or non-http(s) URLs are skipped;
when none is left the call answers 400 URL_LENGTH_CANNOT_BE_ZERO. Not billed.


Request
cURL
curl https://scraping-api.datafuel.ai/api/v1/filter/check \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: df_key_your_key_here' \
  --data '[
    "https://example.com/product/1"
  ]'
Response
[
  {
    "host": "example.com",
    "path": "/product/1",
    "protection_type": "cloudflare"
  }
]
{
  "code": "INVALID_ATTRIBUTES",
  "message": "Invalid attributes for selected task type"
}
{
  "code": "UNAUTHORIZED",
  "message": "You are not authorized to perform this action"
}