Errors, pagination & limits

How the User API and Proxy API report problems, how to page through lists and how much you can send.

This page covers the User API and Proxy API (api.datafuel.ai). The Scraping API reports errors as {code, message}; see its responses on each endpoint.

Error format

Every error answers with the same JSON shape:

error: a stable, machine-readable code such as insufficient_scope. Branch on this, not on the message.
message: a human-readable explanation. For 400 bad_request it is an object that maps each invalid field to its problem.
meta: details that depend on the error, for example required, order, issues or retry_after.

JSON

{
  "error": "bad_request",
  "message": {
    "idempotency_key": "An Idempotency-Key header is required: one value per order, sent again only to retry it."
  }
}

Status codes

Status	Meaning
400	The request is malformed. `message` maps each invalid field to its problem, or is a sentence when the body cannot be read at all.
401	`invalid_api_key`: the key is missing, invalid, expired or revoked.
402	The payment was declined; `error` says why and `meta.order` is the failed order.
403	`insufficient_scope`, `ip_not_allowed`, `spend_limit_exceeded` or `account_suspended`.
404	No such resource, or it does not belong to your account.
409	The request conflicts with the resource’s current state; `error` names why. For orders: `in_progress`, `idempotency_conflict` or `duplicate_request`.
422	`not_purchasable` (with `meta.issues` when the order itself is at fault, e.g. out of stock), `promo_code_invalid`, `amount_too_small` or `payment_rejected`.
429	`rate_limit_exceeded`: wait for the `Retry-After` seconds.
500	Something failed on our side and was logged. Read the resource before you retry a change.
502	`upstream_error`: the provider refused the request. Nothing was changed.
503	Temporarily unavailable, for example during maintenance. Try again shortly.
504	`outcome_unknown`: the provider stopped answering, so the change may or may not have happened. Read the resource before you retry.

Handling errors

Print the status code next to the body to see both:

cURL

curl -sS -w "\n%{http_code}\n" https://api.datafuel.ai/api/v1/isp/plans/PLAN_ID \
  -H "Authorization: Bearer $DATAFUEL_API_KEY"

Pagination

Lists take ?limit= (1 to 100, 50 by default) and ?offset=, and answer {"results": [...], "count": 120, "limit": 50, "offset": 0}, where count is the total number of matches.

cURL

curl "https://api.datafuel.ai/api/v1/orders?limit=100&offset=100" \
  -H "Authorization: Bearer $DATAFUEL_API_KEY"

Rate limits

Limits apply to each API key separately:

About 20 requests per second, with bursts of up to 40.
10 orders (POST …/orders) per minute, with bursts of up to 3.

Over a limit the API answers 429 rate_limit_exceeded with a Retry-After header and meta.retry_after, both in seconds. Wait that long before sending again; a retried order keeps its Idempotency-Key.