Scraping API
Help center

Ask AI engines

Send a prompt to ChatGPT, Gemini, Google AI Mode, Perplexity, Copilot or duck.ai with type llm_scraping and get the answer with its citations as JSON.

type: llm_scraping sends a prompt to a public AI engine and returns the answer as that engine shows it to a visitor, with its citations. Use POST /task for one prompt and wait for the answer, or POST /job for a batch.

Minimal request

Shell
curl https://scraping-api.datafuel.ai/api/v1/task \
  --request POST \
  --header "Content-Type: application/json" \
  --header "X-API-Key: df_key_your_key_here" \
  --header "Idempotency-Key: crm-perplexity-us-2026-11-02" \
  --data '{
    "type": "llm_scraping",
    "proxy_country": "us",
    "attributes": {
      "prompt": "best crm for startups",
      "engine": "perplexity",
      "websearch": true
    }
  }'

The call blocks until the engine has answered. Set your client timeout to at least 120 seconds; AI engines often take tens of seconds. If the API answers 202 TASK_STILL_PROCESSING, poll GET /task/{id} with the id from the body.

Pick the engine

engineEngine
openaiChatGPT
geminiGemini
google_ai_modeGoogle AI Mode
perplexityPerplexity
copilotMicrosoft Copilot
duck_aiduck.ai

Engines can be switched off at runtime, for example while an upstream engine is down. GET /api/v1/config/capabilities (no key needed) lists which ones accept work right now. A call to a switched-off engine answers 503 ENGINE_UNAVAILABLE and charges nothing. An unknown engine answers 400 INVALID_ATTRIBUTES.

Shape the question

AttributeMeaning
promptThe question. Required on a task; a job takes prompts instead.
engineRequired. One of the engines above.
websearchtrue lets the engine browse the web while answering.
follow_up_promptA second prompt sent in the same conversation. Works on openai and perplexity only.
proxy_countryThe country the engine answers from, as an ISO code. Falls back to the top-level proxy_country; with neither, any country.
locationWhere the engine should answer for, used by gemini and perplexity. Defaults to the exit country.
modelduck_ai only: the model that answers. See below.

With follow_up_prompt, result.data holds the answer to the follow-up. On engines other than openai and perplexity the task fails and is refunded.

proxy_type is ignored: every engine uses the same pool and the same price. result_format is ignored too; the answer is always JSON.

duck.ai models

On duck_ai, model picks the model that answers:

  • gpt-6-luna (default)
  • gpt-5.4-mini
  • claude-haiku-4-5
  • mistral-small-2603
  • tinfoil/gpt-oss-120b
  • tinfoil/gemma4-31b

Any other value, or a model on another engine, answers 400 INVALID_ATTRIBUTES. On duck.ai, websearch switches web search on or off for the tinfoil/* models only; the other models decide on their own whether to search. Prompts that ask for an image are not supported: the task fails and is refunded.

Read the answer

result.data is an object:

FieldContent
textThe answer as the engine wrote it. [n] markers point at citations[].n.
citationsThe sources the answer cites: n, kind (source, or product for a product card), title, url. Left out when there are none.
productsopenai only, when ChatGPT shows product cards: title, url, price, merchants, rating, num_reviews, image, and offers (merchant, price, url).
JSON
{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "status": "completed",
  "credits_used": 100,
  "duration_ms": 18210,
  "engine": "perplexity",
  "result": {
    "data": {
      "text": "For most startups HubSpot's free CRM is the easiest start [1]; Pipedrive suits sales-led teams [2].",
      "citations": [
        { "n": 1, "kind": "source", "title": "The 10 best CRM tools for startups", "url": "https://example.com/best-crm" },
        { "n": 2, "kind": "source", "title": "Pipedrive review", "url": "https://example.org/pipedrive" }
      ]
    }
  }
}

The envelope carries engine, the engine the task asked for. There is no status_code or final_url: no target site is fetched. See Read the result envelope.

Many prompts

A job takes prompts instead of prompt, at least two. Every other attribute applies to each prompt, and each prompt is its own task.

JSON
{
  "type": "llm_scraping",
  "multithreaded": true,
  "proxy_country": "us",
  "attributes": {
    "prompts": ["best crm for startups", "top project management software"],
    "engine": "perplexity",
    "websearch": true
  }
}

Poll and fetch it like any job: see Scrape a list of URLs.

Cost

100 credits per request, whatever the engine. A request that fails is refunded. See Pricing and credits.