Platform / Capabilities

Structured data extraction from web pages

Extract typed fields, lists, and object collections with RunHarvester. Reuse recipes and validate structured results alongside the original page.

By · Updated

Ask for the fields your application needs

Structured extraction appends a typed data object to a single Data API request. You can specify fields like title, numeric price, and URL, or utilize an existing active recipe. Design the schema with the downstream use case in mind: attempting to store every possible field complicates validation and upkeep.

Supported scalar types include string, number, integer, boolean, date, and URL. Lists use multiple; object collections describe repeated records with scalar fields. Nested object collections are not supported.

A small product schema

This illustrative request asks for a product title and price. It is a starting schema, not a promise that the example target contains a product. Validate it against a target you are authorized to collect.

Request / responseExample
{
  "url": "https://example.com/product",
  "mode": "auto",
  "extraction": {
    "fields": [
      { "name": "title", "type": "string", "required": true },
      { "name": "price", "type": "number", "required": true }
    ]
  }
}

Page completion and extraction validity are different checks

Read the returned data and extraction run state. Confirm required fields, types, currency context, and record identity before writing to a database. A completed page acquisition is not evidence that the extracted values are correct.

Use individual submissions for structured extraction: Data API batches accept raw scrape jobs only. Reuse recipes for repeatable targets and re-check representative pages after layout changes.

A few useful answers

Common questions

Can extraction return a list of products?

Yes. An object collection employs type object, sets multiple to true, and includes a fields list outlining its scalar attributes. Nested object collections aren't allowed. Field names need to be unique at their level; check current limits before creating a complex schema.

Read the documentation
What is an extraction recipe?

A recipe serves as reusable extraction settings. Use an owned active recipe if you need to run the same extraction again. When the target layout shifts, test sample pages and confirm required fields; relying solely on reuse doesn't ensure accurate data.

Read the documentation
Can I submit structured extraction in a Data API batch?

Data API batches accept raw scrape jobs only. Submit structured extraction as individual requests. Browser Workflow batches belong to a separate API and should not be confused with Data API batches.

Read the documentation
Is structured extraction billed separately?

The documented pricing model includes structured extraction in request pricing. The selected route can affect the total request cost. Check the returned billing metadata and current billing documentation before estimating a workload.

Read the documentation

Put it to work

Start with one real request.

Inspect the result. Refine your workflow. Choose a plan for your workload.