RunHarvester frequently asked questions

RunHarvester serves as a web-data collection platform. Its asynchronous Data API fetches pages via HTTP or browser routes, returning HTML, text, and typed extraction results. You can inspect requests through the client cabinet, whereas Browser Workflows manages a separate versioned automation lifecycle.

Choose HTTP if the required content appears in the initial response. Opt for browser mode when JavaScript execution or page interaction is necessary. Auto permits the service to select a route. Compare actual output and billing metadata on representative pages before committing to a fixed mode.

Browser mode runs the page within a browser environment. Set up supported wait conditions for the desired content and verify the outcome. Rendering a page doesn't ensure all lazy-loaded elements are visible or that the target service accepted the request.

No. Keep the request ID, check its status, and get the result once finished. States like Pending, running, and solving_captcha mean it's not done yet. Fetching a result prematurely yields RESULT_NOT_READY; check for errors rather than assuming every response is scraped data.

Provide extraction.fields for a specific request or link an active recipe you own using extraction.recipe_id. Fields can define scalar values, lists, and object groups. Check the returned data and extraction status before saving the result.

Yes. An object collection employs type object, sets multiple to true, and includes a fields list outlining its scalar attributes. Nested object collections aren't allowed. Field names need to be unique at their level; check current limits before creating a complex schema.

A recipe serves as reusable extraction settings. Use an owned active recipe if you need to run the same extraction again. When the target layout shifts, test sample pages and confirm required fields; relying solely on reuse doesn't ensure accurate data.

Data API batches accept raw scrape jobs only. Submit structured extraction as individual requests. Browser Workflow batches belong to a separate API and should not be confused with Data API batches.

Browser mode acts as a path for a Data API request. Browser Workflows handles reusable automation via workspaces, projects, environments, fixed published versions, and executions. Its basic structure is a linear sequence of typed steps, excluding arbitrary JavaScript, loops, or branching graphs.

Data retention is governed by your specific service and plan settings. Make sure to review the expiry details provided and fetch any necessary results prior to expiration. If a Data API result expires, it will return RESULT_EXPIRED. Ensure you export necessary data to your personal storage instead of relying on retained results as a permanent archive.

Generate an Idempotency-Key before submission and store it with the exact request body. Reuse it when retrying the same submission after a network interruption. A new logical request requires a new key. Retrying a failed request creates a new request ID; preserve the original for diagnosis.

Data API capabilities include both residential and residential_browser routing modes. Available geographic options and inventory vary based on your deployment and account setup. Review the active routing contract and metadata; noting a geographic preference does not guarantee localized content delivery from the target site.

No. Targets are capable of rejecting, challenging, or altering their responses. While browser and proxy routing aid acquisition, you must still verify results and public errors. Set a request credit limit where possible to control costs.

Store API keys securely in a server-side secret manager and assign only the necessary scopes for the integration. Avoid placing credentials in public frontend code, documentation examples, or logs. Browser Workflow keys and scopes are scoped to their specific project and environment.

Request cost depends on the active route and caching behavior. Auto serves as a routing policy, not a fixed price. Rely on response billing metadata and the current pricing version as the definitive source; plan prices and allowances are sourced from the live catalog.

The documented pricing model includes structured extraction in request pricing. The selected route can affect the total request cost. Check the returned billing metadata and current billing documentation before estimating a workload.

Requests that fail validation incur no fees. If a request fails completely, base debits are refunded; the final credit adjustment depends on the applicable billing rules. Cancellations in pending or running states may resolve at varying times. Cross-reference billing metadata and the returned token state to match up charges.

Make sure to attach the request or run ID, a rough timestamp, the projected outcome, and the public error code. Do not include API keys, cookies, signed URLs, or any sensitive collected data. Contact details are listed on the RunHarvester contact page.