# Product data monitoring with repeatable web collection

Design a product-price and availability collection workflow with typed extraction, explicit scheduling, validation, and request-level diagnostics.

Canonical: https://runharvester.com/use-cases/product-data-monitoring

By: [RunHarvester](https://runharvester.com/about-us)

Updated: 2026-10-03

## Collect a useful product record

Treat this as an implementation scenario rather than a customer case study. A catalog-monitoring pipeline may gather product URL, title, price, currency, availability, and observation time. Select stable identifiers to ensure a renamed item isn't mistaken for a new product.

Store the raw observation separately from derived price-change alerts. A missing price is not zero, an empty response is not proof of removal, and a locale change can alter the displayed currency.

1. Maintain an authorized list of target URLs and the intended locale.
2. Use your scheduler or queue to submit individual extraction requests.
3. Poll completion, inspect extraction state, and validate the record.
4. Persist valid observations with request ID and timestamp.
5. Compare equivalent records and flag anomalies for review.

## Schedule around the business decision

Determine collection frequency according to your application's reaction speed requirements, available capacity, and access constraints. Do not assume a landing-page example includes a scheduler. Your integration manages the queue and resubmission policy.

Keep repeated submissions idempotent within each scheduled job, but use a new logical identity for a new observation. Reusing the same identity indefinitely can replay old work rather than create the observation you intended.

## Measure valid observations, not only successful requests

Track missing fields, unexpected record counts, stale observations, and changes in effective route cost. Save diagnostic identifiers when collection fails. Test layout variants before applying a recipe to an entire catalog.

Inspect cache behavior when freshness matters and use the documented cache controls deliberately. Target terms, permissions, privacy requirements, and the meaning of the collected data remain part of the integration design.

- [Extraction validation guide](https://runharvester.com/guides/structured-extraction)
- [Billing and cache behavior](https://docs.runharvester.com/concepts/billing)

## What is an extraction recipe?

A recipe serves as reusable extraction settings. Use an owned active recipe if you need to run the same extraction again. When the target layout shifts, test sample pages and confirm required fields; relying solely on reuse doesn't ensure accurate data.

[Documentation](https://docs.runharvester.com/concepts/structured-extraction)

## Can I submit structured extraction in a Data API batch?

Data API batches accept raw scrape jobs only. Submit structured extraction as individual requests. Browser Workflow batches belong to a separate API and should not be confused with Data API batches.

[Documentation](https://docs.runharvester.com/concepts/structured-extraction)

## What determines the cost of a request?

Request cost depends on the active route and caching behavior. Auto serves as a routing policy, not a fixed price. Rely on response billing metadata and the current pricing version as the definitive source; plan prices and allowances are sourced from the live catalog.

[Documentation](https://docs.runharvester.com/concepts/billing)

## Continue reading

- [Structured extraction](https://runharvester.com/structured-data-extraction)

- [Retry policy](https://runharvester.com/guides/async-scraping-retries)

[Source: RunHarvester documentation](https://docs.runharvester.com/concepts/structured-extraction)