Poor data quality costs organisations real money. Forrester found that more than a quarter lose over $5 million a year, and 7% lose $25 million or more. The bill lands as wasted engineer time, wrong pricing decisions, and lost revenue, and it runs highest when the data was already wrong before any warehouse saw it. Proxy-Cheap sees this from the collection layer, where the record goes wrong first.
Poor data quality practices cost money at two layers. The downstream layer is well documented: Forrester found more than a quarter of organisations lose over USD 5 million a year, and 7% lose USD 25 million or more. The upstream layer is not. When your data comes from the public web, the record is already wrong before it reaches a warehouse, and no downstream check will catch it.
That Forrester split is the freshest hard number on the table, and it is not the one the search results lead with.
Ownership has moved up the org chart, too. In IBM's 2025 CDO Study, 43% of chief operations officers named data quality issues as their most significant data priority, so this is an operations problem now, not just an IT one.
For most organisations, reliable data is a crucial input to strategy and strategic decisions, and low quality data quietly raises compliance risk while dragging on growth and performance.
Here is a provenance problem worth knowing before you quote a number. Most coverage of this topic opens with "Gartner: USD 12.9 million a year." That figure comes from a July 2021 Gartner post by senior director analyst Melody Chien.
The catch is that competing articles cite USD 12.9 million, USD 15 million, and USD 9.7 million against the same broad attribution, which is the first sign a number is not load-bearing. Gartner's live public material on the subject, the May 2023 release 12 actions to improve data quality, carries no per-organisation cost figure at all, and the 2021 page no longer resolves.
So the number is not wrong so much as old, unresolvable, and not the one to plan against. For the current hard figure, use Forrester's research on data quality losses instead.
The cost then splits cleanly into two layers. The warehouse layer is measured, tooled, and covered by every article in this space. The collection layer, where web data is gathered, is unmeasured and untooled, and it is the subject of the rest of this piece.
Four collection-layer failures produce data that is technically valid and factually wrong: a response served for the wrong market, a response truncated mid-page, a cached response served from an edge node, and a challenge page parsed as content. Each returns HTTP 200. Each passes schema validation, null checks, and freshness checks. Each writes a plausible value into your dataset.
These four failures are why data collection workflows can report green while the numbers underneath are bad data. None of them errors. All of them lie.
The same URL returns different content by IP location: currency, price, availability, ranking, and copy all shift. A request sent from a datacenter in one country returns that country's version of the page. If your dataset is labelled as another market, every downstream number is inaccurate data and nothing flags it, because the response was a clean HTTP 200.
This is a location-accuracy problem, not an access problem. The control is direct: route the request through an IP in the market being measured. Residential proxies route traffic through consumer ISP networks for location-accurate collection, which is also the basis for verifying how a page renders in a given market.
Partial content returns parseable HTML that is missing the bottom of the page. The parser succeeds on the first N items and silently drops the rest, still on an HTTP 200.
Pagination and lazy-loaded sections make this systematic rather than random, so the same categories go missing every run. The symptom is record counts that look stable but are consistently short, with no error anywhere in the pipeline. That is incomplete data reported as a clean pull.
CDN edge nodes in different regions serve different cached versions, so your response may reflect the page as it was hours ago, again returned as HTTP 200. For any fast-moving field such as price, stock level, or availability, a cached response is a wrong response wearing a fresh timestamp, because the timestamp records when you fetched it, not when the source changed.
Here is the trap: freshness checks measure ingestion time. They cannot see source-change time, so data decay slips straight through.
This is the most expensive of the four. The target returns a verification interstitial with HTTP 200, the parser extracts the interstitial's text, and the pipeline reports success. Downstream, a price field may end up holding text, a coerced zero, or a null that a default quietly fills.
In practice, most "my data looks wrong" reports trace back to one of these four modes, and this one hides the longest because the row looks populated. The control is to validate the response body against expected shape and range, not the status code.
The Collection-Layer Cost Ladder has four rungs: request, record, dataset, decision. A wrong response costs almost nothing at the request rung. Once it is written, it costs a review cycle. Once it is aggregated, it costs a rebuild plus every report already published from it. Once someone prices, buys, or ships against it, it costs whatever that decision was worth.
This ladder is our model, and it explains why the four failures above are so expensive. The cost climbs at each rung:
The point is that the four collection failures are invisible at rung one, the only rung where they are cheap. They surface at rung three or four, if they surface at all.
That is what "shift left" means when your data comes off the public web: left of the pipeline is the request. Fixing quality there, before a wrong record reaches market research built on collected data, is the whole game.
Price your collection layer on cost per usable record, not cost per request. Take your monthly proxy spend, divide by the number of records that survive validation against the source, and compare that against the same figure for a different product type or market. A pool that costs less per gigabyte and more per usable record is the more expensive pool.
A usable record is one that survived a validation pass against the live source page, not one that merely got written.
The trap is that USD per gigabyte is the number providers compete on, us included, and it is the wrong number in isolation. Bandwidth spent on failed and wrong responses is still bandwidth billed. So run the arithmetic in a spreadsheet in five minutes.
| Input | How to get it | Output |
|---|---|---|
| Monthly collection spend | Billing dashboard | Spend for the period |
| Records written | Pipeline row count | Cost per record written |
| Records passing source validation | Sample and re-verify N records against the live page | Cost per usable record |
The gap between the two output figures is the whole point. Pay-as-you-go is what makes this measurable at all: per-gigabyte metering on rotating products makes spend-per-market directly attributable, while per-IP monthly on static products makes it fixed and predictable.
From there, do not chase a benchmark for a good cost per usable record. Establish your own baseline, watch the trend, and treat any market whose figure drifts up as a signal. If bandwidth is the bottleneck, unlimited bandwidth proxies change the arithmetic for high-volume price monitoring across markets.
Fix collection-layer data quality with three controls. Route each request through an IP in the market whose data you want, so the response you parse is the response a local user sees. Validate the response body, not just the status code. Match the product type to the target: rotating residential for high-volume market data, static residential or ISP for session-bound collection.
Here is what each control looks like in practice:
| Failure mode | Collection-layer control | Fit |
|---|---|---|
| Wrong-market response | Country, region, or city targeting on the request | Rotating residential; rotating mobile |
| Truncated response | Response-size assertion plus pagination completeness check | Any; a stable session helps |
| Edge-cached response | Fresh session per fetch; compare against a second exit IP | Rotating residential |
| Challenge page as content | Response-body validation; consistent session identity | Static residential (ISP) for session-bound targets |
For session-bound targets that must hold one identity, Proxy-Cheap's ISP proxy line keeps a fixed IP on pay-as-you-go or per-IP monthly billing with no monthly commitment. If you are still deciding, our guide to choosing a proxy for collection work maps each product type to a target class.
Poor data quality is data that does not meet the requirements of the operation using it. The standard dimensions are accuracy, completeness, timeliness, and consistency. A record can score well on all four and still be wrong, because those dimensions describe the record's shape, not its provenance. Web-collected data adds a fifth question: was this response the right response?
That fit-for-purpose definition is the consensus, and it holds. The four standard dimensions catch duplicate records, a stale row from a legacy system, or a value that contradicts its own table, and data engineers spend real resources on data observability tools to flag those data inaccuracies across datasets.
What they cannot catch, though, is a clean, consistent, well-shaped record that came from the wrong response. That is why provenance is the missing dimension for anyone collecting from the web. For the infrastructure side of that gap, our datacenter proxy provider comparison covers where high-throughput collection fits.
Standard cost models measure detection and repair time. Monte Carlo's data downtime formula multiplies the number of incidents by mean time to detection plus mean time to resolution. It is a sound model with one assumption baked in: that the incident starts inside your pipeline. For web-collected data, the incident starts at the request, which no pipeline metric observes.
Written out, data downtime equals the number of incidents multiplied by mean time to detection plus mean time to resolution. Monte Carlo pairs it with an estimate that data teams spend 30% to 40% of their time handling data quality issues instead of revenue work.
To be fair, the model is correct for warehouse-origin defects; the limitation is scope, not rigour. What mean time to detection cannot see is a defect that never becomes an incident, because nothing ever errored.
A wrong-market price that parses cleanly is exactly that: one of many data incidents with no detection event, so the average time to detect it never starts counting, and it sits in your analytics as an incorrect value with full confidence attached.
