Compare Scraping APIs by Cost per Delivered Record, Not Request Price

A scraping API that costs $0.50 per 1,000 requests can still be more expensive than one that advertises a higher price. The difference shows up when targets block you, return CAPTCHA pages, require browser rendering, or force retries that still consume credits.

Price per request is the wrong unit

For high-volume scraping, the useful metric is cost per delivered record.

A request is not the same thing as a usable result. A request can fail with a 403, time out, return a login wall, or return a CAPTCHA page with HTTP 200. If your pipeline treats all 200 responses as success, you may pay for junk and store junk.

A better unit is:

cost_per_delivered_1k = total_spend / successful_records * 1000

That metric includes the parts pricing pages usually hide:

• Failed requests that still consume credits
• Retries that count as additional requests
• JavaScript rendering multipliers
• Residential proxy premiums
• Parsing failures after the fetch succeeds

If vendor A charges $0.50 per 1,000 requests and vendor B charges $0.70, vendor A only wins if both deliver records at roughly the same success rate and both bill failures the same way.

Model failures before comparing vendors

Say you submit 1,000,000 URLs at $0.50 per 1,000 requests.

At a 10% failure rate, 900,000 URLs return usable data and 100,000 fail. With no refund policy, you still pay for all 1,000,000 attempts:

1,000,000 / 1,000 * $0.50 = $500
$500 / 900,000 * 1,000 = $0.56 per delivered 1K

With pay-per-success or automatic failure refunds, you pay only for the 900,000 successful records:

900,000 / 1,000 * $0.50 = $450
$450 / 900,000 * 1,000 = $0.50 per delivered 1K

That is only $50 per million requests at a 10% failure rate. At 10 million requests per month, it becomes $500. At 20% failures, which is common on job boards, marketplaces, and bot-protected product pages, the difference gets large enough to change the vendor decision.

Wire fits this specific failure-cost problem by running extraction as jobs and refunding failed runs, so blocked or timed-out jobs do not become paid records.

Here is a small calculator I use when comparing providers:

def delivered_cost(
    requests: int,
    price_per_1k: float,
    failure_rate: float,
    refund_failures: bool,
    js_multiplier: float = 1.0,
    retry_multiplier: float = 1.0,
):
    successful = requests * (1 - failure_rate)

    if refund_failures:
        billable = successful
    else:
        billable = requests * retry_multiplier

    spend = (billable / 1000) * price_per_1k * js_multiplier
    return {
        "successful_records": int(successful),
        "spend": round(spend, 2),
        "cost_per_delivered_1k": round(spend / successful * 1000, 4),
    }

print(delivered_cost(
    requests=1_000_000,
    price_per_1k=0.50,
    failure_rate=0.10,
    refund_failures=False,
))

print(delivered_cost(
    requests=1_000_000,
    price_per_1k=0.50,
    failure_rate=0.10,
    refund_failures=True,
))

This is not a benchmark. It is a sanity check. You still need to measure your own target set.

JavaScript rendering changes the math

Browser rendering is the other place costs jump.

Many APIs charge a multiplier when you enable headless Chrome or Playwright. A 5x multiplier turns $0.50 per 1,000 requests into $2.50 per 1,000 requests. Some providers use higher multipliers for premium proxies, browser rendering, or both together.

The mistake is enabling browser rendering globally because a few targets need it.

Split your targets into modes:

{
  "static_http": [
    "https://example.com/products/1",
    "https://example.com/products/2"
  ],
  "needs_browser": [
    "https://example.com/search?q=laptop"
  ]
}

Then test both paths. If the price is present in the initial HTML, use HTTP. If the response contains only an app shell like this, you probably need rendering:

<div id="root"></div>
<script src="/assets/app.bundle.js"></script>

Even then, check whether the frontend calls a JSON endpoint you can request directly. Browser rendering should be the fallback, not the default.

Session-based browser billing can be cheaper for workflows that reuse cookies, perform login, paginate, or extract several pages in one browser session. It can be worse for short one-page jobs if the provider rounds session time up to a minimum billing interval. Measure both patterns.

Test with a small target set

Before signing an annual contract or migrating a pipeline, run a representative sample. I usually want at least a few hundred URLs per target type:

• Static product pages
• Search result pages
• Geo-specific pages
• Authenticated pages
• Known bot-protected pages

Log outcomes separately. Do not only log HTTP status.

import requests

BAD_MARKERS = [
    "captcha",
    "access denied",
    "verify you are human",
    "unusual traffic",
]

def classify_response(resp):
    text = resp.text.lower()

    if resp.status_code in (401, 403, 429):
        return "blocked"

    if any(marker in text for marker in BAD_MARKERS):
        return "blocked_content"

    if resp.status_code >= 500:
        return "server_error"

    if len(resp.text) < 500:
        return "suspiciously_small"

    return "usable"

resp = requests.get("https://example.com", timeout=30)
print(resp.status_code, classify_response(resp))

This catches the annoying case where the provider returns HTTP 200, but the page content is a CAPTCHA or bot challenge. If the API bills that as success, your real cost is higher than the invoice suggests because your parser still has no usable data.

For teams that need reliable extraction rather than raw HTML fetches, Wire is relevant because its job model makes failure handling, polling, and result accounting explicit instead of hiding them behind a single request count.

What to ask before committing

Ask vendors specific billing questions. Vague answers are a signal to keep testing.

• Does a 403 consume credits?
• Does a timeout consume credits?
• Does a CAPTCHA page returned as HTTP 200 count as success?
• Are internal retries billed separately?
• What is the multiplier for JavaScript rendering?
• Are residential proxies included or billed at a premium?
• Can I export per-request status and credit usage?

The cheapest scraping API is usually not the one with the lowest advertised request price. It is the one with the lowest cost per usable record for your target mix.

Take a sample of your real URLs, run them through two or three providers, classify the responses yourself, and calculate cost per delivered 1,000 records before you compare pricing pages.