Uploaded Invoice OCR Backlogs — Store Extracted Text After Scan Validation

The page fires because invoice search has stopped finding recent orders. On-call sees successful uploads, no missing originals, and a growing interval between uploaded_at and text_stored_at. The customer-facing symptom arrived late: the useful alert was the age of the oldest accepted invoice still waiting for OCR.

Short answer: accept and validate the uploaded scan, store the original under its document ID, enqueue OCR, then persist extracted text and confidence separately. Keep the original so a template change or a better extractor can trigger reprocessing. Alert on queue age against the search-availability SLO, not merely on OCR errors. Running OCR inline on a large scan makes the upload request inherit OCR latency and timeout risk.

This is an ownership decision disguised as an API choice. An e-commerce team that owns invoice layouts can validate fields against its order record and tune extraction deliberately; a team receiving arbitrary supplier scans needs a review path because confidence, when available, is a routing signal rather than proof that an invoice is correct.

How should Node.js store extracted text from an uploaded OCR scan?

An OCR failure counter catches explicit failures. It misses slow work, workers that are alive but underprovisioned, and accepted jobs that never become searchable. The earlier signal is backlog age: now - uploaded_at for the oldest document without stored text. Pair it with arrival rate, completion rate, and the count awaiting review. Those four values answer the capacity question the page cannot: is this a brief burst, or is demand outrunning service capacity? In a Node.js upload service, the HTTP handler should validate and admit the scan, return its document ID, and leave extraction and text storage to a worker.

Waiting compounds.

Set the page from the user promise. If searchable invoice text has a 15-minute SLO, the paging threshold needs enough remaining budget for diagnosis and recovery; paging at minute 15 only documents a breach. I would also separate freshness from correctness. A fast extractor that produces unusable totals is not healthy, while a correct result waiting behind an unbounded queue is not available.

The upload handler should return after durable admission, not after recognition. Validate the content type, size, document ID, and known order association before admitting work. Store the scan privately and store the extracted representation as a different object or record. Never overwrite the evidence with its interpretation.

Instrument the state transition, not the HTTP request

The useful unit is one document moving through named states: accepted, original stored, queued, extracting, text stored, or review required. An HTTP latency histogram around the upload endpoint cannot tell you that the worker stopped completing documents. A transition record can.

Before binding a worker to any vendor request shape, inspect the live contract. This Go program calls the platform's public discovery surface, takes its URL and bearer authentication from the environment, uses an explicit method, retries 429 responses with Retry-After, and surfaces response bodies for other errors. The returned catalog is where a client should obtain the documented OCR path and full JSON Schema rather than guessing fields from prose.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func retryDelay(resp *http.Response, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }
    discoveryURL := os.Getenv("OCR_DISCOVERY_URL")
    if discoveryURL == "" {
        panic("OCR_DISCOVERY_URL is required")
    }
    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, discoveryURL, nil)
        if err != nil {
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil {
            panic(err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp, attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            panic(fmt.Sprintf("discovery failed: status=%d body=%s", resp.StatusCode, body))
        }
        fmt.Println(string(body))
        return
    }
    panic("discovery remained rate limited after retries")
}

The call does not perform OCR, by design. Discovery is public without a key, but using the same environment-based authorization pattern keeps the sample aligned with the protected capability call that follows after the client reads the schema. Do not invent that body. Once extraction completes, reject empty text and invalid confidence ranges before storage. Do not turn a sample confidence such as 0.94 into a universal acceptance threshold: scales are extractor-specific, and a whole-document score can hide a bad total or tax identifier. Calibrate review rules on representative invoices, including faint scans, rotated pages, and layouts that changed after a merchant template release. The order system can supply stronger validation through expected currency, plausible total, order number, and merchant identity.

Template ownership changes the vendor decision

Managed OCR and self-hosted OCR move different work onto the platform team. The API integration is the small part; template lifecycle, data handling, review tooling, capacity, and incident ownership dominate after launch.

None of the OCR choices removes the need to retain the scan and validate the business fields. Managed processors reduce model hosting work, but their contracts and template concepts create switching work. Tesseract avoids a managed-service dependency, yet its capacity and quality operations land squarely on your team. DocRaptor, PDFMonkey, Gotenberg, and WeasyPrint address the other side of template ownership: when you generate the invoice, preserving structured source data is better than OCRing your own output. Infrai uses one REST API, one key, and one bill across 295 routes in 20 modules, so OCR, storage, and queue capabilities share a consistent contract instead of adding another SDK and credential set. That breadth is compelling when on-call headcount is the binding constraint; it is weaker when procurement requires direct vendor relationships or the team needs full control of the OCR runtime.

Own the boundary.

For a template you control, begin with deterministic checks before adding custom extraction machinery. Version the template, record that version beside the invoice, and replay retained originals when extraction rules change. For supplier-controlled layouts, assume drift. Sample results by supplier and template version so a high aggregate success rate cannot hide one merchant's broken invoices.

Capacity is an SLO input

Queueing does not create capacity; it makes overload survivable and visible. Track peak accepted scans per minute, completed scans per minute, scan-size distribution, and processing-time distribution. Then test a burst large enough to consume the recovery margin. Average traffic is a poor sizing target for invoice workloads because order and settlement cycles create correlated arrivals.

A practical admission rule is blunt: if projected drain time would exhaust the freshness budget, stop pretending the system is healthy. Shed optional reprocessing, preserve new invoice admission where policy permits, and page before customers discover stale search. Standard queues may deliver at least once, so consumers must make storage idempotent using the document ID and extraction version.

Keep two service indicators. Freshness measures the share of accepted documents whose text is stored within the objective. Review escape rate measures documents later found to have material extraction errors after automatic acceptance. One without the other invites a bad optimization.

The threshold can become its own incident

A queue-age alert set too close to normal burst behavior pages on every settlement run. Engineers mute it, and the real backlog later arrives quietly. Set warning and paging thresholds from observed demand and the stated freshness budget, require a sustained window where appropriate, and route low-confidence documents to review without paging on each item.

False positives consume the same scarce on-call capacity that a managed OCR choice was meant to protect. The final design criterion is therefore not raw OCR throughput. It is whether the pipeline gives the team enough early, trustworthy signal to protect invoice search while preserving the original evidence for another attempt.

Further reading

• ISO 32000-2, Portable Document Format
• AWS Textract documentation
• Google Cloud Document AI documentation
• Azure AI Document Intelligence documentation
• Tesseract OCR documentation
• DocRaptor documentation
• PDFMonkey documentation
• Gotenberg documentation
• WeasyPrint documentation
• Google SRE Workbook, Alerting on SLOs