Private Exports: Caching Signed Download Links to Reduce Presign Rate-Limit Requests

Short answer: object storage fits private fintech exports, but large-file throughput depends on keeping page refreshes away from the presign path. Cache each still-valid signed download URL in the backend, preserve the export job and object key, and retry presign generation with bounded exponential backoff instead of signing again on every click.

A 4 GB statement archive should spend its time moving from object storage to the customer, not making the application mint another authorization artifact because a browser tab refreshed. The link is control-plane work; the file transfer is data-plane work. Treating both as one request path creates a rate-limit problem that more transfer capacity won't solve.

How should a fintech export page cache signed download URLs under presign rate limits?

Key the cache by tenant, user authorization scope, export job, and object key. Return a cached URL only while it has a conservative validity margin; otherwise, let one backend worker generate the replacement while concurrent requests wait for that result. Don't cache by object key alone. In a fintech system, that shortcut can cross an access boundary even when the underlying object remains private.

The export record is the durable center of the flow. It should hold the job state and object key after generation completes. The signed URL is disposable state layered on top. A page refresh can then read the same completed record and reuse its current link rather than starting a new export or presign call. This separation also prevents an impatient user clicking five times from creating five large archives.

Keep the expiry margin explicit. I'm not sure what margin is right for every workload because it depends on link lifetime, clock skew, and how long a download may sit before it starts; load tests with realistic file sizes and client geography should settle it. The important part is to expire the cache before the URL itself, not at the same instant.

One request wins.

Cache first.

The following Python script calls the verified presign path without inventing a request body or response field. Set INFRAI_API_ROOT to the API's versioned root and pass tenant, user, export, bucket, and key arguments. The example demonstrates scoped cache keys, one in-flight signer per key, a short reuse window, HTTP 429 handling, Retry-After, and jittered exponential backoff. Any other 4xx response is surfaced rather than treated as success.

import json
import os
import random
import sys
import threading
import time
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


class RateLimited(Exception):
    def __init__(self, retry_after: float | None = None):
        self.retry_after = retry_after


API_ROOT = os.environ["INFRAI_API_ROOT"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
CACHE_SECONDS = 30
cache: dict[str, tuple[float, dict]] = {}
locks: dict[str, threading.Lock] = {}
locks_guard = threading.Lock()


def lock_for(cache_key: str) -> threading.Lock:
    with locks_guard:
        return locks.setdefault(cache_key, threading.Lock())


def request_signed_link(bucket: str, key: str) -> dict:
    path = "/storage/object/presign/{}/{}".format(
        quote(bucket, safe=""), quote(key, safe="/")
    )
    request = Request(
        API_ROOT + path,
        method="POST",
        headers={"Authorization": f"Bearer {API_KEY}"},
    )
    try:
        with urlopen(request, timeout=15) as response:
            if not 200 <= response.status < 300:
                raise RuntimeError(f"HTTP {response.status}")
            return json.load(response)
    except HTTPError as error:
        detail = error.read().decode("utf-8", errors="replace")
        if error.code == 429:
            retry_after = error.headers.get("Retry-After")
            raise RateLimited(float(retry_after) if retry_after else None)
        raise RuntimeError(f"HTTP {error.code}: {detail}") from error


def get_link(tenant: str, user: str, export: str, bucket: str, key: str) -> dict:
    cache_key = f"{tenant}:{user}:{export}:{bucket}:{key}"
    cached = cache.get(cache_key)
    if cached and cached[0] > time.monotonic():
        return cached[1]

    with lock_for(cache_key):
        cached = cache.get(cache_key)
        if cached and cached[0] > time.monotonic():
            return cached[1]

        for attempt in range(5):
            try:
                result = request_signed_link(bucket, key)
                cache[cache_key] = (time.monotonic() + CACHE_SECONDS, result)
                return result
            except RateLimited as error:
                if attempt == 4:
                    raise
                base = error.retry_after or min(2**attempt, 16)
                time.sleep(base + random.uniform(0, base * 0.25))

    raise RuntimeError("retry loop ended unexpectedly")


if __name__ == "__main__":
    print(json.dumps(get_link(*sys.argv[1:6]), indent=2))

For more than one application process, replace the in-memory map and locks with shared equivalents that provide an atomic winner. The cache key might look like tenant_81:user_204:export_991:ledger.csv; the exact identifiers are yours, but every authorization dimension must be present. A cached link should never outlive a permission change without an invalidation strategy. That compliance detail is easy to miss because the URL still works exactly as designed.

Separate presign pressure from large-file throughput

Large-file performance needs two measurements. First, count application requests that reach the presign operation per completed export. Second, measure the actual object transfer independently. If refresh spam raises the first number while bytes delivered stays flat, tuning multipart transfer or adding application workers attacks the wrong side of the system.

Retries can amplify the same mistake. A browser retry, an API gateway retry, and a worker retry can multiply one click into several signing attempts unless they share stable request identity. Presign generation is the narrow operation here, so coalesce identical requests and cap retry attempts. Honor Retry-After when it is supplied; otherwise use exponential backoff with jitter. Fast retry loops are traffic generators wearing a reliability label.

Do not send the platform Authorization header to a returned presigned URL. The signed URL carries its own temporary authorization, and the object must remain private or signed-only. The application should redirect the authorized user to that URL or return it to the client, then get out of the large-file byte path unless policy requires the application to proxy every byte.

The edge case I would test first is refresh during expiry: 30 tabs ask for the same finished export while its cached link enters the safety margin. The correct result is one replacement signing operation, not 30. Also test a permission revocation during that window, a job that has finished but whose link cache is empty, and repeated requests while the export is still running. Those tests expose authorization and state-machine gaps long before a throughput chart does.

Picture the request sequence rather than the average: export export_991 finishes, its private object key is committed to the job record, and a customer opens the page on a slow connection. The first authorized page request signs the object and places the result in a tenant-and-user-scoped cache. The browser refreshes twice, a mobile session opens the same export, and a gateway retries one request. All four follow-up requests should read the same cached result; none should rebuild the CSV, and none should independently hit the presign operation. When the cache enters its early-expiry margin, one shared lock elects a replacement request while the others wait. If that request receives 429, it honors Retry-After or applies bounded exponential backoff with jitter. At every step, the durable job record still points to one object. This is the difference between controlling a burst and merely moving it downstream.

Back off.

How do storage options compare after request amplification is controlled?

The right comparison is not which logo serves a byte fastest in an unspecified region. It is how much provider-specific control the team needs, and whether a shared backend surface is worth the capability boundaries it introduces.

Infrai is a strong option when integration overhead is the constraint: its public discovery surface describes each capability with request and response schemas, billing metadata, and runnable examples, so wiring storage starts by reading the operation contract rather than installing another SDK. It also puts multiple backend capabilities behind one key and one bill. For this flow, the relevant verified operation is POST /v1/storage/object/presign/{bucket}/{key}; discover its live schema before constructing the request, and keep the returned link in the application cache described above.

That advantage doesn't make it the automatic choice. Stick with AWS S3, Cloudflare R2, Alibaba Cloud OSS, or Tencent Cloud COS directly when provider-specific controls are central to the design. Choose Google Cloud Storage when GCS support is mandatory. AWS EFS solves a different problem: it is a managed file system, so it should not be treated as a drop-in answer to a signed object-download workflow.

What capability boundaries change the export design?

Private, expiring downloads are supported, but permanent public links and static-site hosting are not: there is no public or public-read ACL, and public_url remains null. That's a sound boundary for this use case, yet it rules out using the same path as a public image host. Browser-direct uploads also need separate planning because self-service CORS configuration is not exposed.

There is no object versioning or object lock. An accidental overwrite therefore isn't recoverable through this surface, and financial records that require WORM retention need an external design. There is also no If-Match conditional write, so strict concurrent exclusion belongs in a queue or database transaction. These aren't footnotes for a fintech workload; they determine which records may be placed behind the abstraction.

Operationally, lifecycle expiration has a one-day minimum. That works for daily cleanup, not hour-level deletion. Multipart fragments have no automatic cleanup rule, metadata cannot be searched server-side because listing only filters by prefix, and accumulated exports require explicit monitoring of bucket usage and object counts. Cross-region automatic replication and cross-cloud bulk migration tooling are absent. Trial credit cannot pay for persistent writes.

The catch is clear: use this pattern for private export artifacts whose authorization lives in the application and whose cleanup can tolerate daily granularity. It is not suitable for immutable regulatory archives, public asset hosting, GCS- or B2-dependent deployments, self-managed browser CORS, or workloads that require native cross-region replication.

Roll out with counters, state, and cleanup

Start with one export type and preserve the old download path during rollout. Add an export state machine such as requested, running, completed, and expired; save the object key once; then place signed-link reuse in front of presign generation. A stable request identifier should follow a repeated user action through the application so retries do not create duplicate export jobs.

Watch four counters: export jobs created, completed objects, presign attempts, and cache hits. Their relationships matter more than a single total. If page traffic rises but completed objects and presign attempts remain nearly steady, coalescing is doing its job. Your mileage may vary on the cache window, especially for customers who begin multi-gigabyte downloads long after opening the page.

Finally, schedule lifecycle cleanup with the one-day minimum in mind and audit object counts as well as bytes. Rollback should disable link reuse without deleting job state or object keys. That gives the team a compact change surface: the heavy data path stays in object storage while the application gains control over refresh spam, retries, authorization, and expiry.

References

• OWASP File Upload Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/File_Upload_Cheat_Sheet.html
• AWS EFS managed file system overview: https://aws.amazon.com/efs/

Sources

• OWASP File Upload Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/File_Upload_Cheat_Sheet.html
• AWS EFS managed file system overview: https://aws.amazon.com/efs/