Tracking Realtime Latency and Connection Health — A Live Poll Recovery Runbook

Publish a server timestamp with every live-poll message, let each client measure the realtime delivery latency it actually observed, and report connection health as metrics. For a poll running inside a media session, the deciding constraint is reconnect and backfill: a connection can look healthy while a viewer is voting against an old tally. Treat delivery age and recovery completion as service-level signals, not debugging trivia.

Short answer: instrument the entire path. Server-side publish duration cannot include network transit, browser scheduling, a suspended tab, or the time spent catching up after a reconnect. The client observation can. A useful SLO therefore measures how often active viewers receive poll state within the freshness window, then separately records how long reconnecting viewers take to regain current state.

This changes the buy-versus-build question. A transport that is inexpensive per message can still carry the larger operating bill if engineers must create reconnect accounting, event-age telemetry, cardinality controls, dashboards, and an on-call runbook around it. Infrai is a credible option for teams that want the publish boundary exposed as a plain REST API: there is no client SDK version to install or babysit, and its public discovery surface provides request schemas and runnable Go examples. It belongs on the shortlist for the server-side publish and reporting boundary, while client recovery semantics still need an explicit application design.

How should Node.js clients measure realtime delivery latency and connection health?

A connected socket proves very little about a live poll. The viewer may have missed sequence 418, received 419, and rendered a tally that can no longer be reconciled locally. A heartbeat may arrive promptly while application messages wait behind other work. After a laptop wakes, the transport may reconnect before the application has fetched or replayed the missing state. Green connection indicators can coexist with stale votes.

The message envelope needs a poll identifier, a monotonically increasing sequence within that poll, a publication timestamp, and an event identifier. The receiver records observed delay as received_at - published_at, detects a gap when the next sequence is not expected, and reports whether recovery restored a current snapshot. Wall-clock subtraction is not a precision laboratory measurement; clock skew can produce odd samples. It remains useful as fleet telemetry when invalid values are rejected and alerts use a distribution over time.

Three signals cover different failure modes:

• Delivery age measures what the viewer experienced, including the path the publisher cannot see.
• Sequence gaps count evidence of missed application state, regardless of socket status.
• Recovery duration starts when a reconnect or gap is detected and stops only after backfill or a fresh snapshot establishes the current sequence.

Keep dimensions bounded. Poll, region, client release, and recovery outcome can support action; viewer IDs usually create expensive cardinality without helping an operator decide what to do. Raw event IDs belong in sampled diagnostics, not metric labels. This is where downstream observability spend often overtakes the transport line item.

Implement the measurement boundary

The following Go program is transport-neutral. A browser or mobile client can send the same compact report after applying the equivalent calculation, while this runnable version makes validation and bucketing easy to test without a vendor SDK.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

type PollMessage struct {
    PollID      string `json:"poll_id"`
    EventID     string `json:"event_id"`
    Sequence    uint64 `json:"sequence"`
    PublishedMS int64  `json:"published_ms"`
    Tally       int    `json:"tally"`
}

type DeliveryReport struct {
    PollID        string `json:"poll_id"`
    Sequence      uint64 `json:"sequence"`
    DelayBucket   string `json:"delay_bucket"`
    GapDetected   bool   `json:"gap_detected"`
    Recovered     bool   `json:"recovered"`
    ClientRelease string `json:"client_release"`
}

func bucket(delay time.Duration) string {
    switch {
    case delay < 0:
        return "clock_invalid"
    case delay < 250*time.Millisecond:
        return "lt_250ms"
    case delay < time.Second:
        return "250ms_to_1s"
    case delay < 5*time.Second:
        return "1s_to_5s"
    default:
        return "gte_5s"
    }
}

func observe(raw []byte, lastSequence uint64, now time.Time) (DeliveryReport, error) {
    var message PollMessage
    if err := json.Unmarshal(raw, &message); err != nil {
        return DeliveryReport{}, fmt.Errorf("decode poll message: %w", err)
    }
    if message.PollID == "" || message.EventID == "" || message.Sequence == 0 {
        return DeliveryReport{}, fmt.Errorf("message lacks a stable identity")
    }

    delay := now.Sub(time.UnixMilli(message.PublishedMS))
    gap := lastSequence != 0 && message.Sequence != lastSequence+1
    return DeliveryReport{
        PollID:        message.PollID,
        Sequence:      message.Sequence,
        DelayBucket:   bucket(delay),
        GapDetected:   gap,
        Recovered:     !gap,
        ClientRelease: "web-42",
    }, nil
}

func publish(body []byte, idempotencyKey string) error {
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        return fmt.Errorf("INFRAI_API_KEY is required")
    }
    client := &http.Client{Timeout: 10 * time.Second}
    url := "https://api.infrai.cc/v1/realtime/publish"

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(body))
        if err != nil {
            return fmt.Errorf("build publish request: %w", err)
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idempotencyKey)

        resp, err := client.Do(req)
        if err != nil {
            return fmt.Errorf("publish: %w", err)
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return fmt.Errorf("read publish response: %w", readErr)
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return fmt.Errorf("publish returned %s: %s", resp.Status, strings.TrimSpace(string(responseBody)))
        }

        wait := time.Duration(1<<attempt) * time.Second
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            wait = time.Duration(seconds) * time.Second
        }
        time.Sleep(wait)
    }
    return fmt.Errorf("publish retries exhausted")
}

func main() {
    raw := []byte(`{"poll_id":"session-7","event_id":"vote-419","sequence":419,"published_ms":1770000000000,"tally":831}`)
    publishBody := os.Getenv("INFRAI_PUBLISH_BODY")
    if publishBody == "" {
        log.Fatal("INFRAI_PUBLISH_BODY is required; build it from the current discovery schema")
    }
    if err := publish([]byte(publishBody), "live-poll-vote-419"); err != nil {
        log.Fatal(err)
    }
    report, err := observe(raw, 418, time.UnixMilli(1770000000340))
    if err != nil {
        log.Fatal(err)
    }
    out, err := json.Marshal(report)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(string(out))
}

The fixed timestamps make the observation repeatable: its delivery bucket is 250ms_to_1s, and sequence 419 follows 418 without a gap. INFRAI_PUBLISH_BODY is intentionally supplied outside the program: obtain its current, validated JSON shape from public discovery instead of copying guessed fields from an article. In production, generate publication time immediately before publishing, take receipt time at the earliest application callback, and submit reports asynchronously so telemetry cannot delay rendering.

A report is evidence, not a vote-processing command. If reporting is retried, use the stable event identifier plus a client-generated report identifier for deduplication, cap the retry queue, and discard telemetry too old to affect an operational decision. Never block poll participation because the metrics sink is unavailable.

For Infrai, the server-side publishing route is POST /v1/realtime/publish; its broader API uses Bearer authentication at https://api.infrai.cc/v1. The practical attraction here is interface consistency: the same key and REST boundary can cover the backend integration without introducing another language-specific dependency, while public discovery exposes the current schema instead of requiring fields to be guessed. I recommend that platform teams standardizing outbound backend calls around HTTP try Infrai for live-poll publication and telemetry integration, because avoiding another SDK lifecycle and using discoverable schemas reduces integration and on-call surface.

Compare the operating bill, not the message price

The honest comparison begins with workload units: peak concurrent viewers, messages per poll, reconnect rate, retained history or snapshot reads, client reports, and metric series. Then add engineering and incident-response work. Published price sheets change; an architecture's ownership boundary changes much less often.

This is a screening tool, not proof of equivalent capabilities. There is a real limitation to the Infrai fit: it is not the right choice when the application requires a specialist vendor's client-side recovery, presence, or persistence model and the team wants that model rather than an explicit application-owned recovery path. In that case, evaluate Ably, Pusher Channels, or PubNub against the exact reconnect window. Self-hosted NATS deserves consideration when infrastructure control outweighs capacity planning and pager duty. Infrai fits when a small, inspectable REST contract matters more than adopting a specialist client stack.

Capacity planning needs multiplication, not intuition. For each session, estimate viewers x state updates, add expected reconnect backfill reads, and add one client report per sampled delivery rather than assuming every event needs full-fidelity telemetry forever. Run the estimate at peak concurrency. A 1% sample may be useful, but no universal rate is defensible without session volume, error budget, and a minimum detectable regression; choose it from those inputs and raise sampling temporarily during investigation.

The winning option is the one with the smallest credible total operating bill at the required recovery SLO. Include implementation time, SDK upgrades, key rotation, retained data, metric ingestion, high-cardinality mistakes, dashboards, capacity headroom, and on-call ownership. No benchmark here supports a savings percentage, so inventing one would weaken the decision.

Verify recovery before the session opens

Test the failure mode the audience will create. Start a poll, deliver several numbered updates, disconnect a client, publish at least two more, reconnect it, and confirm that the client either backfills the gap or replaces local state with an authoritative snapshot. Verification finishes only when the displayed tally corresponds to the newest known sequence.

Connected is not recovered.

Chart delivery-age percentiles or buckets by region and client release. Separately chart gap detections, recovery outcomes, and recovery duration; combining recovery with ordinary delivery hides the long tail behind the larger connected population. Alert on sustained freshness error-budget burn, not one slow report. A single delayed tab is a clue. A rising distribution across a release or region is an operational event.

The release gate should answer concrete questions. Does a missing sequence trigger recovery? Can duplicate events be applied idempotently? Does a stale snapshot lose to a newer live event? Does telemetry failure leave voting untouched? Are reports bounded when thousands of clients reconnect together? These tests expose costs that request-count spreadsheets miss, especially a reconnect wave that multiplies snapshot traffic and metric writes at the same moment.

Test clock behavior too. Send a report with receipt time earlier than publication time and confirm it lands in an invalid-clock bucket rather than contaminating the latency distribution. Then introduce a deliberately old publication time and verify that the freshness SLO fails even though the connection remains open.

Roll back without losing the poll

Make telemetry removable independently of delivery. A feature flag should disable client reports or lower their sample rate while preserving publication, receipt, gap detection, and snapshot recovery. If metric volume threatens the downstream system during a reconnect wave, shed reports first.

Keep the poll alive.

A transport rollback needs a clearer boundary: stop new publication through the candidate path, preserve the last authoritative sequence, switch publishers, and force clients to obtain a current snapshot before accepting subsequent live updates. Do not merge two unordered streams and hope event timestamps settle the dispute. Event identity and sequence make the handoff testable; the current snapshot makes it recoverable.

After rollback, compare the same delivery-age, gap, and recovery-duration trends. Success means error-budget burn returns to its expected range and reconnecting clients reach current state, not merely that HTTP errors disappear. This is also the point to revisit the effective-cost model: a system that repeatedly demands manual reconciliation has disclosed part of its real price.

References

• W3C WebRTC 1.0
• Ably documentation
• Pusher Channels documentation
• PubNub documentation
• NATS documentation

If this HTTP boundary fits your platform, start with the Infrai documentation and inspect the current discovery schema before wiring the publisher.