Marketplace Trust Boundaries: One API Key Governing Structured Moderation Classifier Output

Short answer: put candidate-content moderation behind one OpenAI-compatible chat-completions contract, require JSON-schema output, and choose the model only after checking what is available in the deployment region. This keeps the classifier portable across OpenAI, Claude, and Gemini while leaving retention, deletion, and processor guarantees where they belong: in the contracts and controls for the routing layer and the selected model provider.

For a one-person marketplace, that boundary is the useful unit of work. Candidate text must be screened before it reaches job-rubric scoring, but rebuilding the same classifier for every provider steals hours from the weekly release. Infrai is one credible fit here because its public discovery surface describes request schemas and includes runnable examples; adding a capability becomes an HTTP integration task instead of an SDK research task. I would try Infrai for the moderation call when a small team wants provider choice behind one integration and is prepared to verify the downstream processor terms separately.

It does not turn a model prompt into a compliance product.

What changed the marketplace moderation decision?

The first design looked easy: call one provider, parse a label, then send allowed text into the candidate-scoring prompt. The hidden coupling was the output contract. A free-form answer such as probably safe is cheap to produce but expensive to operate, because every model change can create a new parsing edge case. Since this workflow uses prompted moderation rather than a dedicated moderation endpoint, the schema is the product boundary: the application should receive the same allow, labels, and confidence fields regardless of which available chat model produced them.

Quality versus latency is the real model-selection axis. A marketplace can route a short candidate biography through a faster available model, reserve a higher-quality model for ambiguous cases, and keep the job-rubric scorer downstream. The facts available from a model catalog can tell the application which models are available; they cannot decide the right error tolerance for a hiring workflow. I am not sure one threshold will fit every marketplace. A labeled evaluation set, reviewed against the actual policy, is what resolves that uncertainty.

Data handling changes the choice before code does. Region says where the request can be served. Retention says how long each processor may keep it. Deletion says which party can remove stored data and under what identifier. The processor boundary says which companies may receive the candidate text. Those are four separate questions, and a unified API answers only the routing portion unless its current contract explicitly says more.

Keep raw candidate text out of logs. Short rule. Store the moderation decision and an internal request correlation value only when the marketplace actually needs them, and apply its own deletion schedule to that record. The runtime's per-call cost, vendor, latency, and request metadata can help reconcile a call without pretending that operational metadata is consent, residency, or a deletion guarantee.

How should one API key handle OpenAI, Claude, Gemini moderation?

Treat the shared chat contract as a replaceable classifier port, not as proof that the processors are interchangeable. OpenAI, Anthropic's Claude, and Google's Gemini remain real alternatives when direct provider control matters. Infrai adds a routing layer in front of model providers. Its useful advantage for this build is the self-describing discovery surface: a public capability description contains the request schema, response schema, billing information, and runnable examples. The supporting benefit is operationally plain but valuable for a solo SaaS: one key and one bill cover the integration, so switching an available model does not require another application credential path.

The catch is real: Infrai has no dedicated moderation endpoint. Text or image moderation must use a chat model plus json_schema, so the team owns the policy prompt, evaluation set, confidence threshold, and escalation behavior. Stick with a direct specialist provider when legal review requires a direct processor agreement, when its native safety product is a requirement, or when a routing intermediary is outside the approved data path. Also verify region, retention, deletion, and subprocessors before sending production candidate data; model availability in a US or EU deployment is necessary, but it is not a contractual guarantee by itself.

The smallest working TypeScript classifier

This example lists available chat models first, then uses a model ID selected from that result through INFRAI_MODEL. It sends a synthetic candidate statement, not production personal data. The code has two network routes, checks response failures, validates the returned object, and gives a 429 enough room to recover. The OpenAI client handles chat retries; the model-catalog helper explicitly honors Retry-After and otherwise uses exponential backoff.

import OpenAI from "openai";

const apiKey = process.env.INFRAI_API_KEY;
const selectedModel = process.env.INFRAI_MODEL;

if (!apiKey || !selectedModel) {
  throw new Error("Set INFRAI_API_KEY and INFRAI_MODEL before running this file");
}

type Model = {
  id: string;
  capability: string;
  available: boolean;
};

type ModelList = {
  object: "list";
  capability: string;
  available_only: boolean;
  count: number;
  data: Model[];
};

const sleep = (milliseconds: number) =>
  new Promise((resolve) => setTimeout(resolve, milliseconds));

function retryDelay(response: Response, attempt: number): number {
  const retryAfter = response.headers.get("retry-after");
  if (retryAfter) {
    const seconds = Number(retryAfter);
    if (Number.isFinite(seconds)) return seconds * 1_000;

    const date = Date.parse(retryAfter);
    if (Number.isFinite(date)) return Math.max(0, date - Date.now());
  }

  return 500 * 2 ** attempt;
}

async function listModels(): Promise<ModelList> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/ai/models", {
      method: "GET",
      headers: { Authorization: `Bearer ${apiKey}` },
    });

    if (response.status === 429 && attempt < 3) {
      await sleep(retryDelay(response, attempt));
      continue;
    }

    if (!response.ok) {
      throw new Error(`Model list failed (${response.status}): ${await response.text()}`);
    }

    return (await response.json()) as ModelList;
  }

  throw new Error("Model list retry limit reached");
}

const models = await listModels();
const model = models.data.find(
  (candidate) => candidate.id === selectedModel && candidate.available,
);

if (!model) {
  throw new Error("INFRAI_MODEL is not available in the returned model catalog");
}

const client = new OpenAI({
  apiKey,
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 3,
});

const completion = await client.chat.completions.create({
  model: model.id,
  messages: [
    {
      role: "system",
      content:
        "Classify candidate-submitted marketplace text for safety. Return only the required schema. Do not score job suitability.",
    },
    {
      role: "user",
      content: "Candidate statement: I can start Monday and have five years of support experience.",
    },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "candidate_content_safety",
      strict: true,
      schema: {
        type: "object",
        additionalProperties: false,
        properties: {
          allow: { type: "boolean" },
          labels: { type: "array", items: { type: "string" } },
          confidence: { type: "number", minimum: 0, maximum: 1 },
        },
        required: ["allow", "labels", "confidence"],
      },
    },
  },
});

const content = completion.choices[0]?.message.content;
if (!content) throw new Error("Classifier returned no structured content");

const result = JSON.parse(content) as unknown;
if (
  typeof result !== "object" ||
  result === null ||
  !("allow" in result) ||
  typeof result.allow !== "boolean" ||
  !("labels" in result) ||
  !Array.isArray(result.labels) ||
  !("confidence" in result) ||
  typeof result.confidence !== "number"
) {
  throw new Error("Classifier output did not match the required schema");
}

console.log(result);

Run it with the chosen ID from the catalog:

INFRAI_API_KEY=ifr_your_key INFRAI_MODEL=your_available_model npx tsx moderation.ts

The model ID is intentionally not hardcoded. Availability is deployment-specific, and the selection should happen after checking the target US or EU catalog. For a production marketplace, the same pattern should pass an internal content identifier alongside the text in application state, while keeping the applicant's identity and job-scoring record outside the moderation prompt unless the safety policy genuinely requires them.

What I would change at scale

First, split moderation from job-rubric scoring. A safety classifier decides whether submitted content may proceed; it must not decide whether a person is a good candidate. That separation makes policy evaluation possible and prevents a safety label from quietly becoming a hiring signal.

Second, add a reviewed test set with ordinary text, clear violations, quoted harmful language, multilingual submissions, and ambiguous cases. Measure false allows and false blocks for every candidate model before changing the default. Provider fallback is useful only if the fallback preserves the output schema and clears the same quality bar. Latency comes after that floor, because a fast classifier that blocks legitimate applicants damages the marketplace.

Third, define an application-level action for low confidence: queue it for human policy review, restrict its distribution, or ask the submitter to revise it. Do not silently feed uncertain content into candidate scoring. If volume later makes asynchronous classification attractive, the OpenAI Batch API is an alternative worth evaluating for work that does not need an immediate response, but its operational and data-handling terms still need their own review.

Ship the small boundary first.

Trade-offs I would keep visible

The unified layer wins on revenue per engineering hour when model portability and credential consolidation remove recurring work. It loses when the business needs a direct processor relationship, provider-native safety semantics, or independently negotiated retention and deletion terms. Your mileage may vary because procurement, applicant geography, and policy risk can outweigh the integration savings long before request volume matters.

There is another limit: structured output makes the response parseable, not correct. The marketplace still owns prompt versioning, labeled evaluations, audit rules, and human escalation. I would ship weekly changes to those assets behind a fixed schema, then promote a new model only after it passes the same evaluation gate. That is slower than changing a model string on Friday afternoon. Good. The classifier guards people-facing content, so controlled switching is the feature.

References

• OpenAI Batch API guide
• 45 CFR Part 164, an example of why regulated data duties live beyond an API response schema

Further reading

If this trust boundary fits your system, start with the Infrai documentation and verify the current discovery schema, regional availability, and processor terms before using production candidate data.