{
  "id": "pct-2026-11-06-spotlight-c405e6f869",
  "content_type": "article",
  "desk": "spotlight",
  "title": "OpenAI Batch API: buying the catalogue update instead of the GPU hour",
  "publication_date": "2026-11-06",
  "observation_date": "2026-11-06",
  "editorial_voice": "AI-assisted Spotlight Desk",
  "body": "A catalogue has 100,000 descriptions waiting for cleaner categories and short summaries. Nobody needs the answer to the first record in three seconds. They need a complete, usable dataset before the next publishing cycle. That distinction changes the compute purchase.\n\nOpenAI’s Batch API packages inference around deferred delivery. The supplier’s April 23, 2024 enterprise announcement described a 50% discount for asynchronous workloads, including classification and evaluation. This is an established service, not a fresh November launch. The useful question is what a builder must do to turn that discount into a dependable production result. [S1]\n\n## A workload you can picture\n\nOpenAI’s cookbook, dated April 24, 2024, demonstrates extracting movie categories and one-sentence summaries from descriptions. It also includes image captioning. The page is now explicitly marked archived, so treat it as a concrete workflow illustration rather than a current implementation recipe or performance benchmark. [S2]\n\nThe catalogue example here is PCT’s own planning case. Assume the operator has rights to process the text and a fixed category vocabulary. Each record can be processed independently. The output should contain the record identifier, permitted categories and a short description supported by the input. No production catalogue or customer order was inspected for this article.\n\nThis is attractive builder territory because the job is easy to make visible. You can count the records, inspect samples, test the category rules and decide whether the result is good enough before committing the whole collection. A cheaper run that produces inconsistent labels still leaves an expensive cleanup job.\n\n## A batch needs a receipt\n\nThe current guide permits up to 50,000 requests per batch and a maximum 200 MB input file. It also imposes model-specific queued-input-token limits. Our 100,000-record case therefore needs at least two batches by request count alone; file size and account limits may require more. Splitting the work does not create extra quota.\n\nOutputs may arrive in a different order from inputs. The guide tells developers to join them using custom identifiers, and failed requests have a separate error file. A practical receipt should reconcile every submitted identifier with a successful response, an error or unresolved work. [S3]\n\nThat receipt is valuable beyond avoiding a spreadsheet accident. It lets the operator rerun only the records needing attention and preserve accepted results. Replaying an entire collection because one slice failed is an avoidable cost, and can produce inconsistent versions of records that were already usable.\n\nPCT would also keep the prompt version, category vocabulary, chosen model version and acceptance checks beside the output. A clean audit trail makes it possible to explain why a record changed when the next catalogue refresh arrives.\n\n## Put numbers on the choice\n\nHere is illustrative arithmetic, not an OpenAI quote. Assume 100,000 requests, each with 2,000 input tokens and 200 output tokens. That is 200 million input tokens and 20 million output tokens. Suppose the eligible synchronous route costs $1 per million input tokens and $4 per million output tokens. Its token bill would be $280: $200 for input and $80 for output.\n\nApply the documented 50% Batch discount to those assumed rates and the token estimate becomes $140. The gross saving is $140, before taxes, preparation, evaluation, storage, retries or engineering work. The token counts include the prompt, not just the catalogue description, and no cache discount or tool use is assumed. Actual model selection still requires quality and eligibility checks. [S4]\n\nNow suppose this particular implementation requires three additional staff hours at an assumed internal cost of $75 an hour. That adds $225. For one run, the batch workflow costs $365 against the $280 baseline under these assumptions. It has not paid back its incremental setup effort. After two equivalent runs, token savings of $280 exceed that one-time cost by $55, assuming no extra ongoing overhead.\n\nThere is a good commercial lesson here. A 50% token discount does not mean a 50% reduction in project cost. Repeat frequency, existing tooling and the amount of review required can matter more than the posted rate. An established batch pipeline may need little additional setup; a one-off migration may need substantially more than this example.\n\n## The deadline and retention tests\n\nThe current FAQ specifies a 24-hour processing window, but also describes expiration: unfinished work is canceled, completed results are returned and completed work is charged. Treat partial completion as an outcome the application must handle. Submitting at the last possible moment leaves no room for correction or another attempt. [S4]\n\nFor a publishing pipeline, PCT would choose an internal submission cutoff earlier than the actual deadline and define what happens if some records remain missing. The operator might retain the previous accepted description or hold those records out of the release. That is a product decision to make before the run, not an emergency policy to improvise afterward.\n\nData policy can disqualify the route entirely. OpenAI’s FAQ states that zero data retention does not apply to Batch, even for organizations with that setting enabled. Inputs and outputs follow the applicable retention policies. Buyers with strict requirements must verify the actual service terms and configuration before uploading anything. [S4]\n\n## What the market can learn\n\nThis service sells a different delivery condition for model inference. It does not give the buyer a transferable reservation on named GPUs, and the published discount does not reveal a market-clearing price for physical capacity. PCT’s interpretation is narrower: some workloads can trade immediate response for lower token costs without managing their own training checkpoints or GPU fleet.\n\nThe worthwhile experiment is a small, rights-cleared sample with explicit acceptance criteria and a complete result ledger. Measure accepted records, elapsed time, review effort and actual charges. The opportunity is real enough to test. Whether it is economical for a specific catalogue remains unproven until those measurements replace the assumptions.",
  "sources": [
    {
      "id": "S1",
      "title": "Introducing more enterprise-grade features for API customers",
      "publisher": "OpenAI",
      "published": "2024-04-23",
      "url": "https://openai.com/index/more-enterprise-grade-features-for-api-customers/",
      "accessed": "2026-11-06",
      "access_note": "Opened primary source and checked relevant passages."
    },
    {
      "id": "S2",
      "title": "Batch processing with the Batch API",
      "publisher": "OpenAI Cookbook",
      "published": "2024-04-24",
      "url": "https://developers.openai.com/cookbook/examples/batch_processing",
      "caveat": "Archived recipe; used for workflow illustration only, not current code or performance.",
      "accessed": "2026-11-06",
      "access_note": "Opened primary source and checked relevant passages."
    },
    {
      "id": "S3",
      "title": "Batch API guide",
      "publisher": "OpenAI",
      "published": null,
      "url": "https://developers.openai.com/api/docs/guides/batch",
      "accessed": "2026-11-06",
      "access_note": "Opened primary source and checked relevant passages."
    },
    {
      "id": "S4",
      "title": "Batch API FAQ",
      "publisher": "OpenAI Help Center",
      "published": null,
      "url": "https://help.openai.com/en/articles/9197833-batch-api-faq",
      "accessed": "2026-11-06",
      "access_note": "Opened primary source and checked relevant passages."
    }
  ],
  "facts": [
    {
      "id": "F1",
      "claim": "April 23, 2024 announcement describes discounted asynchronous inference.",
      "source_ids": [
        "S1"
      ],
      "evidence_type": "dated_primary_publication"
    },
    {
      "id": "F2",
      "claim": "The April 24, 2024 cookbook demonstrates category extraction, summaries and image captioning; currently marked archived.",
      "source_ids": [
        "S2"
      ],
      "evidence_type": "primary_example_not_independently_reproduced"
    },
    {
      "id": "F3",
      "claim": "Current batch limits include 50,000 requests and 200 MB per input file, plus queued token limits per model.",
      "source_ids": [
        "S3"
      ],
      "evidence_type": "current_documentation"
    },
    {
      "id": "F4",
      "claim": "Output ordering can differ from input; custom identifiers and error files support reconciliation.",
      "source_ids": [
        "S3"
      ],
      "evidence_type": "current_documentation"
    },
    {
      "id": "F5",
      "claim": "FAQ states 50% discount against synchronous API pricing and a 24-hour processing window with partial expiry behavior.",
      "source_ids": [
        "S4"
      ],
      "evidence_type": "current_documentation"
    },
    {
      "id": "F6",
      "claim": "Zero data retention does not apply to Batch.",
      "source_ids": [
        "S4"
      ],
      "evidence_type": "current_documentation"
    }
  ],
  "inferences": [
    {
      "id": "I1",
      "claim": "Count accepted records and review effort when evaluating savings.",
      "label": "PCT analysis"
    },
    {
      "id": "I2",
      "claim": "100,000 requests require at least two batches by request count; size and quota can require more.",
      "basis_fact_ids": [
        "F3"
      ],
      "label": "derived constraint"
    },
    {
      "id": "I3",
      "claim": "Illustrative token costs are $280 synchronous versus $140 Batch; $225 incremental setup gives $365 first-run Batch cost and $55 net savings after two equivalent runs.",
      "label": "illustrative arithmetic, not provider quote or measured result",
      "assumptions": [
        "100,000 requests",
        "2,000 input and 200 output tokens per request",
        "Synchronous rates assumed $1/M input and $4/M output",
        "50% Batch discount",
        "Three incremental setup hours at $75/hour",
        "No cache discount, tools, taxes, retries or other ongoing differences"
      ]
    },
    {
      "id": "I4",
      "claim": "Batch is a model-service delivery condition, not a tradable GPU reservation or physical capacity clearing-price observation.",
      "label": "PCT market interpretation"
    }
  ],
  "unknowns": [
    "Live account quotas and eligible model versions",
    "Current exact model quote",
    "Measured input/output token counts",
    "Accuracy and accepted-record yield",
    "Actual completion distribution",
    "Retry rate and review effort",
    "Applicable retention configuration",
    "Rights to any proposed dataset"
  ],
  "correction_history": [],
  "license_status": "Original PCT editorial text; public license not selected. Source rights remain separate.",
  "public_url": null,
  "desk_extensions": {
    "id": "pct-2026-11-06-spotlight-batch-api",
    "title": "OpenAI Batch API: buying the catalogue update instead of the GPU hour",
    "section": "Product spotlight",
    "publisher": "Peterson Compute Trading",
    "editorial_voice": "Spotlight Desk | AI-assisted, builder-focused",
    "as_of_date": "2026-11-06",
    "status": "desk_research",
    "subject": "OpenAI Batch API for catalogue enrichment",
    "editorial_word_count": 965,
    "limitations": [
      "No paid execution, upload or account-specific quote",
      "No interviews or independently reproduced benchmark",
      "Worked economics use explicit assumed rates, not live OpenAI prices",
      "Historical cookbook is marked archived; current guide controls mechanics"
    ],
    "distinct_from_previous_issue": "New tangible inference-enrichment workflow. Previous spotlights covered AWS Capacity Blocks, Mooncake Store and SageMaker Managed Spot Training.",
    "copyright_licensing_status": "Original PCT draft with factual paraphrase and source links; no copied datasets or third-party media. Public license not selected.",
    "public_url": null
  },
  "public_projection": {
    "version": "1.0.0",
    "date": "2026-11-10",
    "changes": [
      "Internal workspace artifact paths omitted where present. Historical license/status fields retain original provenance; current public reuse policy is stated separately."
    ],
    "rights": "PCT reserves rights in original editorial content; third-party rights retained. No general source-data reuse license."
  }
}