A GPU has to earn its place in an audio transcription queue

Product spotlight | Peterson Compute Trading | November 8, 2026
AI-assisted public-source analysis. No transcription benchmark, cloud deployment or purchase was performed. All prices and calculations are in US dollars.

Consider a hypothetical archive of a thousand hours of recorded lectures that needs searchable transcripts by next week. The recordings are owned or appropriately licensed, the speakers have consented, and the collection contains no sensitive material. This is a useful inference workload: it can wait, it divides into independent jobs, and a failed attempt should not require starting the whole collection again.

The first purchasing question is whether a GPU saves money after its supporting CPU and memory are counted. Using Modal's public rates, the two configurations below put that hurdle at about 3.60 times the CPU worker's throughput. A hypothetical fourfold speedup clears it narrowly. That small margin is the interesting part.

Start with a transcript worth keeping

The proposed engine is faster-whisper, a CTranslate2 implementation of Whisper. Begin with the same small English checkpoint on both machines, the same beam size, language setting, timestamp requirements and silence filtering. CPU INT8 and GPU INT8 with FP16 arithmetic are candidate execution modes, not a guarantee of identical output. Choose acceptable transcription quality before selecting the cheapest configuration. Project documentation

Create a representative evaluation set before processing the archive: quiet narration, unfamiliar technical names, poor microphones, accents, music and long pauses. Compare against human-checked reference transcripts. Record word error rate, missing passages, invented passages and timestamp failures. A cheap transcript that requires extensive correction can be more expensive than the compute it saved. Whisper's model card explicitly warns about hallucinated text and uneven performance across languages and speakers. Whisper model card

The project's published performance table is useful orientation, but it compares a small model on an Intel i7-12700K with large-v2 on an RTX 3070 Ti in separate sections. Those rows do not establish a speedup for this CPU-versus-L4 experiment. Timing must also consume faster-whisper's output iterator: transcription work happens during iteration, so timing only the initial call can produce a misleadingly tiny number. Benchmark and usage notes

Price the whole worker

Modal's November 8 rate card lists CPU at $0.0000131 per physical-core-second, memory at $0.00000222 per GiB-second, and an L4 at $0.000222 per second. The planning configurations are four physical CPU cores plus 8 GiB, or one L4 plus one physical core and 8 GiB. Multiplying by 3,600 gives $0.252576 and $0.910296 per worker-hour respectively. These use default Function rates with automatic placement, not a regional premium or a Sandbox. Published prices

The resulting GPU-to-CPU price ratio is 3.604. For equal accepted output and no extra overhead, GPU throughput must exceed that multiple to reduce compute spending. Actual CPU and memory usage can change the arithmetic: Modal bills the higher of requested resources and actual usage, and its CPU setting denotes physical cores rather than vCPUs. The proposed allocations are sizing assumptions to test, not measured requirements. Resource accounting

A fourfold speedup can save only five dollars

For illustration only, assume the CPU consumes 0.20 worker-hours per audio hour and the GPU consumes 0.05. These are invented planning inputs, not measured or vendor-promised speeds. Processing 1,000 audio hours would then consume 200 CPU worker-hours or 50 GPU worker-hours, costing $50.52 or $45.51 before other charges. The GPU saves about $5.00, or 9.9%, despite finishing the compute four times faster at equal worker counts.

Now add ten billed worker-hours of overhead to each option across the collection. CPU compute becomes $53.04; GPU compute becomes $54.62. The ranking reverses. Across 6,000 ten-minute shards, ten hours is an average of six additional billed seconds per shard. This sensitivity test does not predict a cold start on every shard. Containers can be reused, and the mix of initialization, downloading, retries and processing will determine actual overhead. Cold-start behavior

These are gross compute scenarios. They exclude credits, taxes, storage, network transfer, transcription review, orchestration and engineering time. Queueing can affect the deadline without being equivalent to billed worker time. Modal documents no minimum usage-time increment; an hourly equivalent is a comparison unit, not a one-hour reservation. Billing documentation

Design for interruption

Modal Functions are preemptible by default and restart with the same input after an interruption. Its documentation recommends small tasks or checkpoints and retry-safe work. The nonpreemptible option adds a CPU-and-memory premium but is not supported for GPU Functions. An L4 worker therefore needs an interruption plan rather than a checkbox promising uninterrupted execution. Preemption documentation

A practical design begins with roughly ten-minute audio shards, adjusted around utterance boundaries. Any overlap must be reconciled when timestamps and text are joined, and the extra decoding belongs in the bill. Give each shard a stable identity derived from the source content hash, time range, model revision and decoding options. Store complete outputs durably under that identity and use conditional creation or an equivalent atomic commit. A retry checks for an accepted result before doing expensive work again.

Keep a separate manifest for accepted, failed and pending shards. Do not mark a shard complete until its output is durable and validated. Quarantine corrupt inputs, cap retries and reconcile the manifest against outputs at the end. This proposed design limits repeated work to unfinished shards, subject to correct persistence and concurrency handling. It has not been deployed or tested here.

A small trial before the archive

Measure both configurations on the same sample, separating cold and warm runs. Record completed, quality-accepted audio hours per billed worker-hour, peak memory, retries and elapsed completion time. Cache the selected model revision, reuse initialized workers where appropriate, and repeat runs enough to expose variation. If batching improves throughput, rerun quality checks and retain its memory and startup costs.

The faster-whisper implementation and upstream Whisper code and weights carry MIT licenses. Preserve required notices when distributing software, and separately verify rights to the audio and any additional components. A software license does not grant permission to process someone else's recordings. faster-whisper license, Whisper licensing

The decision rule is simple: buy the configuration with the lowest total cost per accepted audio hour that can meet the deadline. This case gives the GPU a measurable hurdle, not an endorsement. If a representative trial barely clears 3.60 times the throughput, small overhead or quality differences deserve more attention than the GPU's headline speed.