How to Deploy a Whisper Transcription API

# How to Deploy a Whisper Transcription API

OpenAI's Whisper is an excellent open-weights speech-to-text model, and you can self-host it instead of paying per-minute API fees. The catch: transcription is compute-heavy and slow for long files, so a naive synchronous endpoint will time out. This guide builds a transcription API that handles real audio.

Use faster-whisper, not the reference implementation

The original whisper package works but is slow and memory-hungry. faster-whisper (built on CTranslate2) is several times faster on the same hardware and uses less memory — it's the practical choice for a service:

# transcribe.py
from faster_whisper import WhisperModel

# "base"/"small" run fine on CPU; "medium"/"large-v3" want a GPU
model = WhisperModel("small", device="cpu", compute_type="int8")

def transcribe(path: str):
    segments, info = model.transcribe(path, beam_size=5)
    text = " ".join(s.text for s in segments)
    return {"language": info.language, "text": text.strip()}

compute_type="int8" quantizes the model for CPU inference, trading a little accuracy for a big speed and memory win — often the right call for non-GPU deployments.

Synchronous for short clips, async for long ones

A voice note of a few seconds can be transcribed inline. An hour-long podcast cannot — it will blow past any HTTP timeout. Offer both paths:

from fastapi import FastAPI, UploadFile, BackgroundTasks
import uuid, tempfile, os

app = FastAPI()
JOBS = {}  # use Redis/Postgres in production

@app.post("/transcribe")
async def sync_transcribe(file: UploadFile):
    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
        tmp.write(await file.read())
        path = tmp.name
    try:
        return transcribe(path)
    finally:
        os.unlink(path)

@app.post("/jobs")
async def create_job(file: UploadFile, bg: BackgroundTasks):
    job_id = str(uuid.uuid4())
    data = await file.read()
    JOBS[job_id] = {"status": "processing"}
    bg.add_task(run_job, job_id, data)
    return {"job_id": job_id}

@app.get("/jobs/{job_id}")
def get_job(job_id: str):
    return JOBS.get(job_id, {"status": "not_found"})

For anything serious, replace the in-memory JOBS dict with a managed datastore and move work to a real worker/queue — the dict doesn't survive restarts or span replicas.

Don't forget ffmpeg

Whisper needs ffmpeg to decode most audio formats. It's the most common reason a containerized Whisper service fails at runtime with a cryptic error:

FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Choose the right model and compute tier

This is where transcription deployments succeed or fail on cost:

On PandaStack, match the model to a compute tier. CPU inference with small + int8 is comfortable on a c1/c2 compute-optimized tier (up to 8 CPU / 16GB on C2-2XCompute). Larger models that load a lot of weights benefit from m1/m2 memory-optimized tiers. Don't overprovision: a base model on a small tier is plenty for short voice notes.

Deploy on PandaStack

1. 1Push the repo to GitHub.
2. 2Create a container app in the [dashboard](https://dashboard.pandastack.io) connected to the repo. The Dockerfile (with ffmpeg) builds via rootless BuildKit and deploys with an HTTPS URL.
3. 3Pick a compute tier that matches your model — start with compute-optimized for CPU inference.
4. 4For async jobs, provision a managed PostgreSQL or Redis to track job state; DATABASE_URL is auto-wired.

A note on cold starts and big models

Loading a Whisper model takes time and memory. On the free tier's scale-to-zero, the first request after idle pays both the cold start *and* the model load — potentially many seconds. For a responsive API, run on a paid tier so the model stays resident in a warm instance. The free tier is great for testing the pipeline first.

Verify

curl -X POST https://<app>/transcribe -F 'file=@sample.wav'

Tail live logs to confirm ffmpeg decoded the file and the model loaded without an out-of-memory kill. OOM kills are the signal you need a larger memory tier or a smaller model.

Cost: self-host vs. API

Self-hosting wins when you transcribe steady, high volume — you pay for compute time, not per minute of audio. The hosted API wins for spiky, low volume where idle compute would be wasted. With scale-to-zero on the free tier you can even self-host bursty workloads cheaply, accepting cold starts.

References

• [OpenAI Whisper](https://github.com/openai/whisper)
• [faster-whisper](https://github.com/SYSTRAN/faster-whisper)
• [CTranslate2 quantization](https://opennmt.net/CTranslate2/quantization.html)
• [ffmpeg documentation](https://ffmpeg.org/documentation.html)

Self-hosted Whisper is one of the best cost wins in the AI stack if you have steady volume. PandaStack lets you pick a compute-optimized tier, bundle ffmpeg in your image, and track jobs in an auto-wired database. Start at [dashboard.pandastack.io](https://dashboard.pandastack.io).