How to Deploy a LangChain API to Production

A LangChain "API" is just a Python web service that happens to call an LLM. That sounds obvious, but it's the thing most deployment guides get wrong — they focus on chains and prompts and hand-wave the parts that actually break in production: dependency pinning, streaming over HTTP, persisting conversation state, and not baking your API keys into a Docker image. Here's the full path, end to end, using FastAPI as the web layer.

Project structure

Keep it boring:

langchain-api/
├── app/
│   ├── __init__.py
│   ├── main.py        # FastAPI app + routes
│   ├── chain.py       # LangChain chain definition
│   └── history.py     # Postgres chat history
├── requirements.txt
├── Dockerfile
└── .env               # local only — never committed

The chain

A minimal but real chain in app/chain.py. LCEL composition (prompt | llm | parser) is stable across recent LangChain versions:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a concise support assistant for our docs."),
    ("human", "{question}"),
])

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

chain = prompt | llm | StrOutputParser()

ChatOpenAI reads OPENAI_API_KEY from the environment. Do not pass keys in code, and do not put them in the Dockerfile — set them as environment variables on the platform.

The API, with streaming done right

Two endpoints: a plain JSON one, and a streaming one using server-sent events. Users perceive an LLM app as fast or slow almost entirely based on time-to-first-token, so streaming is not optional for anything interactive.

# app/main.py
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel

from app.chain import chain

app = FastAPI()

class Ask(BaseModel):
    question: str

@app.get("/healthz")
async def healthz():
    return {"ok": True}

@app.post("/ask")
async def ask(body: Ask):
    answer = await chain.ainvoke({"question": body.question})
    return {"answer": answer}

@app.post("/ask/stream")
async def ask_stream(body: Ask):
    async def gen():
        async for chunk in chain.astream({"question": body.question}):
            yield f"data: {chunk}\n\n"
        yield "data: [DONE]\n\n"
    return StreamingResponse(gen(), media_type="text/event-stream")

Two details that matter:

• The health check must not call the LLM. Your platform will hit it constantly; you don't want to pay per-token for liveness probes, and you don't want a provider outage to make your pods look unhealthy.
• Use the async methods (ainvoke, astream). LLM calls are I/O-bound — one async uvicorn worker will happily hold many concurrent requests open while waiting on the model provider. A pile of sync workers just burns memory.

Run it locally:

uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

Pin your dependencies — LangChain moves fast

This is the single most common LangChain production failure: an unpinned pip install langchain in CI pulls a newer minor version than you developed against, an import path or behavior changed, and your build breaks on a Friday deploy. The ecosystem is split across langchain, langchain-core, and per-provider packages (langchain-openai, langchain-postgres, …) that version independently.

List what you need in requirements.txt:

fastapi
uvicorn[standard]
langchain
langchain-openai
langchain-postgres
psycopg[binary]

Then, once it works, freeze the exact resolved versions and commit that:

pip freeze > requirements.txt

Yes, it's less elegant than ranges. It's also the difference between reproducible builds and archaeology.

Persisting chat history in Postgres

In-memory conversation history dies on every restart and doesn't survive scaling past one replica. Postgres is the pragmatic answer, and langchain-postgres ships a message history class that uses plain tables — no extensions required, so it works on any managed Postgres.

# app/history.py
import os
import psycopg
from langchain_postgres import PostgresChatMessageHistory

TABLE = "chat_history"

conn = psycopg.connect(os.environ["DATABASE_URL"])

# One-time setup — run at startup or as a release step
PostgresChatMessageHistory.create_tables(conn, TABLE)

def get_history(session_id: str) -> PostgresChatMessageHistory:
    return PostgresChatMessageHistory(TABLE, session_id, sync_connection=conn)

Notes from doing this in practice:

• psycopg (v3) accepts both postgres:// and postgresql:// URLs directly, so a platform-injected DATABASE_URL works as-is. If you route through SQLAlchemy instead, you'll need the postgresql+psycopg:// scheme prefix.
• create_tables is idempotent — safe to call at boot — but if you're running multiple replicas, prefer running it once as a release step so two pods don't race on DDL.
• Session IDs are your responsibility. Generate a UUID per conversation on the client and pass it with each request.

The Dockerfile

FROM python:3.12-slim

WORKDIR /app
ENV PYTHONUNBUFFERED=1

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000
CMD ["sh", "-c", "uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-8000}"]

The ${PORT:-8000} dance matters: most platforms tell your container which port to bind via a PORT environment variable. Bind 0.0.0.0, not 127.0.0.1, or the platform's router will never reach you. PYTHONUNBUFFERED=1 makes your logs show up in real time instead of arriving in confusing bursts.

Deploying on PandaStack

This is where the pieces come together with the least ceremony:

1. 1Provision the database. Create a managed PostgreSQL instance (14.x or 16.x) from the dashboard and attach it to your app. DATABASE_URL is injected into the container automatically — the history code above needs zero configuration.
2. 2Connect the repo. Add your GitHub repo as a container app. With the Dockerfile present, PandaStack builds exactly that; without one, the Python buildpack is auto-detected. Builds run in rootless BuildKit inside ephemeral Kubernetes Job pods — no host Docker socket involved.
3. 3Set your secrets. Add OPENAI_API_KEY (and any other provider keys) as environment variables in the app settings. They reach the container at runtime and never touch the image.
4. 4Push. git push triggers the build; the build logs stream live so you can watch pip resolve your pinned dependencies rather than staring at a spinner.

One caveat worth knowing: on the free tier, idle apps scale to zero and cold-start on the next request. For a demo or internal tool that's free compute; for a user-facing chat API, the first request after idle pays both the container cold start and your LLM provider's latency stacked together, so a paid tier with an always-on instance is the right call once real users show up.

Production gotchas, in order of how often they bite

1. 1Unpinned LangChain versions. Covered above. Freeze them.
2. 2Streaming through buffering proxies. If tokens arrive all at once instead of incrementally, something between the client and uvicorn is buffering. SSE with text/event-stream usually passes through cleanly, but test the deployed URL, not just localhost.
3. 3Request timeouts. A long generation can run 60+ seconds. Check both your platform's ingress timeout and your client's — streaming helps here too, since bytes flowing keep idle-timeout counters happy.
4. 4No token limits on input. Cap the length of question at the API layer. Otherwise someone will paste a novel and you'll pay for it.
5. 5Health checks hitting the LLM. Costs money, adds noise, couples your uptime to a third party.

None of this is exotic — it's the same discipline as any Python service, plus an expensive, occasionally slow dependency in the hot path. Get the boring parts right and the chain itself is the easy bit.

If you want to try the flow with the database wired in automatically, it's a git push away at [pandastack.io](https://pandastack.io).