How to Deploy a LlamaIndex App with Postgres in Production

LlamaIndex is a Python framework, so "deploying a LlamaIndex app" really means deploying a Python web service that loads an index and answers queries. That sounds simple until you hit the three things the quickstart never mentions: where the index lives between restarts, how much memory it eats, and what happens when your ingestion pipeline and your API server are the same process. Here's how to structure it properly and ship it.

The shape of a production LlamaIndex app

The quickstart pattern — read documents, build an index, query it — works in a notebook and falls apart in production:

# Fine for a notebook. Not fine behind a load balancer.
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

Every restart re-embeds every document. That costs real money (embedding API calls), takes minutes on a decent corpus, and means two replicas each build their own copy of the index. The fix is to split the app in two:

1. 1An ingestion script that embeds documents and writes vectors to a database. Runs on demand or on a schedule.
2. 2A query server that connects to that database and serves requests. Stateless, fast to boot, safe to run with multiple replicas.

Dependencies

LlamaIndex is modular now. Install only what you use instead of the full llama-index meta-package — your Docker image will thank you:

# requirements.txt
llama-index-core
llama-index-llms-openai
llama-index-embeddings-openai
llama-index-vector-stores-postgres
fastapi
uvicorn[standard]

Skip anything that pulls in torch unless you're running local embeddings. The difference between an OpenAI-embeddings image and a local-embeddings image is often 2 GB.

Storing vectors in Postgres

LlamaIndex ships PGVectorStore, which keeps embeddings in a regular Postgres table using the pgvector extension. This is the pragmatic choice for most apps: one database for your vectors and your application data, one backup story, no extra vendor.

Most managed platforms hand you a single DATABASE_URL. Parse it into components rather than passing the string through — this also sidesteps the old postgres:// vs postgresql:// scheme fight, because only the parsed parts are used:

# db.py
import os
from sqlalchemy import make_url
from llama_index.vector_stores.postgres import PGVectorStore

def get_vector_store() -> PGVectorStore:
    url = make_url(os.environ["DATABASE_URL"])
    return PGVectorStore.from_params(
        database=url.database,
        host=url.host,
        port=url.port or 5432,
        user=url.username,
        password=url.password,
        table_name="doc_embeddings",
        embed_dim=1536,  # text-embedding-3-small
    )

One check before you rely on this: connect with psql and run CREATE EXTENSION IF NOT EXISTS vector;. If your Postgres build doesn't ship pgvector, use the local-persistence fallback described below instead of fighting it.

Also make sure embed_dim matches your embedding model. If you switch models later, the stored vectors are useless — you re-ingest. Treat the pair (model, table) as one unit.

The ingestion script

# ingest.py
from llama_index.core import (
    SimpleDirectoryReader,
    StorageContext,
    VectorStoreIndex,
)
from db import get_vector_store

def main():
    documents = SimpleDirectoryReader("data").load_data()
    storage_context = StorageContext.from_defaults(
        vector_store=get_vector_store()
    )
    VectorStoreIndex.from_documents(
        documents,
        storage_context=storage_context,
        show_progress=True,
    )
    print(f"Ingested {len(documents)} documents")

if __name__ == "__main__":
    main()

Run it once locally against the production database, and re-run it whenever the source documents change. Because it's a plain script, it slots straight into a scheduled job — more on that below.

The query server

The server never builds an index. It attaches to the existing vector store, which is nearly instant:

# main.py
from contextlib import asynccontextmanager
from fastapi import FastAPI
from pydantic import BaseModel
from llama_index.core import VectorStoreIndex
from db import get_vector_store

engine = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    global engine
    index = VectorStoreIndex.from_vector_store(get_vector_store())
    engine = index.as_query_engine(similarity_top_k=4)
    yield

app = FastAPI(lifespan=lifespan)

class Query(BaseModel):
    question: str

@app.post("/query")
def query(q: Query):
    response = engine.query(q.question)
    return {
        "answer": str(response),
        "sources": [n.node.metadata for n in response.source_nodes],
    }

@app.get("/healthz")
def healthz():
    return {"ok": True}

from_vector_store doesn't load embeddings into memory — retrieval happens in Postgres at query time. That keeps the service light: the RAM cost of your corpus lives in the database, not in every replica.

Fallback: local persistence baked into the image

No pgvector? Persist the index to disk at build time and load it at boot:

index.storage_context.persist(persist_dir="storage")

from llama_index.core import StorageContext, load_index_from_storage

storage_context = StorageContext.from_defaults(persist_dir="storage")
index = load_index_from_storage(storage_context)

The tradeoff is real: the index becomes part of the container image, so updating documents means rebuilding and redeploying, and the whole index loads into RAM. Fine for a docs bot over a few hundred pages; wrong for anything that changes daily.

The Dockerfile

FROM python:3.12-slim
WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Nothing clever. The only rule that matters: bind to 0.0.0.0, not 127.0.0.1, or the platform's router will never reach you.

Environment variables

Two matter, and both fail in annoying ways if missing:

• OPENAI_API_KEY — the default LlamaIndex Settings use OpenAI for both the LLM and embeddings. Without it, the app may boot fine and then throw on the first query. Fail fast instead: read it at startup and raise if absent.
• DATABASE_URL — the Postgres connection string.

Deploying on PandaStack

This layout maps cleanly onto PandaStack:

1. 1Provision a managed PostgreSQL instance (14.x or 16.x are available) from the dashboard.
2. 2Connect your repo as a container app. With the Dockerfile above it builds exactly what you tested locally; the build runs in rootless BuildKit inside an ephemeral Kubernetes Job, and you can watch the live build logs while it goes.
3. 3Attach the database to the app — DATABASE_URL is injected automatically, so the only secret you manage by hand is OPENAI_API_KEY, which you add in the app's environment variables.
4. 4Run ingest.py against the database once. For a corpus that changes on a schedule, register the same script as a cronjob so re-indexing happens without you.

One honest note on the free tier: apps scale to zero when idle, so the first query after a quiet period pays a cold start. For a demo or an internal tool that's fine — and it's exactly why the Postgres-backed layout beats the baked-in index here, since there's no big index to reload on every cold boot. For user-facing latency, use a paid tier that stays warm.

What usually breaks, and in what order

• First deploy, first query fails: missing OPENAI_API_KEY. Validate at boot.
• Answers are empty: ingestion wrote to a different table name than the server reads. Keep table_name in one shared module (that's why db.py exists).
• Dimension mismatch errors: you changed embedding models without re-ingesting. Drop the table, re-run ingest.py.
• Memory creep: someone switched to local embeddings and the model now loads in every replica. Check your image size diff before merging that PR.

None of these are exotic — they're the same four failures in roughly every LlamaIndex deployment I've seen. Structure the app as ingestion plus a stateless server and you've already dodged the worst of them.

If you want to try this end to end — managed Postgres, auto-injected DATABASE_URL, git-push deploys — it takes a few minutes on https://pandastack.io.