How to Deploy a Dramatiq Worker with PandaStack

Dramatiq is what a lot of Python teams reach for after getting burned by Celery's configuration surface: a task queue with sane defaults, reliable at-least-once delivery, and retries that work out of the box. Deploying a Dramatiq worker is different from deploying a web app, though — there's no port, no HTTP, no health-check endpoint, and a couple of platform behaviors that are perfect for web apps will quietly kill a queue worker. Here's the full path to production, including the traps.

A minimal worker

A Dramatiq project needs a broker and some actors. Both live happily in one module:

# tasks.py
import os

import dramatiq
from dramatiq.brokers.redis import RedisBroker

broker = RedisBroker(url=os.environ["REDIS_URL"])
dramatiq.set_broker(broker)


@dramatiq.actor(max_retries=3, time_limit=600_000)
def generate_report(org_id: int):
    data = fetch_rows(org_id)
    build_pdf(data)

Your web app enqueues work by importing the actor and calling .send():

from tasks import generate_report

generate_report.send(42)

And the worker process is just the Dramatiq CLI pointed at the module:

dramatiq tasks --processes 2 --threads 4

That's the whole architecture: web app pushes messages to the broker, worker pulls and executes. The web app and the worker are two separate deployments sharing one codebase and one broker — resist the urge to run the worker as a subprocess inside your web container. When the web app restarts mid-task, you'll lose work and spend a day figuring out why.

Choosing a broker: Redis or RabbitMQ

Dramatiq supports both. RabbitMQ is the more featureful broker (priorities, better back-pressure semantics); Redis is the one you probably already have. For most workloads — emails, report generation, image processing, webhook fan-out — Redis is entirely adequate and one less system to run.

On PandaStack, Redis is one of the managed database engines (alongside PostgreSQL, MySQL, and MongoDB), so the broker can live on the same platform as the worker: provision a managed Redis from the dashboard and put its connection string in the worker's environment as REDIS_URL. If you'd rather use RabbitMQ, the only code change is the import:

from dramatiq.brokers.rabbitmq import RabbitmqBroker

broker = RabbitmqBroker(url=os.environ["RABBITMQ_URL"])
dramatiq.set_broker(broker)

Everything else in this post stays the same.

Talking to PostgreSQL from actors — and the connection math

Most real actors touch a database. When you attach a managed PostgreSQL instance to your PandaStack app, DATABASE_URL is injected into the environment automatically. The important part is *how* you create the engine, because Dramatiq runs multiple worker processes:

# db.py
import os

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

_engine = None
_Session = None


def get_session():
    global _engine, _Session
    if _engine is None:
        url = os.environ["DATABASE_URL"]
        if url.startswith("postgres://"):
            url = url.replace("postgres://", "postgresql://", 1)
        _engine = create_engine(url, pool_size=4, max_overflow=0, pool_pre_ping=True)
        _Session = sessionmaker(bind=_engine)
    return _Session()

Create the engine lazily, inside the process that uses it — never at module import time in a way that gets shared across forked processes. A SQLAlchemy connection pool inherited across a fork produces corrupted-protocol errors that look like database bugs and are miserable to diagnose.

Then do the connection math, because workers multiply fast:

2 processes × 4 threads × pool_size 4 = up to 32 connections

PandaStack's free tier database allows 50 connections, Pro allows 300. A worker with default-ish settings plus your web app can eat the free-tier limit by itself. Keep pool_size small and max_overflow=0 on workers — a background task waiting a few milliseconds for a pooled connection costs nothing; a database refusing connections costs everything. And pool_pre_ping=True matters more here than in web apps, because worker connections sit idle between jobs and stale ones will otherwise greet you with dropped-connection errors after every quiet period.

If your project has Alembic migrations, run them from the web app's release step, not the worker — one owner for schema changes, no races.

The Dockerfile

Workers make for pleasantly boring Dockerfiles:

FROM python:3.12-slim

WORKDIR /app
ENV PYTHONUNBUFFERED=1

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["dramatiq", "tasks", "--processes", "2", "--threads", "4"]

Two things to note. PYTHONUNBUFFERED=1 so log lines show up in the platform's log stream as they happen, not in buffered bursts after the fact. And size --processes/--threads to the container, not the defaults — Dramatiq defaults to one process per CPU and 8 threads each, which is tuned for a beefy VM, not a 0.5-CPU container. Each process is a full Python interpreter with its own memory footprint; for I/O-heavy tasks (API calls, emails), fewer processes and more threads is the right trade. For CPU-heavy tasks (PDF rendering, image work), it's the reverse.

Deploying on PandaStack

1. 1Provision a managed Redis (the broker) and a managed PostgreSQL if your tasks need one. Attach the Postgres to the worker app so DATABASE_URL is injected; add the Redis connection string as a REDIS_URL environment variable.
2. 2Connect the repo as a container app. The Dockerfile is picked up and built with rootless BuildKit in an ephemeral build pod; the build logs stream live, so a broken pip install is visible in seconds rather than after a mystery timeout.
3. 3Push to Git. The worker builds, deploys, and starts consuming.

Now the trap: scale-to-zero. PandaStack's free tier scales idle apps to zero based on incoming HTTP traffic and cold-starts them on the next request. That's exactly right for a web app and exactly wrong for a queue worker — a worker receives no HTTP traffic ever, so the platform sees it as permanently idle. Free-tier containers also run on preemptible nodes that can be reclaimed. Run workers on a paid compute tier, where the container stays up continuously on stable nodes. This isn't a PandaStack quirk so much as a category truth — any platform with HTTP-based autoscaling has some version of it — but it bites Dramatiq users specifically because the failure mode is silent: no errors, no crashes, just a queue that quietly grows.

Dramatiq's delivery guarantees are your safety net for everything else. Delivery is at-least-once: if a worker dies mid-task during a deploy or a node event, the message is redelivered and retried with exponential backoff. Two implications: write actors to be idempotent (running twice must be safe — check-then-insert, upserts, deduplication keys), and let max_retries do its job instead of catching every exception yourself.

Shutdowns, retries, and scheduled work

Dramatiq handles SIGTERM gracefully — on a deploy, the worker stops pulling new messages and finishes what it's running before exiting. Pair that with sane time_limit values per actor so a hung task can't block a shutdown indefinitely:

@dramatiq.actor(max_retries=5, min_backoff=15_000, time_limit=300_000)
def sync_customer(customer_id: int):
    ...

For periodic work, you don't need to bolt a scheduler thread onto the worker. Keep the worker dumb and use a scheduled trigger to enqueue: PandaStack has cronjobs as a first-class primitive, so a small cron task that calls nightly_cleanup.send() on a schedule keeps scheduling concerns out of your worker entirely — and means a worker restart can never lose the schedule.

The end state is a genuinely low-maintenance system: a stateless worker container, a managed Redis broker, a managed Postgres, and retries covering the gaps. If you want to stand one up without wiring three services together by hand, https://pandastack.io is a reasonable place to start.