Deploy a Dagster Data Pipeline on PandaStack (2026)

If you've ever inherited a pile of cron-triggered Python scripts that quietly stopped producing correct data three weeks ago and nobody noticed, you already understand the case for an orchestrator. Dagster (https://dagster.io) is a modern one built around software-defined assets — you declare the tables, files, and models your pipeline produces, and Dagster tracks their lineage, freshness, and materialization history. It's testable, observable, and far friendlier to debug than a wall of cron entries.

This guide runs Dagster on PandaStack: the webserver (UI + GraphQL), the daemon (schedules + sensors), and a managed PostgreSQL instance as the run/event store.

The moving parts

A production Dagster deployment has three pieces:

1. 1Webserver (dagster-webserver) — the UI and API. This is the container PandaStack exposes publicly.
2. 2Daemon (dagster-daemon) — runs schedules and sensors. Without it, nothing fires on time.
3. 3A storage backend — Dagster defaults to SQLite, which does not survive container restarts and can't be shared between the webserver and daemon. You must point it at PostgreSQL.

That third point is the one that trips people up. SQLite works on your laptop and fails in the cloud.

Step 1: Define an asset

my_pipeline/assets.py:

import pandas as pd
from dagster import asset

@asset
def raw_events() -> pd.DataFrame:
    # In reality: pull from an API or your app's Postgres
    return pd.DataFrame({"user": ["a", "b", "a"], "action": ["click"] * 3})

@asset
def events_per_user(raw_events: pd.DataFrame) -> pd.DataFrame:
    return raw_events.groupby("user").size().reset_index(name="count")

Dagster infers that events_per_user depends on raw_events from the function signature — that dependency graph is your lineage, for free.

Step 2: Point storage at PostgreSQL

Create dagster.yaml — this is the file that saves you from the SQLite trap:

storage:
  postgres:
    postgres_db:
      username:
        env: DAGSTER_PG_USER
      password:
        env: DAGSTER_PG_PASSWORD
      hostname:
        env: DAGSTER_PG_HOST
      db_name:
        env: DAGSTER_PG_DB
      port: 5432

run_coordinator:
  module: dagster.core.run_coordinator
  class: QueuedRunCoordinator

Set DAGSTER_HOME to the directory containing this file so Dagster finds it.

Step 3: A schedule

my_pipeline/__init__.py:

from dagster import Definitions, define_asset_job, ScheduleDefinition, load_assets_from_modules
from . import assets

all_assets = load_assets_from_modules([assets])
daily_job = define_asset_job("daily_refresh", selection="*")

daily_schedule = ScheduleDefinition(job=daily_job, cron_schedule="0 6 * * *")

defs = Definitions(
    assets=all_assets,
    jobs=[daily_job],
    schedules=[daily_schedule],
)

Step 4: requirements.txt

dagster>=1.7
dagster-webserver>=1.7
dagster-postgres>=0.23
pandas>=2.2

Step 5: Dockerfile

Both the webserver and daemon run from the same image; the start command differs per app.

FROM python:3.12-slim

ENV DAGSTER_HOME=/opt/dagster
WORKDIR /opt/dagster

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY dagster.yaml /opt/dagster/dagster.yaml
COPY my_pipeline /opt/dagster/my_pipeline

EXPOSE 3000

Step 6: Create the PostgreSQL database

Dashboard (https://dashboard.pandastack.io) → Databases → New Database → PostgreSQL. Grab the connection details — you'll feed the individual parts (host, user, password, db) to Dagster via env vars because dagster.yaml references them by field.

Step 7: Deploy the webserver

1. 1Push the repo to GitHub.
2. 2New App → Container App, connect the repo, container port 3000.
3. 3Set the start command to:

dagster-webserver -h 0.0.0.0 -p 3000 -m my_pipeline

1. 1Environment variables:

1. 1Deploy. The Dagster UI is now at https://your-dagster.pandastack.io.

Step 8: Deploy the daemon (this is what makes schedules run)

Create a second container app from the same repo, but:

• No public port needed (it doesn't serve HTTP).
• Start command:

dagster-daemon run -m my_pipeline

• Same four DAGSTER_PG_* environment variables (it must share the storage backend with the webserver).

Without this second app, your schedules show up in the UI but never fire. This is the single most common "why isn't my pipeline running" cause.

Step 9: Verify

Open the UI, go to Assets, and click Materialize all. You'll see the lineage graph, run logs, and materialization history. Then check Overview → Schedules and confirm daily_schedule is toggled on and shows a next-tick time — that confirms the daemon is alive and connected.

Should you use Dagster or a PandaStack cronjob?

Be pragmatic. If you have one script on a simple schedule, a PandaStack cronjob is far less machinery — one container, one cron expression, done. Reach for Dagster when you have many interdependent data assets, need lineage and backfills, or want a UI where non-engineers can see pipeline health. Don't stand up a webserver + daemon + Postgres to run a single nightly export.

Wrap-up

Dagster turns a fragile pile of cron scripts into an observable, testable asset graph. On PandaStack that's two container apps (webserver + daemon) sharing one managed Postgres store — just don't forget the daemon, and don't forget to move off SQLite. For simpler needs, a cronjob is the honest answer. Docs: https://docs.pandastack.io. Start free at https://dashboard.pandastack.io.