Creating a new Dagster project

The easiest way to start building a Dagster project is by using the create-dagster CLI. This CLI tool allows you to create a special type of Python package, called a project, that defines a Dagster code location.

Prerequisites

Before creating a Dagster project, you must do one of the following:

• If you will be using uv as a package manager, install uv (Recommended).
• If you will be using pip as a package manager, install the create-dagster CLI with Homebrew, curl, or pip.

For more information, see the Installation guide.

Step 1. Scaffold a new Dagster project

uv (recommended)

1. Scaffold a new Dagster project, replacing <project_name> with the name of your project:

   uvx -U create-dagster project <project_name>

2. Respond y to the prompt to run uv sync after scaffolding:

   

3. Change to the project directory:

   cd <project-name>

4. Activate the virtual environment:

MacOS/Unix

source .venv/bin/activate

Windows

.venv\Scripts\activate

pip

1. Scaffold a new Dagster project, replacing <project_name> with the name of your project:

create-dagster project <project_name>

2. Change to the project directory:

cd <project_name>

3. Create and activate a virtual environment:

MacOS/Unix

python -m venv venv

source .venv/bin/activate

Windows

python -m venv venv

.venv\Scripts\activate

4. Install required dependencies:

pip install dagster dagster-webserver dagster-dg

5. Install your project as an editable package:

pip install --editable .

info

The create-dagster project command creates a directory with a standard Python package structure with some additions. For more information on the files and directories in a Dagster project, see the Dagster project file reference.

Step 2. Add assets

Assets are the core abstraction in Dagster and can represent logical units of data such as tables, datasets, or machine learning models. Assets can have dependencies on other assets, forming the data lineage for your pipelines. To add assets to your project, see Defining assets.

Step 3: View assets in the UI

To start the Dagster UI, run:

dg dev

To see your assets, navigate to http://localhost:3000.

Step 4: Continue development

• Adding new Python dependencies
• Use environment variables and secrets
• Add and run unit tests

Add new Python dependencies

You can specify new Python dependencies in pyproject.toml.

Use environment variables and secrets

Environment variables, which are key-value pairs configured outside your source code, allow you to dynamically modify application behavior depending on environment.

Using environment variables, you can define various configuration options for your Dagster application and securely set up secrets. For example, instead of hard-coding database credentials - which is bad practice and cumbersome for development - you can use environment variables to supply user details. This allows you to parameterize your pipeline without modifying code or insecurely storing sensitive data.

For more information and examples, see Using environment variables and secrets.

Add and run unit tests

Tests can be added to the tests directory and run using pytest:

pytest tests

For more information on testing, see the following docs:

• The Testing guides have guidance on asset checks, data freshness checks, unit testing assets and ops, and testing partitioned config and jobs.
• Testing component definitions contains testing best practices for definitions scaffolded existing components.
• Testing your component has best practices for testing custom components.

Next steps

• Add integrations to your project
• Create your own Dagster Components to share with your team
• Deploy your project to Dagster+ or your own infrastructure