App service
Services/Apache Airflow

Apache Airflow service

Apache Airflow workflow orchestration service built from your Git repository, with your DAGs, plugins, and Python packages in every Airflow component and linked PostgreSQL metadata storage.

Use it as a versioned stack building block that can build from source, plug into CI/CD, and connect to external providers without hardcoding secrets into the codebase.

On Wodby

How Apache Airflow runs on Wodby

What Wodby sets up for Airflow on this service. It runs the official apache/airflow image built with the app's repository. Configuration is passed as AIRFLOW__<SECTION>__<KEY> environment variables, which take precedence over airflow.cfg.

Processes

One pod runs four containers on the same volume: the API server (main container, port 8080, also the web interface), scheduler, dag-processor and triggerer. Before they start, an init container migrates the metadata database (_AIRFLOW_DB_MIGRATE) and creates the administrator (_AIRFLOW_WWW_USER_CREATE). The executor is LocalExecutor: tasks run inside the scheduler container, so their memory comes from that container's limit, which is set with the other containers' resources on the app service. Airflow itself doesn't cap a task's memory; AIRFLOW__CORE__PARALLELISM and DAG-level max_active_tasks limit how many tasks run at once. The service is not scalable.

Linked services

LinkVariables
PostgreSQL (required)AIRFLOW__DATABASE__SQL_ALCHEMY_CONN

It is the metadata database of Airflow. Do not set the connection elsewhere.

Generated credentials

TokenVariableUse
admin_username, admin_password_AIRFLOW_WWW_USER_USERNAME, _AIRFLOW_WWW_USER_PASSWORDadministrator created by the init container; an existing user is left as it is
fernet_keyAIRFLOW__CORE__FERNET_KEYencrypts connection passwords stored in the database
api_secret_keyAIRFLOW__API__SECRET_KEYAPI server sessions
api_auth_jwt_secretAIRFLOW__API_AUTH__JWT_SECRETtokens between Airflow components and for the API

Users are managed by the FAB auth manager (AIRFLOW__CORE__AUTH_MANAGER). Example DAGs are off (AIRFLOW__CORE__LOAD_EXAMPLES).

Image and DAGs

The image is built from the app's Git repository on top of the apache/airflow image of the service's version, so the service's versions are Airflow versions. Start the repository from the Airflow boilerplate. The build:

  • copies the repository to /opt/airflow-app, outside the /opt/airflow data volume, which would hide files baked into the image;
  • installs the packages in requirements.txt, when the repository has one, with Airflow pinned to the image's version, so packages that need another Airflow fail the build;
  • sets AIRFLOW__CORE__DAGS_FOLDER to /opt/airflow-app/dags and AIRFLOW__CORE__PLUGINS_FOLDER to /opt/airflow-app/plugins.

Every Airflow component runs the built image: the API server, scheduler, DAG processor and triggerer are build containers of the workload, and the setup step gets the image through helm.image.references. DAG and package changes reach Airflow with the next deployment. Airflow's version comes from the service version and the image, not from the repository.

Put DAG files in dags/ and plugins in plugins/ of the repository; the dags and plugins directories on the data volume are not read. The data volume is mounted at /opt/airflow, the Airflow home, in every container and keeps Airflow's logs.

Changing configuration

Add or change AIRFLOW__* environment variables on the service and deploy it. The scheduler, DAG processor and triggerer get only the service's secret variables, from the environment Secret, plus the executor, examples and auth manager settings declared in the manifest; after a change, check that it reached the container that uses it.

Check the result

/api/v2/monitor/health on port 8080 reports the state of the metadata database, scheduler, DAG processor and triggerer.

Overview

Technical profile and packaging

This page shows how Apache Airflow is packaged and versioned as a Wodby service, what it exposes at runtime, and which source artifacts it builds on.

Service type

App service

Versions

3.3

Ports and protocols

8080/HTTP

Labels

airflow, workflow, orchestration

Persistent volumes

Airflow data and logs

Stacks

Use it inside ready-made stacks or your own

These public stacks already include Apache Airflow as part of their default service model. You can start there and customize the stack, or compose your own stack around the same building block.

CI/CD

Build from source and ship changes without extra glue work

Apache Airflow can build from source as part of a stack. Start from a compatible public boilerplate or connect your repository and keep CI/CD and image storage inside the same Wodby workflow.

Vanilla Airflow project

Start from a working boilerplate instead of wiring the build process from scratch.

wodby/airflow-boilerplate

Git providers

Connect repositories from supported Git providers when this service should build from your own codebase.

CI providers

Wodby CI is the default path, but external CI providers can fit the same build and release workflow when that suits your team better.

Container registries

Keep built images in Wodby Registry by default or store them in another registry without changing how the service is built.

Runtime wiring

Reuse connectivity, storage, and runtime defaults

This service can expose ports, connect to other services, attach persistent storage, and define runtime defaults that become reusable inside stacks rather than being rebuilt case by case.

Endpoints

8080/HTTP

Technical domains

HTTP endpoints from this service can use technical domains with auto-renewed TLS certificates on Wodby.

Volumes

Airflow data and logs

Service links

These links define which other services can connect to Apache Airflow when it is used inside a stack.