Service type
App service
Apache Spark worker service
Apache Spark worker service for executing data processing tasks assigned by a linked Spark master.
Use it as a reusable, versioned service inside custom or predefined stacks so teams can keep the runtime and operational model consistent.
On Wodby
How Apache Spark worker runs on Wodby
What Wodby sets up for a Spark standalone worker on this service. It runs the official
apache/spark image with org.apache.spark.deploy.worker.Worker.Linked services
| Link | Variables |
|---|---|
| Apache Spark master (required) | SPARK_MASTER_URL |
The worker registers with the master at that address when it starts. Do not set the master address elsewhere.
Settings
| Setting | Variable | Meaning |
|---|---|---|
| Worker CPU cores | SPARK_WORKER_CORES | cores Spark applications may use on this worker |
| Worker memory | SPARK_WORKER_MEMORY | memory Spark applications may use on this worker |
These are what the worker offers to the master. They are separate from the container's CPU and memory resources: keep them within the container's limits.
Other facts
- The service is scalable: each replica is one worker.
- It has no endpoint and no Kubernetes service. The worker's web interface (port 8081) is not reachable from other services.
- It has no volume: application work directories are lost when the pod is replaced.
Overview
Technical profile and packaging
This page shows how Apache Spark worker is packaged and versioned as a Wodby service, what it exposes at runtime, and which source artifacts it builds on.
App service
Versions
4.1
Source repository
Public Helm chart
Labels
spark, data-processing, worker
Stacks
Use it inside ready-made stacks or your own
These public stacks already include Apache Spark worker as part of their default service model. You can start there and customize the stack, or compose your own stack around the same building block.
Runtime wiring
Reuse connectivity, storage, and runtime defaults
This service can expose ports, connect to other services, attach persistent storage, and define runtime defaults that become reusable inside stacks rather than being rebuilt case by case.
Settings
2 settings can be overridden per environment.
Service links
These links define which other services can connect to Apache Spark worker when it is used inside a stack.
master
Eligible linked services