App service
Services/Apache Spark worker

Apache Spark worker service

Apache Spark worker service for executing data processing tasks assigned by a linked Spark master.

Use it as a reusable, versioned service inside custom or predefined stacks so teams can keep the runtime and operational model consistent.

On Wodby

How Apache Spark worker runs on Wodby

What Wodby sets up for a Spark standalone worker on this service. It runs the official apache/spark image with org.apache.spark.deploy.worker.Worker.

Linked services

LinkVariables
Apache Spark master (required)SPARK_MASTER_URL

The worker registers with the master at that address when it starts. Do not set the master address elsewhere.

Settings

SettingVariableMeaning
Worker CPU coresSPARK_WORKER_COREScores Spark applications may use on this worker
Worker memorySPARK_WORKER_MEMORYmemory Spark applications may use on this worker

These are what the worker offers to the master. They are separate from the container's CPU and memory resources: keep them within the container's limits.

Other facts

  • The service is scalable: each replica is one worker.
  • It has no endpoint and no Kubernetes service. The worker's web interface (port 8081) is not reachable from other services.
  • It has no volume: application work directories are lost when the pod is replaced.

Overview

Technical profile and packaging

This page shows how Apache Spark worker is packaged and versioned as a Wodby service, what it exposes at runtime, and which source artifacts it builds on.

Service type

App service

Versions

4.1

Labels

spark, data-processing, worker

Stacks

Use it inside ready-made stacks or your own

These public stacks already include Apache Spark worker as part of their default service model. You can start there and customize the stack, or compose your own stack around the same building block.

Runtime wiring

Reuse connectivity, storage, and runtime defaults

This service can expose ports, connect to other services, attach persistent storage, and define runtime defaults that become reusable inside stacks rather than being rebuilt case by case.

Settings

2 settings can be overridden per environment.

Service links

These links define which other services can connect to Apache Spark worker when it is used inside a stack.