Introduction
On this page
A backend in Apalis is the bridge between your application and a task store. It is responsible for delivering tasks to workers, signalling liveness, providing middleware hooks, and encoding task data for storage. Any queue technology — Redis, PostgreSQL, SQLite, in-memory — becomes an Apalis backend by implementing two core traits: Backend and BackendExt.
In other ecosystems you may know this concept as a broker, queue, or transport. In Apalis, the backend abstraction is deliberately generic so the same worker logic runs against any storage layer.
The Backend Trait
Backend defines the runtime contract between a worker and a task store. It covers three responsibilities:
- Polling — producing a stream of tasks for a worker to execute
- Heartbeating — emitting periodic liveness signals so the system can detect stalled workers
- Middleware — exposing a composable layer for cross-cutting concerns (retries, tracing, rate limiting, etc.)
pub trait Backend {
/// The type of job arguments this backend delivers to workers.
type Args;
/// The type used to uniquely identify each task (e.g. `Uuid`, `i64`).
type IdType: Clone;
/// Arbitrary context attached to every task — for example, priority,
/// scheduled time, or retry count. Must implement `Default` so the
/// backend can construct context for new tasks.
type Context: Default;
/// The error type returned by backend operations such as polling
/// or heartbeating.
type Error;
/// The stream of tasks returned by `poll`. Each item is:
/// - `Ok(Some(task))` — a task is available for processing
/// - `Ok(None)` — the backend is idle (no tasks right now)
/// - `Err(e)` — a backend error occurred
type Stream: Stream
Item = Result<Option<Task<Self::Args, Self::Context, Self::IdType>>, Self::Error>,
>;
/// The stream returned by `heartbeat`. Each `Ok(())` item represents
/// a successful liveness tick. An `Err` signals that the heartbeat
/// mechanism has failed.
type Beat: Stream<Item = Result<(), Self::Error>>;
/// The middleware layer type for this backend. Used by the Apalis
/// service builder to compose cross-cutting behaviour (e.g. retries,
/// timeout, tracing) around task execution.
type Layer;
/// Produces a heartbeat stream for the given worker. The worker
/// drives this stream to periodically register its liveness with
/// the backend, enabling dead-worker detection and task requeuing.
fn heartbeat(&self, worker: &WorkerContext) -> Self::Beat;
/// Returns the backend's middleware layer. This is called once
/// during worker setup and composed into the service stack.
fn middleware(&self) -> Self::Layer;
/// Begins polling the backend for tasks assigned to this worker,
/// returning a decoded task stream. For most use cases you will
/// drive this stream inside a [`Worker`].
///
/// If you need access to the raw encoded representation, see
/// [`BackendExt::poll_compact`].
fn poll(self, worker: &WorkerContext) -> Self::Stream;
}Associated Types at a Glance
| Type | Purpose |
|---|---|
Args | The decoded job argument type your handler receives |
IdType | Unique task identifier — often Uuid or a database row ID |
Context | Per-task metadata (priority, attempt count, scheduled time, …) |
Error | Unified error type for all backend I/O operations |
Stream | Async stream of decoded tasks for a worker |
Beat | Async stream of heartbeat ticks |
Layer | Middleware type composed into the worker's service stack |
The BackendExt Trait
BackendExt extends [Backend] with serialization awareness. While Backend works in terms of fully decoded Args, BackendExt exposes the compact (encoded) representation that the backend actually stores. This separation keeps storage concerns out of your handler logic while still allowing low-level access when needed.
pub trait BackendExt: Backend {
/// The codec used to encode `Args` into `Compact` for storage and
/// decode them back again on the worker side. See [`Codec`] for details.
type Codec: Codec<Self::Args, Compact = Self::Compact>;
/// The encoded representation of task arguments as stored in the
/// backend — for example, a `Vec<u8>` for binary formats or a
/// `serde_json::Value` for JSON-based stores.
type Compact;
/// Like `Backend::Stream`, but yields tasks in their *encoded* form.
/// Useful for inspecting or forwarding tasks without a full decode
/// round-trip, or for implementing custom deserialization paths.
type CompactStream: Stream
Item = Result<Option<Task<Self::Compact, Self::Context, Self::IdType>>, Self::Error>,
>;
/// Returns the [`Queue`] identifier associated with this backend
/// instance — typically the queue name or topic the backend is
/// bound to.
fn get_queue(&self) -> Queue;
/// Like `Backend::poll`, but yields tasks in their compact (encoded)
/// form without deserializing `Args`. Use this when you need direct
/// access to the raw payload — for example, when routing tasks to
/// different workers based on the encoded content.
fn poll_compact(self, worker: &WorkerContext) -> Self::CompactStream;
}Associated Types at a Glance
| Type | Purpose |
|---|---|
Codec | Handles encoding Args → Compact and decoding Compact → Args |
Compact | The storage-level representation (e.g. Vec<u8>, String) |
CompactStream | Async stream of encoded tasks — bypasses deserialization |
How Backend and BackendExt Relate
┌─────────────────────────────────────────┐
│ BackendExt │
│ ┌───────────────────────────────────┐ │
│ │ Backend │ │
│ │ heartbeat() middleware() poll() │ │
│ └───────────────────────────────────┘ │
│ get_queue() poll_compact() Codec │
└─────────────────────────────────────────┘
Backendis the minimum required interface for a working job queue. All workers depend only onBackend.BackendExtadds serialization details needed by [TaskSink] (for pushing tasks) and advanced consumers that need raw payloads.- A type that implements
BackendExtautomatically satisfiesBackend, sinceBackendExt: Backend.
Implementing a Custom Backend
Any type can become an Apalis backend. At minimum you implement Backend; add BackendExt when the type also needs to act as a [TaskSink] target or expose its codec.
use apalis::prelude::*;
struct MyBackend { /* ... */ }
impl Backend for MyBackend {
type Args = MyJob;
type IdType = uuid::Uuid;
type Context = MyJobContext;
type Error = MyBackendError;
type Stream = /* ... */;
type Beat = /* ... */;
type Layer = /* ... */;
fn heartbeat(&self, worker: &WorkerContext) -> Self::Beat { /* ... */ }
fn middleware(&self) -> Self::Layer { /* ... */ }
fn poll(self, worker: &WorkerContext) -> Self::Stream { /* ... */ }
}For most use cases, you will use one of Apalis's built-in backends rather than writing your own. See the Backends overview for the full list.
Implementations
| Backend | Crate | Best For |
|---|---|---|
| PostgreSQL | apalis-postgres | Durable jobs, existing Postgres infra |
| MySQL/MariaDB | apalis-mysql | Durable jobs, existing MySQL infra |
| SQLite | apalis-sqlite | Low-traffic or single-node deployments |
| Redis | apalis-redis | High-throughput, low-latency job queues |
| AMQP | apalis-amqp | Message broker-based architectures |
| PGMQ | apalis-pgmq | Postgres-native message queues with at-least-once delivery |
| NATS | apalis-nats | Distributed messaging, cloud-native and edge deployments |
| RSMQ | apalis-rsmq | Redis-backed simple message queues |
| Cron | apalis-cron | Schedule-driven jobs — use with pipe_to for persistence |
Summary
| Trait | Responsibility |
|---|---|
Backend | Task polling, heartbeating, middleware composition |
BackendExt | Codec configuration, queue identity, encoded task access |
Together these two traits form a clean boundary between how tasks are stored and retrieved and how they are processed — letting you swap backends without touching your job handler logic.