A worker holds no state.That is the whole design.
The queue of what to do next, a join's partial arrivals, a suspended approval, a repeating timer — every one of them is a row in the database. A process contributes a worker loop and nothing else. Everything on this page follows from that, including the parts that are inconvenient.
Six places the engine keeps its promises, none of them in a process
Events are leased, not held
claim · lease · reclaimA worker claims an event with a lease. If it finishes, the event is completed and its consequences are queued before the claim is released — so there is never a moment where the queue looks empty while work is still in flight. If the worker dies, the lease expires and another worker picks the event up. Delivery is at-least-once, and the engine says so plainly rather than implying exactly-once.
A gather's arrivals are rows
one row per arrival, never an arrayThe obvious implementation of a fan-in appends to an array on one record. That record becomes a contention point, and under a forty-way fan-out the losers of each write conflict lose their items silently. Here each arrival is its own row at its own id, and completion is claimed by taking the rows: exactly one worker gets them, the rest stand down.
A join completes once, on whichever node
count, then claimBranches of a join arrive on different workers at the same instant, by design. The arrival that completes it is decided by a conditional claim rather than by a read-and-check, because a read-and-check is the same race one layer up. This is where the engine has been wrong before, and both bugs are pinned by tests that run ten branches finishing together, five times over.
Waiting costs nothing
wf_wait · wf_timerAn approval, an inbound signal, a delay, an interval: each is a row with a status or a time on it. No worker is parked, no thread is held, no memory is pinned. A run can wait days for a person and survive every replica restarting in the meantime — and the answer can land on whichever replica the person happens to reach.
Scaling out is running more of them
shared queue, no leaderEvery replica runs the same loop against the same database. There is no coordinator to elect and no shard to assign: they compete for events, and the claim decides. A schedule's occurrence is claimed the same way, so four replicas do not mean four runs at 07:00.
Secrets are not in the graph
AES-256-GCM at restA connector node names a credential; the credential holds the value, sealed, and no endpoint returns it — not to the UI, not to the API, not to an agent. That is what lets the same graph be exported, reviewed in a pull request and imported into another deployment without a secret travelling with it.
Every run is a document, and it stays one
A run keeps its definition, its per-task state, its result, its log and its event ledger. That is what makes 'what did this actually do' answerable a week later rather than a matter of reconstruction — and what makes the figure on a published page traceable to the execution that produced it.
- The graph as it was when the run started, not as it is now
- Per-task status, repetition count, last result and error
- The event ledger, from which the run's statistics are counted rather than accumulated
- The log, with the widgets a graph published replayed into it
- Records a graph wrote carry the run id, and artifacts carry the run that filed them
What this design costs, said out loud
- At-least-once delivery
- A node that dies between doing its work and recording it will run again. Idempotency is therefore the author's job where it matters, and the catalog's connector nodes are written to make it easy — upserts rather than inserts, named artifacts rather than appended ones. The engine will not pretend to exactly-once semantics it cannot provide.
- The database is the floor
- Throughput is bounded by SurrealDB, because every event, arrival, wait and timer is a write. That is a deliberate trade: the failure modes it removes are worth more than the ceiling it imposes for the work this engine is for. It is not the right engine for a million events a second.
- An `all` join trusts the author
- A join that lists two branches only one of which ever fires waits for the other for ever. The engine cannot tell those apart from two branches that both fire slowly, so it does not guess — the run finishes with the branch below the join not having run, and the task's state says so.
- Sandboxed, not isolated
- A JavaScript node runs in a Bun worker with a timeout, which contains a mistake and an infinite loop. It is not a security boundary against a hostile author, and the deployment's own controls — who may write a graph — are what stand in that place.
- Local files are off by default
- The file nodes are confined to a configured root, and with that root unset they are disabled rather than unrestricted. A workflow engine whose graph authors can read
/etc/shadowby typing a path is not a feature anybody asked for.
What an operator gets
- Deploy
- Docker Swarm. A self-contained single-host stack, or the cluster delta that puts it on djinious-core's shared SurrealDB and object store.
- Scale
- More replicas of the same image. They share the queue; nothing is pinned to a node and no replica is special.
- Roles
- admin, user, viewer — checked next to the route handlers, mirrored in the UI, never only in the UI.
- Tokens
- Scoped
djwf_tokens for scripts, CI and agents. Minted and revoked per consumer. - Object storage
- Optional. With no S3 endpoint configured, artifact bytes stay in the database and everything still works.
- Outbound HTTP
- An allow-list, when you want one:
WORKFLOW_HTTP_ALLOWLISTbounds where a graph may call.
Ask us the awkward operational question
What happens when a worker dies mid-fan-out, what a rejected approval leaves behind, how a schedule behaves with four replicas. We would rather answer those in a demo than in week three.