Backends & clustering

A backend decides where collection state lives, how durable it is, and how it replicates across a cluster. Backends are drop-in — swapping one is a one-line change; nothing above it moves.

There are two backend seams, because the two consistency models are stored differently:

• collections: — the row collection backend (one per server, a single transaction domain → atomic cross-collection batches).
• crdtCollections: — the CRDT document collection backend (separate; CRDT never joins a cross-collection atomic batch).

Just want to pick one?

This page is the reference — every capability, every option, every limit. For the decision itself, read choose a collection backend first; it gets you to a name in two questions.

The axes

A row backend is described by three properties. The first is obvious; the other two are declared on the backend itself and decide what it can promise you across nodes.

Durability

Whether state outlives the process: in-memory, a local file, or a central database.

Clustering: relay vs. self

• relay — each node holds a full replica and replicates writes over the server↔server adapter (Redis/libp2p/RabbitMQ/ZeroMQ). Simple, adapter-driven.
• self — the backend owns a central Postgres and a per-node Electric-synced replica, and needs no adapter at all. Cross-node convergence rides Postgres + Electric instead of super-line's fan-out.

Don't confuse self clustering with the transport or the adapter — a self backend bypasses the adapter entirely.

Coordination: local vs. cluster

Clustering says how a write reaches other nodes. Coordination says whether the cluster can agree at the moment of the write — which is a stronger and separate guarantee:

• coordination: 'local' (memory · sqlite · cloudflare) — a conditional write is checked against this node's rows. Two nodes can each find the same condition satisfied and both proceed.
• coordination: 'cluster' (pglite) — conditional writes are serialized through the central Postgres, so exactly one node wins a contested claim.

Adding an adapter does not upgrade local to cluster: the adapter carries fan-out, not serialization. This distinction is what makes plugin-queue single-node on memory and sqlite, and cluster-wide on pglite.

Row backends

	collections-memory	collections-sqlite	collections-pglite	collections-cloudflare
Durability	process memory	SQLite file (WAL)	central Postgres	the Durable Object's SQLite
clustering	relay	relay	self	relay
coordination	local	local	cluster	local
Survives a restart	no	yes	yes	yes — and every wake
Converges across nodes	via adapter	via adapter	via Electric	— (one node)
Cluster-wide conditional writes	no	no	yes	— (one node)
A new node sees pre-existing state	no	its own file only	yes	— (one node)
External infrastructure	none	a writable path	Postgres + Electric	a SQLite-backed Durable Object
Factory	sync	sync	async	sync
Needs the contract collections map	no	yes	yes	yes
Query execution	JS evaluator	SQL when compilable, else JS	SQL narrowing, JS authoritative	as sqlite
Row timestamps in Control Center	yes	yes	yes	yes

// swap the backend — nothing else changes
import { sqliteCollections } from '@super-line/collections-sqlite'
createSuperLineServer(api, { /* … */, collections: sqliteCollections({ file: './data.db', collections: api.collections }) })

memoryCollections()

Takes no arguments. Holds every collection's rows in nested maps and applies batches atomically through an undo log, so a failed batch leaves nothing behind. Zero dependencies.

sqliteCollections(options)

Synchronous. Sets journal_mode = WAL and synchronous = NORMAL.

Option	Type	Notes
file	string	Path to the database file. :memory: for an ephemeral store.
collections	Record<string, CollectionDef>	Required. The contract's post-plugin-merge collections map — each LWW collection gets its own typed table.

pgliteCollections(options)

Async — await it. Writes and strong reads go to the central Postgres; change delivery arrives through this node's Electric-fed local replica.

Option	Type	Notes
pgUrl	string	Required. Central Postgres — source of truth for writes and strong reads.
electricUrl	string?	The Electric shape endpoint streaming each table into this node's replica. Required in practice: without a fed replica onChange never fires. Omit only if you pass db and feed the replica yourself.
collections	Record<string, CollectionDef>	Required. As above.
tablePrefix	string?	Prefix for the per-collection tables and the <prefix>meta fingerprint table. Defaults to col_.
db	PGliteWithLive?	Advanced/testing: supply the local replica yourself (needs the live extension).

Postgres must run with wal_level=logical for Electric to replicate.

cloudflareCollections(storage, collections)

Synchronous, positional, and only usable inside a SQLite-backed Durable Object — the node of a Cloudflare deployment. It lays tables out exactly as sqliteCollections does and pushes the same queries to SQL.

Argument	Type	Notes
storage	DurableObjectStorage	Required. The object's ctx.storage.
collections	Record<string, CollectionDef>	Required. As above.

It opens lazily, on the first operation, so building it in the object's constructor never throws. A collection that can't open — its schema changed non-additively, or it plans more than 98 typed columns — fails only its own operations with an INTERNAL error naming it and the fix; additive drift is altered in as with SQLite. Every row must fit the platform's 2 MB, and a snapshot filter needing more than 98 bound values scans and filters in JS. See change a schema on a deployed object.

remoteRows(options)

Also in @super-line/collections-cloudflare: one read-only collection serving the tables of other Durable Objects — sources whose tables are created at runtime by arbitrary SQL — to the node's subscribers. Each source wraps its SQLite in syncedSql(ctx), which captures every row change with triggers and pushes each committed batch to the node; the contract declares the collection with remoteRowsCollection(name) from the /contract subpath, as rows { key, storeId, table, id, data } with each row's columns opaque under data. It declares clustering: 'self', because its changes arrive from the sources rather than from super-line's fan-out, and it serves alongside the node's own backend through composeCollections({ default, [name]: rows }) from @super-line/server. A subscription must pin storeId and table with eq(), writes fail with FORBIDDEN, and a source whose history breaks (a dropped or altered table, a reset) sends its subscribers back for a fresh snapshot. Its changes carry no prior row, so it declares storeId and table as fixed fields (fixedFields) — fields whose value never changes for a key: an insert or update then reaches only the subscribers whose filter and policy admit that store and table, while a delete reaches every subscriber of the collection as a key-only delete. See serve another Durable Object's tables.

Option	Type	Notes
collection	string	Required. The name given to remoteRowsCollection.
storage	RemoteRowsStorage	Required. The node's ctx.storage — where the last clock applied per source is kept.
stub	(storeId) => RemoteSource	Required. The source object for a store id.
onWipe	(storeId) => unknown	A source's history broke; close its readers' sockets (4409) so they resubscribe. Default: nothing.
snapshotLimit	number?	The most rows a snapshot reads from the source table. Default 1000. Live changes aren't capped, but every snapshot is, including a resubscribe's, so a subscription to a table over the limit can shrink on any reconnect.

Typed tables

The SQL backends store each LWW collection in its own typed table (col_<name>): scalar schema fields become real columns, everything else a per-field JSON column, derived from the contract — so every SQL factory takes the contract's collections map. collections-cloudflare runs the same table layout and query compiler as collections-sqlite. collections-sqlite compiles the query IR against those columns (an exactly-compilable query runs entirely in SQL, and so do count's COUNT(*) and countBy's GROUP BY); collections-pglite is the self-clustering tier (central Postgres + one Electric shape per table streaming into each node's replica).

Which validators get typed columns

super-line reads a schema's shape through Standard JSON Schema, the companion spec to Standard Schema — so it never assumes a particular library and every implementer plans identically. That covers Zod 4.2+, Zod Mini 4.2+, ArkType 2.1.28+, VineJS 4.3+, Sury 11+, stnl 2.1+, and Valibot 1.2+ (wrap the schema in toStandardJsonSchema() from @valibot/to-json-schema).

A schema whose shape can't be read as one object still works — its collection falls back to the key column plus one _sl_data JSON column. That happens when the validator's library hasn't implemented the companion spec, or when the root isn't an object (a union, such as z.discriminatedUnion). Everything is still validated, stored and queried correctly; you only lose SQL-level column typing and pushdown. The same fallback applies per-field to anything JSON Schema can't express, such as a .transform(). A root .meta({ id }) doesn't count: it plans the same columns as the object without one.

What keeps a query out of SQL

An exactly-compilable query runs entirely in SQL: the filter, the orderBy on number and text columns, and limit/offset. That covers comparisons on scalar columns (text ranges such as ISO dates included), in, like (as GLOB), ilike with an ASCII pattern (as LIKE) and has on a top-level array field (through json_each). A few things SQLite would answer differently from the JS evaluator, so collections-sqlite uses SQL only to narrow the scan, then filters, sorts and windows the rest in JS:

• like with _, or ilike with a non-ASCII character — _ is one UTF-16 code unit to the evaluator but one character to SQLite, and SQLite's LIKE folds ASCII case only.
• A filter on a JSON-backed field — a non-scalar schema field (record/union/nested/array/optional+nullable) or a dot path, except has on a top-level array. neq there isn't used even to narrow: SQL's 1/0 and true/false collide under IS NOT.
• orderBy on a boolean or JSON-backed field.

None of this affects correctness — the JS evaluator stays authoritative either way — but it's a perf cliff: the snapshot reads every row the narrowed WHERE admits (the whole table, if nothing narrows it) and sorts them in JS, and count/countBy count that snapshot instead of running COUNT(*)/GROUP BY. countBy on a dot-path field always counts a snapshot.

collections-pglite works differently: it compiles a superset-safe WHERE to narrow what it reads from Postgres, then always re-applies the full query in JS. The SQL is an optimization that never has to be exact, so there is no equivalent cliff — but also no full pushdown of sorting or limits.

CRDT backends

	collections-crdt-memory	collections-crdt-libsql	collections-crdt-pglite	collections-crdt-cloudflare
Durability	process memory	libsql / Turso, snapshot per doc	central Postgres Yjs op-log	the Durable Object's SQLite, snapshot per doc
clustering	relay	relay	self	relay
Survives a restart	no	yes	yes	yes — and every wake
Converges across nodes	via adapter	via adapter	via Electric	— (one node)
External infrastructure	none	a file, or Turso/sqld	Postgres + Electric	a SQLite-backed Durable Object
Factory	sync	async	async	sync
Boot cost	none	rehydrates every document	replica sync	none — each document loads on first touch
Write persistence	—	debounced snapshot	op-log append	debounced snapshot

CRDT backends expose clustering but no coordination flag. There are no conditional writes to serialize — merge is the conflict-resolution model — so the axis doesn't apply.

Validate-before-commit isn't a backend axis either: all four behave identically, validating at the ingress node while relay nodes trust an already-checked delta. Whether a document is validated at all is declared on the contract per collection — crdt: { validate: false } makes the server pass no validator, so the backend skips the merge-and-check entirely. See turning ingress validation off.

collections-crdt-memory also exports the universal crdtCollectionsClient() — one client engine pairs with every backend tier (the client only merges opaque deltas).

crdtMemoryCollections()

Takes no arguments. The Yjs merge engine every other tier wraps or mirrors.

crdtLibsqlCollections(options)

Async — it rehydrates every document (history-preserving) before returning a ready backend, so boot time and memory both scale with total document count. Persistence is a debounced onChange subscriber, keeping the apply hot path synchronous.

Option	Type	Notes
url	string	Required. file:x.db, :memory:, libsql:// (Turso), or http(s):// (sqld).
authToken	string?	Auth token for Turso Cloud.
table	string?	Table this backend owns. Defaults to crdt_docs.
debounceMs	number?	Coalesce rapid edits into one snapshot write. Defaults to 250.
docOptions	(collection: string) => DocOptions | undefined	Needed to rehydrate each document's Yjs state on boot. Derive it from the contract: (n) => contract.collections[n]?.crdt.

A clean close() flushes pending edits; a hard kill loses at most debounceMs of writes.

crdtPgliteCollections(options)

Async. Client writes are validated at the ingress node, then appended as opaque Yjs deltas to an append-only op-log in Postgres, which Electric streams to each node's replica.

Option	Type	Notes
pgUrl	string	Required. Central Postgres — the op-log and existence source of truth.
electricUrl	string?	Shape endpoint. Required in practice: without a fed replica no document change reaches a subscriber.
table	string?	Creates <table> (existence + materialized snapshot) and <table>_updates (the op-log). Defaults to crdt_docs.
docOptions	(collection: string) => DocOptions | undefined	Must agree with the collection's crdt options on the contract.
compact	false | { everyNUpdates?, debounceMs? }	Op-log compaction: fold the log, materialize a baseline, trim superseded rows. Bounds op-log growth. false disables it.
db	PGliteWithLive?	Advanced/testing: supply the local replica yourself.
onError	(err, ctx) => void	Called when a background op-log append fails. Server co-writes are synchronous and can't reject to the caller, so this is the only place that failure is observable. Defaults to console.error.

See validate-before-commit for why op-log compaction demands tolerant schemas.

crdtCloudflareCollections(storage, collections, options?)

Synchronous, and only usable inside a SQLite-backed Durable Object. It wraps crdtMemoryCollections's merge engine — so @super-line/collections-crdt-memory is a peer dependency — and snapshots each document to one row of _sl_crdt in the object's storage.

Argument	Type	Notes
storage	DurableObjectStorage	Required. The object's ctx.storage.
collections	Record<string, CollectionDef>	Required. The contract's collections map — each document is rebuilt with its collection's crdt options.
options.debounceMs	number?	Coalesce rapid edits into one snapshot write. Defaults to 250.

Unlike libsql it rehydrates nothing at boot: a document loads on its first touch, so a wake from hibernation costs nothing until a document is used. A pending write keeps the object awake until it lands, so hibernation never loses an edit; a deploy or crash loses at most debounceMs. A document whose encoded state outgrows the platform's 2 MB row is refused with an INTERNAL error naming it, and storage keeps the last state that fit — split large content across documents.

Cold start & convergence

relay clustering fans out live batches over the adapter. It performs no state transfer to a node that joins — a node starts from whatever its own local store holds and converges only forward from that point:

• memory · crdt-memory — a restarted node comes back empty and never receives anything written before it booted.
• sqlite · crdt-libsql — a restarted node comes back with its own file, missing everything written while it was down.

That is fine on a single node, and fine for state that is naturally re-created (presence, ephemeral sessions, caches). It is a real hazard for long-lived rows across a multi-node relay deployment.

One file per relay node

Don't point two nodes at the same SQLite file on a shared volume. A relay backend expects a node-local store: each node applies every relayed batch to its own copy, so a shared file receives the same batch once per node. If you want a genuinely shared database, that is the self tier.

The self tier has no cold-start gap. Each node's replica is fed from the central Postgres, so a fresh node syncs current state on connect and reads stay strongly consistent against the centre.

Advisory foreign keys

references on a contract collection is metadata; opt into an existence check with checkReferences: true on the server. It's advisory:

• best-effort under relay clustering (no global serialization point),
• no cascades,
• doesn't resolve intra-batch parent-then-child references.

For strict referential integrity, use a self backend (a central Postgres serialization point) or route the write through a request handler that checks explicitly. The metadata still feeds the Control Center schema graph and the TanStack adapter's join hints regardless of whether the check is on.

Next

• Choose a collection backend — the decision guide, if you landed here still undecided.
• Row-level security & policies — how routing respects read filters per backend.
• Run queues across a cluster — plugin-queue stores jobs, schedules and concurrency slots in collections, so the backend you pick here is what decides whether a queue coordinates one node (memory · sqlite) or the whole cluster (pglite).
• Choose an adapter — the server↔server fan-out that relay backends ride.
• Transports vs. adapters — where the self tier sits relative to both.