API reference › @evolu/common › local‑first/Protocol
Evolu Protocol
Evolu Protocol is a local-first, end-to-end encrypted binary synchronization protocol optimized for minimal size and maximum speed. It enables data sync between a client and a relay. In the future, direct peer-to-peer (P2P) sync between clients will be possible without a relay.
Relays don't need to sync with each other—clients using those relays will sync them eventually. If a relay is offline (e.g., for maintenance), it will sync automatically later via client sync logic. For relay backup using SQLite, see https://sqlite.org/rsync.html (uses a similar algorithm to Evolu RBSR).
Evolu Protocol is designed for SQLite but can be extended to any database. It implements Range-Based Set Reconciliation. To learn how RBSR works, check Negentropy. Evolu Protocol is similar to Negentropy but uses different encoding and also provides data transfer, ownership, real-time broadcasting, request-response semantics, and error handling.
Message structure
| Field | Notes |
|---|---|
| Header | |
| - protocolVersion | |
| - OwnerId | Owner |
| - messageType | MessageType |
| Request (messageType=0) | |
| - hasWriteKey | 0 = no, 1 = yes |
| - OwnerWriteKey | If hasWriteKey = 1 |
| - subscriptionFlag | SubscriptionFlags |
| Response (messageType=1) | |
| - ProtocolErrorCode | |
| Broadcast (messageType=2) | |
| - (no additional fields) | |
| Messages | |
| - NonNegativeInt | A number of messages. |
| - EncryptedCrdtMessage | |
| Ranges | |
| - NonNegativeInt | Number of ranges. |
| - Range |
Range upper bounds must not decrease. The last range always has InfiniteUpperBound, which is not encoded. A message with decreasing bounds, or with a hasWriteKey or subscriptionFlag value not listed above, is rejected as ProtocolInvalidDataError. So is a request with a change shorter than 41 bytes, the smallest EncryptedDbChange. A client passes such a change to storage, which skips a change it cannot read.
WriteKey validation
The initiator sends a hasWriteKey flag and optionally a WriteKey. The WriteKey is required when sending messages as a secure token proving the initiator can write changes. It's ok to not send a WriteKey if the initiator is only syncing (read-only) and not sending messages. The non-initiator decodes the whole request first and then validates the WriteKey, before storing any messages or reconciling ranges. Only the subscriptionFlag takes effect before that, because subscribing needs no WriteKey: broadcasts carry only encrypted changes. So a request rejected with ProtocolWriteKeyError still changes its subscription.
Synchronization
- Messages: Sends EncryptedCrdtMessages in either direction.
- Ranges: Determines messages to sync. Usage varies by transport—e.g., sent only on WebSocket connection open or with every fetch request.
Synchronization involves an initiator and a non-initiator. The initiator
is typically a client, and the non-initiator is typically a relay. Each
side processes the received message and responds with a new ProtocolMessage
if further sync is needed or possible, continuing until both sides are
synchronized.
The non-initiator answers every request it can decode to provide sync
completion feedback, even with empty messages containing only the header and
no error. This allows the initiator to detect when synchronization is
complete. A request it cannot decode, or one larger than the relay's
totalMaxSize, gets no answer and has no effect.
Ranges are compared by their Fingerprint, which anyone who can write for an owner can make collide. Its documentation describes the consequences.
Both Messages and Ranges are optional, allowing each side to send, sync, or only subscribe data as needed.
When the initiator sends data, the OwnerWriteKey is required as a
secure token proving the initiator can write changes. The non-initiator
responds without a OwnerWriteKey, since the initiator’s request
already signals it wants data. If the non-initiator detects an issue, it
sends an error code via the Error field in the header back to the
initiator. In relay-to-relay or P2P sync, both sides may require the
OwnerWriteKey depending on who is the initiator.
Protocol errors
A Response carries a ProtocolErrorCode in its header. The initiator
reports every code except NoError as an error with the OwnerId:
- ProtocolWriteKeyError: The WriteKey is invalid, or missing from a request with messages.
- ProtocolWriteError: The relay failed to store the messages or to validate the WriteKey.
- ProtocolQuotaError: Storage or billing quota exceeded.
- ProtocolSyncError: The relay's storage failed while it reconciled the ranges.
The initiator also reports ProtocolVersionError, with the OwnerId,
for a reply of another version, and ProtocolInvalidDataError, without
it, for a message it cannot decode, including one with an unknown code.
A relay answers every request it can decode within its totalMaxSize. It
decodes a whole request before acting on it, so for a malformed one it
returns ProtocolInvalidDataError without subscribing, storing, or
broadcasting anything, and sends no reply. When its storage throws while
validating the WriteKey, for example because SQLite cannot store a new
owner's key on a full disk, the relay logs the error and answers with
ProtocolWriteError, like a failed write. Likewise, a client applies nothing
from a message it cannot decode.
decryptAndDecodeDbChange returns ProtocolInvalidDataError,
ProtocolTimestampMismatchError, or
DecryptWithXChaCha20Poly1305Error for a change it cannot read. These
describe a change rather than a protocol message, and client storage skips
such a change.
Message size limit
The protocol enforces a strict maximum size for all messages, defined by ProtocolMessageMaxSize. This ensures every ProtocolMessage is less than or equal to this limit, enabling stateless transports, simplified relay implementation, and predictable memory usage. When all messages don't fit within the limit, the protocol automatically continues synchronization in subsequent rounds using range-based reconciliation.
Each mutation is limited to maxMutationSize, so every change fits one message of defaultProtocolMessageMaxSize next to the largest ranges section. Changes saved before that limit existed, and crafted changes a relay stores, can be larger. Sync skips a stored change that cannot fit an empty message after a pending Skip range, which is the message a later round is guaranteed to reach, and reports it as a ProtocolChangeTooLargeError.
Why Binary?
The protocol avoids JSON because:
- Encrypted data doesn’t compress well, unlike plain JSON.
- Message size must be controlled during creation.
- Sequential byte reading is faster than parsing and avoids conversions.
It uses structure-aware encoding, significantly outperforming generic binary serialization formats with the following optimizations:
- NonNegativeInt: Up to 33% smaller than MessagePack.
- DateIso: Up to 75% smaller.
- Timestamp Encoding: Delta encoding for milliseconds and run-length encoding (RLE) for counters and NodeIds.
- Small Integers (0 to 19): Reduces size by 1 byte per integer.
To avoid reinventing serialization where it’s unnecessary—like for JSON and certain numbers—the Evolu Protocol relies on MessagePack.
Versioning
Every message of every protocol version begins with the version and the
OwnerId, so a peer can route and report a message of any version. Nothing
negotiates a version, and no side falls back to the other's.
A non-initiator answers a request of another version with only its own
version and that OwnerId. The initiator reports it as
ProtocolVersionError for that owner, whose isInitiator tells which
side is older, and stops syncing the owner through that relay. Clients from
@evolu/common 8.0.0 before 8.11.0 drop that reply and stop syncing the
owner through that relay without reporting anything, while 7.x clients report
it as a ProtocolVersionError. So a relay that moves to another version must
keep answering version 1 while such clients remain.
Credible exit
The protocol specification is intentionally non-configurable to ensure universal compatibility. This design allows applications (users) to switch between any compliant relay without negotiation or compatibility checks beyond version matching. Relays are generic infrastructure that any application can use interchangeably making exit from any single provider technically feasible and economically viable.
Ranges
| Interface | Description |
|---|---|
| TimestampsRangeWithTimestampsBuffer | Common shape of every Range. |
Functions
Interfaces
| Interface | Description |
|---|---|
| ApplyProtocolMessageAsClientBroadcast | Result of applyProtocolMessageAsClient: a Broadcast message whose messages were written. |
| ApplyProtocolMessageAsClientConverged | Result of applyProtocolMessageAsClient: the response needed nothing more, because its ranges all matched or it carried none. |
| ApplyProtocolMessageAsClientFailed | Result of applyProtocolMessageAsClient: the protocol logged an exception thrown by calling the storage's writeMessages (Write) or a failed range reconciliation (Sync) to the console. An exception while the returned Task runs is a defect that aborts the Run instead. A rejected or failed write, which the built-in storages return when SQLite fails it, gives the original StorageWriteMessagesError through Result. Reconciliation can fail after messages have been committed; that failure does not roll back the write. |
| ApplyProtocolMessageAsClientOptions | - |
| ApplyProtocolMessageAsClientReadonly | Result of applyProtocolMessageAsClient: the messages were written, but without a write key no ranges are reconciled. |
| ApplyProtocolMessageAsClientResponse | Result of applyProtocolMessageAsClient: a continuation for the non-initiator. |
| ApplyProtocolMessageAsRelayOptions | Options for applyProtocolMessageAsRelay. The callbacks must not throw, because a throw is a defect that aborts the Run. |
| ApplyProtocolMessageAsRelayResult | Result type for applyProtocolMessageAsRelay. |
| ProtocolChangeTooLargeError | Error for a stored change that sync skipped because it cannot fit an empty ProtocolMessage after a pending Skip range, which is the message a later round is guaranteed to reach. |
| ProtocolHeader | The routing prefix of a ProtocolMessage. |
| ProtocolInvalidDataError | Error for a malformed ProtocolMessage or EncryptedDbChange, with its bytes as data and the decoding error as error. |
| ProtocolMessageBuffer | Mutable builder for constructing ProtocolMessage respecting size limits. |
| ProtocolQuotaError | Error when storage or billing quota is exceeded. |
| ProtocolSyncError | Error indicating a serious relay-side synchronization failure. Sync state shows it as the failure of that relay's route; apps show a generic sync error for the owner's Error status. |
| ProtocolTimestampMismatchError | Error when embedded timestamp doesn't match expected timestamp in EncryptedDbChange. Indicates potential tampering or corruption of CRDT messages. |
| ProtocolVersionError | Represents a version mismatch in the Evolu Protocol. Occurs when the initiator and non-initiator are using incompatible protocol versions. |
| ProtocolWriteError | Error indicating a serious relay-side write failure. Sync state shows it as the failure of that relay's route; apps show a generic sync error for the owner's Error status. |
| ProtocolWriteKeyError | Error when a OwnerWriteKey is invalid, missing, or fails validation. |
| TimestampsBuffer | - |
Type Aliases
| Type Alias | Description |
|---|---|
| ApplyProtocolMessageAsClientResult | - |
| MessageType | - |
| ProtocolError | - |
| ProtocolErrorCode | The error code in the header of a Response. |
| ProtocolMessage | Evolu Protocol Message. |
| ProtocolMessageMaxSize | Protocol message maximum size, from 1MB to 100MB. |
| ProtocolMessageRangesMaxSize | Protocol message ranges maximum size. |
| SubscriptionFlag | - |
Variables
| Variable | Description |
|---|---|
| defaultProtocolMessageMaxSize | Default ProtocolMessageMaxSize (1MB). |
| defaultProtocolMessageRangesMaxSize | Default ProtocolMessageRangesMaxSize (30KB). |
| MessageType | - |
| ProtocolErrorCode | The error code in the header of a Response. |
| ProtocolMessageMaxSize | Protocol message maximum size, from 1MB to 100MB. |
| ProtocolMessageRangesMaxSize | Protocol message ranges maximum size. |
| ProtocolValueType | - |
| protocolVersion | Evolu Protocol version. |
| SubscriptionFlags | - |