API Reference / @evolu/common / Evolu/Protocol
Evolu/Protocol
Evolu Protocol
Evolu Protocol is a local-first, end-to-end encrypted binary synchronization protocol optimized for minimal size and maximum speed. It enables data sync between a client and a relay, clients in a peer-to-peer (P2P) setup, or relays with each other.
Evolu Protocol is designed for SQLite but can be extended to any database. It implements Range-Based Set Reconciliation by Aljoscha Meyer.
To learn how RBSR works, check Negentropy. Evolu Protocol is similar to Negentropy but uses different encoding and also provides data transfer and ownership.
Message Structure
| Field | Notes |
|---|---|
| Header | |
| - protocolVersion | |
| - OwnerId | |
| - ProtocolErrorCode | In non-initiator response. |
| Messages | |
| - NonNegativeInt | A number of messages. |
| - EncryptedCrdtMessage | |
| - WriteKey | In initiator request. |
| Ranges | |
| - NonNegativeInt | Number of ranges. |
| - Range |
Every protocol message belongs to an owner.
Synchronization
- Messages: Sends EncryptedCrdtMessages in either direction.
- Ranges: Determines messages to sync. Usage varies by transport—e.g., sent only on WebSocket connection open or with every fetch request.
Synchronization involves an initiator and a non-initiator. The initiator
is typically a client, and the non-initiator is typically a relay. Each
side processes the received message and responds with a new ProtocolMessage
if further sync is needed or possible, continuing until both sides are
synchronized.
Both Messages and Ranges are optional, allowing each side to send, sync, or only subscribe data as needed.
When the initiator sends data, the WriteKey is required in
Messages as a secure token proving the initiator can write changes. The
non-initiator responds without a WriteKey, since the initiator’s
request already signals it wants data. If the non-initiator detects an issue
(e.g., an invalid WriteKey causing a ProtocolWriteKeyError, or
a write failure causing a ProtocolWriteError), it sends an error code
via the Error field in the header back to the initiator. In relay-to-relay
or P2P sync, both sides may require the WriteKey depending on who is
the initiator.
Message Size Limit
The protocol enforces a strict maximum size for all messages, defined by
maxProtocolMessageSize. This ensures every ProtocolMessage is less
than or equal to this limit, eliminating the need for applications to
fragment and reconstruct messages during transmission.
Why Binary?
The protocol avoids JSON because:
- Encrypted data doesn’t compress well, unlike plain JSON.
- Message size must be controlled during creation.
- Sequential byte reading is faster than parsing and can avoid conversions.
It uses structure-aware encoding, significantly outperforming generic binary serialization formats with the following optimizations:
- NonNegativeInt: Up to 33% smaller than MessagePack.
- Base64Url Strings: Up to 25% size reduction.
- DateIso: Up to 75% smaller.
- Timestamp Encoding: Delta encoding for milliseconds and run-length encoding (RLE) for counters and NodeIds.
- Small Integers (0 to 19): Reduces size by 1 byte per integer.
To avoid reinventing serialization where it’s unnecessary—like for JSON and certain numbers—the Evolu Protocol relies on MessagePack.
Versioning
The initiator sends a versioned ProtocolMessage. If the non-initiator uses
a different version, it responds with a message containing only its protocol
version—without an ownerId. This allows the initiator to check protocol
compatibility, for example, by sending version-only messages to multiple
relays before starting synchronization.
Storages
TODO: Explain Evolu Protocol Storages.
Interfaces
| Interface | Description |
|---|---|
| ApplyProtocolMessageAsClientOptions | - |
| ApplyProtocolMessageAsRelayOptions | - |
| CrdtMessage | A CRDT message that combines a unique Timestamp with a DbChange. |
| DbChange | A DbChange is a change to a table row. Together with a unique Timestamp, it forms a CrdtMessage. |
| EncryptedCrdtMessage | An encrypted CrdtMessage. |
| FingerprintRange | - |
| ProtocolInvalidDataError | Error for invalid or corrupted protocol message data. |
| ProtocolMessageBuffer | Mutable builder for constructing ProtocolMessage respecting size limits. |
| ProtocolSyncError | Error indicating a synchronization failure during the protocol exchange. Used for unexpected or generic sync errors not covered by other error types. |
| ProtocolUnsupportedVersionError | Represents a version mismatch in the Evolu Protocol. Occurs when the initiator and non-initiator are using incompatible protocol versions. |
| ProtocolWriteError | Error when a write fails due to storage limits or billing requirements. Indicates the need to expand capacity or resolve payment issues. |
| ProtocolWriteKeyError | Error when a WriteKey is invalid, missing, or fails validation. |
| SkipRange | - |
| Storage | Evolu Protocol Storage |
| StorageDep | - |
| TimestampsBuffer | - |
| TimestampsRange | - |
| TimestampsRangeWithTimestampsBuffer | - |
Type Aliases
| Type Alias | Description |
|---|---|
| Base64Url256 | - |
| Base64Url256Variant | Union type for all variants of Base64Url strings with limited length. All these types use Base64Url alphabet and are < 256 characters. |
| BinaryId | Binary representation of Id. |
| BinaryOwnerId | Binary representation of OwnerId. |
| ColumnName | - |
| DbIdentifier | - |
| EncryptedDbChange | Encrypted DbChange |
| Fingerprint | A cryptographic hash used for efficiently comparing collections of BinaryTimestamps. |
| InfiniteUpperBound | - |
| ProtocolError | - |
| ProtocolMessage | Evolu Protocol Message. |
| Range | - |
| RangeType | - |
| RangeUpperBound | Union type for Range's upperBound: either a BinaryTimestamp or InfiniteUpperBound. |
| TableName | - |
Variables
| Variable | Description |
|---|---|
| Base64Url256 | Base64Url string with maximum length of 256 characters. Encoding strings as Base64UrlString saves up to 25% in size compared to regular strings. |
| binaryIdLength | - |
| ColumnName | - |
| DbIdentifier | DbIdentifier is used for database (tables and columns) names. It enforces that the string length is at least 1 character and does not exceed 42. |
| decodeLength | - |
| fingerprintSize | - |
| InfiniteUpperBound | - |
| maxProtocolMessageRangesSize | Maximum size of the ranges portion (timestamps, types, and payloads) in bytes (50 KB). |
| maxProtocolMessageSize | Maximum size of the entire protocol message (header, messages, and ranges) in bytes (1 MB). |
| ProtocolErrorCode | - |
| ProtocolValueType | - |
| protocolVersion | Evolu Protocol version. |
| RangeType | - |
| TableName | - |
| zeroFingerprint | A fingerprint of an empty range. |