API reference › @evolu/common › local‑first/Storage › BaseSqliteStorage

Defined in: packages/common/src/local-first/Storage.ts:560

Common interface for both client and relay SQLite storages.

Evolu uses a Skiplist, which leverages SQLite indexes. It runs in Evolu's SQLite core as SQL functions, such as evolu_skiplist_insert, which Evolu's SQLite drivers register on every connection, so a query walks the Skiplist without a roundtrip per node. A SqliteDriver without them fails with "no such function".

The Skiplist data structure is well explained in this Stack Overflow answer. The logic resembles Negentropy's C++ storage, except we use a Skiplist to leverage SQLite indexes, which makes the code simpler.

The ideal storage for a Relay should use an architecture like strfry (a KV storage), but with Skiplist to ensure that insertion order doesn't matter (local-first apps can often write in the past.)

Scaling

The load can be distributed by deploying multiple relays, synchronized with each other, if necessary. One relay should handle hundreds of thousands of users, and when it goes down, nothing happens, because it will be synchronized later.

Extends

Properties

deleteOwner

readonly deleteOwner: (ownerId: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">) => void;

Defined in: packages/common/src/local-first/Storage.ts:588

Deletes the owner's timestamps and usage.


findLowerBound

readonly findLowerBound: (ownerId: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">, begin: number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">, end: number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">, upperBound: RangeUpperBound) => number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">;

Defined in: packages/common/src/local-first/Storage.ts:190

Returns the index of the owner's first timestamp not below upperBound, counted from the owner's first timestamp, or end when there is none or upperBound is InfiniteUpperBound.

begin must be 0 or the result for an earlier bound not above upperBound, and end the owner's size, so the result lies in [begin, end]. When end is 0 or equals begin, the result is end. Sync meets this because the protocol rejects range upper bounds that decrease.

Inherited from

Storage.findLowerBound


fingerprint

readonly fingerprint: (ownerId: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">, begin: number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">, end: number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">) => Fingerprint;

Defined in: packages/common/src/local-first/Storage.ts:155

Returns the Fingerprint of the owner's timestamps from index begin up to, but not including, end, where begin is at most end.

Inherited from

Storage.fingerprint


fingerprintRanges

readonly fingerprintRanges: (ownerId: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">, buckets: readonly [number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">, number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">], upperBound?: RangeUpperBound) => readonly FingerprintRange[];

Defined in: packages/common/src/local-first/Storage.ts:174

Computes fingerprints with their upper bounds in one call.

Each bucket is the end index of a range that starts at the previous bucket, or at 0 for the first. There is at least one, they never decrease, and none exceeds the owner's size, which only the last may equal: a range's upper bound is the timestamp at its bucket, and the last range's is upperBound.

This function can be replaced with many fingerprint/findLowerBound calls, but implementations can leverage it for batching and more efficient fingerprint computation.

Inherited from

Storage.fingerprintRanges


getExistingTimestamps

readonly getExistingTimestamps: (ownerIdBytes: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">, timestampsBytes: readonly [Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"TimestampBytes">, Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"TimestampBytes">]) => readonly (Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"TimestampBytes">)[];

Defined in: packages/common/src/local-first/Storage.ts:582

Efficiently checks which timestamps already exist in the database using a single CTE query instead of N individual queries.


getSize

readonly getSize: (ownerId: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">) => number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">;

Defined in: packages/common/src/local-first/Storage.ts:149

Returns the number of the owner's timestamps.

Inherited from

Storage.getSize


insertTimestamp

readonly insertTimestamp: (ownerId: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">, timestamp: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"TimestampBytes">) => boolean;

Defined in: packages/common/src/local-first/Storage.ts:573

Inserts a timestamp for an owner into the skiplist-based storage.

Returns whether the timestamp was new. An existing timestamp is left unchanged.

It must run in a transaction, which the caller rolls back when it throws, because a failed insert can leave part of its changes.


iterate

readonly iterate: (ownerId: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"IdBytes"> & Brand<"OwnerIdBytes">, begin: number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">, end: number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">, callback: (timestamp: Uint8Array<ArrayBufferLike> & Brand<"Length16"> & Brand<"TimestampBytes">, index: number & Brand<"NonNaN"> & Brand<"Finite"> & Brand<"Int"> & Brand<"NonNegative">) => boolean) => void;

Defined in: packages/common/src/local-first/Storage.ts:204

Calls callback with the owner's timestamps and their indexes from begin up to, but not including, end, where begin is at most end, until the callback returns false. No timestamp after the one it stops at is read. The callback can read, but must not write, the storage's database, as Sqlite.iterate requires.

Inherited from

Storage.iterate