API reference / @evolu/common / local-first / BaseSqliteStorage
Interface: BaseSqliteStorage
Defined in: packages/common/src/local-first/Storage.ts:332
Common interface for both client and relay SQLite storages.
Evolu uses a Skiplist, which leverages SQLite indexes. The core logic is implemented in SQL, so it doesn't have to make roundtrips to the DB.
While the SQL implementation may look sophisticated, it's conceptually simple and LLMs can explain how it works. The Skiplist data structure is well explained in this Stack Overflow answer. The logic resembles Negentropy's C++ storage, except we use a Skiplist to leverage SQLite indexes, which makes the code simpler.
Note: A paid review by the SQLite team is planned, as they use the same algorithm for their rsync tool.
The ideal storage for a Relay should use an architecture like strfry (a KV storage), but with Skiplist to ensure that insertion order doesn't matter (local-first apps can often write in the past.)
The ideal client implementation should probably use the SQLite extension instead of SQL or even a KV storage, when such a thing for browsers/native will exist and will be faster than SQLite.
Scaling
The load can be distributed by deploying multiple relays, synchronized with each other, if necessary. One relay should handle hundreds of thousands of users, and when it goes down, nothing happens, because it will be synchronized later.
Extends
Pick<Storage, |"getSize"|"fingerprint"|"fingerprintRanges"|"findLowerBound"|"iterate"|"deleteOwner">
Extended by
Properties
| Property | Modifier | Type | Description | Inherited from | Defined in |
|---|---|---|---|---|---|
deleteOwner | readonly | (ownerId) => boolean | Delete all data for the given Owner. Returns true on success, false on failure. | Storage.deleteOwner | packages/common/src/local-first/Storage.ts:167 |
findLowerBound | readonly | (ownerId, begin, end, upperBound) => | number & Brand<"Int"> & Brand<"NonNegative"> | null | - | Storage.findLowerBound | packages/common/src/local-first/Storage.ts:113 |
fingerprint | readonly | (ownerId, begin, end) => | Fingerprint | null | - | Storage.fingerprint | packages/common/src/local-first/Storage.ts:94 |
fingerprintRanges | readonly | (ownerId, buckets, upperBound?) => | readonly FingerprintRange[] | null | Computes fingerprints with their upper bounds in one call. This function can be replaced with many fingerprint/findLowerBound calls, but implementations can leverage it for batching and more efficient fingerprint computation. | Storage.fingerprintRanges | packages/common/src/local-first/Storage.ts:107 |
getExistingTimestamps | readonly | (ownerIdBytes, timestampsBytes) => Result<readonly Uint8Array<ArrayBufferLike> & Brand<"TimestampBytes">[], SqliteError> | Efficiently checks which timestamps already exist in the database using a single CTE query instead of N individual queries. | - | packages/common/src/local-first/Storage.ts:353 |
getSize | readonly | (ownerId) => | number & Brand<"Int"> & Brand<"NonNegative"> | null | - | Storage.getSize | packages/common/src/local-first/Storage.ts:92 |
insertTimestamp | readonly | (ownerId, timestamp, strategy) => Result<void, SqliteError> | Inserts a timestamp for an owner into the skiplist-based storage. | - | packages/common/src/local-first/Storage.ts:343 |
iterate | readonly | (ownerId, begin, end, callback) => void | - | Storage.iterate | packages/common/src/local-first/Storage.ts:120 |