Enable history sync for apps built with XMTP

Enable the history sync feature to give your users a way to sync an archive (decrypted historical data) from an existing app installation to a new app installation.

This historical data includes:

Conversations
Conversation messages
Consent state
HMAC keys (for push notifications)

History sync enables your users pick up conversations where they left off, regardless of the app installation they use. All they need is a pre-existing and online app installation to provide the archive.

Enable history sync

History sync is enabled by default. When creating a client, the historySyncUrl client option is dynamically set to a default history sync server URL based on your env client option setting.

When env is set to dev, the historySyncUrl is set to https://message-history.dev.ephemera.network/
When env is set to production, the historySyncUrl is set to https://message-history.production.ephemera.network

These default servers are managed and operated by XMTP Labs, a steward of the development and adoption of XMTP.

You can choose to run your own server and set the historySyncUrl to your server's URL.

If needed, you can turn off history sync by setting the historySyncUrl client option to an empty string.

How history sync works

When you send a sync request (via sendSyncRequest()) and a historySyncUrl client option is present, the receiving installation creates an encrypted archive.

Diagram showing step 1 of history sync: client initialization triggers sync request and creates encrypted archive

History sync then uploads the archive, sends a sync reply, and pulls all conversation state history into the new app installation, merging it with the existing app installations in the sync group.

Diagram showing step 2 of history sync: archive upload, sync reply, and merging conversation state history across app installations

Ongoing updates to history are streamed automatically. Updates, whether for user consent preferences or messages, are sent across the sync group, ensuring all app installations have up-to-date information.

History syncs are accomplished using these components:

Sync group
Sync worker
History server

Sync group

A sync group is a special Messaging Layer Security group that includes all of the user's devices. The sync group is used to send serialized sync messages across the user's devices. The sync group is filtered out of most queries by default so they don't appear in the user's inbox.

Sync worker

A sync worker is a spawned background worker that listens for sync events, and processes them accordingly. This worker:

Emits and updates consent
Creates and consumes archives for and from other devices
Keeps preferences synced across devices

History server

A history server acts as a bucket that holds encrypted sync archives. The URL location of these archives and the password (cipher encryption key) to decrypt these archives is sent over the sync group for the recipient to decrypt.

Control history sync

Use these methods to manage the sync flow.

Archive options

The following methods accept an ArchiveOptions parameter that controls what data is included in the sync archive:

Option	Default	Description
`startNs`	`nil`/`null`	Start timestamp in nanoseconds to filter the archive to a time range
`endNs`	`nil`/`null`	End timestamp in nanoseconds to filter the archive to a time range
`archiveElements`	`[messages, consent]`	What to include in the archive: `messages`, `consent`, or both
`excludeDisappearingMessages`	`false`	Whether to exclude disappearing messages from the archive

Using ArchiveOptions() with no arguments includes all messages and consent data.

Send a sync request

Send a request to other installations to sync conversations and messages to the current installation. Call this after creating a client on a new installation to pull in historical data.

Browser

await client.sendSyncRequest();

Send a sync archive

Send an archive to the sync group without waiting for a request from another installation. The pin is used as a reference when importing the archive on another installation.

Kotlin

// With default options
client.sendSyncArchive(pin = "my-pin")
 
// With custom options
client.sendSyncArchive(
    pin = "my-pin",
    opts = ArchiveOptions(archiveElements = listOf(ArchiveElement.CONSENT))
)

Sync device sync groups

Sync all device sync groups from the network.

Kotlin

val summary = client.syncAllDeviceSyncGroups()

List available archives

Get a list of archives available for import from the sync group. This method reads from the local database only and does not sync from the network. If you believe your data may be out of date, call syncAllDeviceSyncGroups() first to pull the latest state.

Kotlin

// List archives from the last 30 days
val archives = client.listAvailableArchives(daysCutoff = 30)

Process a sync archive

Process an available archive from the sync group. If no pin is provided, it processes the last archive sent.

Kotlin

// Process a specific archive by pin
client.processSyncArchive(archivePin = "my-pin")
 
// Process the last archive sent
client.processSyncArchive()

FAQ

A user logged into a new app installation and doesn't see their conversations. What's going on?

Ensure that you've called sendSyncRequest() after creating the client on the new installation, and that a pre-existing app installation is online to receive and respond to the request.

A user logged into a new app installation and sees their conversations, but no messages. What's going on?

Ensure that you've initiated a call to sync messages and that the pre-existing app installation is online to receive the sync request, process and encrypt the archive, upload it to the history server, and send a sync reply message to the new app installation.

I called a sync method (messages, consent state, or conversations), but nothing is happening. What's going on?

After calling sendSyncRequest(), ensure that the pre-existing app installation is online to receive the sync request, process and encrypt the archive, and send a sync reply.