Struct MonitorUpdatingPersister

pub struct MonitorUpdatingPersister<K, L, ES, SP>
where
    K: Deref,
    L: Deref,
    ES: Deref,
    SP: Deref,
    <K as Deref>::Target: KVStore,
    <L as Deref>::Target: Logger,
    <ES as Deref>::Target: Sized + EntropySource,
    <SP as Deref>::Target: Sized + SignerProvider,
{ /* private fields */ }

Implements Persist in a way that writes and reads both ChannelMonitors and ChannelMonitorUpdates.

Overview

The main benefit this provides over the KVStore’s Persist implementation is decreased I/O bandwidth and storage churn, at the expense of more IOPS (including listing, reading, and deleting) and complexity. This is because it writes channel monitor differential updates, whereas the other (default) implementation rewrites the entire monitor on each update. For routing nodes, updates can happen many times per second to a channel, and monitors can be tens of megabytes (or more). Updates can be as small as a few hundred bytes.

Note that monitors written with MonitorUpdatingPersister are not backward-compatible with the default KVStore’s Persist implementation. They have a prepended byte sequence, MONITOR_UPDATING_PERSISTER_PREPEND_SENTINEL, applied to prevent deserialization with other persisters. This is because monitors written by this struct may have unapplied updates. In order to downgrade, you must ensure that all updates are applied to the monitor, and remove the sentinel bytes.

Storing monitors

Monitors are stored by implementing the Persist trait, which has two functions:

• Persist::persist_new_channel, which persists whole ChannelMonitors.
• Persist::update_persisted_channel, which persists only a ChannelMonitorUpdate

Whole ChannelMonitors are stored in the CHANNEL_MONITOR_PERSISTENCE_PRIMARY_NAMESPACE, using the familiar encoding of an OutPoint (for example, [SOME-64-CHAR-HEX-STRING]_1).

Each ChannelMonitorUpdate is stored in a dynamic secondary namespace, as follows:

• primary namespace: CHANNEL_MONITOR_UPDATE_PERSISTENCE_PRIMARY_NAMESPACE
• secondary namespace: [the monitor’s encoded outpoint name]

Under that secondary namespace, each update is stored with a number string, like 21, which represents its update_id value.

For example, consider this channel, named for its transaction ID and index, or OutPoint:

• Transaction ID: deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef
• Index: 1

Full channel monitors would be stored at a single key:

[CHANNEL_MONITOR_PERSISTENCE_PRIMARY_NAMESPACE]/deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef_1

Updates would be stored as follows (with / delimiting primary_namespace/secondary_namespace/key):

[CHANNEL_MONITOR_UPDATE_PERSISTENCE_PRIMARY_NAMESPACE]/deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef_1/1
[CHANNEL_MONITOR_UPDATE_PERSISTENCE_PRIMARY_NAMESPACE]/deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef_1/2
[CHANNEL_MONITOR_UPDATE_PERSISTENCE_PRIMARY_NAMESPACE]/deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef_1/3

… and so on.

Reading channel state from storage

Channel state can be reconstructed by calling MonitorUpdatingPersister::read_all_channel_monitors_with_updates. Alternatively, users can list channel monitors themselves and load channels individually using MonitorUpdatingPersister::read_channel_monitor_with_updates.

EXTREMELY IMPORTANT

It is extremely important that your KVStore::read implementation uses the io::ErrorKind::NotFound variant correctly: that is, when a file is not found, and only in that circumstance (not when there is really a permissions error, for example). This is because neither channel monitor reading function lists updates. Instead, either reads the monitor, and using its stored update_id, synthesizes update storage keys, and tries them in sequence until one is not found. All other errors will be bubbled up in the function’s Result.

Pruning stale channel updates

Stale updates are pruned when a full monitor is written. The old monitor is first read, and if that succeeds, updates in the range between the old and new monitors are deleted. The lazy flag is used on the KVStore::remove method, so there are no guarantees that the deletions will complete. However, stale updates are not a problem for data integrity, since updates are only read that are higher than the stored ChannelMonitor’s update_id.

If you have many stale updates stored (such as after a crash with pending lazy deletes), and would like to get rid of them, consider using the MonitorUpdatingPersister::cleanup_stale_updates function.

Implementations

impl<K, L, ES, SP> MonitorUpdatingPersister<K, L, ES, SP>

where K: Deref, L: Deref, ES: Deref, SP: Deref, <K as Deref>::Target: KVStore, <L as Deref>::Target: Logger, <ES as Deref>::Target: Sized + EntropySource, <SP as Deref>::Target: Sized + SignerProvider,

pub fn new( kv_store: K, logger: L, maximum_pending_updates: u64, entropy_source: ES, signer_provider: SP, ) -> MonitorUpdatingPersister<K, L, ES, SP>

Constructs a new MonitorUpdatingPersister.

The maximum_pending_updates parameter controls how many updates may be stored before a MonitorUpdatingPersister consolidates updates by writing a full monitor. Note that consolidation will frequently occur with fewer updates than what you set here; this number is merely the maximum that may be stored. When setting this value, consider that for higher values of maximum_pending_updates:

• MonitorUpdatingPersister will tend to write more ChannelMonitorUpdates than ChannelMonitors, approaching one ChannelMonitor write for every maximum_pending_updates ChannelMonitorUpdates.
• MonitorUpdatingPersister will issue deletes differently. Lazy deletes will come in “waves” for each ChannelMonitor write. A larger maximum_pending_updates means bigger, less frequent “waves.”
• MonitorUpdatingPersister will potentially have more listing to do if you need to run MonitorUpdatingPersister::cleanup_stale_updates.

pub fn read_all_channel_monitors_with_updates<B, F>( &self, broadcaster: &B, fee_estimator: &F, ) -> Result<Vec<(BlockHash, ChannelMonitor<<<SP as Deref>::Target as SignerProvider>::Signer>)>, Error>

where B: Deref, F: Deref, <B as Deref>::Target: BroadcasterInterface, <F as Deref>::Target: FeeEstimator,

Reads all stored channel monitors, along with any stored updates for them.

It is extremely important that your KVStore::read implementation uses the io::ErrorKind::NotFound variant correctly. For more information, please see the documentation for MonitorUpdatingPersister.

pub fn read_channel_monitor_with_updates<B, F>( &self, broadcaster: &B, fee_estimator: &F, monitor_key: String, ) -> Result<(BlockHash, ChannelMonitor<<<SP as Deref>::Target as SignerProvider>::Signer>), Error>

where B: Deref, F: Deref, <B as Deref>::Target: BroadcasterInterface, <F as Deref>::Target: FeeEstimator,

Read a single channel monitor, along with any stored updates for it.

It is extremely important that your KVStore::read implementation uses the io::ErrorKind::NotFound variant correctly. For more information, please see the documentation for MonitorUpdatingPersister.

For monitor_key, channel storage keys be the channel’s transaction ID and index, or OutPoint, with an underscore _ between them. For example, given:

• Transaction ID: deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef
• Index: 1

The correct monitor_key would be: deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef_1

Loading a large number of monitors will be faster if done in parallel. You can use this function to accomplish this. Take care to limit the number of parallel readers.

pub fn cleanup_stale_updates(&self, lazy: bool) -> Result<(), Error>

Cleans up stale updates for all monitors.

This function works by first listing all monitors, and then for each of them, listing all updates. The updates that have an update_id less than or equal to than the stored monitor are deleted. The deletion can either be lazy or non-lazy based on the lazy flag; this will be passed to KVStore::remove.

Trait Implementations

impl<ChannelSigner, K, L, ES, SP> Persist<ChannelSigner> for MonitorUpdatingPersister<K, L, ES, SP>

where ChannelSigner: WriteableEcdsaChannelSigner, K: Deref, L: Deref, ES: Deref, SP: Deref, <K as Deref>::Target: KVStore, <L as Deref>::Target: Logger, <ES as Deref>::Target: Sized + EntropySource, <SP as Deref>::Target: Sized + SignerProvider,

fn persist_new_channel( &self, funding_txo: OutPoint, monitor: &ChannelMonitor<ChannelSigner>, _monitor_update_call_id: MonitorUpdateId, ) -> ChannelMonitorUpdateStatus

Persists a new channel. This means writing the entire monitor to the parametrized KVStore.

fn update_persisted_channel( &self, funding_txo: OutPoint, update: Option<&ChannelMonitorUpdate>, monitor: &ChannelMonitor<ChannelSigner>, monitor_update_call_id: MonitorUpdateId, ) -> ChannelMonitorUpdateStatus

Persists a channel update, writing only the update to the parameterized KVStore if possible.

In some cases, this will forward to MonitorUpdatingPersister::persist_new_channel:

• No full monitor is found in KVStore
• The number of pending updates exceeds maximum_pending_updates as given to Self::new
• LDK commands re-persisting the entire monitor through this function, specifically when update is None.
• The update is at CLOSED_CHANNEL_UPDATE_ID