Skip to main content

hdds-persistence

Persistence service providing TRANSIENT and PERSISTENT durability QoS support with SQLite and RocksDB backends.

Overview​

hdds-persistence is a standalone service that stores DDS samples to disk and replays them to late-joining readers. It implements the DDS durability contract for topics that require data to survive beyond the lifetime of the original publisher.

  • SQLite Backend -- Zero-dependency, production-ready persistent storage (default)
  • RocksDB Backend -- High-performance embedded database (optional feature flag)
  • Late-joiner Support -- Automatically replays historical samples to new readers
  • Retention Policies -- Time-based, count-based, and size-based limits

Architecture​

PersistenceService
+-- DurabilitySubscriber  (listens to TRANSIENT/PERSISTENT topics)
+-- LateJoinerPublisher   (replays history to new readers)
+-- PersistenceStore      (SQLite or RocksDB backend)

Installation​

cargo install hdds-persistence

Or build from source:

cd crates/hdds-persistence
cargo build --release

Quick Start​

# Run with default settings
hdds-persistence --db hdds_persist.db

# Filter specific topics
hdds-persistence --topics "State/*" --retention-count 1000

# Specify DDS domain and retention
hdds-persistence --domain 0 --topics "*" --retention-count 10000

CLI Reference​

hdds-persistence [OPTIONS] [COMMAND]

Options:
      --db <PATH>             Database file path [default: hdds_persist.db]
  -t, --topics <PATTERN>      Topic filter (supports wildcards) [default: *]
  -r, --retention-count <N>   Max samples per topic [default: 10000]
      --retention-time <SEC>  Max sample age in seconds (0 = infinite) [default: 0]
      --retention-size <BYTES> Max storage size in bytes (0 = infinite) [default: 0]
      --domain <ID>           DDS domain ID [default: 0]
  -n, --name <NAME>           Participant name [default: PersistenceService]
      --mock                  Use mock DDS interface (testing)
      --mode <MODE>           Service mode: all, subscriber, publisher [default: all]
  -h, --help                  Print help
  -V, --version               Print version

Commands:
  replay    Replay stored samples for a topic
  stats     Show storage statistics
  clear     Clear all stored samples (requires --confirm)
  list      List stored topics

Subcommands​

List Stored Topics​

hdds-persistence --db hdds_persist.db list

Output:

Stored topics:
  sensor/temperature (1523 samples)
  robot/state (892 samples)
  status/battery (45 samples)

Replay Samples​

hdds-persistence --db hdds_persist.db replay sensor/temperature

Output:

Replaying 1523 samples for topic 'sensor/temperature':
  seq=1, ts=1705312200000000000, size=24 bytes
  seq=2, ts=1705312200100000000, size=24 bytes
  ...

Storage Statistics​

hdds-persistence --db hdds_persist.db stats

Clear Storage​

hdds-persistence --db hdds_persist.db clear --confirm

Retention Policies​

Retention policies prevent unbounded storage growth:

PolicyFlagDescription
Count--retention-countKeep N most recent samples per topic
Time--retention-timeDelete samples older than N seconds
Size--retention-sizeLimit total storage to N bytes

Policies are enforced periodically by the service. Multiple policies can be combined; the most restrictive wins.

Service Modes​

The service can run in three modes:

ModeDescription
allRun both subscriber and publisher (default)
subscriberOnly store incoming samples
publisherOnly replay to late-joiners
# Run only the subscriber (storage node)
hdds-persistence --mode subscriber --db /data/persist.db

# Run only the publisher (replay node)
hdds-persistence --mode publisher --db /data/persist.db

Storage Backends​

SQLite (default)​

Zero-dependency, uses rusqlite with bundled SQLite. Suitable for most deployments.

hdds-persistence --db /var/lib/hdds/persist.db

RocksDB (optional)​

Higher throughput for write-heavy workloads. Requires the rocksdb-backend feature:

[dependencies]
hdds-persistence = { path = "../hdds/crates/hdds-persistence", features = ["rocksdb-backend"] }

Library API​

use hdds_persistence::{PersistenceService, Config, SqliteStore, HddsDdsInterface};
use hdds::{Participant, TransportMode};

let config = Config::builder()
    .topic_filter("State/*")
    .retention_count(1000)
    .retention_time_secs(3600)
    .domain_id(0)
    .build();

let store = SqliteStore::new("hdds_persist.db")?;

let participant = Participant::builder("PersistenceService")
    .with_transport(TransportMode::UdpMulticast)
    .domain_id(0)
    .build()?;

let dds = HddsDdsInterface::new(participant)?;

let service = PersistenceService::new(config, store, dds);
service.run().await?;

PersistenceStore Trait​

Custom backends can implement the PersistenceStore trait:

pub trait PersistenceStore {
    fn save(&self, sample: &Sample) -> Result<()>;
    fn load(&self, topic: &str) -> Result<Vec<Sample>>;
    fn query_range(&self, topic: &str, start_ns: u64, end_ns: u64) -> Result<Vec<Sample>>;
    fn apply_retention(&self, topic: &str, keep_count: usize) -> Result<()>;
    fn count(&self) -> Result<usize>;
    fn clear(&self) -> Result<()>;
}