Pebblify

Pebblify is a high-performance migration tool that converts LevelDB databases to PebbleDB format, specifically designed for Cosmos SDK and CometBFT (formerly Tendermint) blockchain nodes.

PebbleDB offers significant performance improvements over LevelDB, including better write throughput, more efficient compaction, and reduced storage overhead. Pebblify makes it easy to migrate your existing node data without manual intervention.

⚠️ This tool is still in the early stages of development and may contain bugs or be unstable. If you notice any unusual behavior, please open an issue.

Features

  • Fast parallel conversion — Process multiple databases concurrently with configurable worker count
  • Crash recovery — Resume interrupted migrations from the last checkpoint
  • Adaptive batching — Automatically adjusts batch sizes based on memory constraints
  • Real-time progress — Live progress bar with throughput metrics and ETA
  • Data verification — Verify converted data integrity with configurable sampling
  • Disk space checks — Pre-flight validation to ensure sufficient storage
  • Docker support — Multi-architecture container images (amd64/arm64)

How It Works

  1. Scanning — Pebblify scans the source directory to discover all LevelDB databases and estimates key counts
  2. Conversion — Each database is converted in parallel (up to the worker limit), with adaptive batching to optimize memory usage
  3. Checkpointing — Progress is saved periodically, enabling crash recovery
  4. Finalization — Once all databases are converted, the output is moved to the final destination
  5. Cleanup — Temporary files are removed automatically

State Management

Pebblify maintains a state file (.pebblify-tmp/state.json) that tracks:

  • Source and destination paths
  • Status of each database (pending, in_progress, done, failed)
  • Last checkpoint key for each database
  • Migration statistics and metrics

This enables seamless recovery from any interruption.

Requirements

  • Go 1.25+ (for building from source)
  • Sufficient disk space — Approximately 1.5x the source data size is recommended during conversion
  • Source database — Must be a valid LevelDB directory structure (Cosmos/CometBFT data/ format)

Next Steps