Pipe Network CDN for Solana Snapshots
Pipe Network’s CDN provides Solana Mainnet snapshots and how using our snap-fetch tool can dramatically save bandwidth, reduce sync times, and increase efficiency for Solana validators.
Pipe Network CDN for Solana Snapshots
Why Use a Snapshot CDN?
• Save Bandwidth: Instead of repeatedly transferring large snapshots among validators and RPC providers, snapshots come directly from Pipe Network’s CDN. This reduces the load on the entire network.
• Faster Sync Times: Our high-performance CDN and snap-fetch tool can saturate connections of 10–20 Gbps, letting you catch up to the network far more quickly.
• Lower Infrastructure Costs: With less bandwidth usage on the validator side, you can reduce operational expenses and improve node uptime.
How It Works
1. Snapshots Hosted on Pipe Network CDN
We regularly host the latest full and incremental Solana Mainnet snapshots on our globally distributed CDN.
2. High-Performance snap-fetch
A specialized downloader that fetches the snapshot files in parallel. It uses multi-threading and chunked downloads to maximize throughput and minimize sync times.
3. Solana Validator Auto-Detection
Once snapshots are in your validator’s snapshots folder, the Solana validator automatically:
• Loads the most recent full snapshot
• Applies incremental snapshots on top
• No manual unzipping or processing is needed
Getting Started
Step 1: Install or Build snap-fetch
To build the Snap Fetch project from source, follow these steps:
Clone the repository:
Navigate into the project directory:
Build the project in release mode:
The optimized binary will be in target/release/snap-fetch.
Step 2: Download Snapshots to Your Validator
By default, Solana validators store ledger data here:
• Linux: ~/.local/share/solana/validator/ledger
• macOS: ~/Library/Application Support/solana/validator/ledger
• Custom: Pass --ledger /path/to/ledger to solana-validator
Inside your ledger path, there’s a snapshots directory. Simply run:
snap-fetch -o /path/to/ledger/snapshots --skip-existing
This command:
• Downloads the latest snapshot files to the specified snapshots folder
• Skips files that have already been downloaded (so you don’t waste bandwidth re-downloading)
Step 3: Start Your Validator
Launch your validator, pointing it to the snapshots folder. For example:
solana-validator
--ledger /path/to/ledger
--snapshot-fetch
...other options...
Solana automatically detects and applies the snapshots. No extra steps needed!
Recommended Settings & Tips
Basic Usage
snap-fetch
By default, downloads the latest Solana Mainnet snapshot files.
Speeding Up Downloads
• Increase concurrency (-c 20)
• Larger chunk size (-s 20971520 for 20 MB chunks)
• More chunks per file (--chunks-per-file 30 or higher)
Example:
snap-fetch -c 20 -s 20971520 --chunks-per-file 30
Using Alternative Servers
Should our primary CDN be temporarily slow in your region, you can point snap-fetch to an alternative server:
snap-fetch --base-url "http://207.121.20.172:8080"
Why This Matters for the Solana Network
Massive Bandwidth Savings
If the majority of Solana nodes fetch snapshots directly from Pipe’s CDN rather than from each other or from RPC nodes, the network could save over 3 petabytes (3PB) in data transfers every week.
Improved Validator Performance
• Faster Catch-Ups: Reduced time spent waiting for large file downloads.
• Network Offload: Frees up validators to focus on processing transactions rather than serving snapshots.
Lower Costs & Greater Stability
When fewer snapshots are served among peers, node operators see reduced egress costs and higher reliability overall. This is especially beneficial for community validators with limited resources.
Hardware & Performance Notes
Validator Requirements
• CPU: At least 8 cores
• RAM: Minimum 128GB (256GB+ recommended)
• Disk: High-speed (NVMe SSD), 1TB+ free space
• Network: 10 Gbps or better for rapid downloading
snap-fetch Performance
• Optimized for 10–20 Gbps connections
• Uses parallel chunk downloads
• Works best with servers supporting HTTP byte-range requests
• Adapts chunk sizes automatically (--dynamic-chunks on by default)
Summary
Pipe Network’s CDN offers a robust and efficient way to get Solana Mainnet snapshots. Using snap-fetch and our global servers:
• Minimizes load on the broader Solana network
• Cuts your sync time and bandwidth usage
• Frees up RPC and validator nodes, contributing to a more scalable and cost-effective ecosystem
If you’re running a Solana validator, consider pointing your snapshot fetches to Pipe Network’s CDN. With widespread adoption, we can collectively save 3PB+ of bandwidth every week—benefiting the entire Solana community.
Last updated