The Ritvik Blog
Cover image for Understanding & Evaluating SIMD-223

Understanding & Evaluating SIMD-223

Solana's SIMD-223 upgrade marks a significant evolution in the blockchain's architecture, enhancing scalability and efficiency. By removing the legacy Accounts Delta Hash and fully embracing the Accounts Lattice Hash introduced in SIMD-215, Solana transitions from a Merkle-tree based state hashing model to a novel homomorphic hashing scheme. This shift eliminates computational bottlenecks, enabling full-state hashing in each block and paving the way for a future with billions of accounts. The upgrade requires careful coordination and adoption by all validators, but the payoff promises a leaner, faster, and more scalable Solana network.

Case Study: Understanding and Evaluating SIMD-223

This article provides a comprehensive technical analysis of Solana's transition away from Merkle-based state hashing, focusing on the implications of SIMD-215 and SIMD-223.

1. System Architecture & Technical Analysis

Accounts Delta Hash – Role and Initial Rationale: Solana's initial design incorporated the Accounts Delta Hash, a Merkle root of accounts modified within a block. This served as a state commitment, balancing security with the performance demands of Solana's high throughput. A full state Merkle root for every block was computationally prohibitive. The Delta Hash, embedded in the block hash, provided a cryptographic record of state changes within each slot. Concurrently, the Epoch Accounts Hash, a full state Merkle root, was computed infrequently, offering an eventual consistency check. This dual-hash system addressed the performance limitations of full-state hashing per block.

SIMD-215 and the Accounts Lattice Hash – New Hashing Model: SIMD-215 introduced the Accounts Lattice Hash, a paradigm shift in Solana's state management. This homomorphic hashing scheme enables incremental updates to a hash representing the entire account state. Instead of a Merkle tree, the lattice hash employs algebraic properties. Each account a is assigned a cryptographic hash lthash(a). The global state hash is the sum of all lthash(a) within a finite field. Modifications within a block are reflected by subtracting the old account hashes and adding the new hashes. This homomorphic update is computationally inexpensive, scaling linearly with the number of modified accounts, not the total number of accounts. Consequently, each block can now include a full-state commitment via the Accounts Lattice Hash. This rendered the Epoch Accounts Hash redundant, as the lattice hash provides a continuous, up-to-date full-state representation. While SIMD-215 retained the Accounts Delta Hash for backward compatibility, it fundamentally altered Solana's hashing model, embedding the full-state Lattice Hash within each block's bank hash. The lattice hash, leveraging a lattice-based cryptographic accumulator, offers 128-bit security, including post-quantum resistance.

Removal of the Accounts Delta Hash (SIMD-223) – Changes and Effects: Building upon SIMD-215, SIMD-223 eliminates the legacy Accounts Delta Hash. This simplification streamlines consensus: the block's bank hash now solely relies on the Accounts Lattice Hash. The code responsible for constructing the per-block Merkle tree is removed. This significantly impacts block processing, reducing the computational burden on validators. Removing the Delta Hash is a consensus-breaking change, requiring all validators to upgrade. Furthermore, data structures like snapshot formats are adjusted to utilize the Lattice Hash directly, accelerating snapshot generation and verification. In essence, SIMD-223 completes Solana's transition to a unified, efficient, full-state hashing mechanism.

bank-hash-calculation-update
Legacy vs. New Bank Hash Calculation Approaches

2. Validator Performance & Scalability

Block Production Efficiency Gains: Eliminating the Merkle-based delta hash calculation dramatically improves validator performance. The previous process involved collecting, sorting, and hashing modified accounts, a computationally intensive task. The lattice hash update, in contrast, is a linear operation involving simple field arithmetic. This reduces CPU overhead and latency in block production, enabling faster block sealing and lower block completion time variability. Validators benefit from reduced compute cycles per block, freeing resources for transaction processing and network operations. This efficiency gain directly translates to higher throughput, allowing Solana to maintain its slot time even with growing state and potentially accommodate more transactions per block.

Hashing Overhead Before vs After (SIMD-215/223): Previously, Solana's hashing overhead scaled poorly with state size. The per-block Merkle tree calculation for modified accounts, approximately O(m log m) for m modified accounts, posed a significant bottleneck. The Epoch Accounts Hash calculation was even more demanding. SIMD-215 drastically reduced this overhead. The lattice hash update scales linearly with the number of modified accounts. SIMD-223 further reduces overhead by eliminating the delta hash calculation altogether. The per-block state commitment now reuses the already-maintained lattice hash. The computational burden is shifted to transaction execution, where account hashes are computed and applied to the accumulator. This represents a significant algorithmic improvement, effectively removing a major scalability bottleneck.

Impact on Solana’s Scalability: These changes significantly enhance Solana's ability to scale. By adopting the Accounts Lattice Hash, Solana mitigates the "state growth problem." The network can accommodate billions of accounts without compromising block production efficiency. Removing hashing overhead increases the throughput ceiling. Faster block processing and verification potentially reduce confirmation latency. The reduced computational burden also promotes decentralization by lowering hardware requirements for validators. This aligns with Solana's emphasis on parallelism, allowing the runtime to focus on transaction parallelization and signature verification. The transition to the lattice hash ensures that state commitment will not be a limiting factor for future growth, solidifying Solana's position as a high-performance L1.

3. Comparison with Other Blockchains

State Models: Solana vs Ethereum vs Polkadot (and others): Solana's approach to state management and hashing diverges significantly from other L1s like Ethereum and Polkadot. Ethereum utilizes an account-based model with a Merkle-Patricia Trie. Each block header contains the state root, the root hash of the global trie encompassing all accounts and storage. Every account update necessitates trie modifications, incurring significant overhead. Polkadot (Substrate) also employs a Merkle Trie for state representation, with a state_root in each block header. Other L1s like Near, Cosmos, and Aptos/Diem utilize variations of Merkle trees for state commitment. All these systems include a full-state commitment (Merkle root) in each block, enabling light-client verification at the expense of performance.

Solana historically prioritized performance, employing the Accounts Delta Hash and infrequent full hashes. The introduction of the Accounts Lattice Hash with SIMD-215 brings Solana closer to other chains by providing a full-state commitment per block, albeit through a different mechanism. Solana's lattice hash is an algebraic accumulator, not a tree. It avoids the overhead of tree updates, preserving Solana's performance advantage.

State Update Mechanisms Comparison: Ethereum's state updates are synchronous with transaction execution, requiring trie node recalculations for each modification. Polkadot/Substrate similarly updates the state trie. Solana's lattice hash approach is more efficient. Account changes are hashed in isolation, and the global accumulator is updated in constant time. Solana avoids the multiple hashing of each change inherent in Merkle trie approaches. Furthermore, Solana's model allows for parallel computation of account hashes and lattice updates, aligning with its parallel execution model.

Removal of Merkle-based Hashing & High-Throughput Alignment: Removing the Accounts Delta Hash eliminates the last Merkle-based component from Solana's consensus-critical path. This aligns with Solana's focus on high throughput. Merkle trees are performance bottlenecks for high-TPS systems. By eliminating Merkle hashing, Solana avoids this bottleneck. The trade-off is the loss of direct inclusion proofs. However, Solana is exploring alternative state proof mechanisms. Solana's new approach provides a full-state commitment without a Merkle tree, enabled by the homomorphic lattice hash. This underscores Solana's commitment to maintaining its performance lead.

4. Risk and Trade-Offs

Backward Compatibility and Consensus Impact: Removing the Accounts Delta Hash is a hard fork, requiring all validators to upgrade. Backward compatibility is not maintained. Mishandled activation could lead to network stalls or consensus splits. Careful rollout and communication are crucial.

Validator Requirements for Adoption: Validators must upgrade to the Solana release incorporating SIMD-215 and SIMD-223. Precomputation of the lattice hash is recommended. Validators must coordinate on the activation slot. Post-upgrade, validators should adapt related procedures, such as snapshot verification.

Risks and Challenges of Removing the Delta Hash: The novelty of the lattice hash presents a potential risk, although mitigated by its design, collision resistance and the robust mathematical properties of lattice-based cryptography—offering post-quantum security. The loss of Merkle proofs could impact certain use cases. Engineering risks during rollout include ensuring consistent lattice hash computation across validators. Thorough testing and incremental rollout are essential.

Trade-offs and Alternatives: Keeping the Merkle system was deemed unscalable. Verkle trees were considered but deemed too slow. Other alternatives, like elliptic curve-based hashes and XOR accumulators, were also evaluated and rejected for performance or security reasons. The lattice-based hash offers a balance of speed and security, trading off Merkle proofs. This implies a shift towards reliance on full nodes. The complexity of the new scheme might present a challenge for decentralization of expertise. Maintaining both systems was deemed too complex.

Resources: