hyperscale-rs

Introduction

hyperscale-rs is a community-built Rust implementation of the Hyperscale consensus protocol for the Radix DLT ecosystem. The project's stated goal is to produce a viable Xi'an candidate — the next-generation sharded consensus layer that will enable Radix to become what its community describes as "the world's first linearly scalable Layer 1 network."

Led by flightofthefox of proven.network, the project was publicly announced in late 2025 with the opening of its source code and an invitation for community review and contribution. By April 2026 the lead developer reported that "all major foundational consensus pieces are pretty solid now," and a formal RFC for delivering Xi'an was submitted to the Radix governance forum on 20 April 2026 with an 18-month mainnet target. In May 2026 a community consultation vote approved funding for the first milestone, and the Radix Foundation agreed to issue that payment directly — ahead of the community DAO (MIDAO) being finalised. Milestone 1 — dynamic topology has been in active development since mid-May 2026 and was reported as nearly feature-complete by mid-June.

The project significantly diverges from both the original Cerberus design and the Foundation's Hyperscale reference implementation. As the lead developer put it: "It's very different and throws out almost all designs from both Cerberus and the original Hyperscale repo... which is probably surprising to people if they think this is only a rust port." In a May 2026 discussion in the main Radix Telegram he was blunter still, stating that "Hyperscale never really used Cerberus" and that "hyperscale-rs has thrown out pretty much all of the original design of Hyperscale," and characterising Cerberus itself as "never a fully thought through design... more like a vague beginnings of an idea."

Cerberus remains the consensus securing the current Babylon mainnet as a single, unsharded instance, and its sharded form was peer-reviewed (Journal of Systems Research, 2023), demonstrating linear scaling to over ten million transactions per second on 1,024 shards. Xi'an, however, does not implement Cerberus at all — its per-shard consensus is a HotStuff-2–derived two-chain commit and its control plane is a leaderless prefix-consensus beacon chain (see Architecture and Dynamic Topology and the Beacon Chain below). Community members have accordingly argued that Radix's public materials should stop framing the network's scalability around Cerberus.

Background: Hyperscale & Xi'an

Radix's long-term roadmap centres on achieving linear scalability — the ability to increase throughput proportionally by adding more shards to the network. The Hyperscale Alpha consensus mechanism (formerly known as Cassandra) represents Radix's approach to this problem, combining principles from Nakamoto consensus and classical Byzantine Fault Tolerant (BFT) protocols.

In public testing, the Foundation's Hyperscale reference implementation sustained over 500,000 transactions per second with peaks exceeding 700,000 TPS across more than 590 participating nodes. Private testing demonstrated linear scaling at roughly 250,000 TPS on 64 shards and maintained the same per-shard throughput at 500,000 TPS on 128 shards. However, those tests used very small per-shard committees; hyperscale-rs targets meaningful committee sizes (~100 validators per shard), which fundamentally changes the design space.

The Xi'an production track implements this hybrid consensus mechanism into a production network candidate. After the interim Hyperscale phase closed in February 2026, hyperscale-rs has emerged as the leading community-led candidate to deliver Xi'an.

Architecture

hyperscale-rs is architected as a pure consensus layer — deliberately containing no I/O, no locks, and no async code in the consensus core. This design decision enables deterministic simulation testing as a first-class design principle, allowing the entire consensus logic to be tested without non-determinism from network or disk operations.

A defining feature of the codebase is the systematic pairing of production and simulation backends behind common traits. This includes network-libp2p and network-memory, storage-rocksdb and storage-memory, dispatch-pooled and dispatch-sync, and metrics-prometheus and metrics-memory — meaning the same consensus code can be exercised either against real infrastructure or inside a deterministic harness that can inject faults, partitions, and adversarial timing.

Consensus Mechanism

The protocol uses a faster two-chain commit derived from HotStuff-2. Key consensus features include:

• Optimistic pipelining — proposers can submit new blocks immediately after quorum certificate (QC) formation, without waiting for the previous block to be fully committed
• One-round finality — BFT provides finality with no possibility of reorganisation after QC
• Decoupled execution and consensus — transactions can start at block height N and finalise at N+1 or later; blocks contain certificates so execution does not need to occur before voting on a block
• Two-phase commit for cross-shard atomicity, where a coordinator sends prepare messages, shards lock resources, then commit or abort
• Aggregated provisions — one provision message per block per height covering all touching transactions, with proofs aggregated via verkle-tree inclusion proofs

Fault Model and Committees

Each shard runs a committee of approximately 100 validators using the n = 3f+1 model, requiring 2f+1 (~67) votes for QC formation. A separate beacon-chain-like control plane of all validators tracks which shards exist, which committees are responsible for each, and which portion of the state space each owns. Validators on a shard act as light clients of remote shards: they verify cross-shard messages using BLS signatures on gossiped headers and state-root checks, removing the single point of failure on remote proposers.

Radix Engine Integration

Unlike the Foundation's reference implementation, hyperscale-rs integrates directly with the real Radix Engine for smart contract execution, providing actual transaction processing rather than simulated execution. This is the basis for the lead developer's preference for atomic composability: in a sharded system without atomic commit, developers reliably collapse all components into a single shard, defeating the purpose of sharding.

Crate Structure

The project is organised as a Cargo workspace of 28 crates, each handling a specific responsibility. The structure matured significantly through April–June 2026: networking, sync and dispatch concerns were extracted into dedicated modules, the earlier bft, messages and topology crates were consolidated into a single shard consensus crate, and a new beacon crate was added for the global control plane (see the Dynamic Topology and the Beacon Chain section below).

Core Consensus

• types — fundamental data structures: cryptographic hashes, blocks, votes, quorum certificates
• core — trait-based architecture foundation, shared state machines and wire formats
• shard — per-shard Byzantine fault-tolerant consensus: block proposal, voting, view changes and committee enforcement (consolidating the former bft, messages and topology crates)
• beacon — the global beacon chain: shard topology, committee membership, leaderless prefix consensus, shuffling and market-driven economic parameters

Execution

• engine — Radix Engine integration adapters
• execution — transaction processing with two-phase commit coordination
• mempool — transaction pool administration and shard-specific queuing
• provisions — batched cross-shard provision messages and verification
• remote-headers — header sync for light-client verification of remote shards

Networking

• network — common networking traits
• network-libp2p — production networking via libp2p with Yamux auto-tuning
• network-memory — in-process simulation networking with fault injection

Storage

• storage — storage abstraction layer
• storage-rocksdb — production persistence via RocksDB
• storage-memory — in-memory storage for simulation
• jmt — Jellyfish Merkle Tree for state-root computation

Dispatch & Metrics

• dispatch, dispatch-pooled, dispatch-sync — task scheduling abstraction with parallel and synchronous backends
• metrics, metrics-prometheus, metrics-memory — observability with Prometheus integration and in-memory recording for simulations

Node, Simulation, Tooling

• node — integrates all subsystems into a complete validator
• production — production wiring of node + libp2p + RocksDB
• simulation, simulator — deterministic simulation harness with configurable network conditions, used for routine 300-node validation runs
• spammer — load generation utility for performance evaluation and benchmarking
• test-helpers — shared utilities across test suites

Performance

The project routinely runs full simulations against the deterministic harness to validate design changes. A 300-node simulation reported on 13 April 2026 demonstrated the operating characteristics targeted for production:

Nodes	300
Transactions submitted	30,000
Transactions completed	23,950 (within 30s window)
Average TPS	798
Peak TPS	1,147
Latency p50	6.35 s
Latency p99	7.75 s
Lock contention	0.00%
Total messages	9,438,658

Per-shard throughput targets are roughly 1,000 TPS at ~5 second finality, which the lead developer describes as a deliberate trade-off: rather than chase sub-second finality (which is bounded by the irreducible complexity of atomic commit), the project targets validator hardware that home users on consumer fibre can operate, with running costs estimated at "a few dollars a month." Higher per-shard throughput (10,000 TPS) is technically configurable but would require fibre and multi-core machines beyond the home-validator profile. Linear network scale comes from adding shards rather than scaling individual shards harder.

Transaction Flow

The transaction lifecycle follows a 14-step pipeline from user submission to finality, spanning three phases:

Pre-Consensus (Steps 1–6)

A user signs a transaction externally, which is submitted via an RPC gateway. The node receives the raw bytes, converts them to internal events, and performs cross-shard analysis to determine which NodeIDs (components, resources, packages, accounts) are touched. Transactions enter shard-specific mempools; cross-shard transactions are propagated to all involved shards via libp2p Gossipsub.

BFT Consensus (Steps 7–11)

Proposer selection occurs deterministically per round (e.g., round-robin by validator identity). The proposer builds a block from mempool transactions. Validators authenticate the block and broadcast votes. A quorum certificate is formed when 2f+1 (~67 of 100) votes are collected — notably, the QC is not sent as a separate message but is formed by the next proposer from collected votes. The block is committed once the commit rule is satisfied.

Execution & Finality (Steps 12–14)

Committed transactions are executed per shard using the Radix Engine. Cross-shard coordination uses a two-phase commit protocol where the coordinator sends prepare messages, shards lock resources without visible state changes, then commit or abort with state applied in an agreed order. BFT provides one-round finality with no possibility of reorganisation.

Improvements Over Reference Implementation

According to the project's design notes, hyperscale-rs aims to improve upon the official Hyperscale reference implementation in several areas:

• Better architected — modular workspace of 28 crates with paired production/simulation backends, replacing monolithic "kitchen drawer" crates
• Better tested — deterministic simulation harness was the first thing written for the project, and 300-node simulations are part of routine iteration cycles rather than rare set-piece events
• Validators as light clients of remote shards — only the remote proposer sends provisions (not M-to-N), with header-and-state-root verification removing the single point of failure on the proposer
• Aggregated provisions and verkle-tree inclusion proofs — one provision message per block per height carries all transactions, dramatically reducing N→M messaging overhead at large committee sizes
• Real Radix Engine — uses the actual Radix Engine for smart contract execution rather than an alternative environment, so atomic composability across shards behaves the same as same-shard composition from a developer's perspective
• Decoupled consensus and execution — durable persistence is batched into a single fsync per block and execution is moved off the consensus dispatch pool; voting on a block does not require executing it first
• Batch sequencing on contended state — beacon-coordinated sequencing of hot spots lifts contended-state throughput from the sub-1-TPS ceiling imposed by state locks toward the shard's aggregate throughput
• Safety-violation fixes — addresses several serious safety violations identified in the reference implementation

The lead developer notes that the reference implementation could only sustain its headline numbers with very small per-shard committees, where the inter-shard messaging cost is hidden. At hyperscale-rs's target committee size (~100 nodes per shard), the older design would saturate the bandwidth of a single data centre at a fraction of the throughput hyperscale-rs achieves on consumer hardware.

Xi'an RFC and Funding (April–May 2026)

On 20 April 2026, flightofthefox posted an RFC for delivering Xi'an for Radix to the governance forum. The proposal frames hyperscale-rs as the production candidate for the Xi'an mainnet release and structures funding around six delivery milestones, with the existing codebase to be relicensed under Apache 2.0 on acceptance.

Headline Terms

Total budget	$300,000 USD-equivalent across acceptance and five delivery milestones, paid in XRD (30-day TWAP per payment)
Bonus	50M XRD on Milestone 6 (mainnet launch)
Timeline	18 months to mainnet-ready delivery, plus a 12-month post-launch support window. Best case: testnet Q4 2026, mainnet Q1 2027
Governance	Milestones signed off by the Radix Accountability Council; neutral third-party arbitration for disputes
Termination	No exit fees if the project is paused or terminated at any milestone boundary

Milestones

• M1 — Validator lifecycle and consensus engine: dynamic topology, node shuffling, validator joining/leaving, the staking model
• M2 — Engine–Gateway alignment: API breaking changes required for sharded operation
• M3 — Gateway rewrite for sharded data
• M4 — Alpha desktop validator application (cross-platform), targeted at home validators on consumer hardware
• M5 — Post-quantum cryptography integration
• M6 — Mainnet launch with 12-month support window

Solo-Developer Risk

The proposal acknowledges that the project remains a solo effort and that adding contributors mid-flight would extend the timeline rather than accelerate it. Three options are offered for managing key-man risk: a larger proposal employing a second person, DAO-purchased key-man insurance for the duration of the project, or accepting the single point of failure given the milestone-based payment structure. The lead developer's stated preference is that hyperscale-rs's documentation and design discussions remain public so that other parties can build the depth of knowledge to maintain the network long-term — "we have to get to a world where multiple entities have a stake (and the knowledge to) maintain the network."

Status

In May 2026 the Milestone 1 funding moved through the community's Consultation V2 process: a temperature check (hosted by Mountaintop) passed within two days and was promoted to a binding proposal requiring a roughly 940M XRD quorum. Because the Foundation's and RDXH's holdings sit in custodial vaults and cannot vote, reaching quorum depended on community turnout; the Foundation confirmed (via community manager Andy) that it could and would issue the Milestone 1 payment directly, contingent on clear community consensus, rather than wait the minimum two months for the DAO to be constituted. With that consensus secured, work on Milestone 1 began in mid-May 2026.

As of mid-June 2026 the formal community DAO (MIDAO) had still not been finalised — its legal setup having run for several months — but development was funded and proceeding regardless via the direct Foundation grant, to the point where community members joked that Xi'an's first milestone might be finished before the DAO is even established. The later milestones remain dependent on the DAO and its treasury coming online, and the existing source-available codebase is slated to be relicensed under Apache 2.0 on full acceptance.

Dynamic Topology and the Beacon Chain (Milestone 1)

Milestone 1 of the Xi'an RFC — the validator lifecycle and dynamic-topology layer — began in mid-May 2026. By mid-June the lead developer reported that the major pieces were feature-complete, with only network-parameter governance and batch sequencing on contended state remaining, followed by "a month or two of pure QA, testing, benchmarking, optimization." The work resolves the parts of the design that earlier reference implementations left untested, and is described by the lead developer as "far and away the most sophisticated sharded L1 design ever implemented."

The Beacon Chain

A dedicated beacon crate implements a global beacon chain — a slow-ticking control-plane chain, run by all validators, that roots trust for every shard. It tracks which shards exist, which committee is responsible for each, and which portion of the state space each owns, so that any node can validate whether a cross-shard message represents a genuine quorum. The beacon chain carries no user transactions and grows very slowly (estimated under 100 MB per year); a new validator can verify roughly three years of beacon history in about five minutes, after which it can trust every shard's committee. Node software ships with embedded beacon checkpoints so new joiners sync from a recent point rather than from genesis.

Consensus on the beacon chain uses Strong Prefix Consensus, a leaderless Byzantine-fault-tolerant design (Prefix Consensus for Censorship-Resistant BFT), scaled to thousands of participants with VRF sortition that samples a small committee (~100) per slot. It is strongly censorship-resistant: each member proposes its own view of the shard headers at the epoch boundary together with a verifiable VRF reveal, all proposals are merged into a vector, and a few voting rounds converge on their common prefix. Because headers are self-authenticating via their quorum certificates and the VRF reveals are verified, a Byzantine majority cannot forge headers, bias the randomness, or drop an honest member's proposal — so a single honest committee member per epoch is enough to keep the chain live and fair. The beacon-chain state machine was first prototyped in a separate repository, POLARIS, then integrated into hyperscale-rs.

Lifting and Lowering

Shards and the beacon chain communicate through a "lifting and lowering" mechanism. Topological signals originating on a shard — staking and unstaking, validator registration and deregistration, jailing and unjailing, and missed proposals — are committed into that shard's block headers via a Merkle root and "lifted" to the next beacon block. The beacon block then applies any topology changes and "lowers" the updated committee assignments back to the shards. Jailing, the most time-sensitive operation, was implemented in hyperscale-rs in roughly a day, compared with being written off as impractical to retrofit onto Babylon.

Stake Pool and Market-Driven Parameters

A stake pool model governs how many nodes an operator may run: more stake unlocks the ability to run additional nodes (or virtual nodes). Economic parameters are self-regulating and market-driven — when the supply of nodes rises above target the minimum stake increases, and when too few nodes are available it falls. This keeps enough validators in the pool to support shard splits without manual governance intervention.

Shuffling

For its full end-to-end safety model the network shuffles roughly 20% of each shard's population per day, continuously churning committee membership. Shuffling raises the cost of attacking a single shard toward the cost of controlling 2f+1 of the entire network, rather than merely 2f+1 of one committee — taking economic security well beyond a simple division of staked value by the number of shards. It also removes any lasting incentive to misrepresent a shard's resource utilisation in order to game split/merge decisions, since a validator churns out of the shard within a few epochs regardless.

Dynamic Sharding and Live Resharding

The topology adjusts itself to load. Shards reconfigure only at epoch boundaries (about every five minutes), splitting when a shard grows too large or too busy and merging again when demand subsides. Split triggers can be driven by fees processed (a proxy for compute) and/or by state/storage size, with target sizes set by governance and tuned over time to keep a shard comfortably runnable on home-validator hardware. State is stored as a prefix subtree of a single global Merkle trie, so splits and merges do not require re-indexing the underlying data — only the physically unavoidable movement of bytes. Live resharding (in progress as of June 2026) lets a shard split in two without halting: the splitting shard coasts to a clean stop at a certified boundary, hands its state and in-flight work to its two child shards, and the children pick up live. Combined with enshrined snapshots checkpointed on the beacon chain and a "snap-sync" process, a validator shuffled into a new shard can download a recent checkpoint and replay only the small number of blocks since.

The lead developer contrasts this with peers: MultiversX has discussed adaptive state sharding for years without shipping it, and NEAR's dynamic sharding requires rewriting storage at a split and can only split, not merge back.

Virtual Nodes

Operators can run virtual nodes (vnodes) — multiple logical validators inside a single process. When several of an operator's nodes are shuffled into the same shard they share execution, storage and networking work rather than duplicating it, amortising hardware cost. The lead developer routinely runs eight nodes on a single sixteen-core machine while spamming far more than current Radix throughput.

Batch Sequencing on Contended State

One of the last Milestone 1 items, batch sequencing on contended state, addresses what the lead developer calls "the largest design issue with Hyperscale in general": even if total network throughput reaches millions of transactions per second, a single hot spot such as a popular DEX pool would otherwise be throttled to under one transaction per second by state locks. Using the beacon chain to form network-wide agreement, sequencing work for a contended piece of state can be delegated so that its throughput approaches the aggregate throughput of the shard the state lives on.

Development & Community

hyperscale-rs is developed openly with an active Telegram community that has grown from 319 members at the project's public unveiling in February 2026 to 451 members by June 2026. Day-to-day development is tracked through automated GitHub commit notifications in the channel via a Kit-Watcher bot.

Contributors

• flightofthefox (proven.network) — lead developer, ~2,000 commits, sole driver of the architecture
• kaldeberger — secondary contributor, ~70 commits
• shambupujar, cyril88888, pprogrammingg — community contributors
• wizzl0r — channel owner, PR reviews and testing infrastructure
• Radical — code reviewer and author of the community transaction-flow documentation

Recent Development (March–May 2026)

Between 20 March and 5 May 2026, the project recorded 376 commits, all on the main branch and almost entirely from flightofthefox. CI passed 642 runs by mid-April. Headline themes:

• Foundational consensus declared complete (13 April 2026) — "all the major foundational consensus pieces are pretty solid now. Not anticipating any more major changes just light tidy ups and doc work from here"
• Networking refactor — extraction of networking from runner crates into dedicated network-libp2p / network-memory crates; switch to default Yamux configuration with auto-tuning
• Persistence and sync — decoupled consensus advancement from durable persistence, batched block persistence into a single fsync, reorganised protocols module into sibling sync and fetch modules, reclamation of freed sync slots
• BFT hardening — vote validation rejects out-of-range committee indices, topology snapshots enforce committee/validator-info consistency, block transactions wrapped in Arc to remove duplicate allocations, integer-overflow safety in quorum arithmetic
• Storage — separated SST and WAL paths in RocksDB to allow operators to host them on different filesystems, refactored typed column-family encoding into DbEncode + DbCodec
• Dispatch — moved state-root verification onto its own dispatch pool, removed direct rayon dependency from the production build, eliminated double mutex locks in SubstateOverlay rebuild

Milestone 1 Development (May–June 2026)

Milestone 1 got underway in mid-May 2026 and dominated activity through June, with the lead developer summarising "650+ commits" over roughly a month. Headline work: a new beacon chain built on leaderless prefix consensus; the stake-pool model and self-regulating, market-driven economic parameters; integration of shards with the beacon chain via header attestations (the lifting/lowering process); enshrined snapshots and a snap-sync join process; the 20%-per-day shuffling model; a state tree reorganised into a prefix subtree of one global Merkle trie so splits and merges need no re-indexing; first-cut virtual nodes (vnodes); and a first cut of live shard splitting and merging. Remaining Milestone 1 items as of mid-June are network-parameter governance and batch sequencing on contended state, after which the lead developer plans one to two months of QA, testing, benchmarking and optimisation.

Roadmap and Public Testing

The Milestone 1 timeline is roughly four months (a target of around end of August 2026, possibly a month earlier). The first public tests are targeted for Q3 2026; rather than repeat the Foundation's one-off set-piece tests, the project intends to run longer-lived testnets that participants can register nodes into and out of at will — a model that was blocked until the beacon chain existed to manage which validators are registered. Early tests will be limited because the Radix Engine has not yet been adapted for sharding (that is Milestone 2): expect simple transactions and validator fees rather than full Scrypto support.

Migration from Babylon

The eventual switch from Babylon to Xi'an will not be an in-place upgrade. The lead developer expects a process modelled on the Olympia→Babylon transition: the network is halted at an agreed epoch by vote, state is dumped and imported into the new client (programmatically, after many dry runs), with validators running both clients across the handover epoch. State is re-indexed on ingestion as part of a move from an Aptos-style Jellyfish Merkle Tree to a pure binary JMT better suited to multiproofs, and the hashing algorithm is changed in the same step.

Funding History

Until April 2026 the project was sustained by community donations through the FoxFund initiative. In April 2026 the lead developer asked supporters to redirect their support into voting for the Xi'an RFC rather than continuing direct donations: "no more donations tho lads — just support foxy proposal for xi'an if you want to give back. it will have a hefty enough price tag."

External Links

• GitHub Repository — hyperscalers/hyperscale-rs
• RFC: Xi'an — Delivering Hyperscale for Radix (radixtalk)
• Telegram Community — hyperscale-rs for Radix
• Transaction Flow Documentation
• POLARIS — beacon-chain prototype
• Prefix Consensus for Censorship-Resistant BFT (paper)
• Radix Labs Roadmap — To Hyperscale and Beyond
• Hyperscale Alpha Part I — The Inception of a Hybrid Consensus Mechanism
• Hyperscale Update — 500k+ Public Test Done
• Wiki: Radix Mainnet (Xi'an)
• Wiki: Radix Accountability Council