Secure, scalable and upgradeable Web3 infrastructure

Summary

With the rise of the new Internet infrastructure blockchain, developers are deploying tens of thousands of decentralized applications at a rapid pace. Unfortunately, due to poor stability, high cost, low throughput, and some security issues, blockchain has not yet been widely adopted. In order to be widely used in the Web3 era, blockchain infrastructure should emulate the characteristics of cloud infrastructure, that is, provide a credible, scalable, cost-effective and continuously optimized platform for many decentralized applications.

To address these challenges, we launched the Aptos blockchain with scalability, security, reliability, and upgradeability as core design principles. The Aptos blockchain has been jointly developed by more than 350 developers around the world over the past three years [1]. It offers new innovations in consensus, smart contract design, system security, performance, and decentralization. The combination of these technologies will provide a solid foundation to bring Web3 to a wider audience:

1. The Aptos blockchain natively integrates and uses the Move language to achieve fast and secure transaction execution [2]. Move prover, a formal verification tool for smart contracts developed in the Move language, provides additional guarantees for contract constants and operations. This security-focused approach allows developers to better protect software from malicious entities.

2. The Aptos data model allows for flexible key management and hybrid hosting options. This, together with pre-signature transaction transparency and a practical light client protocol, together provide a safer and more trustworthy user experience.

3. In order to achieve high throughput and low latency, the Aptos blockchain uses a pipelined and modular approach at key stages of transaction processing. Specifically, operations such as transaction distribution, block metadata ordering, parallel transaction execution, batch storage, and ledger verification run concurrently. This approach makes full use of all available hardware resources, improves hardware efficiency, and achieves a high degree of parallel processing.

4. Unlike the parallel execution engine that needs to obtain the data to be read and written before reading and writing, which destroys the atomicity of the transaction, the Aptos blockchain does not set such restrictions on developers. It provides applications with higher throughput and lower latency, and simplifies development by guaranteeing the atomicity of complex transactions.

5. The Aptos modular architecture ensures the flexibility of the client and is optimized for frequent upgrades. Additionally, to rapidly deploy new technological innovations and support new Web3 use cases, the Aptos blockchain provides an embedded on-chain change management protocol.

6. The Aptos blockchain is experimenting with future initiatives beyond the performance of a single validator: its modular design and parallel execution engine support the internal sharding of validators, while homogeneous state sharding (homogeneous state sharding) provides horizontal throughput. The potential to scale without introducing additional complexity to node operators.

[1] Legal disclaimer: This white paper and its contents are not an offer to sell any tokens, nor an offer to induce the purchase of any tokens. We publish this white paper solely to accept public feedback and comments. Nothing in this document should be construed as a guarantee or promise as to how the Aptos blockchain or its tokens (if any) will develop, utilize or accumulate value. Aptos has only outlined its current plans, which are subject to change at its discretion and whose success will depend on a number of factors outside its control. Such future statements necessarily involve known and unknown risks, which may cause actual performance and results in future periods to differ materially from those described or implied by us in this white paper. Aptos undertakes no obligation to update its plans. There can be no assurance that any statement in the white paper will prove to be accurate, as actual results and future events may differ materially. Please do not place undue reliance on future statements.

1. Preamble

In the Web2 era, services such as communication, social media, finance, games, shopping, and audio and video streaming media are provided by centralized companies that have user data permissions (such as Google, Amazon, Apple, and Meta). These companies leverage application-specific software to optimize development infrastructure for targeted use cases and deploy those applications to users using cloud infrastructure. Cloud infrastructure provides access to virtual or physical infrastructure services, such as virtual machine (VM) rentals and bare-metal hardware running in data centers around the world (such as AWS, Azure, and Google Cloud). As a result, building Web2 Internet services capable of scaling to billions of users has never been easier than it is today. However, Web2 requires users to explicitly trust centralized entities, a requirement that is increasingly causing social concern.

To address this concern, a new Internet era has begun: Web3. In the Web 3 version of the Internet, the blockchain has emerged to provide a decentralized, immutable ledger, allowing users to communicate with each other safely and reliably without the need to trust a controlling middleman or centralized entity. Similar to how Web2 Internet services and applications rely on cloud infrastructure, decentralized applications can use blockchain as a decentralized infrastructure layer to reach billions of users around the world.

However, despite the existence of many blockchains, Web3 has not yet been widely adopted [3]. While technology continues to advance the industry, existing blockchains are still unreliable. Expensive transaction fees, low throughput, frequent asset losses due to security issues, and inability to support real-time response. Compared with cloud infrastructure empowering Web2 services and successfully reaching billions of people, the blockchain has not yet brought Web3 applications to the same height.

2. Aptos Vision

Aptos' vision is to provide a blockchain that can bring mainstream adoption to Web3 and empower an ecosystem of decentralized applications to solve real-world user pain points. Our mission is to drive new heights in blockchain reliability, security and performance by providing a flexible and modular blockchain architecture. The architecture should support frequent upgrades, rapid adoption of the latest technologies, and best-in-class support for emerging use cases.

We envision building decentralized, secure and scalable networks run by community governance. As demand for infrastructure grows around the world, blockchain computing resources scale horizontally and vertically to meet those demands. As new use cases and technological advancements emerge, the network should be upgraded frequently and seamlessly without disrupting users. Let users no longer pay attention to infrastructure-related issues. Developers and users will have access to many different options for key recovery, data modeling, smart contract standards, resource usage tradeoffs, privacy, and composability. Users are confident that their assets are safe, available, and accessible at virtually no cost. Anyone can securely and easily conduct immutable transactions with untrusted parties around the world. Blockchain will be as ubiquitous as cloud infrastructure.

To realize this vision, significant advances in technology must be made. Our experience developing, upgrading, and deploying the Diem blockchain (predecessor to the Aptos blockchain) over the past three years has demonstrated that the network can continuously upgrade the protocol without disrupting clients [4]. In early 2020, the Diem mainnet was deployed to a dozen nodes with multiple wallet providers. In the following year, our team carried out two major upgrades to the consensus protocol and the core framework. Both upgrades completed without any downtime for users. In the Aptos blockchain, we have made a series of radical improvements to the technology stack, while also taking security, transparency, and frequent upgrades as core functions inspired by the Diem blockchain. We place particular emphasis on new methods of transaction processing (as described in Section 7) and new approaches to decentralization and network governance.

With the continuous improvement and development of the Aptos blockchain, we will continue to update the protocol and design, and release the latest version of the white paper at that time. In the following, we describe the current state of the Aptos blockchain and plans for the future.

3. Overview

As shown in Figure 1, the Aptos blockchain consists of a group of validators (Validators), who use Byzantine Fault Tolerance (BFT) and Proof of Stake (POS) consensus mechanisms to receive and process user transactions. Token holders lock or pledge their tokens in their selected validators (Validator). The consensus voting weight of each validator is proportional to the amount of staked tokens. A validator can be active and participate in consensus decisions. Similarly, if a verification node does not have enough pledged tokens, or is rotated out of the validator set, or is offline when synchronizing the blockchain state, or the consensus protocol rejects its participation in the consensus due to poor historical performance, then The validator may also be inactive.

A client is any part of the system that needs to submit transactions or query the state and history of the blockchain. Clients can choose to download and verify data that has been signed and verified by validators. A *Full Node* is a client that replicates transactions and blockchain state from a validator node or other full nodes in the network. They may prune some transaction history and blockchain state records as needed to regain sufficient storage space. Light clients only maintain a set of current validating nodes and can safely query partial blockchain states from full nodes. Wallets are common examples of light clients.

To meet the need for a secure, fast, reliable and scalable Web3 infrastructure for widespread adoption, the Aptos blockchain is built on the following core design principles:

• Through the new smart contract programming language Move [5], fast and secure execution of on-chain logic, as well as simple auditability and procedural analyzability. The Aptos blockchain inherits from the Diem blockchain and continues to develop Move.
• Extremely high throughput and low latency are achieved through batched, pipelined, and parallelized transaction processing methods.
• Unlike the existing parallel execution engine that pre-identifies the data to be read/written and destroys the atomicity of transactions, the Aptos blockchain innovatively uses Block-STM technology as a parallel execution engine, effectively supporting the atomicity of arbitrary complex transactions sex.
• Optimize performance and decentralized governance through fast, staking validator rotation and reputation tracking of validators.
• Upgradability and configurability are the most important design principles, so that the infrastructure can embrace new use cases and the latest technologies.
• Passed rigorous component-level testing such as threat modeling and a modular design for seamless deployment, ensuring high security and reliability of operations.
• Guaranteed decentralized horizontal throughput scalability. Sharding, derived from the program and data model, is an important concept in horizontal scaling.

Chapter 4 explains how developers can interact with the Aptos blockchain through the Move language. Chapter 5 describes the logical model. Chapter 6 details how the Aptos blockchain enables a secure user experience through strong verification methods. Chapter 7 describes key performance innovations around pipelining, batching, and parallelization. Chapter 8 details the various options for different types of clients to synchronize state with other nodes. Chapter 9 describes our plans for community ownership and governance. Finally, Chapter 10 discusses future performance directions while maintaining decentralization.

4. Move programming language

Move is a new smart contract programming language that focuses on security and flexibility. The Aptos blockchain uses Move's object model to represent its ledger state (see Section 5.5), and uses Move code (modules) to encode the rules for state transitions. Transactions submitted by users can include publishing new modules, upgrading existing modules, executing interface functions defined in modules, and scripts that can directly interact with the public interface of the module.

The Move ecosystem includes a compiler, a virtual machine, and many other development tools. Move is inspired by the Rust programming language, which clarifies the ownership of data through concepts such as linear types. Move emphasizes the scarcity, preservation, and access control of resources. The Move module defines the lifecycle, storage and access modes of each resource. This ensures that resources like Coin cannot be minted without proper credentials, cannot be double spent, and cannot disappear.

Even in the presence of untrusted code, Move can still utilize bytecode verification tools to ensure type and memory safety. To help write more believable code, Move includes a type verifier, Move Prover [6], which can verify the functional correctness of Move programs according to a given specification. This type verification function has been integrated into the Move language .

In addition to user accounts and corresponding account contents, the state of the distributed ledger also contains the on-chain configuration of the Aptos blockchain. This network configuration includes the set of currently active validators, the attributes of the pledge, and the configuration of various services within the Aptos blockchain. Move's support for module upgradability and full programmability enables seamless configuration changes, as well as support for upgrades to the Aptos blockchain itself (both types of upgrades have been performed multiple times on the private mainnet, and there is no downtime records).

The Aptos team has further added Move features to support a wider range of Web3 use cases. As described in Section 5.5 below, the Aptos blockchain enables fine-grained resource control. This feature not only effectively supports parallel execution, but also nearly fixes the cost of accessing and changing data. In addition, the Aptos blockchain provides table support built on top of fine-grained storage, which enables large-scale data sets (for example, a large collection of NFTs) to be implemented in a single account. At the same time, Aptos supports shared or automated accounts that are fully embodied on-chain. This enables complex decentralized autonomous organizations (DAOs) to share accounts and use those accounts as containers for heterogeneous collections of resources.

5. Logical Model

The ledger state of the Aptos blockchain represents the state of all accounts on the chain. The ledger status uses an unsigned 64-bit integer for version division, corresponding to the number of transactions executed by the current system. Anyone can submit transactions to the Aptos blockchain to modify the state of the ledger. After the transaction is executed, a transaction output is generated. The output of a transaction consists of zero or more operations to manipulate the ledger state (called write sets), a resulting set of events (see Section 5.1.1), the gas consumed, and the state of the executed transaction.

5.1 Transactions

A signed transaction contains the following information:

• Transaction Authenticator: The sender uses a transaction authenticator that includes one or more digital signatures to verify that the transaction has been authenticated.
• Sender Address: The account address of the sender.
• Payload: A payload either refers to an existing interface function on-chain, or contains a function (called a script) to be executed as inline bytecode. Additionally, a set of input parameters is encoded in a byte array. For peer-to-peer transactions, the input parameters include the recipient's information and the transfer amount.
• Gas Price (in specified currency/Gas unit): This is the amount per unit of Gas that the sender is willing to pay to execute the transaction. Gas fee refers to the payment of computing, networking and storage fees. Gas is an abstract unit of calculation with no inherent real value.
• Maximum gas amount: Maximum gas is the maximum number of gas units allowed to be consumed before the transaction is aborted. There must be at least the balance of the Gas unit price multiplied by the maximum number of Gas units in the account, otherwise the transaction will be terminated during the verification process.
• SequenceNumber: The sequence number of the transaction. The sequence number in the transaction must match the sequence number stored in the sender's account when the transaction was executed. After the transaction is successfully executed, the account sequence number is incremented to prevent replay attacks.
• Expiration time: A timestamp after which the transaction will no longer be valid.
• Blockchain ID: Identifies the validity of transactions in the blockchain, providing users with further protection against signing errors.

At each version i, the state change is represented by a tuple (Ti, Oi, Si), containing the transaction, transaction output, and ledger state after the transaction, respectively. Given a deterministic function Apply, execute transaction Ti, ledger state Si-1, generate transaction output Oi and new ledger state Si. That is, Apply(Si-1*,Ti*) → 〈Oi,Si〉.

5.1.1 Events

Events are emitted during transaction execution. Each Move module can define its own events and choose when to emit them at execution time. For example, during a money transfer, the sender's and receiver's accounts will emit SentEvent and ReceivedEvent respectively. The data will be stored in the ledger and can be queried through Aptos nodes. Each registered event has a unique index that can be used to query event details.

Multiple events emitted to the same event index result in an event stream, which is a list of events with each entry containing a number, type, and data that increment from 0. Every event must be defined with some type. There can be multiple different events defined by the same or similar type, especially when using generics. Events have associated data. A general principle for developers of Move modules is to include all necessary data in order to understand the underlying resource changes caused before and after transaction execution, which changed data and sent events.

Transactions can only generate events, not read them. This design allows transactions to be executed only with the current state and inputs of the current transaction (rather than historical information, e.g., previously generated events).

5.2 Account

Each account is identified by a unique 256-bit value called the account address. Create a new account in the ledger state (see Section 5.5), which can be created by calling the create_account(addr) Move function when an existing account sends a transaction. This usually happens when a transaction attempts to send Aptos tokens to an account address that has not yet been created. For convenience, Aptos also supports a transfer(from, to, amount) function, which will create an account by default if it does not exist before the transfer.

To create a new account, the user first generates a signing key pair: (vk,sk). Next, the signature scheme identifier (ssid) is spliced together with the public key vk, and the new account address of the specific signature scheme is obtained through the encrypted hash H: addr=H(vk, ssid).

After creating a new account at address addr, the user can sign transactions to be sent from the account at addr, using the private key sk to sign. Users can also rotate sk, either actively changing sk or responding to possible private key leaks. The operation of changing the private key will not change the account address, because the account address is created only from the public key.

The Aptos blockchain does not link accounts to real-world identities. A user can create multiple accounts by generating multiple key pairs. Accounts controlled by the same user are not intrinsically linked to each other.

However, for ease of asset management, a single user can still manage multiple accounts in one wallet. This flexibility provides users with anonymity, and we are trying to use more privacy-preserving primitives in future versions (privacy- preserving primitives). As described in Section 7.4, multiple accounts owned by a user or group of users also provide avenues for increased execution concurrency.

5.3 Move module

A Move module contains Move bytecode that declares data types (structures) and procedures. It is identified by the address of the account declaring the module along with the module name. For example, the identifier for the first currency module in Figure 2 is 0x1::coin. A module can depend on other modules on the chain, as shown in the wallet module in Figure 2, and the code of other modules can be reused.

A module must be unique within an account, that is, the module name must remain unique under each account. For example, the account with address 0x1 in Figure 2 cannot claim another module named coin. On the other hand, the account at address 0x3 could declare a module named coin, and the identifier of this module would be 0x3:coin. Note that 0x1::coin::Coin and 0x3::coin::Coin are different types and cannot be used interchangeably nor do they share common module code. In contrast, 0x1::coin::Coin and 0x1::coin::Coin> are different instances of the same generic type and cannot be used interchangeably, but can share common module code.

Modules under the same address will be merged into the same *package (package)* The owner of this address publishes the package as a whole on-chain, including bytecode and package metadata. Package metadata determines whether a package is upgradeable or immutable. For upgradeable software packages, a compatibility check is performed before the upgrade is allowed: existing interface functions cannot be changed, and no resources can be stored in memory. However, upgrades can add new functions and resources.

The Aptos framework consists of the core library and configuration of the Aptos blockchain, defined as a regularly upgradeable module package (see Section 9.2).

Figure 3: Example of on-chain data

5.4 Resources

Similar to modules, account addresses can also have data values associated with them. In each account address, the data value is determined by its type, and under each account, each type of data value should remain unique. Using Figure 3 as an example, address 0x50 holds a single value, where 0x3::coin::Coin is a fully qualified type. 0x3 is the address of the Coin module, Coin is the name of the module, and Coin is the name of the data type. Generic data values can also be used, and different generic instances are treated as different types. This is critical for scalability, allowing different instances to share the same function code.

The rules for changing, deleting, and publishing a value are encoded in the modules that define the data type. Move's security and validation rules prevent other code or entities from directly creating, modifying, or deleting instances of data types defined in other modules.

Having at most one top-level value of each type under an address sounds restrictive. However, this is not a problem in practice, because developers can avoid any restrictions by defining different wrapper types with the same type of data as member variables. The Wallet structure (in Figure 3) is an example of how to use wrapper types.

It should also be noted that not all data types can be stored on-chain. In order for a data instance to qualify as a top-level value, the data type must have the Key capability. Also, the Store capability is required for nested values. Data types with two capabilities are also called resources.

5.5 Ledger Status

From the perspective of the Move virtual machine (Move VM), each account consists of a set of values and key-value data structures. These data structures are called entries and are stored in Binary Normalized Serialization format (BCS). This data structure design allows developers to write smart contracts that can be executed efficiently when a small amount of data is copied to a large number of accounts; they can also write smart contracts that can be executed efficiently when a large amount of data is concentrated on a small number of accounts. Move modules are stored similarly to account data, but under a separate namespace. The Genesis ledger state defines the initial set of accounts and their related states when the blockchain is initialized.

At launch, the Aptos blockchain will be represented by a single ledger state. However, as usage spreads and technology evolves, Aptos will expand the number of shards to increase throughput (i.e. enable multiple ledger states) and support transactions that move or access assets across shards. Each ledger state will maintain all on-chain assets of a specific shard, and provide the same account model, as well as fine-grained key-value data storage, providing a near-fixed cost for storage access.

6. Safe user experience

In order to reach billions of Internet users, the user experience of Web3 must be secure and convenient. In this section, we will describe several innovations of the Aptos blockchain to achieve this goal.

6.1 Transaction Feasibility Protection

Signing a transaction means that the signer authorizes the blockchain to submit and execute that transaction. Sometimes users sign transactions without being careful or realizing that they are being manipulated. To reduce this risk, the Aptos blockchain places limits on the feasibility of transactions and protects signers from being locked into infinite confirmation operations. Currently Aptos provides three different protection measures: the sender's serial number, transaction expiration time and specified chain ID.

A transaction's sequence number can only be submitted once per sender's account. Therefore, if the sender finds that the current account sequence number ≥ the sequence number of transaction t, then t has already been committed, or t will never be committed (because the sequence number used by t is already taken by another transaction).

Blockchain time is advanced with high precision and high frequency (typically sub-second), see Section 7.3.1 for details. If the blockchain time exceeds the expiry time of transaction t, then again, either t has been committed, or t will never be committed.

Each transaction has an assigned chain ID to prevent replay attacks by malicious entities between different blockchain environments (e.g. across testnet and mainnet).

6.2 Move-based key management

As mentioned in Section 5.2, Aptos accounts support key rotation (key rotation), which is an important feature that reduces the risk of private key leakage, remote attacks, and future cracking of existing cryptographic algorithms. In addition, Aptos accounts are flexible enough to support a new hybrid custody model, where users can delegate the ability to rotate account private keys to one or more custodians and other trusted entities. A policy is then defined through the Move module that enables these trusted entities to rotate keys under certain circumstances. For example, an entity might be a kout-of-n multisig key held by many trusted parties, thus providing a key recovery service in case a user key is lost (e.g. 20% of bitcoins are currently locked in the account [7]).

In addition, although many wallets provide various key recovery schemes such as cloud-backed private keys, multi-party computation, and social recovery, these schemes are not based on blockchain implementations (i.e., off-chain). Therefore, each wallet must implement its own key management scheme, and for users, key management becomes a black box. On the contrary, the key management function in Aptos provides a complete and transparent key management operation, and at the same time greatly reduces the difficulty of implementing wallet key management solutions.

6.3 Pre-signed transaction transparency

Today, wallets are mostly opaque to the transactions they sign. Therefore, users are often deceived by malicious transactions, resulting in loss of funds and a series of serious consequences. Even a blockchain that can query the data of each transaction on the chain cannot avoid this loss. There are currently few user safeguards in place, exposing users to a wide variety of attacks.

To solve this problem, the Aptos ecosystem provides a transaction pre-execution service: the result of a transaction (in human-readable form) is provided to the user before they sign it. Aptos will help reduce fraud by combining transaction pre-execution services with previously known attacks and malicious smart contracts. Additionally, Aptos allows wallets to impose restrictions on transactions during execution. Violation of these restrictions will result in aborted transactions to further protect users from malicious applications or social engineering attacks.

6.4 Practical Light Client Protocol

Simply relying on the API provider's TLS/SSL certificate to establish trust between the blockchain client and server does not adequately protect the client. Even if a valid certificate exists, wallets and clients cannot guarantee the authenticity and integrity of the data provided to them. Therefore, API providers may return false or malicious blockchain data, deceive third parties and perform double spend attacks.

Figure 4: All phases of the transaction processing life cycle are completely independent and can be parallelized

To prevent this from happening, Aptos provides proof-of-state and light-client verification protocols that wallets and clients can use to verify the validity of data provided by untrusted third-party servers. Additionally, as described in Section 7.6.2, using timestamp-based proofs of state, light clients can either track changes in the network configuration (*epoch changes)* or synchronize from currently trusted nodes (*anchors)* Up-to-date state [8], to keep the account state real-time (e.g., within seconds). With high-frequency timestamps and low-cost state proofs, Aptos provides secure services to clients.

In addition, Aptos nodes also provide a wealth of high-performance storage interfaces. After further fine-tuning these interfaces in the future, they will support proof of specific data and accounts on the subscription chain. This allows light clients to retain only minimal verifiable data in the future without running a full node or processing a large number of transactions.

7. Streamlined, batch, and parallel transaction processing

In order to maximize throughput, increase concurrency, and reduce engineering complexity, the transaction processing process of the Aptos blockchain is divided into different stages. Each stage is completely independent and can be parallelized independently, similar to modern superscalar processor architectures. This not only provides significant performance benefits, but also enables the Aptos blockchain to offer new validator-client interaction models. For example:

• Clients are notified when a given transaction is included in a batch of reserved transactions. Transactions that are on hold and valid are generally committed immediately.
• Clients are notified when a batch of reserved transactions is ordered. Therefore, to reduce the delay in determining the outcome of a transaction to execute, a client can execute a transaction locally instead of waiting for a remote validating node to complete execution.
• Clients can choose to wait for the verifier to execute the authenticated transaction, and then synchronize the completed transaction status (see Section 8).

Aptos module design helps to increase development speed and shorter release cycle, and developers can optimize individual modules instead of the whole system. Similarly, the modular design provides a structured path to scale the validator, allowing the validator to utilize the computing, network and storage resources of multiple machines. Figure 4 shows the various processing phases of the transaction processing lifecycle.

7.1 Batch processing

Batching is an efficiency optimization important to every stage of Aptos processing. When broadcasting transactions, validators group transactions into batches; during the consensus phase, batches are merged into blocks. The execution, storage, and ledger authentication phases also batch operations, providing opportunities for reordering, reducing operations (such as double computation or signature verification), and parallel execution.

Batching transactions may incur a small amount of delay, such as waiting 200ms for a batch to fill up before propagating the transaction. However, the maximum waiting time for batching and the maximum batch size are configurable, allowing the decentralized network to automatically optimize latency and efficiency. Batching enables efficient fee market prioritization of transactions while avoiding denial of service (DoS) attacks by malicious clients.

7.2 Continuous transaction broadcast

According to the main viewpoint of Narwhal & Tusk [9], transaction propagation and consensus in Aptos are decoupled. Validators are constantly transferring batches of transactions to each other while utilizing all available network resources. Each batch distributed by verifier v is persisted and a signed batch digest is sent back to v. Any 2f + 1 stake-weighted signatures on the batch digest form a Proof of Availability (PoAv) according to the consensus requirements defined in Section 7.3. Such a proof (PoAv) guarantees that at least f+1 weighted honest validators have stored the batch, so all honest validators will be able to retrieve it before execution.

Unlimited pending transactions could create a DoS attack causing validator nodes to run out of storage and crash. To prevent this, each batch of transactions has an associated timestamp. Timestamps on batches allow efficient garbage collection by validators. In addition, a quota mechanism for single validator nodes is also designed to ensure that even in the most extreme cases (such as potential Byzantine attacks), the space will not be exhausted. At the same time, there is a limit on the amount of data in batches, and Aptos will verify the data size in persistent storage. Finally, through some optimizations for transaction cache and redundant transaction data cleaning, not only the storage cost is reduced, but also the high-performance integration of parallel execution engines is guaranteed.

7.3 Block metadata sorting

There is a common misconception that slow consensus speed is the main bottleneck for high throughput and low latency of blockchains. One of the key innovations of the Aptos blockchain is the decoupling of non-protocol related tasks from the consensus stage, such as transaction propagation, transaction execution/storage, and ledger authentication. By decoupling transaction propagation from the consensus phase, ordering can be performed at extremely low bandwidth (block metadata and proofs only), resulting in high transaction throughput and minimal latency.

Today, the Aptos blockchain uses the latest version of DiemBFTv4 [10], an optimistically responsive BFT consensus protocol. Usually, consensus only requires two network round trips (the whole network round trip time is usually less than 300 milliseconds), and abnormal validators are dynamically adjusted through the leader reputation mechanism [11]. The on-chain leader reputation mechanism will promote the reputation of validators who successfully submitted blocks during the "window period", and demote validators who did not participate. This novel mechanism significantly improves the performance of decentralized systems, provides proper incentives for nodes, and minimizes the impact of inactive validators on throughput and latency.

DiemBFTv4 guarantees liveness under partial synchronization and security under asynchrony. When the total validator stake is greater than or equal to 3f + 1, there can be at most f stake-weighted wrong validators. Since 2019, DiemBFTv4 has undergone extensive testing with dozens of node operators and a multi-wallet ecosystem in multiple iterations. We are also experimenting with our recent research (e.g. Bullshark [12]) and other protocols that rely on block history and related communications to determine block metadata ordering and finality.

Consensus blocks and proposal timestamps are proposed by the leader and agreed upon by other validators, as shown in Figure 5. It is important to note that each consensus block only contains metadata and proofs of the batch. No actual transactions are required in consensus blocks, as PoAV ensures that batches of transactions are available at the execution stage after block metadata ordering (see Section 7.2). Validators can vote on the leader's proposal after the proof is verified and the block metadata criteria are met (e.g., the proposal timestamp cannot exceed the block expiration time).

7.3.1 Blockchain time

The Aptos blockchain assigns each proposed block and all corresponding transactions within that block an approximate, agreed upon physical timestamp. This timestamp supports a number of important use cases. For example:

• Time-related logic in smart contracts. For example, a developer wants to code that all bids for an auction must be received by 12 noon on a Thursday.
• As oracles publish on-chain data, accurate and trusted on-chain timestamps are required to correlate events and handle latency from real-world data.
• Clients can tell how up-to-date they are on the blockchain. For security reasons, to avoid stale data and remote attacks, clients should have access to a high-precision timestamp of when the account state was updated.
• Auditing the blockchain with trusted timestamps can provide strong correlation to off-chain events, such as ensuring that legally enforced spending is as expected.
• Transaction expiration is based on the most recent commit timestamp. As an additional protection for client-side transactions, clients can choose the expiration time of the transaction, as described in Section 6.1.

The Aptos blockchain provides the following guarantees for the timestamping of all transactions within a block:

• In the blockchain, the timestamp is monotonically increasing. Suppose block B1
• If the timestamp T of a transaction block is confirmed, then at least f + 1 honest validators believe that the time point T has passed. Honest verifiers can only vote on blocks when their own clock ≥ block timestamp T. See Section 7.2.
• If a transaction block has a certain number of consensus signatures on the timestamp T, then the honest verifier will

The most recent timestamp is updated on each committed block and used as the timestamp for all transactions in that block. When the network is in sync, a block of transactions is committed for every network roundtrip, providing fast updates and highly reliable timing. A finer-grained ordering within transaction blocks can be determined if desired.

7.4 Parallel transaction execution

Once the consensus block metadata is sorted, any validator node, full node, or client can execute the transaction. At least 2f+1 weighted validators have verifiably ongoing transactions on the proposed batch. Since transaction propagation is continuous, other honest validators will receive batches of transactions over time. If an honest validator node has not received an ordered batch of transactions by the time it reaches the execution phase, it can download them from 2f+1 stake-weighted validators, known to have at least f+1 stake-weighted validators (no less than half of the stake-weighted PoAV signers) are honest.

An important goal of blockchains is to achieve as much parallel processing as possible. Aptos blockchain starts from two aspects of data model and execution engine.

7.4.1 Parallel Data Model

The Move language data model natively supports global addressing of data and modules. Transactions with non-overlapping and conflicting data and accounts can be executed in parallel. Given the pipeline design used in Aptos, the reordering of transactions can reduce conflicts and thus improve concurrency.

Even if transactions modify the same set of on-chain values, most transactions can still be executed in parallel. The Aptos blockchain introduces a new concept, delta writes, which describe modifications to the account state rather than the modified account state (e.g. incrementing an integer rather than simply determining the final value). All transaction processing can be done in parallel, and then delta writes to conflicting values are done in the correct order to ensure deterministic results.

Over time, the Aptos blockchain will continue to enhance the data model by increasing concurrency (e.g., utilizing read/write hints) and improving the developer experience, allowing developers to create, modify and combine on-chain values more naturally . Move provides flexibility for language-level and platform-specific feature enhancements.

7.4.2 Parallel execution engine

The Block-STM parallel execution engine detects and manages conflicts of ordered transactions while performing optimistic concurrency control to achieve maximum parallelism under a specific order [13].

Batch transactions are parallelized using optimistic locks and verified after execution. Validation failures will result in a re-execution. Block-STM uses multi-version data structures to avoid write-write conflicts. All writes to the same location are stored with their version, which contains their ID and the number of times they were optimistically retried. When the transaction tx reads the memory data, it will obtain the transaction with the highest block height appearing before tx from the multi-version data structure in the preset order, and write the value of the transaction and its related version.

Block-STM has been integrated into the Aptos blockchain To understand the performance potential of Block-STM, we use an in-memory database, with meaningful (non-trival) point-to-point Move transactions (eg: 8 reads and 5 write) as a standalone, execution-only (not end-to-end) benchmark. In Fig. 6, we show the execution results of Block-STM. Each block contains 10,000 transactions, and the number of accounts determines the degree of conflict and dispute.

At low contention, Block-STM's TPS is 16 times that of 32-thread linear execution, and at high contention, Block-STM's TPS is also increased by 8 times. Unlike other blockchain parallel execution engines, Block-STM is able to dynamically and transparently (without user prompting) capture the intrinsic parallelism of any workload. BlockSTM can simultaneously support more complex transactions than parallel execution environments that require advance knowledge of the location of data to be read or written. This property results in fewer but more efficient transactions, reduces costs, and provides lower latency for users. More importantly, splitting things into multiple small transactions breaks the atomicity of transactions (all operations are indivisible, either all operations succeed or all fail). Combining expressive transactional semantics with parallel execution in Block-STM enables developers to have the best of both worlds.

Note that the block metadata ordering step does not preclude transaction reordering during the parallel execution phase. To optimize concurrency performance for parallel execution, transactions can be reordered across blocks. The only requirement is that the reordering of all honest validators must be irreversible. Optimizing parallel execution and adding randomization to the reordering can improve performance and potentially prevent miner extractable value (MEV) technology from being reordered by validator transactions with ulterior motives. In this pipeline design, an anti-MEV strategy of "order-then-reveal" (Order-then-reveal) can also be added.

Block-STM and transaction reordering are complementary techniques to increase execution concurrency. They can be combined with transactional read/write access hints for additional concurrency.

7.5 Bulk storage

The result of the parallel execution phase is the write set of all transactions in the group. These write sets can be stored in memory for maximum execution speed and then used as a cache for the next block or set of blocks to be executed. Any overlapping writes only need to be written to stable storage once. If a validator fails before storing the in-memory writeset, it can simply resume parallel execution from the block metadata ordering phase. Separating the bulk storage of write sets from the parallel execution step ensures that parallel execution can run efficiently. In summary, batching write sets reduces the number of storage operations and takes advantage of more efficient, larger I/O operations.

The amount of memory reserved for the write-set cache can be manually configured on a per-machine basis and provides a natural back-pressure mechanism. The granularity of batch processing can be different from the granularity of parallel execution blocks if tuning is required for specific I/O and memory environments.

7.6 Ledger Authentication

At this point (batch storage), each individual validator on the pipeline has computed a new state for the committed transaction block. However, to efficiently support authenticated light clients and state synchronization, the Aptos blockchain implements ledger authentication for ledger history and ledger state. A key difference with the Aptos blockchain is that ledger authentication is not on the critical path of transaction processing, and can even run completely out-of-hand if required.

7.6.1 Account history verification

Validators append transactions, along with their execution outputs, into a globally authenticated ledger data structure. Part of the transaction output is the set of state writes, including changes made to the global state accessible by the Move. The short validator for this data structure is a binding commitment to the ledger history, which includes a newly executed batch of transactions. Similar to transaction execution, the generation of this data structure is deterministic.

Each verifier signs the short authenticator into the resulting new version of the database. Validators share with each other their most recent set of signed short validators, collectively aggregate quorum-signed short certifiers, and also share with each other the most recent quorum-signed short validators.

According to the properties of the BFT protocol, using this collective signature, the client can trust that the database version represents a complete, valid and irreversible ledger history. Clients can query any validator (or any third-party copy of the database, such as a full node) to read database values, and use the validator and a proof of the required data to verify the result.

7.6.2 Periodic status certification

The entire global state accessible by the Move language can be aggregated at any point in history into a short validator, similar to a summary of a ledger's history. Due to the random-access nature of the global state (unlike the appendix-only ledger history), maintaining this certification is costly. However, when updating data structures in large batches, we can compute the updates in parallel and also take advantage of any overlap between the parts that must be updated when each individual state value changes. The Aptos blockchain consciously uses periodic validation of global state to reduce duplicate shared updates.

During a deterministic and configurable period of time, the network publishes state collation point transactions, the result part of which includes the global state authenticator. This version is called a state checkpoint. The larger the gap between two collation points, the lower the amortized cost of updating the state authentication data structure with each transaction.

With a state collation point, people can read any state value in a trustless way without storing all the global state. This capability works well for applications such as incremental state synchronization, sharded storage across validator nodes, stateless validator nodes, and storage-constrained light clients.

However, since state checkpoints are periodic, obtaining proofs of a particular version of the ledger state either requires alternately executing additional transactions for the missing states, or obtaining proofs containing them from the verified ledger history.

State checkpoints are associated with specific transaction versions in the ledger history, and thus tied to the timestamps associated with transaction batches mentioned in Section 7. Timestamps allow light clients to understand the timeliness of proven state values. In the absence of timestamps, light-client proofs can only ensure the validity of previous states from a long time ago, which provides little guarantee of associativity. Additionally, timestamping of state proofs is necessary for tracking historical access and auditing (eg: calculating the hourly average balance of tokens in the token reserve).

State checkpoints can be generated based on previous state checkpoints and subsequent state changes in transaction results. Therefore, persisting state checkpoints to stable storage does not need to be on the critical path of transaction processing. In addition, the retention of state verification points also has a beneficial batch processing effect. Caching the most recent state checkpoints (or rather the deltas between them) in memory, and dumping only periodic state checkpoints to stable storage, can greatly reduce consumption of storage bandwidth. The choice of how the checkpoints are retained does not affect the computation of the verified data structures. Therefore, it is a choice for each node: the node operator can make the appropriate trade-off between memory capacity and storage bandwidth.

8. State Synchronization

The Aptos blockchain aims to provide a high-throughput, low-latency system for all participants in the ecosystem. Therefore, the blockchain must provide an efficient state synchronization protocol to propagate, verify, and persist blockchain data to light clients, full nodes, and verification nodes [14]. In addition, considering the different hardware resources of users, the synchronization protocol must also be compatible with resource limitations and differences existing in the network. For example, it must allow archival full nodes to verify and store the entire blockchain state and history, while also allowing light clients to efficiently access only a small portion of the blockchain state.

To achieve this feature, the Aptos blockchain utilizes certified ledger history and state proofs provided by validators, full nodes, and other synchronizers (see Section 7.6.1) to implement a flexible and configurable synchronization protocol . Specifically, network participants can choose different synchronization strategies to optimize their own use cases and needs.

For example, Aptos provides a variety of synchronization strategies for full nodes, including a full synchronization strategy; and a strategy that ignores historical records and uses anchors to only synchronize the latest blockchain state. Aptos provides a variety of synchronization strategies for light clients, including a partial synchronization strategy that can synchronize specific accounts or data; and a get verified data strategy that can get verified account balances. In all cases, Aptos allows participants to acquire, process and store specific quantities and versions of data through configuration configurations.

Through this flexible and configurable state synchronization protocol, Aptos can meet the needs of various customers and provide more advanced and efficient synchronization strategies in the future.

9. Community Governance

The Aptos blockchain will be owned, operated and governed by a broad and diverse community. The native Aptos token will be used for transaction and network fees, protocol upgrades and governance voting for on-chain/off-chain processes, as well as securing the blockchain through a proof-of-stake (PoS) model. The specific economic model of Aptos tokens will be released later.

9.1 Transaction and Network Fees

All Aptos transactions have a fee unit price (specified in the Aptos token), allowing validators to prioritize the highest value transactions in the network. In addition, at each stage of the pipeline model, there are multiple opportunities to abandon low-value transactions (as far as possible to ensure that the blockchain can still run efficiently at the maximum system capacity). Over time, the deployment of fees will ensure that the cost of using the Aptos blockchain is proportional to the actual cost of hardware deployment, maintenance, and node operations. In addition, applications designed by developers can make trade-offs between computing, storage, and networking according to different costs.

9.2 Network Governance

Every major feature optimization and iteration on the Aptos blockchain will go through several stages, including proposal, implementation, testing and deployment. This structure provides opportunities for interested parties and stakeholders to provide feedback, share concerns and make suggestions. As the final stage, deployment is usually done in two steps. First, a software version with a new feature will be deployed to each node, and second, the feature will be turned on, for example, via a feature flag or an on-chain configuration variable.

Every software deployment by node operators must be backward compatible to ensure that new software is interoperable with supported versions. The process of deploying a new software version may take several days to account for operators in different time zones and any external issues. Once a sufficient number of nodes have been upgraded, the enablement of new features can be triggered by a synchronization point, such as a pre-agreed block height or epoch switch. In emergency situations (e.g. when downtime is unavoidable), restarts can be enabled through manual and forced changes by node operators, and in worst cases enabled by hard forks in the network.

In contrast to other blockchains, the Aptos blockchain encodes its configuration on-chain. Each validator is able to synchronize with the current state of the blockchain and automatically select the correct configuration (eg, consensus protocol and Aptos framework version) based on current on-chain values. Based on this functionality, upgrades in the Aptos blockchain are seamless and instant.

To enable flexibility and configurability during the enablement process, the Aptos blockchain will support on-chain governance where token holders can vote based on the weight of their staked tokens. On-chain voting protocols are public, verifiable and can be instant. On-chain governance also enables non-binary outcomes without software deployment. For example, on-chain leader election protocol parameters can be modified through on-chain governance, while pre-known synchronization points cannot handle dynamic modification, because all changes must be known in advance.

On-chain governance can be deployed throughout the upgrade management process over time. For example:

1. Token holders vote on-chain to transition to a new quantum-resistant signature scheme

2. The developer implements and verifies the new signature plan and creates a new software version.

3. Validators upgrade their software to newer versions.

4. Token holders vote on-chain to enable a new signature scheme, the on-chain configuration is updated, and the changes take effect.

As an open source project, the governance of the Aptos blockchain relies heavily on strong community feedback and on-chain governance. In some cases, it may still be necessary to enable off-chain upgrades, but this will be minimized over time.

9.3 Proof of Stake Pledge Consensus

To participate in transaction validation on the Aptos blockchain, a validator must have a minimum stake of Aptos tokens. During the leader election process in transaction propagation, voting weight and block metadata ordering, the stake amount affects 2f + 1 stake weight PoAv proportionally. Validators decide the distribution of rewards between themselves and their respective stakers. Stakers can choose any number of validators to stake their tokens for a pre-agreed distribution of rewards. At the end of each epoch, validators and their respective stakers will be rewarded via the associated on-chain Move module.

Any validator with sufficient collateral is free to join the Aptos blockchain. All parameters, including the minimum required stake value, can be set by the on-chain enabler described in Section 9.2.

10. Performance

As described in Section 7, the Aptos blockchain is able to achieve optimal throughput and hardware efficiency through its parallel, batch-optimized, and modular transaction processing pipeline. Additional performance initiatives such as consensus upgrades, delayed writes, transaction hints, and critical path caching will continue to increase throughput and improve efficiency over time.

Today, blockchain throughput is often measured in transactions per second. However, this is an imprecise way of comparing systems given the varying levels of transaction and infrastructure cost and complexity. Transaction delays are also flawed, as commits to finality start and end differently in different experiments.

Additionally, some systems require prior knowledge of transaction inputs and outputs and force logical transactions to be broken down into smaller, less complex transactions. Splitting transactions results in a poor user experience and artificially affects latency and throughput, ignoring what the developer is trying to achieve. In contrast, Aptos' approach gives developers unlimited freedom to build and measure throughput and latency against real-world use cases rather than synthetic transactions.

Aptos Blockchain will continue to optimize the performance of individual validators, as well as experiment with scaling techniques that add more validators to the network. There are obvious trade-offs in both directions. Any blockchain capable of parallel execution can support greater concurrency by requiring more powerful hardware or even constructing each validator as a separate cluster of machines. However, there is a practical limit to the number of global validators, which corresponds to the cost and complexity of validator operators. The rise and popularity of serverless databases in cloud services has illustrated that there are few entities capable of effectively deploying and maintaining these types of complex distributed systems.

10.1 Homogeneous state sharding

Initially, the Aptos blockchain will launch with a single ledger state. Over time, the Aptos network will take a unique approach to horizontal scalability while still remaining decentralized. This will be achieved through the state of multiple sharded ledgers, each of which provides a homogeneous API and uses sharding as a main concept. Aptos tokens will be used for transaction fees, deposits and governance of all shards.

Data can be transferred between shards through homogeneous bridges. Users and developers can choose their own sharding scheme according to their needs. For example, a developer can propose a new shard, or group users within an existing shard to achieve high connectivity within a shard. Additionally, shards may have different system characteristics. One shard can be optimized for computing with SSDs, and the other shard can be optimized for low-performance large storage. By providing hardware flexibility between different shards, developers can maximize the utilization of system resources for their applications.

In summary, homogeneous state sharding offers the potential for horizontal throughput scaling, allows developers to program with a single common state across shards, and enables wallets to easily incorporate shard data for their users. This provides significant performance advantages, combined with the simplicity of a unified Move smart contract platform.

References

1. "Aptos-core," 2022.
   Online documentation. Available at: https://github.com/aptos-labs/aptos-core
   
2. "Move," 2022.
   Online documentation. Available at: https://github.com/move-language/move
   
3. Matsuoka, C. Dixon, E. Lazzarin, and R. Hackett. (2022) Presentation of the State of Encryption 2022 report. Online documentation. Available at: https://a16z.com/tag/state-crypto-2022/
4. Amsden, R. Arora, S. Bano, M. Baudet, S. Blackshear, A. Bothra, G. Cabrera, C. Catalini, K. Chalkias, Cheng, A. Ching, A. Chursin, G. Danezis, GD Giacomo , DL Dill, H. Ding, N. Doudchenko, Gao, Z. Gao, F. Garillot, M. Gorven, P. Hayes, JM Hou, Y. Hu, K. Hurley, K. Lewi, C. Li, Z . Li, Malkhi, S. Margulis, B. Maurer, P. Mohassel, L. de Naurois, V. Nikolaenko, T. Nowacki, O. Orlov, Perelman, A. Pot, B. Proctor, S. Qadeer, Rain, D. Russi, B. Schwab, S. Sezer, A. Sonnino, H. Venter, L. Wei, N. Wernerfelt, B. Williams, Q. Wu, X. Yan, T. Zakian and R. Zhou, “libra Blockchain," 2019.
   Online documentation. Available at: https://developers.diem.com/papers/the-diem-blockchain/2020-05-26.pdf
   
5. Blackshear, E. Cheng, DL Dill, V. Gao, B. Maurer, T. Nowacki, A. Pott, S. Qadeer, DR Rain, S. Sezer, T. Zakian, and R. Zhou, “Move: A The Language of Programmable Resources," 2019.
   [Online]. Available at: https://developers.diem.com/papers/diem-move-a-language-with-programmableresources/2019-06-18.pdf
   
6. Dill, W. Grieskamp, J. Park, S. Qadeer, M. Xu, and E. Zhong, "Fast and Reliable Formal Verification of Smart Contracts with the Move Validator," published in Tools and Algorithms for the Construction and Analysis of Systems, D. Fisman and G. Rosu, Eds. Cham: Springer International Publishing, 2022, pp. 183-200.
7. Popper. (2021) Lost passwords lock up millionaires' bitcoin fortunes.
   Online documentation. Available at: https://www.nytimes.com/2021/01/12/technology/bitcoin-passwords-wallets-fortunes.html
   
8. Diem Group, "State Synchronization and Verification of Commitment Information in Reconfigured Systems", 2020.
   Online documentation. Available at: https://github.com/aptos-labs/aptoscore/blob/main/documentation/tech-papers/lbft-verification/lbft-verification.pdf
   
9. Danezis, L. Kokoris-Kogias, A. Sonnino, and A. Spiegelman, "The narwhal and the tusk: dag-based mempool and efficient bft consensus," published in Proceedings of the Seventeenth European Conference on Computer Systems, ser. EuroSys' 22. New York, NY, USA: Association for Computing Machinery, 2022, p. 34-50.
   Online documentation. Available at: https://doi.org/10.1145/3492321.3519594
   
10. Diem Group, "Diembft v4: State Machine Replication in the diem Blockchain", 2021.
    Online documentation. Available at: https://developers.diem.com/papers/diem-consensus-state-machine-replication-in-the-diemblockchain/2021-08-17.pdf
    
11. Cohen, R. Gelashvili, L. Kokoris-Kogias, Z. Li, D. Malkhi, A. Sonnino, and A. Spiegelman, “Be known of your leaders,” CoR, vol. abs/2110.00960, 2021.
    Online documentation. Available at: https://arxiv.org/abs/2110.00960
    
12. Spiegelman, N. Giridharan, A. Sonnino, and L. Kokoris-Kogias, “Bullshark: A practical implementation of the Dag bft protocol,” in Proceedings of the 20th Conference on Computer and Communications Security (CCS), ser. CCS [22]. Los Angeles, CA, USA: Association for Computing Machinery, 2022.
13. Gelashvili, A. Spiegelman, Z. Xiang, G. Danezis, Z. Li, Y. Xia, R. Zhou, and D. Malkhi, “Block-stm: Scaling blocks by turning the curse of sorting into a blessing for performance Chain of Execution," 2022.
    Online documentation. Available at: https://arxiv.org/abs/2203.06871
    
14. Lind, "The Evolution of State Synchronization: A Path to 100,000+ Transactions per Second, Sub-Second Latency in aptos," 2022.
    Online documentation. Available at: https://medium.com/aptoslabs/52e25a2c6f10
    

Article source: Buidler DAO