Sharding in TON: How a 90s idea helped scale blockchain and solve its main problem

Telegram Open Network (TON), conceived by Pavel Durov along with his brother Nikolai, a candidate of physical and mathematical sciences and a brilliant programmer. It was Nikolai who developed the first version of the Telegram messenger for communication with Pavel, and later, the MTProto encryption protocol used by Telegram.

Nikolai Durov outlined his vision for the TON blockchain in a white paper in 2019. The main goal of TON, as envisioned by the brothers, was to monetize Telegram. However, the platform failed to launch, and the Durovs abandoned the idea, making the blockchain platform's code open to all interested parties.

From that point on, independent developers from around the world began to develop the product. Later, they formed the TON Foundation community, which today supports technology and projects within the TON ecosystem.

TON is a crypto ecosystem developed by an independent community based on technology originally developed by Telegram.

TON has its own currency called Toncoin.

We provided more detailed information about TON in this material.

TON positions itself as an open network for everyone — from startups to crypto enthusiasts. Source: ton.org

Tightly integrated with Telegram, next-generation smart contracts, anonymity, transaction speed, and low fees make TON a unique and promising technology, setting it apart from others.

Integration with Telegram. The integration with the popular messenger provides convenient and quick access to TON for millions of users worldwide.

Next-generation smart contracts. TON utilizes an advanced smart contract system enabling the creation of complex decentralized applications (DApps) and automating financial operations, opening up new opportunities for business and technology.

Built-in anonymity. One of TON's key distinctions from many other blockchains is its focus on transaction privacy and user anonymity. Various encryption and anonymization technologies make TON attractive to users who prioritize confidentiality.

Low fees. TON users can conduct micro-payments with low fees, making this blockchain an ideal platform for mass microtransactions on the internet, such as payments for content and services with pay-per-action models.

Participation in the ecosystem. In addition to the TON cryptocurrency, TON offers users other opportunities: cloud storage, anonymous networks, DNS, and other technologies. This makes TON not just a blockchain for transactions, but also a complete ecosystem for the development and use of various decentralized services.

Flexibility and adaptability. Thanks to its unique architecture, TON can adapt to current market needs and technological trends, providing flexibility and the ability to implement new features without significantly impacting network performance and security.

Transaction speed. TON is officially recognized as the fastest blockchain in the world. In 2023, TON set a world record for transaction speed: the network processed over 104 thousand transfers per second. This became possible thanks to sharding.

Sharding is the division of data into several parts to reduce network load and increase transaction processing speed. These data parts are called shards. Each shard contains a unique set of data, making the database more efficient and scalable.

The concept of sharding emerged in the late 1990s when this method was first used to manage centralized databases.

Before blockchain, sharding was applied to distribute databases across different servers. For example, in businesses, when information from one server was transferred to other servers, organizing it based on specific criteria. This reduced the load on a specific server, enabling scalability.

To understand the principle of sharding, let's consider a simple example.

Sharding is like dividing a large pie into pieces. The number and size of the pieces into which we cut the pie can be absolutely arbitrary.

Another example. Let's say blockchain is like a huge library. If we keep all the books in one giant hall, it will be difficult to find what we need. That's why we divide the library into several rooms, or shards. Each room stores books of a specific genre: in one room — science fiction, in another — historical novels. And so on.

When a visitor comes to the library for a specific book, they are directed to the corresponding room. This greatly speeds up the search process, as there is no need to search through the entire library — just search in one small room. The library administrators can add new rooms as needed to expand the library and accommodate more books without sacrificing performance.

The creators of blockchain as an innovative technology did not focus on the scalability issue. At that time, it was unclear whether the idea would work or not, so it was more important to devise mechanisms that provide maximum security while favoring speed and energy efficiency.

When independent developers took on TON, they faced several key problems.

• Scalability. In a traditional blockchain system, every transaction is processed by the entire network. This limits the number of transactions that can be processed in a unit of time and increases the time and cost of transactions as the network grows.
• Performance. With the growth of users and transactions in traditional blockchains, network performance begins to decrease due to overload. This leads to transaction processing delays and increased fees.
• Energy consumption. Large and centralized blockchain networks often face high energy consumption because they need to maintain the operation of all network nodes. This is not only expensive but also harmful to the environment.
• Availability. As network load increases, the likelihood of failure or slowdown in network operation increases. As a result, the blockchain becomes less accessible to users.

Because of these problems, it would be difficult to use TON on a global scale. But developers found a solution in the form of sharding.

In addition to this, sharding provides the following capabilities to the TON blockchain:

• Transaction processing is distributed among different nodes, or shards, reducing the load on each individual node and speeding up transaction processing overall.
• With each added shard, the network's throughput increases, allowing it to process more transactions simultaneously, making the blockchain available to a growing number of users and operations.
• Thanks to more efficient transaction processing, transaction costs are reduced, and the blockchain becomes more economically viable for users.
• Distributed architecture through sharding increases the network's resilience to failures and attacks, providing a higher level of availability and security for users.

Important: Sharding in blockchain is not unique to TON. For example, Ethereum also has such a tool in its roadmap towards transitioning to an updated version of the ecosystem.

The approach used by TON is based on sharding and is further reinforced by a unique blockchain architecture known as infinite sharding.

The paradigm of infinite sharding allows the blockchain to simultaneously process different transactions and smart contracts and scale almost infinitely. This ensures high transaction speeds and availability of the technology as the number of users continues to grow.

The main idea is to divide data and transactions into many small blockchains, or shards, which can be processed in parallel. This significantly improves the overall performance of the network, as each shard is capable of processing transactions independently of the others.

As the network load increases, the data is divided into smaller parts. When the load decreases, the shards are merged again. The division and merging happen automatically.

The TON blockchain is divided into three levels: the masterchain, workchains, and shardchains.

The masterchain is the primary blockchain. It is singular and the entire system is subservient to it.

Workchains are the operational blockchains responsible for processing specific types of transactions. A workchain is responsible for grouping data into shardchains.

Shardchains are small blockchains, each handling transactions for its own set of accounts. It is these shardchains that provide TON with conditional infinite scalability by splitting into multiple shards as system load increases and then rejoining when load decreases.

Each node in TON is responsible for storing and verifying only its own data, not the data of the entire blockchain. blog.ton.org

Here are examples that will help understand how sharding works in TON, how the division into shards occurs, and how shards are merged.

Example 1: General principle of sharding in TON. Let's imagine that TON is a large city. Sharding in TON is like dividing a large city into districts. Instead of having one mayor manage the entire city, each district has its own manager who takes care of local affairs.

Masterchain: This is the city administration. It coordinates the work of all districts, ensuring overall order and consistency of actions.

Workchains: These are different departments within the district administration. Each department is responsible for its specific area, such as education and healthcare.

Shardchains: These are the city districts. Each district handles its own affairs, but they all adhere to common rules and the city administration.

Example 2: Division into shards in TON. Let's say there is a large apartment building where each staircase represents a shard.

Residents in each staircase communicate with each other, solve everyday issues—these are transactions within the shard. However, all staircases are in the same building and adhere to the common rules set by the tenants' council, i.e., the masterchain.

If something important happens in one staircase that affects the entire building, the information is passed to the tenants' council, which then informs the other staircases, i.e., the shards.

Example 3: Merging shards in TON. Sometimes, there is a need to address an issue that requires the agreement or involvement of residents from all staircases. For example, coordinating the installation of a new playground.

Information and decisions are gathered from representatives of each staircase and passed to the tenants' council.

The tenants' council then coordinates actions and ensures that the decision is executed in the interest of the entire building.

Thus, each staircase, which is also a shard, operates autonomously, but important decisions are coordinated and made at the level of the entire building, i.e., the masterchain.

• The idea of TON emerged in 2018 by Pavel Durov, but he soon stepped away from the project, and independent developers from around the world began its development.

• The main advantages of TON are its integration with Telegram, next-generation smart contracts, anonymity, transaction speed, and low fees.

• In 2023, TON became the fastest blockchain in the world.

• Sharding is the division of data into multiple parts to reduce network load and increase transaction processing speed.

• The concept of sharding predates blockchain and has been successfully used in business, electronics, and other fields.

• The approach used by TON is based on sharding and further strengthened by a unique blockchain architecture known as infinite sharding.

• With dynamic sharding, data is automatically divided and merged depending on the load on the blockchain.

• Thanks to sharding, TON processes transactions and other operations more efficiently, allowing many people to use the blockchain simultaneously.

• Sharding in TON is like dividing a large city into districts. Instead of one mayor managing the entire city, each district has its own manager who takes care of local affairs.