Sharding Explained in Simple Terms

As it stands, bitcoin is currently able to process a maximum of 7 transactions per second, while ethereum tops this with its 15 transactions per second. However, both of these blockchains don’t even come close to Visa’s 1,700 transactions per second. This makes it obvious that public blockchains have a scaling problem. And while many solutions have been proposed, including increasing block sizes and lowering block times, sharding has emerged to become the front-runner.

What you’ll learn

• What database sharding is and how it came to be.
• How sharding can be used to solve blockchain’s scalability problem.
• The downsides of sharding.
• Examples of blockchain networks already using sharding.

What is sharding

Sharding is the process of splitting a large database into smaller more-easily manageable components (shards). The goal is to lower the load on the database by separating it into many parts and hosting each part on a separate server, thus speeding up query times. Sharding is known as a horizontal type of partitioning, meaning it splits a database by rows, not by columns (vertical).

For example, when an application becomes popular and its database gets overloaded due to the increase in traffic, sharding is one of the best ways to ease the sudden increase in load.

Sharding and blockchain

The application of sharding in blockchains is very similar to its application in databases. Instead of splitting a database into many parts, it splits the nodes into different groups (shards). The different groups of nodes are then assigned only to a specific section of the network, eliminating the need for each node to store the entire blockchain locally. Let’s use the ethereum blockchain and its planned sharding update to illustrate exactly how sharding solves the scalability problem of public blockchains.

At the moment, all nodes on the ethereum network, currently a little over 7,000, have to validate every single transaction in each new block appended to the blockchain. This process obviously spends a lot of computational power, but also secures the network via proof of work.

The proposal for the implementation of sharding in ethereum, which should go live along with the deployment of ethereum 2.0, is to split the nodes into many shards, with each of the shards dealing only with a certain portion of the transactions on the network. That way, parallel execution and verification of transactions becomes possible.

In an official AMA session focused on the ethereum 2.0 update, Justin Drake, researcher at the Ethereum Foundation, stated that, if the number of nodes remains the same and the network switches to a full proof-of-stake consensus algorithm, the ethereum network would be able to handle about 10,000 transactions per second:

Rough ballpark figures. 1024 shards * 10 transactions per second per shard ~= 10k transactions per second.

Downsides

One of the bigger downsides of sharding is the dip in security. As it stands, the ethereum network is vulnerable to a 51% attack in terms of blockchain integrity. However, let’s say the network is split into 100 shards. Each shard handles a different subset of computations and each shard, for the purposes of this example, represents 1% of the total hash power of the network. This means that a 1% attack becomes possible i.e. it would take only 1% of the total network hash rate for a malicious actor to take over a certain shard and do with it whatever they please. In a case like that, data could be irreversibly lost.

To handle this issue, ethereum will introduce a random component to sharding, meaning nodes will be randomly assigned to a shard when they join the network, and they will also be periodically and randomly re-assigned to other shards on the network.

Another downside is the fact that each shard becomes somewhat of a closed environment. Nodes working within a certain shard will only be able to communicate with other nodes within the same shard. To solve this, ethereum will use a separate protocol that will allow cross-shard communication.

In short, in terms of downsides, sharding heightens the complexity of the architecture of a blockchain network, making it more prone to unforeseen exploits. That is part of the reason the ethereum 2.0 update has been postponed so many times. The researchers and developers are trying to design, implement and test everything in a way that makes ethereum 2.0 bulletproof. Hopefully, when they deploy it, that will be the case.

Blockchain projects that already use sharding

The two most notable examples are Zilliqa and NEAR Protocol.

Zilliqa is the first blockchain project to deploy sharding into its network. Binance research suggests that Zilliqa can currently handle about 2,800 transactions per second, with the project itself claiming a range between 2,000 and 3,000 – depending on the number of nodes and shards. To be more specific, here’s an excerpt from the official Zilliqa website:

For instance, if 6 shards (each with 600 nodes) process transactions individually, then all the shards will collectively be able to process around 2828 transactions per second. The sharded network architecture also allows the throughput of the platform to increase almost linearly as the network expands.

The NEAR Protocol is a developer-friendly, sharded, proof-of-stake public blockchain. It uses a novel sharding and consensus mechanism titled Nightshade. Similar to ethereum, NEAR Protocol also focuses on decentralized applications, though they pride themselves on making dapps “nearly as easy to build and use as today’s web apps”.

Summary

Sharding, alongside off-chain storage of data, is at the forefront of solving blockchain’s scalability problem. It provides a proven method to segment a blockchain into many small parts (shards), lowering computational load exerted on each node and allowing parallel execution of transactions, all while preserving the decentralized nature of distributed ledgers intact.

While there are certainly downsides to sharding, blockchain researchers and developers are working on different applications of the mechanism, such that 1% attacks and other issues remain only theoretical. With the deployment of ethereum 2.0 closing in, perhaps the biggest production-scale test of sharding is about commence.