Sharding has morphed from an obscure concept in 2017 into the buzz-word of 2018. The need for blockchain scalability has become glaringly obvious, and several projects have turned to Sharding as a solution.

However, Sharding a blockchain is not a simple task. In fact, it poses challenges that have our best thought-leaders scratching their heads. Several projects have made lofty promises of future scalability using Sharding. But, only few have provided a viable Sharding solution for mass adoption. RadixDLT is one of the rare projects that brings forth a novel approach to Sharding –– an approach that seeks to meet the demands of mass adoption

Right from its conception, the goal for Radix was:  Every single person, on every single device using a single protocol simultaneously.

​Every single person, on every single device using a single protocol simultaneously.

The RadixDLT Sharding architecture was designed (unlike other projects that approached Sharding “after-the-matter”)  to allow for unbounded scalability while maintaining security, and maximizing decentralization.

In this post, I will explain how Sharding works in Radix in a simple way – without any technical jargon.

Break a window, and you have shards of glass. Break an iceberg and you have shards of ice. A shard is simply a broken piece of ‘something’.  So, when you “shard” something, you’re simply breaking it into smaller pieces.

But, why do we shard? Well, you usually shard something because it’s easier to manage. For example, we ‘shard’ a large pizza because it’s easier to eat one slice at a time. It also allows us to share (distribute) the pizza with friends a lot easier.

Similarly, when a database gets too large to handle, we shard it and distribute it across multiple computers. There you go –– you now understand distributed computing & Sharding. It’s really that simple.

Sharding has been used to partition databases for a long time. You simply cut the database (think of an excel sheet) horizontally into several pieces and distribute across multiple “database servers” (machines that ‘serve’ you data when you need it). When the data needs to be retrieved or processed, the relevant database server is called upon to do the task.

In Blockchain these “servers” are what we call “nodes”.  However, Sharding a decentralized system isn’t as straightforward as we’d like. There are complications that a centralized system doesn’t need to concern itself with.

Every distributed system requires a Consensus Method. But, developing a Consensus Method for a Sharded blockchain is tricky. You find yourself sacrificing security in favor of scalability.

Why? Well, a huge component of the security comes from the fact that every node stores all the data. Since every node keeps a copy of the entire database – you can’t cheat/lie about past events. But when you shard that database, each node stores partial data. Suddenly, you can tell Node Bob one thing, and tell Node Lisa another.

To understand this better, think back to when you were a kid.

Remember when your Mom grounded you and your Dad didn’t know about it? If you were anything like me, you tried to sneak out by asking your Dad for permission. Dad didn’t have all the info. And you took advantage of it.

Mom and Dad represent a Sharded database. Sure, together they have all the info needed to run the household. But as individuals, they don’t – and can be lied to about past events (like you being grounded or not)

Founder & CTO  Dan Hughes identified the scalability issues that would plague Bitcoin back in 2012. After several attempts at improving the protocol, he realized that the only solution is a brand new architecture and consensus method. Six years and a lot of sweat later, he brought us Radix DLT – a unique Sharding approach and consensus algorithm.

Radix DLT approaches Sharding in a unique but simple manner.  Most projects take existing Consensus Methods and build a Sharding solution on top of it. But as discussed, this leads to sacrificing security. For example, Sharding on PoS network could result in a One Percent Shard Attack.

Radix, however , started with a Sharded network –– and built a unique consensus method on top of that network. This “Sharding-first” approach allowed them to bypass the limitations faced by other consensus methods.

Essentially, Radix began with the end state in mind, which is:  every single person on every single device using a single protocol.

Radix’s pre-sharded network serves as a fundamental around which they have designed their consensus method (Tempo).

The size of each shard, and the number of total shards are predetermined. Nodes then place themselves atop shards – and can overlap with other nodes in layers. 

This is where it gets interesting…

Remember, every shard has already been created. They all live in the same “Universe” and their location ID is known. Every transaction is stamped with a blend of the sender’s ID and shard ID.

This makes it extremely simple to locate the Shard from which the transaction that has been sent.  Now, if Node Bob tries to double-spend his $10, we don’t need to check every other shard to catch him cheating. We can simply check his shard.

To better understand this let’s go back to our Mom & Dad analogy

​Your Mom grounds you. But this time, she stamps your forehead with the word “Grounded”. You now go to your Dad’s study room and ask him if you can out to play. Your Dad simply looks at your forehead and says “Nope”.

He didn’t need to go check with your Mom. He didn’t need all the info – and was still able to stop you from cheating.

Similarly, the overlapping of nodes and easy cross-shard communication allows each node to store partial info. Which essentially means: not all nodes have to store all the data!  This plays a significant part in Radix’s massive scalability without sacrificing security.

(I simplified this, of course. But we will discuss more on Atoms, Universes timestamping & Temporal Proofs in future posts)

As mentioned earlier, the need to store and process every transaction is a huge limitation for blockchains. Radix is a cleverly designed network that avoids this limitation.

Radix has presented an innovative and noble approach to solving the scaling issues of DLTs. 

Over six years of sweat and frustration, Dan Hughes and team remained true to the goal: Every single person, on every single device using a single protocol simultaneously. The project is still in alpha and the team urges us to participate and help find any potential issues or flaws. 

​Although this was an introductory explanation, I’m sure you can’t help but wonder... Will Radix DLT be the answer to the burning question for mass adoption? Time will tell.

One of the most popular Consensus Methods used today is Proof Of Work. As I write this post, Bitcoin and Ethereum both still use the Proof Of Work consensus method.  Ethereum is set to switch to Proof Of Stake in it’s upcoming Casper update. However, the first iteration of this update will be a hybrid of PoW & PoS. Bitcoin will continue to use Proof Of Work for the foreseeable future. 

Regardless of the consensus method used,
Bitcoin & Ethereum are still blockchains. And such, both networks rely on the Longest Chain Rule when coming to consensus.

In this post, I’ll attempt to dive a little bit deeper into The Longest Chain Rule while still keeping things as simple as possible 🙂

In a previous post I discussed the various Consensus Methods - PoW, PoS, Tangle & Tempo.  Since then I’ve covered a few topics on the most well known of the consensus methods (PoW and PoS).  Today, I want to discuss, what I believe is, the most important of the Consensus Methods for us to grasp: Tempo.

Don’t get me wrong –  I’m not claiming that Tempo is “better” than the rest. “Better” is subjective. However, I do believe Tempo is by far the most groundbreaking of the Consensus Methods. 

Tempo tackles the current issues that plague the DLT/Blockchain world  – namely scalability & mass adoption. And it does so effectively. As such, from an enthusiast perspective, Radix’s Tempo deserves our keen attention.

In this post, I want to start with the most important element of Tempo:
The Logical Clocks.

Don’t worry – it’s not as intimidating as it sounds. In fact, we’re going to keep this as simple as possible 🙂

Far too often people misunderstand the “goal” of a Consensus Method. Most people believe that consensus methods exist to “validate” transactions.  But in truth, the focus of a Consensus Method is coming to an agreement on a set of events. And most importantly to:

In distributed systems, the ordering of events (like transactions) can be difficult. You cannot simply attach a timestamp to an event. Due issues like network latency, each system may witness an event at a different time.  

So how are the systems to agree on the different timestamps? How are they to come to  consensus  on the ordering of events?

Logical Clocks – The Heart Of Tempo

Using normal timestamps for events doesn’t work too well in distributed systems. Why? Put simply – while one system may timestamp an event at 12:01 PM, another may timestamp the same event at 12:02 PM.  This will cause inconsistencies – and a failure to agree on the data.

So instead of using a normal clock, we use a ‘logical’ clock. This allows us to place a timestamp that is relative to an occurence of the previous event. The clock cares more about “what happened before” an event than the exact “time” of the event.  

Confusing? Let’s try an analogy....

So, Mike – a Mango reader –  emailed me with this question a few days ago:

“ Hey Shawn, I enjoyed your explanation on Sharding. Thank you. But now I’m  trying to understand what makes Proof Of Stake less resource intensive than Proof Of Work? Doesn’t the verification process need to take place the same way? ”

This is actually a really good question, Mike! It shows that your understanding is growing deeper. Your asking the right questions!

Yes, PoW and PoS conduct the validation/verification of transactions similarly. As such, Proof Of Stake  consumes just as much resources to verifiy transactions as Proof Of Work does.

However, it’s important to note that the “verification of transactions” is not what makes Proof Of Work resource intensive. It is the Cryptographic Puzzle  – required to be solved by all miners –  that consumes a large majority of the resources.

Many of us tend to believe that it's the validation of transactions consumes the resources. In truth, the majority of the resources is consumed while solving the Cryptographic Puzzle - which has nothing to do with the validation/verification of transactions.

In fact, Proof Of Work and other consensus methods are more concerned with the ordering of transactions than the "validation" of transactions. The ordering of transactions is resolved by getting miners to solve the Cryptographic Puzzle.

Miners solve the cryptographic puzzle in order to win the “right” to place their block next on the chain. And it’s the solving for the "cryptographic puzzle" that is computationally intensive - not the verifications transactions.  

What Exactly Is Gas

There seems to be a lot confusion & questions around Ethereum & Gas – and rightfully so. Blockchain veterans & enthusiasts alike can be thrown off by the terminology. Moreover, people often wonder “why” Ethereum chose this route. Why not keep it simple – like Bitcoin?

There are good answers to these questions. But before we dive into the “why & how”  of Gas – let’s briefly go over “what” Gas is.

Contrary to popular belief  – Gas is not a sub currency of Ether.  Gas is more so an “internal” token within the EVM (Ethereum Virtual Machine).  It is used by Ethereum to place a relative cost on each operation within a Smart Contract.

For example, a contract can have the following types of operations and associated costs

This allows Ethereum to “charge” more for contracts that are more complex. This is fair – since a Smart Contract with more/demanding operations will be using more network resources.

When you send a transaction to a Smart Contract, you are required to enter the following:

I get it – it's still confusing. Understanding how "Gas Price" and "Gas Limit" relate to each other can be a task. So let's use an analogy to clear things out.

In the previous post of this Proof Of Work series, we discussed how PoW is used to determine representation of the majority. Determining the majority is one of the keys in being able to come to consensus (or agreement) on decisions. But in order to determine the majority, Proof Of Work uses Cryptographic Puzzles (or Proof of Work Puzzles)  – making them just as important.

But how does this Proof Of Work puzzle work? What kind of puzzle is this? I get these questions a lot. Usually in different forms. 

But the general gist of the question is this:
“What exactly is the Proof Of Work Puzzle? What’s the point?”

In this post, I’ll attempt to explain the proof of work puzzle simply. We’ll also discuss its significant importance in Proof Of Work as a Consensus Method.

As discussed in a previous post, one of the primary goals for a Consensus Method is to facilitate “agreement” on the network. To come to agreement we need a majority of votes. However, determining this majority may not be as easy as you may think.

The difficulty in achieving Consensus in a networked system is actually a fundamental problem in Computer Science. There are various technical reasons for this. But we’re going to keep it simple in this post. 

We will tackle one particular difficulty that may seem easy on first glance:
​"How Do We Determine Who Represents The Majority?"

Let’s take a step back for a moment. Let’s think about how majority decision making is done in the “real” world.

Agreement is usually settled by the “rule of the majority”, right? We’d sit in a big boardroom office, and everyone (who matters) would vote by either a ballot-system or raising their hands.

It’s pretty hard to cheat. (You could try to raise two hands, I suppose. But you’ll probably be found out)

But this boardroom clearly represents a centralized system. How do we achieve this in a decentralized system - like a blockchain? Such a system has the following features:

• People Distributed Around The World
• People Are Pseudo-Anonymous – Using Their Computers To “Vote”

In the office boardroom, the majority is represented by people raising their hands. We need a way to determine the majority in a networked & decentralized system. Essentially, we need a way for people to vote in such a system – and to do so fairly.

The first idea that comes to mind is : IP Addresses.  Each IP address could represents one vote. Not bad – but it won’t work.  It’s far too easy for someone create millions of Virtual IPs. A single person could spin-up several IPs around the world and cast a million votes to further his own agenda. 

A Blockchain Validator is someone who is responsible for verifying transactions within a blockchain. In the Bitcoin Blockchain, any participant can be a blockchain validator by running a full-node.  However,  the  primary incentive to run a full node is that it increases security. Unfortunately, since this is an intangible incentive, it is not enough to prompt someone to run a full node.  As such, Blockchain Validators comprise primarily of miners and mining pools that run full nodes.

Blockchain Validation vs Blockchain Consensus

It’s important to note that “validation” and “consensus” aren’t the same thing. A Blockchain Validator performs validation by verifying that  transactions are legal (not malicious, double spends etc).

However, Consensus involves determining the ordering of events in the blockchain – and coming to agreement on that order. 

Essentially, Consensus involves agreeing on the ordering of of validated transactions. 

The validation precedes the Consensus. We may very well have something that is “valid” that the network does not “agree” upon. How? Let’s go over an example.

How Are Blockchain Transactions Validated?

At this point, you’re probably thinking:
“What the hell? Everyone is building different blocks? Then how will we agree upon a single common ledger!?”

Let’s run through a simplified example of two miners with different blocks.

Miner Bob &  Miner Joe

Let’s say Bob and Joe are two miners on our network. Neither of them are up to any mischief. They are listening to the network and creating blocks with only valid transactions that have not already been spent.

Both of these blocks consist of valid transactions. Great. But we still need to come to “consensus” on who’s block to include onto the chain. Remember, a reward is given out to whoever gets to add their block to the chain. So should Bob get it? Or should Joe? How about both of them?

Adding both of them would be ideal, right? They both get rewards. And all the transactions get included onto the chain. Everybody wins! - But wait...Note that they both contain “Transaction B” in their blocks.

If we let Joe and Bob both include their blocks, Alice will end up paying Jennifer 20 bitcoins (two transactions of 10 btc each) when she intended to pay her only 10!  Furthermore, Alice may only have 10 bitcoin - so the second payment would be invalid.

As you can see, we can add only one of the blocks. And we need to agree upon which one. But how do we do that? 
This is where consensus methods like “Proof Of Work” or “Proof Of Stake” comes into play - Enter Consensus Methods

Using Proof of Work (PoW) for Picking Blocks

Miner Bob and Miner Joe both want their blocks included in the chain – they both want the reward. But only one of them can be picked. To “win” the right to include their block, they go through a consensus process – usually either “Proof Of Work” or “Proof Of Stake”

Food for thought: What if they solve it at the same time - or around the same time? That’s how a ‘fork’ can occur. And the bitcoin protocol resolves these occurrences as well. More on that later

Using Proof Of Stake (PoS) for Picking Blocks

Proof Of Stake involves Bob and Joe putting up their coins into a “jar”.  A coin is randomly picked from the jar. If it belongs to Bob, Bob gets to add his block. If it belongs to Joe, Joe gets to add his block instead.

Once the block is added, the process repeats. Both miners will listen to the network for pending transactions and create a new block. The competition goes on and on.

(Don’t worry, miners aren’t trying to solve the PoW puzzles themselves. Their computers are doing most of the work) 😉

Wrapping Up - Blockchain Validators vs Miners

As you can see, Consensus methods are primarily concerned with coming to an agreement on the ordering of events/transactions (and who gets to add them). Validation of the transactions is initially handled by the miner before they are added to the block.  And then once more by the rest of the Blockchain Validators when a block winner is picked. The miners add the block, and the Blockchain Validators verify that the block is valid. If Consensus is reached, then the network successfully moves on to the next block. (More on that in future posts)

However, as the village began to grow, they found it difficult to keep up with the system.

Maintaining the Security of the ledger was a tedious and time-consuming task.  It was essential that they cross-check & validate the ledgers properly. Any shortcuts would lead to security-vulnerabilities. Eventually, the village grew into a city, and they simply could not keep up with the task of maintaining the ledger anymore.

Essentially, the village wasn’t able to take their system and Scale it in preparation for the growth of the city.

In order to scale, the villagers had to alter their solution. They were wary of having a single middle-man due to their past experience with Mr.Ledger. So they appointed a group of individuals to maintain the ledger and keep a check on each other.

Conclusion​

As things stand, blockchain solutions are struggling the find the right balance balance to the trilemma.  Vitalik Buterin proposes Sharding to theoretically break the trilemma. I explain Sharding (using another analogy) in a previous post. It’s an ingenious solution – but still has some form of sacrifice. 

Other blockchain solutions that have high scalability have done so by sacrificing decentralization. EOS is a notable example that uses Delegated Proof Of Stake. This is essentially much like appointing (delegating) a group of individuals to manage the ledger – much like our village analogy.

Finally, other solutions like Radix, HashGraph and IOTA are not blockchains. They do not have the architectural limitations that blockchains do.

PoW Sharded: Is a 1% Attack Feasible?

Alright, so now we understand how a 51% attack is infeasible in Proof Of Work (if not, don’t hesitate to email me. I understand that this can get confusing). But what if the network is split into groups/pieces (as is done in Sharding)?  

Let’s consider a blockchain network that is split into a hundred pieces (Hundred shards)

If the entire network has 100% of the hash power
​then each of the 100 shards will have 1% of the hash power  

Q. But what if an attacker has even 1% control of the total network hash power? 

He can essentially concentrate his hash power on a single shard. Since each shard is responsible for 1% of the network, the attacker attains 100% control of the shard (Shard = 1% of network. Attacker = 1% of network)