All posts in " validation "

Cryptographic Puzzle – Understanding Proof of Work

In the previous post of this Proof Of Work series, we discussed how PoW is used to determine representation of the majority. Determining the majority is one of the keys in being able to come to consensus (or agreement) on decisions. But in order to determine the majority, Proof Of Work uses Cryptographic Puzzles (or Proof of Work Puzzles)  – making them just as important.

But how does this Proof Of Work puzzle work? What kind of puzzle is this? I get these questions a lot. Usually in different forms. 

But the general gist of the question is this:
“What exactly is the Proof Of Work Puzzle? What’s the point?”

In this post, I’ll attempt to explain the proof of work puzzle simply. We’ll also discuss its significant importance in Proof Of Work as a Consensus Method.

The PoW Puzzle - The ‘Reverse’ Lottery Analogy

To avoid the nitty-gritty technicals, let’s use an analogy. 

Suppose you enter yourself into a lottery.. But this is a special kind of lottery. We’ll call it a “Reverse Lottery".
In this lottery everybody knows the winning numbers. Sounds like an easy lottery to win, right? Not exactly.

You can’t simply “pick” your numbers when buying your ticket. You can only buy a ticket with numbers randomly generated on it. So, the only way for you to win the lottery is to keep buying ticket after ticket after ticket…. Until you buy one that matches the winning numbers you have.

This is precisely what is involved in the Proof Of Work cryptographic puzzle. People use their computers to randomly generate numbers at a rapid pace. (These people are called miners, btw.)  If they generate the “winning” number, they win the lottery.

The Reverse Lottery & PoW : Breaking It Down

Let’s say you know the winning ticket number of a Reverse Lottery to be:  1111111111.

Now it’d be nice if you could go up to the counter and ask for a ticket with those numbers. But you can’t. All you can do is pay for a ticket and hope it generates: 1111111111. If not – tough luck. Buy another ticket and try again.

The good news is that you can try as many times as you can. The bad news is that you have to pay for each new ticket.

Our “Reverse Lottery” is essentially what the cryptographic puzzle is.

  • Player in the Lottery  = Miner in the Network
  • Buying tickets with random numbers = Using Computational Power To ‘Solve’ Algorithms (Hashing)

In the Reverse Lottery, each player is hoping that he generates the winning ticket number before someone else does.  Similarly, each Miner is hoping that he can generate (by ‘hashing’) the winning numbers before other miners do.

  • In the lottery, players are paying for each ticket with money.
  • Miners are using computational power to “hash” their numbers – which translates to electricity costs.

The good news is the miners are using hardware that generate millions of hashes per second.

The miner who finds the winning hash gets to add his “block” to the blockchain. In return he gets a reward. He also gets to collect transaction fees for all transactions listed in the block.

On average, a winning number (or winning hash) is found every 10 minutes. This average can be adjusted but we’ll talk about that later. And the miners move on to compete for the next winning hash. It’s kinda like a new Reverse Lottery every 10 minutes.

The process of trying to find the winning hash is called “Mining”.  The term actually originates from the traditional “mining” of gold. But instead of using a pick-axe to dig for Gold, Miners are using their computers to dig for the winning hash.

That’s pretty much it. You pretty much understand how the process works now.  But I may have left out one aspect. The Nonce.

The Cryptographic Puzzle: Nonce?

Let’s go back to our Reverse Lottery.  Suppose the creators of this lottery decide to spice things up. They realize that they need to make lottery players feel more “involved”.

To achieve this, they add a new element into the Reverse Lottery. Now, instead of simply randomly generating a number they ask the player for the following information first:

  • His Name  (For example :  Bob )
  • His Favourite Color  (For example : Green )
  • A “random number” of his choice  (For example : 7)

The Reverse Lottery will take all three answers –  Bob, Green, 7 – and put it into a special “mixer”.  You can think of Bob, Green and 7 are the “ingredients”.  The mixer will spit out a Lottery Ticket for Bob based on the ingredients he gave it.

(Btw this is precisely what “hashing” is. A function/algorithm that takes “ingredients” and produces an output based the input.)

If the ticket matches the winning number – great! He wins! If not, he will have to try again – but this time with a different “random number”.  This will give him a new lottery ticket number.

Essentially, his name “Bob” and favorite color “Green” will remain fixed as ingredients. His ‘random number’ keep changing until he gets the winning lottery ticket.

In Mining, this “random number” is what we call the nonce. The other “fixed” ingredients are the transaction data, block data, timestamp etc.

The nonce is incremented after each failed attempt. (Fyi, the other ingredients are relatively fixed. They change when necessary – but that’s nitty-gritty stuff we don’t care about at them moment)

Wrapping Up - PoW Cryptographic Puzzle

To sum it all: In Proof Of Work people (miners) are essentially racing against each other to generate the winning numbers for the Reverse Lottery.

Instead of repeatedly buying tickets at a counter, they expend electricity by rapidly running an algorithm on their computer over and over again. When and If they generate the winning number (winning hash) they get to add their “block” of transactions to the blockchain.  In return, they claim a reward and the transaction fees for all the transactions in the added block.

There’s more I’d like to discuss – like Verifiability. I’ll leave that for the next post though. 

For now, I’ll let you ponder over this question:
“How do the other miners verify that he really did find the winning hash?”

We’ll talk about this and more in the next post 🙂

Did you enjoy this post?

Help Us Keep Doing What We Do Best!

Tip Jar 🙂

BTC: 3LrzDr7ZYQ5xWAKnweM1XuUAvU5YEkF7Zb

ETH: 0x87ba0C08910Dbd3b93D74A2A3b61d78A3C2dbDab

Get my upcoming eBook for Free!

"The Mango Guide TO Understanding Blockchain"

Offer Valid For FIRST 500 registrations only

No menu items have been found.
Continue reading >
Share

Proof Of Work: Determining Majority “Power”

As discussed in a previous post, one of the primary goals for a Consensus Method is to facilitate “agreement” on the network. To come to agreement we need a majority of votes. However, determining this majority may not be as easy as you may think.

The difficulty in achieving Consensus in a networked system is actually a fundamental problem in Computer Science. There are various technical reasons for this. But we’re going to keep it simple in this post. 

We will tackle one particular difficulty that may seem easy on first glance:
"How Do We Determine Who Represents The Majority?"

Let’s take a step back for a moment. Let’s think about how majority decision making is done in the “real” world.

Consensus In A Boardroom Office

Agreement is usually settled by the “rule of the majority”, right? We’d sit in a big boardroom office, and everyone (who matters) would vote by either a ballot-system or raising their hands.

It’s pretty hard to cheat. (You could try to raise two hands, I suppose. But you’ll probably be found out)

But this boardroom clearly represents a centralized system. How do we achieve this in a decentralized system - like a blockchain? Such a system has the following features:

  • People Distributed Around The World
  • People Are Pseudo-Anonymous – Using Their Computers To “Vote”

In the office boardroom, the majority is represented by people raising their hands. We need a way to determine the majority in a networked & decentralized system. Essentially, we need a way for people to vote in such a system – and to do so fairly.

Consensus Online:  IP Addresses?

The first idea that comes to mind is : IP Addresses.  Each IP address could represents one vote. Not bad – but it won’t work.  It’s far too easy for someone create millions of Virtual IPs. A single person could spin-up several IPs around the world and cast a million votes to further his own agenda. 

Proof of Work - a simplified example

Suppose we have a network of 1000 participants. Each of them are allowed one vote per IP Address. The participants have to come to a decision between Decision A and Decision B.

999 participants vote for Decision A.  But John – a network participant – wants Decision B for his own agenda. He decides to create 1000 Virtual IPs and casts votes for Decision B from each one of them. The network now thinks it has a total of 2000 participants.

  • Decision A gets 999 votes.  Decision B gets 1001 votes. (1000 fake votes + John’s real vote

  • Decision B goes through. John wins, everybody else loses.

Consensus Online: CPU Power

The “One-IP-One-Vote” solution was simply not going to cut it. It was far too easy for people to manipulate the votes  – and also face no consequence for doing so. Even failed attempts could be brushed off. We needed the following:

  1. Add a difficulty to casting vote
  2. Make it costly for attempting manipulation

That’s where Proof Of Work stepped in as a Consensus Method - Proof Of Work uses CPU Power to determine the majority decision.

In order to show their support, participants have to expend electricity. (They do so using by solving Cryptographic puzzles – which we will discuss in the this post. For now, all you need to know is that they have to expend energy in order to show who they support)

Therefore, the only way someone could manipulate the “votes” is by  having a A LOT of CPU Power. Achieving this will cost them A LOT in:

  1. Initial hardware costs 
  2. Ongoing electricity costs.

Longest Chain: Determining Majority Support

Note: I explain the Longest Chain using a simple analogy in a post here: The Longest Chain - A Simple Farmer's Analogy

This gives us another interesting (and important) feature. Each time a “block” is added to the blockchain, CPU Power/energy is  consumed. Thus, every block preceding it also involved consumption of energy. These are essentially a “chain of blocks” (hence ‘blockchain’). These “chain of blocks” are representative of the TOTAL energy used for that chain.  

  • Remember, energy consumption is a form of indicating “support” by network participants.  So the longer the chain, the more support the chain has by the network participants.

With this, if there is ever a conflict of decision – we can simply look at the Longest Chain to determine which decision has majority support. Why? Because the longest chain has the most energy invested in it. And hence the most support.

Wrapping Up - Proof of Work

Just like any other system, decision conflict needs to be resolved by a majority vote. However, determining that majority can be difficult when dealing with a networked system. Proof Of Work adds a extrinsic cost to each vote (hardware and electricity).  And thus making it increasingly difficult to manipulate the votes.

Energy is expended to show support on each “block” of data added to the ledger. As the ledger (blockchain) increases in size, more energy is expended. The longest chain serves as representation of the majority support.

In future posts I’ll go into a little bit more detail on the Cryptographic Puzzle and The Longest Chain. For now, I hope this post helps.

If you have any questions, don’t hesitate to shoot me an email or PM 🙂

Further Readings:

Did you enjoy this post?

Help Us Keep Doing What We Do Best!

Tip Jar 🙂​​

BTC: 3LrzDr7ZYQ5xWAKnweM1XuUAvU5YEkF7Zb

ETH: 0x87ba0C08910Dbd3b93D74A2A3b61d78A3C2dbDab

Get my upcoming eBook for Free!

"The Mango Guide TO Understanding Blockchain"

Offer Valid For FIRST 500 registrations only

Continue reading >
Share

How Are Blockchain Transactions Validated? Consensus VS Validation

By Shawn Dexter / March 22, 2018

In this post Shawn answers the question: How Are Blockchain Transactions Validated?  He also explains what is Blockchain Validator – and the key difference between Consensus and Blockchain Validation.

How Are Blockchain Transactions Validated?

After my last post on Ethereum Proof Of Stake & Sharding, I received several follow up questions. Many of them could be summarised to one simple question: How Are Blockchain Transactions Validated? Is it done via the Consensus Method? Or does a Blockchain Validator Do It?

As discussed in previous posts, “consensus” is a key aspect in blockchain. It’s imperative that all participants in the network come to an agreement on the state of the ledger.

But what exactly are we trying to achieve consensus on? Are we simply trying to agree that each transaction is “valid”? Or is it more than that?  I believe that this is where a lot of people steer the wrong way.

Blockchain Validators

A Blockchain Validator is someone who is responsible for verifying transactions within a blockchain. In the Bitcoin Blockchain, any participant can be a blockchain validator by running a full-node.  However,  the  primary incentive to run a full node is that it increases security. Unfortunately, since this is an intangible incentive, it is not enough to prompt someone to run a full node.  As such, Blockchain Validators comprise primarily of miners and mining pools that run full nodes.

Blockchain Validation Explained

Blockchain Validation vs Blockchain Consensus

It’s important to note that “validation” and “consensus” aren’t the same thing. A Blockchain Validator performs validation by verifying that  transactions are legal (not malicious, double spends etc).

However, Consensus involves determining the ordering of events in the blockchain – and coming to agreement on that order. 

Essentially, Consensus involves agreeing on the ordering of of validated transactions. 

The validation precedes the Consensus. We may very well have something that is “valid” that the network does not “agree” upon. How? Let’s go over an example.

Validation Without Consensus

How Are Blockchain Transactions Validated?

  • Each time a transaction is made, it’s broadcasted to the entire network. Upon hearing the broadcasts, miners take a bunch of transactions, validate that they are “legitimate” – and put them into a block. (More on how in another post)
  • But miners “hear” about different transactions at different times (due to latency issues etc). Furthermore, they may simply choose different transactions to include in their block based on transaction fees. So essentially, each miner is building his own block. His block may be completely different from the rest of the miners in the network.

At this point, you’re probably thinking:
“What the hell? Everyone is building different blocks? Then how will we agree upon a single common ledger!?”

  • And that’s one of the beautiful things about the protocol.  Miners don’t need to build the same global block. They can each build their own block – consisting of entirely different transactions. And the participants will come to “consensus” on which block is included next.
  • A miner may have a block consisting of completely valid transactions, but his block may still fail to achieve consensus by the network. If someone else is picked, he will simply create a new block and try again.

Let’s run through a simplified example of two miners with different blocks.

A Simple Bitcoin Transaction Example

Miner Bob &  Miner Joe

Let’s say Bob and Joe are two miners on our network. Neither of them are up to any mischief. They are listening to the network and creating blocks with only valid transactions that have not already been spent.

  • Miner Bob creates a block consisting of “Transaction A , Transaction B, Transaction C”
  • Miner Joe creates a block consisting of “Transaction Z, Transaction Y, Transaction B”

Both of these blocks consist of valid transactions. Great. But we still need to come to “consensus” on who’s block to include onto the chain. Remember, a reward is given out to whoever gets to add their block to the chain. So should Bob get it? Or should Joe? How about both of them?

Adding both of them would be ideal, right? They both get rewards. And all the transactions get included onto the chain. Everybody wins! - But wait...Note that they both contain “Transaction B” in their blocks.

  • Let’s suppose that Transaction B is - “Alice pays 10 bitcoin to Jennifer” 

If we let Joe and Bob both include their blocks, Alice will end up paying Jennifer 20 bitcoins (two transactions of 10 btc each) when she intended to pay her only 10!  Furthermore, Alice may only have 10 bitcoin - so the second payment would be invalid.

As you can see, we can add only one of the blocks. And we need to agree upon which one. But how do we do that? 
This is where consensus methods like “Proof Of Work” or “Proof Of Stake” comes into play - Enter Consensus Methods

Using Proof of Work (PoW) for Picking Blocks

Miner Bob and Miner Joe both want their blocks included in the chain – they both want the reward. But only one of them can be picked. To “win” the right to include their block, they go through a consensus process – usually either “Proof Of Work” or “Proof Of Stake”

  • In Proof Of Work, Miner Bob and Miner Joe compete against each other to solve a cryptographic puzzle. Whoever solves it first, gets to add their block to the chain.

Food for thought: What if they solve it at the same time - or around the same time? That’s how a ‘fork’ can occur. And the bitcoin protocol resolves these occurrences as well. More on that later

Using Proof Of Stake (PoS) for Picking Blocks

Proof Of Stake involves Bob and Joe putting up their coins into a “jar”.  A coin is randomly picked from the jar. If it belongs to Bob, Bob gets to add his block. If it belongs to Joe, Joe gets to add his block instead.

Once the block is added, the process repeats. Both miners will listen to the network for pending transactions and create a new block. The competition goes on and on.

(Don’t worry, miners aren’t trying to solve the PoW puzzles themselves. Their computers are doing most of the work) 😉

Wrapping Up - Blockchain Validators vs Miners

As you can see, Consensus methods are primarily concerned with coming to an agreement on the ordering of events/transactions (and who gets to add them). Validation of the transactions is initially handled by the miner before they are added to the block.  And then once more by the rest of the Blockchain Validators when a block winner is picked. The miners add the block, and the Blockchain Validators verify that the block is valid. If Consensus is reached, then the network successfully moves on to the next block. (More on that in future posts)

Get my upcoming eBook for Free!

"The Mango Guide TO Understanding Blockchain"

Offer Valid For FIRST 500 registrations only

No menu items have been found.
Continue reading >
Share
>