How Ethereum Nodes Talk and Sync: Peer-to-Peer in Action

Ethereum is a peer-to-peer network where thousands of computers, called nodes, communicate using shared rules. These rules, known as the "networking layer," help nodes find each other and share information. This communication happens in two ways: broadcasting data to many nodes (gossiping) and sending specific messages between individual nodes.

Ethereum nodes have two main parts: execution clients and consensus clients. Each has its own job and network. Execution clients share transactions in an encrypted P2P network, storing them in a local pool. When a validator creates a new block, these transactions are sent to the consensus client through a local connection. The consensus client packages them into Beacon blocks and shares them on its P2P network. This setup requires two separate P2P networks: one for sharing transactions and another for sharing blocks, ensuring smooth and secure communication between nodes.

The Ethereum execution layer uses two main networking protocol stacks to keep the network running:

1. Discovery Stack: This stack operates on UDP and helps new nodes find and connect to peers in the network.
2. DevP2P Stack: Built on TCP, this stack enables nodes to share and exchange information after connecting.

Both stacks work together in parallel. The discovery stack ensures that new nodes can join the network, while the DevP2P stack handles communication between nodes, enabling them to interact seamlessly. This combination ensures that Ethereum's peer-to-peer network remains decentralized and efficient.

How New Nodes Discover the Ethereum Network?

When a new Ethereum node joins the network, it needs to find other nodes to connect with. This process is called discovery.

Starting with Bootnodes

• What Are Bootnodes?

  
  

  1. Bootnodes are special nodes with hardcoded addresses in Ethereum clients.

  2. Their sole purpose is to introduce new nodes to the network by providing a list of active peers.

  3. They do not participate in tasks like syncing the blockchain and are only used the first time a client connects to the network.

     
     

• How It Works:

  
  

  1. When a new node starts, it contacts a bootnode to receive a list of neighbors (peers) using the discovery protocol.
  2. The node then establishes connections with these neighbors through additional discovery interactions.

The Discovery Protocol

Ethereum nodes use peer discovery protocols to locate and connect with other nodes in the network. This is essential for maintaining Ethereum's decentralized peer-to-peer (P2P) network. Currently, Ethereum uses Discovery Protocol Version 4 (Discv4) and is transitioning to Discovery Protocol Version 5 (Discv5). Here's how these protocols work:

Discv4 (Current Protocol)

• Purpose: Discv4 is the current standard for Ethereum's peer discovery, used in execution clients.

• How It Works: Each node maintains a Kademlia-like Distributed Hash Table (DHT). This table maps Node IDs to IP addresses and ports. Node IDs are derived from public keys. Node "closeness" is determined by the similarity of Node IDs, not by geographic distance. Nodes periodically refresh their DHT to maintain accuracy and security.

• Steps to Discover Peers:

  
  

  1. Initiating Contact:

     • A new node sends a PING message to a bootnode or a known peer.

     • The PING contains:

       • The Hashed Node ID (unique identifier of the node).
       • The Bootnode ID (identifier of the target bootnode).
       • An Expiry Timestamp (indicating the validity of the message).

       
       

  2. Verification:

     • The bootnode responds with a PONG message that includes the hash sent in the PING.

     • If the hashes match, the connection is verified, and the nodes are considered bonded (trusted).

       
       

  3. Requesting Neighbors:

     • Once bonded, the new node sends a FIND_NEIGHBOURS request to the bootnode.

     • The bootnode replies with a list of neighboring peers (nodes that are close to the new node's ID in the network).

       
       

  4. Bonding with Neighbors:

     • The new node performs a PING-PONG exchange with each neighbor received from the bootnode.

     • Successful exchanges bond the new node with these neighbors, allowing them to communicate.

       
       

  5. Expanding the Network:
     • The new node repeats the process (PING, PONG, FIND_NEIGHBOURS) with newly bonded peers to discover additional nodes.
     • This iterative process continues until the new node has a sufficiently diverse list of peers.

  Discv5 (Upcoming Protocol)

  • Enhancements Over Discv4:

    
    

    • Ethereum Node Records (ENR): A structured object introduced in Discv5 to manage identifying information. Key components of ENR include:

      • Signature: A cryptographic hash of the record contents, ensuring data integrity.
      • Sequence Number: Tracks updates to the ENR, enabling dynamic handling and prioritization of the latest records.
      • Key-Value Pairs: Additional information like Node ID, IP address, supported protocols, and client type/version.

    • Dynamic IP Handling: ENR supports nodes with changing IP addresses, improving network flexibility.

    • Future-Proof Design: The ENR format is versatile, accommodating future changes in network structure.

      
      

  • Additional Features:

    
    

    • Nodes can advertise supported subprotocols, allowing for compatibility negotiation.
    • Improved scalability and security using modern cryptographic techniques.

Establishing Connections

After discovering peers, Ethereum nodes use the DevP2P stack, starting with the RLPx protocol to establish secure communication.

RLPx is a key protocol in Ethereum's DevP2P stack that helps nodes connect and communicate securely. It uses RLP (Recursive Length Prefix) encoding to compress and organize data efficiently for transmission. The protocol ensures encrypted and authenticated communication, creating a secure connection between nodes. Once this connection is established, nodes can interact using various subprotocols for exchanging blockchain data or for syncing state, enabling smooth and reliable network operations.

RLPx Protocol

• Cryptographic Handshake: The nodes exchange public keys and initiate encryption using ECIES (Elliptic Curve Integrated Encryption Scheme). This generates a shared secret, ensuring secure communication between the nodes.

  
  

• Hello Exchange: Following the handshake, the nodes send "hello" messages containing key details, including:

  • Supported subprotocols.

  • Client version.

  • Node ID.

  • Port and protocol version information. This allows both nodes to identify each other's capabilities and set up a compatible connection.

    
    

• Session Maintenance:

  
  

  • Ping/Pong Messages: Periodic "ping" messages are sent to confirm the peer's availability, and the receiving node replies with a "pong" message. This ensures that the session stays active and maintains the connection.

  
  

  • Disconnect Message: If a node intends to terminate the session, it can send a "disconnect" message to the peer, notifying them of the upcoming disconnection. This is a polite way of closing the connection gracefully.

Sub-Protocols within DevP2P stack:

1. Transaction and Data Exchange

   After nodes are connected, the wire protocol manages how they communicate. It originally handled tasks like:

   
   

   • Syncing chains: Ensuring all nodes are up-to-date with the blockchain.

   • Propagating blocks: Sharing new blocks across the network.

   • Exchanging transactions: Sharing pending transactions between nodes for future blocks.

     
     

   However, with Ethereum's shift to Proof of Stake (PoS), syncing blocks and chain data is now part of the consensus layer, while transaction exchange continues under execution clients.

   
   

2. Light Ethereum Subprotocol (LES)

   The LES protocol is for light clients, which are nodes that don’t store the whole blockchain. They rely on full nodes to provide data when needed. Light clients save space and resources, but they depend on full nodes to share information with them. However, because full nodes aren’t incentivized to serve light clients, LES has not been used much.

   
   

3. Snap Protocol

   The Snap protocol is an extension that helps Ethereum nodes exchange state snapshots (recent data states). This allows nodes to verify accounts and storage without downloading too much data, speeding up the process of syncing with the network.

   
   

4. Witness Protocol (Wit)

   The Witness Protocol lets Ethereum nodes share state witnesses, helping them stay updated with the latest blockchain state. This is especially useful for clients syncing to the latest block, reducing the amount of data they need to download.

   
   

5. Whisper Protocol (Deprecated)

   The Whisper protocol was designed for secure messaging between nodes, without storing any messages on the blockchain. However, it had problems with scalability and wasn’t widely adopted. As a result, it has been deprecated, and other projects are now working on similar secure messaging solutions.

   
   

The Consensus Layer

The Ethereum consensus layer ensures that all Ethereum nodes agree on the state of the blockchain, keeping everything in sync. It works separately from the execution layer (where transactions are executed) but needs to interact closely with it.

Peer Discovery

Just like the execution clients, the consensus clients need a way to discover other nodes (peers) in the network. They use a protocol called discV5 over UDP. This is different from the old DevP2P system and integrates libP2P, a newer and more flexible technology.

Ethereum Node Records (ENR) for Consensus Nodes

Each consensus node in the Ethereum network has an Ethereum Node Record (ENR), which helps other nodes identify and connect to it. The ENR contains:

• Public Key: The node’s cryptographic key for secure communication.
• IP Address: The node’s internet address.
• UDP and TCP Ports: These specify which ports the node listens to for communication over UDP and TCP.
• Attestation Subnet Bitfield: This field helps nodes find others that are involved in attestation gossip, making it easier to sync with peers that are part of specific attestation sub-networks.
• Eth2 Key: This identifies which Ethereum fork version the node is using. It ensures that nodes only connect to peers running the correct Ethereum version, keeping everything compatible and synchronized.

LibP2P: The Communication Backbone

After discovering peers, the consensus layer uses libP2P to keep the communication going. It uses two main methods for communication:

• Gossip: This helps spread important updates like new blocks and attestations quickly across the network. It’s like passing a note between people in class, making sure everyone gets the same information fast.

  
  

• Request-Response: This is a more specific way for one node to ask another for particular data, like a block with certain information. It’s like requesting a copy of a specific page from a book.

Why SSZ for Consensus?

In Ethereum's consensus layer, SSZ (Simple Serialization) is used instead of the usual RLP (Recursive Length Prefix) format. This is because SSZ is faster and more efficient when dealing with large data and makes it easier to verify information in Merkle trees, which are used to keep the blockchain secure.

Connecting Execution and Consensus Clients

The execution layer (which processes transactions) and the consensus layer (which secures the blockchain) need to talk to each other. This happens through a local RPC connection. They share a common Ethereum Node Record (ENR), which keeps track of information like the client's public key and which layer it belongs to.

When consensus client is not block producer:

• Consensus client receives a block via the block gossip protocol (consensus p2p)
• Consensus client pre-validates the block, i.e. ensures it arrived from a valid sender with correct metadata
• The transactions in the block are sent to the execution layer as an execution payload (local RPC connection)
• The execution layer executes the transactions and validates the state in the block header (i.e. checks hashes match)
• Execution layer passes validation data back to consensus layer, block now considered to be validated (local RPC connection)
• Consensus layer adds block to head of its own blockchain and attests to it, broadcasting the attestation over the network (consensus p2p)

When consensus client is block producer:

• Consensus client receives notice that it is the next block producer (consensus p2p)
• Consensus layer calls create block method in execution client (local RPC)
• Execution layer accesses the transaction mempool which has been populated by the transaction gossip protocol (execution p2p)
• Execution client bundles transactions into a block, executes the transactions and generates a block hash
• Consensus client grabs the transactions and block hash from the execution client and adds them to the beacon block (local RPC)
• Consensus client broadcasts the block over the block gossip protocol (consensus p2p)
• Other clients receive the proposed block via the block gossip protocol and validate as described above (consensus p2p)