Skip to main content

LibP2P Implementation

A peer-to-peer (P2P) network serves as the backbone of decentralized communication and data sharing among blockchain nodes. It enables the propagation of transaction and block information across the network, facilitating the consensus process crucial for maintaining the blockchain's integrity. Without a P2P network, nodes in the Mina blockchain would be isolated and unable to exchange vital information, leading to fragmentation and compromising the blockchain's trustless nature.

The Rust node implements a LibP2P networking stack to ensure compatibility with existing OCaml Mina nodes while providing a foundation for the transition to WebRTC-based communication between Rust nodes.

Why LibP2P?​

For our networking stack, we utilize LibP2P, a modular networking stack that provides a unified framework for building decentralized P2P network applications.

Key Features​

Modularity​

Being modular means that we can customize the stacks for various types of devices, i.e. a smartphone may use a different set of modules than a server.

Cohesion​

Modules in the stack can communicate between each other despite differences in what each module should do according to its specification.

Layers​

LibP2P provides vertical complexity in the form of layers. Each layer serves a specific purpose, which lets us neatly organize the various functions of the P2P network. It allows us to separate concerns, making the network architecture easier to manage and debug.

LibP2P Stack Layers

Above: A simplified overview of the Rust node LibP2P networking stack. The abstraction is in ascending order, i.e. the layers at the top have more abstraction than the layers at the bottom.

Network Architecture​

The following sections describe each layer of the P2P networking stack in descending order of abstraction.

Remote Procedure Calls (RPCs)​

A node needs to continuously receive and send information across the P2P network.

For certain types of information, such as new transitions (blocks), the best tips or ban notifications, Mina nodes utilize remote procedure calls (RPCs).

An RPC is a query for a particular type of information that is sent to a peer over the P2P network. After an RPC is made, the node expects a response from it.

Supported RPCs​

Mina nodes use the following RPCs:

  • get_staged_ledger_aux_and_pending_coinbases_at_hash
  • answer_sync_ledger_query
  • get_transition_chain
  • get_transition_chain_proof
  • Get_transition_knowledge (note the initial capital)
  • get_ancestry
  • ban_notify
  • get_best_tip
  • get_node_status (v1 and v2)
  • Get_epoch_ledger

Peer Discovery with Kademlia​

Overview​

The P2P layer enables nodes in the Mina network to discover and connect with each other. Rust nodes must be able to connect to peers, both other Rust nodes (written in Rust) as well as native Mina nodes (written in OCaml).

To achieve this compatibility, we implement peer discovery via Kademlia as part of our LibP2P networking stack. Previously, we used the RPC get_initial_peers as a workaround to connect nodes. Now, to ensure compatibility with native Mina nodes, we've implemented KAD for peer discovery.

What is Kademlia?​

Kademlia, or KAD, is a distributed hash table (DHT) for peer-to-peer computer networks. Hash tables are a data structure that maps keys to values. In broad terms, think of a hash table as a dictionary, where a word (i.e. dog) is mapped to a definition (furry, four-legged animal that barks).

KAD specifically works as a distributed hash table by storing key-value pairs across the network, where keys are mapped to nodes using the XOR metric, ensuring that data can be efficiently located and retrieved by querying nodes closest to the key's hash.

Distance Measurement via XOR​

XOR is a unique feature of how KAD measures the distance between peers - it is defined as the XOR metric between two node IDs or between a node ID and a key, providing a way to measure closeness in the network's address space for efficient routing and data lookup.

The term "XOR" stands for "exclusive or," which is a logical operation that outputs true only when the inputs differ (one is true, the other is false).

Kademlia Binary Tree

Above: A Kademlia binary tree organized into four distinct buckets (marked in orange) of varying sizes.

The XOR metric used by Kademlia for measuring distance ensures uniformity and symmetry in distance calculations, allowing for predictable and decentralized routing without hierarchical or centralized structures, which enables better scalability and fault tolerance in our P2P network.

LibP2P leverages Kademlia for peer discovery and DHT functionalities, ensuring efficient routing and data location in the network. In Mina nodes, KAD specifies the structure of the network and the exchange of information through node lookups, making it efficient for locating nodes in the network.

Connection Multiplexing with Yamux​

The Challenge​

In a P2P network, connections are a key resource. Establishing multiple connections between peers can be costly and impractical, particularly in a network consisting of devices with limited resources. To make the most of a single connection, we employ multiplexing, which means having multiple data streams transmitted over a single network connection concurrently.

Yamux Multiplexing

Yamux Implementation​

For multiplexing, we utilize Yamux, a multiplexer that provides efficient, concurrent handling of multiple data streams over a single connection, aligning well with the needs of modern, scalable, and efficient network protocols and applications.

Noise Protocol Encryption​

We want to ensure that data exchanged between nodes remains confidential, authenticated, and resistant to tampering. For that purpose, we utilize Noise, a cryptographic protocol featuring ephemeral keys and forward secrecy, used to secure the connection.

Noise Capabilities​

Asynchronous Communication​

Noise supports asynchronous communication, allowing nodes to communicate without both being online simultaneously. It can efficiently handle the non-blocking I/O operations typical in P2P networks, where nodes may not be continuously connected, even in asynchronous and unpredictable blockchain P2P network environments.

Forward Secrecy​

Noise utilizes ephemeral keys, which are random keys generated for each new connection that must be destroyed after use. The use of ephemeral keys provides forward secrecy. This means that decrypting a segment of data does not provide additional ability to decrypt other data. Simply put, forward secrecy means that if an adversary gains knowledge of the secret key, they will be able to participate in the network on behalf of the peer, but they will not be able to decrypt past or future messages.

XX Handshake Pattern​

The Noise protocol implemented by libp2p uses the XX handshake pattern, which happens in the following stages:

Noise Handshake Step 1

Step 1: Alice sends Bob her ephemeral public key (32 bytes).

Noise Handshake Step 2

Step 2: Bob responds to Alice with a message that contains:

  • Bob's ephemeral public key (32 bytes)
  • Bob's static public key (32 bytes)
  • The tag (MAC) of the static public key (16 bytes)
  • A payload of extra data including the peer's identity_key, an identity_sig, Noise's static public key and the tag (MAC) of the payload (16 bytes)

Noise Handshake Step 3

Step 3: Alice responds to Bob with her own message that contains:

  • Alice's static public key (32 bytes)
  • The tag (MAC) of Alice's static public key (16 bytes)
  • The payload, in the same fashion as Bob does in step 2, but with Alice's information instead
  • The tag (MAC) of the payload (16 bytes)

After the messages are exchanged (two sent by Alice, the initiator, and one sent by Bob, the responder), both parties can derive a pair of symmetric keys that can be used to cipher and decipher messages.

Pnet Layer (Private Network)​

We want to be able to determine whether the peer we want to connect to is running the same network as our node. For instance, a node running on the Mina mainnet will connect to other mainnet nodes and avoid connecting to peers running on Mina's testnet.

For that purpose, Mina utilizes pnet, an encryption transport layer that constitutes the lowest layer of libp2p. Please note that while the network (IP) and transport (TCP) layers are lower than pnet, they are not unique to LibP2P.

Chain Identification​

In Mina, the pnet secret key refers to the chain on which the node is running, for instance mina/mainnet or mina/testnet. This prevents nodes from attempting connections with the incorrect chain.

Although pnet utilizes a type of secret key known as a pre-shared key (PSK), every peer in the network knows this key. This is why, despite being encrypted, the pnet channel itself isn't secure - security is achieved via the aforementioned Noise protocol.

Transport Layer​

At the lowest level of abstraction, we want our P2P network to have a reliable, ordered, and error-checked method of transporting data between peers. This is crucial for maintaining the integrity and consistency of the blockchain.

Connection Establishment​

LibP2P connections are established by dialing the peer address across a transport layer. Currently, Mina uses TCP, but it can also utilize UDP, which can be useful when implementing WebRTC-based nodes.

Multiaddress Format​

Peer addresses are written in a convention known as Multiaddress, which is a universal method of specifying various kinds of addresses.

For example, let's look at one of the addresses from the Mina Protocol peer list:

/dns4/seed-1.mainnet.o1test.net/tcp/10000/p2p/12D3KooWCa1d7G3SkRxy846qTvdAFX69NnoYZ32orWVLqJcDVGHW

Breaking down this address:

  • /dns4/seed-1.mainnet.o1test.net/ - States that the domain name is resolvable only to IPv4 addresses
  • tcp/10000 - Tells us we want to send TCP packets to port 10000
  • p2p/12D3KooWCa1d7G3SkRxy846qTvdAFX69NnoYZ32orWVLqJcDVGHW - Informs us of the hash of the peer's public key, which allows us to encrypt communication with said peer

An address written under the Multiaddress convention is 'future-proof' in the sense that it is backwards-compatible. For example, since multiple transports are supported, we can change tcp to udp, and the address will still be readable and valid.

Integration with the Rust Node​

The LibP2P implementation in the Mina Rust Node serves as a compatibility layer, enabling communication between:

  • OCaml Mina nodes ↔ Mina Rust Node instances (via LibP2P)
  • Mina Rust Node instances ↔ Mina Rust Node instances (preferably via WebRTC)

This dual-transport approach allows for gradual migration from the existing OCaml implementation to the new Rust implementation while maintaining network connectivity and compatibility.