Introducing Reframe

kubo v0.14.0 added support (opens new window)for the Reframe protocol (opens new window), which enables users to configure how their kubo node discovers content, peers, and handles IPNS records by just adding an HTTP endpoint to their config file. This means if you have a new content routing system you’d like to try out next to, or instead of, the IPFS Public DHT it’s now simple to do so. Similarly, Reframe starts enabling applications like public IPFS HTTP Gateways to decouple their DHT nodes from their content serving nodes so that they can be scaled and load-balanced independently.

You can see more information and a demo utilizing Reframe in this presentation (opens new window)from IPFS Thing 2022.

#Why Reframe

The IPFS Public DHT provides functionality for peer routing (e.g., what’s the current IP address of this peer), content routing (e.g., finding which peers have a specific piece of content), IPNS record publishing and discovery, and more.

However, there are likely to be other types of systems that fulfill similar roles to the IPFS Public DHT but with different sorts of properties. While common IPFS libraries have long had support for plugging in custom code for new routing systems, it can be more cumbersome to add support for one without calling an out-of-process API. This is similar to how new DNSLink resolvers for custom TLDs can be added simply by plugging in a new DNSLink resolver (opens new window), whether it’s local or remote.

Additionally, there have been proposals for recursive or caching routing systems where some node close to the user gets a request and has the opportunity to respond before falling back to requesting data from another node or system. Making these routing systems composable is much easier when they have a shared API that they can speak.

Reframe was established as a way of resolving both of these problems.

#What is Reframe

Reframe is a request-response protocol with two components:

1. Methods -> the different requests you can make to a Reframe endpoint (e.g., FindProviders which asks for which peers have a given piece of content)
2. Transports -> the way in which requests and responses are made (e.g., HTTP with the data encoded using DAG-JSON)

For example, if you want to send a FindProviders request over HTTP you can encode the request and response as the transport requires.

The Reframe protocol is specified in ipfs/specs (opens new window).

#Reframe flexibility

Reframe’s flexibility in being defined abstractly over methods and transports is that adding new methods, new transports, or even new ways of encoding data within transports is fairly easy to do, while not preventing users from using transport-specific features (e.g., HTTP headers).

#Transports

For example, while using HTTP is desirable in many environments, others prefer gRPC, making some custom protocol over QUIC, or a more general libp2p stream. While only HTTP has been specified, if you find yourself needing something else then start hacking and raise a spec (opens new window)PR.

Similarly, while the initial HTTP transport used the DAG-JSON format to encode requests because HTTP + JSON is familiar to people and works easily with existing tooling, some of the earliest Reframe users also wanted a binary format in an HTTP GET request for better interoperability with caches. Since the method parameters are described abstractly, it was trivial to use the DAG-CBOR format with GETs for when HTTP cacheability is more important than JSON’s ease of use.

#Methods

On the method side, we’ve already added methods (opens new window)beyond the initial FindProviders method, including IPNS Get and Put. We’ve also extended FindProviders itself to support a greater variety of protocols. There are more proposals underway, but if you have a new method that you think Reframe would be useful for, implement and make spec PRs to tell people what you’re working on.

While not every Reframe client or endpoint is going to support every method, having shared specs and tooling can make it easier for Reframe implementations to support what they need to.

#Reframe Implementations

#Server Implementations

Two current Reframe server implementations are StoreTheIndex (opens new window)which supports the Network Indexers (opens new window), and someguy (opens new window)which is a delegated routing endpoint that forwards requests to other services such as the IPFS Public DHT and Network Indexers and streams back the results. The go-delegated-routing (opens new window)library provides tooling to make it easier to create new server implementations.

Protocol Labs is currently running a someguy instance at routing.delegate.ipfs.io (opens new window).

#Client Implementations

There are various clients that utilize the go-delegated-routing (opens new window)library to handle their requests including kubo (opens new window), hydra-booster (opens new window)and someguy (opens new window).

#Example

Querying the Reframe endpoint at routing.delegate.ipfs.io (opens new window)for the providers of a CID will return those providers and some metadata about them such as their addresses and the protocols they support.

https://blog.ipfs.tech/2022-09-02-introducing-reframe/