# Bringing Reconfigurability to the Network Stack

Akshay Narayan${}^{\varheartsuit}$, Aurojit Panda♠, Mohammad
Alizadeh${}^{\varheartsuit}$,
Hari Balakrishnan${}^{\varheartsuit}$, Arvind Krishnamurthy♣, Scott
Shenker${}^{\vardiamondsuit}$
${}^{\varheartsuit}$ MIT CSAIL, ♠ NYU, ♣ University of Washington,
${}^{\vardiamondsuit}$ UC Berkeley and ICSI

###### Abstract

Reconfiguring the network stack allows applications to specialize the
implementations of communication libraries depending on where they run, the
requests they serve, and the performance they need to provide. Specializing
applications in this way is challenging because developers need to choose the
libraries they use when writing a program and cannot easily change them at
runtime. This paper introduces Bertha, which allows these choices to be
changed at runtime without limiting developer flexibility in the choice of
network and communication functions. Bertha allows applications to safely use
optimized communication primitives (including ones with deployment
limitations) without limiting deployability. Our evaluation shows cases where
this results in $16\times$ higher throughput and $63\%$ lower latency than
current portable approaches while imposing minimal overheads when compared to
a hand-optimized versions that use deployment-specific communication
primitives.

##  1 Introduction

Distributed and cloud computing applications require the endpoint network
stack to provide more than just (reliable or unreliable) communication
channels accessed via the sockets API. Many applications use the network to
implement higher-level communication primitives such as remote procedure calls
(RPCs) [9] and publish-subscribe (pub/sub) [8] communication. To perform well
and meet their service-level objectives (SLOs), applications need low latency
and/or high throughput from these libraries.

Operating systems and shared libraries have evolved to make it simpler to
write networked applications that implement complex logic while meeting
stringent performance requirements. For example, new kernel network interfaces
such as io_uring [5] and kernel bypass libraries such as DPDK [12] reduce
software overheads and improve communication performance. Similarly, RPC
libraries such as gRPC [17] and Thrift [3] and communication services such as
Kafka [24] and Google PubSub [16] implement useful communication primitives.
In modern deployments, several network functions are often packaged into
higher-layer (i.e., application-facing) middleware libraries (e.g.,
ServiceRouter [39]) or “sidecar proxies” such as Envoy [13] and Istio [21].

The problem, however, is that these communication libraries often make
deployment assumptions to implement their functionality or meet performance
requirements. For example, kernel bypass libraries (and io_uring under some
settings) achieve low latency by requiring that each application thread be
“affinitized” to a CPU core that is not shared with other threads or
processes, thus limiting what other applications the host can run. Similarly,
most pub/sub libraries rely on an external pub/sub service, and pub/sub
libraries that use cloud provider services are designed assuming that the
application is deployed in the cloud provider’s datacenter.

On the other hand, libraries implemented without such deployment assumptions
generally perform worse and, in cases such as pub/sub, might not be able to
provide equivalent functionality. Thus, it is appealing (and often necessary)
for _developers_ to use libraries that make deployment assumptions.

The use of libraries that make deployment assumptions also constrains an
application _administrator’s_ choices, since changing resource allocations or
deploying in a different environment can require significant changes to
application code. For example, reconfiguring a kernel-bypass application to
allow sharing application cores with other applications would require changing
its code to use a different I/O abstraction (e.g., sockets) or switching from
polling to interrupt-driven I/O. Even when deployed in the same location, an
administrator might want to change an application’s libraries to cope with
changes in workload. For example, an administrator might want applications to
use different pub/sub services (with different cost and scalability trade-
offs) depending on the number of subscribers.

For these reasons, when choosing communication libraries, application
developers must consider not only application requirements but also possible
deployment locations and workloads. Accounting for all of these factors,
however, is complicated and nearly impossible. Thus, developers today must
choose between specialized and performant libraries that limit where an
application is deployed and how it responds to workload changes and general
libraries that provide flexibility but with poorer performance and fewer
features.

This paper presents Bertha 111Named for Tunnel-Boring Machines., a
_reconfigurable_ and _extensible_ network stack that allows developers to use
specialized communication libraries and interfaces that make deployment
assumptions but enable those decisions to be reconfigured both when deploying
an application and at runtime. Bertha does so by allowing application
developers to specify several alternate implementations of the same
functionality and providing mechanisms to switch between these alternatives.

Bertha’s core enabling abstraction is the Chunnel. A Chunnel, similar to a
library, implements some communication functionality, e.g., for kernel-bypass
networking, serialization, and pub/sub communication. A developer can
implement the Chunnel trait (or interface) to expose some functionality to a
Bertha application. Applications can _compose_ a sequence of Chunnels when
creating a connection (Figure 1). We use the term “Chunnel stack” to refer to
the sequence of Chunnels available for a connection; all data sent or received
over the connection is processed in sequence by Chunnels in the stack.

Developers specify reconfiguration choices by specifying multiple Chunnels (or
a composition of Chunnels) at a particular layer of the stack: in this case,
the Bertha runtime chooses one of these when making a connection and provides
a mechanism for the application to change its choice later. We assume that the
application can function correctly with any of these Chunnel choices. The only
constraint Bertha imposes on alternate Chunnels in a layer is type safety
(§3.2). Thus, alternate Chunnels in a layer can make different deployment
assumptions and provide different performance and efficiency trade-offs,
allowing application developers to use feature-rich, high-performance
communication libraries without limiting an administrator’s ability to change
deployment environments.

Figure 1: A Bertha connection specifies a _Chunnel stack_ , which composes
multiple Chunnels. Chunnel stacks can express options (e.g., here between
Kernel and DPDK), and at runtime Bertha can create a connection using one set
of options (here, Kernel) and also _reconfigure_ this choice later (§4).

Chunnels enable reconfigurability and extensibility by providing:  (a) a
uniform interface for receiving and sending data, ensuring that switching
between different Chunnels does not require any changes to application code or
logic;  (b) a typed interface, allowing Bertha to check during compilation
that reconfiguration cannot change the type of input data (e.g., bytes,
strings, or objects) that a Chunnel processes, nor change the type of outputs
it produces; and  (c) embedded runtime type information that allows Bertha to
ensure compatibility between the Chunnels used by a connection’s endpoints.

We have implemented Bertha’s core features in ~5,000 lines of Rust. In §7, we
use an example ETL application implemented using Bertha to evaluate the
performance benefits Bertha can provide by reconfiguring the network stack and
find. Bertha can unlock application optimizations such as using client-side
sharding to achieve $16\times$ higher load meeting a $50$ $\mu$s target
latency for a sharded key-value store (§7.3) or 63% lower latency for a
publish-subscribe benchmark (§7.2). Further, our evaluation in §8 shows that
Bertha’s abstractions that enable reconfigurability and extensibility add no
overhead for most applications and only a 27% throughput overhead in the worst
case (for the highest-performance applications using minimum-sized packets).

##  2 Related Work and Motivation

Reconfigurability and extensibility have been extensively studied in the
systems literature. In this section, we first outline the prior work on
building reconfigurable and extensible systems, then discuss why it is time to
make the network stack extensible and reconfigurable.

Reconfigurability in OSes Prior approaches to program reconfiguration and
extension rely on patching or replacing code in an executable system: Ksplice
[4] and Proteos [15] developed approaches to patching kernel code to address
security concerns, Nooks [42] and Shadow Drivers [41] describe approaches to
replace drivers to avoid failures, and other systems [37, 19] have provided
tools for live-patching running applications. Another line of work,
exemplified by VM-PHUU [38] and NetKernel [32] develops approaches to
reconfigure executing VMs.

Service Mesh Service meshes [39, 13, 21] implement communication functionality
outside the applications, making it easier to reconfigure and change features
such as how connections are encrypted, or what data is logged. Bertha allows
similar functionality, but unlike these, implements mechanisms to ensure that
communicating endpoints have compatible configuration and to allow
applications to vary configuration depending on what endpoint they are
connected to.

Reconfigurability in Traditional Network Stacks A lot of an application’s
communication functionality is split across the kernel and the user-space
(e.g., in communication libraries, service proxies, etc.), and its correctness
can depend on agreement between endpoints (e.g., agreement on serialization
format). Therefore, work on reconfigurable network stacks has largely focused
on individual pieces of functionality: TAPS [44] and the Fox project [7]
sought to modularize transport layer functionality. The x-Kernel’s Protocol
Framework [20] proposed an architecture for an extensible network stack, and
Microprotocols [6] built on this work with a proposal to extend the network
stack with higher-level protocols such as atomic multicast. Meanwhile, another
early-stage workshop paper [31] described a uniform abstraction for using
offloads in the network stack, but without reconfiguration.

Hardware Portability Since the early 1960s, many systems have aimed to provide
_portability_ —rather than reconfigurability—thus allowing the system to work
on a variety of hardware. Early examples include Unix V4, which was written in
C to enable this. Recent examples include DPDK [12] (which supports multiple
NICs), Demikernel [45] (which supports multiple network technologies including
RDMA and kernel bypass networking), and Terraform [18] for configuring
infrastructure across different cloud providers. These approaches allow the
program to run on multiple deployments, but do not specialize the program to
these deployments, and might thus carry a performance or efficiency overhead.

###  2.1 What is different now?

This paper is predicated on the argument that it is now necessary to evaluate
and adopt reconfigurable network stacks. This is because of the following
three changes that applications must tackle:

The “Narrow Waist” has moved up. While traditionally applications only
required TCP/IP connections that they could establish using the socket API,
this is no longer the case [35]; modern applications need to adopt one of a
variety of open-source libraries that implement more advanced communication
abstractions that improve network security, simplify application management,
and enable better logging and debugging. These libraries differ in their
performance, features, and requirements, and thus, the ability to reconfigure
which library is used can significantly reduce management and performance
overheads for applications. For example, applications have several options
when choosing how to serialize data, each of which has different tradeoffs:
Apache Arrow is optimized for fast computation, Capnproto can reduce
serialization overheads but requires the machine architecture to be identical
between communicating hosts, and Protobufs are more general but sacrifice
performance. The best choice varies by workload and deployment and thus
requires assumptions beyond what the application developer might know.

The emergence of deployment specific offloads. Most applications today run in
virtualized environments (e.g., cloud). These environments increasingly offer
specialized services, including hardware offloads (e.g., Google’s offload-
optimized PSP encryption or network sequencers such as Hydra [11]), and
managed communication services (e.g., publish-subscribe) that can improve
application performance. To take advantage of this, applications must
explicitly be designed to work in these environments. One approach to doing so
is to limit where the application can run or to customize and maintain
applications for each runtime environment, but these options decrease utility
and increase cost. Bertha’s reconfiguration can embrace environment-specific
services without limiting an application’s deployment or its network logic.

Application performance and efficiency matter. Applications also have more
stringent performance and efficiency requirements now, and this limits the
ability to use proxies or eschew the use of deployment-specific hardware or
software. Reconfiguration is thus useful to meet performance and efficiency
requirements without imposing a significant developer burden or limiting where
an application is deployed.

Our goal in this paper is to describe a reconfigurable network stack that is
useful for today’s applications and their diverse runtime environments. While
our approach borrows from prior work—e.g., our approach to safe
reconfiguration is similar to the one used by Shadow Drivers and Proteos—our
main contribution is showing how an extensible abstraction for application
networking functionality can enable reconfigurable communication stacks for
modern applications. To build such an abstraction, we addressed three
challenges: (i) ensuring that the component abstraction (we refer to these as
Chunnels) can be composed (we refer to these compositions as Chunnel stacks);
(ii) ensuring that reconfiguration avoids incompatibility between connected
hosts, which we enforce using a negotiation protocol (§5); and (iii)
minimizing overheads introduced our abstractions, which we do by carefully
designing the Chunnel interface and runtime to make use of modern low-cost
language abstractions. To the best of our knowledge, no prior work has used
these or alternate approaches to provide mechanisms that allow administrators
to reconfigure all layers of an application’s network stack.

Chunnel |  A specific piece of network functionality.
---|---
Chunnel stack |  An application’s specification of the set of Chunnels it wants to use.
Reconfiguration |  Picking or changing Chunnel implementations for a connection at runtime.
Negotiation |  Ensuring that Chunnel implementations are compatible across connection endpoints.
Table 1: Glossary of terms used in this paper.

##  3 Bertha Design

Bertha is a _reconfigurable_ and _extensible_ network stack. We start by
defining these terms.

Reconfigurability refers to an application’s ability to choose between
different implementations of the sam functionality both _when_ establishing a
connection and _after_ the connection has been established. In practice, this
means Bertha allows applications to choose among Chunnels that implement
similar functionality, e.g., between DPDK and kernel networking in Figure 1.
As we discussed earlier, choosing Chunnels (i.e., reconfiguration) when
establishing connections allows applications to be specialized to where they
are deployed. Similarly, reconfiguration after a connection has been
established is useful when Chunnel performance and efficiency varies depending
on workload characteristics or resource availability, since it allows
application to respond to changes without disrupting ongong connections. For
example, DPDK Chunnels can be built either so they poll for I/O in the
application thread—which requires pinning the application thread to a
dedicated core but provides low latency—or to use a shared I/O core for
polling—which makes it easier to share cores between applications but
increases latency due to IPC. The choice between these two alternatives
depends on the number of cores that are available and on the observed
workload. Bertha’s runtime reconfiguration process allows administrators (or
orchestrators) to change how I/O is performed at runtime, without disrupting
existing connections.

Extensible refers to application and library developer’s ability to add new
Chunnels to Bertha. These Chunnels can either represent alternate
implementations of the same functionality or entirely new functions.
Extensibility allows Bertha’s reconfiguration to allow changes not just to the
lowest layers of the network stack, but also to allow changes to how more
complex communication primitives are implemented.

Supporting both reconfigurability and extensibility imposes four design
requirements on Bertha:

  1. 1.

Switching between two Chunnels that implement the same functionality should
not require changes to application logic.

  2. 2.

Reconfiguration should be _safe_ ; it should not lead to scenarios where a
connected host can no longer communicate. Some Chunnels, e.g., serialization
Chunnels, require all communicating hosts to use compatible formats, and
Bertha must ensure that this holds.

  3. 3.

Reconfigurability should neither limit what Chunnels an application can
compose nor limit what functionality a Chunnel can implement.

  4. 4.

When possible, reconfiguring an established connection should not require re-
establishing the connection state.

The Chunnel abstraction meets most of these requirements by enforcing a
unified interface that ensures that Chunnels can be composed and reconfigured.
However, the abstraction alone is insufficient for ensuring safety, and the
Bertha runtime implements a negotiation protocol (§5) that checks
compatibility between hosts when establishing or re-configuring connections.
For reference, we provide in Table 1 a glossary of Bertha’s important
concepts.

⬇

// Client-side application code.

let sock = KernelUdpChunnel::new(); // () -> bytes

let shrd = ClientShard::new(cfg); // T -> T + sharding

let ser = SerializeChunnel::new(idl); // bytes -> type T

let conn = bertha::make_stack!(ser, shrd, sock).connect(addr);

Figure 2: An application developer specifies a Chunnel stack (shrd, ser, and
sock). Each of the Chunnels in the stack modify the resulting connection type.
Figure 3: Bertha builds connections by recursively calling Chunnels’
connect_wrap functions. We show reconfiguration in §4.

###  3.1 The Chunnel Abstraction

Chunnels are Bertha’s core abstraction: each Chunnel represents a single
communication function, i.e., logic that can:  (a) Transform data (e.g.,
serializing, encrypting, or compressing data);  (b) Decide where to send data
(e.g., which shard or DHT node should receive a request), or replicate data to
multiple endpoints (e.g., to implement publish-subscribe functionality); or
(c) Send and receive data from hardware (e.g., via DPDK or the OS networking
stack)

Each Chunnel takes as input an existing connection $c$, and produces a new
connection that encapsulates connection $c$ and adds the Chunnel’s
functionality to $c$. Developers of communication libraries implement Chunnels
to expose their library’s functionality to application developers. Chunnels
are composable: composing two Chunnels creates a third Chunnel.222Chunnel
composition is associative but not commutative. By composing Chunnels into
Chunnel stacks, application developers can layer functionality into their
connections: after applying one Chunnel to a connection, Bertha can apply
another Chunnel to the resulting connection to create a connection that
incorporates the features both Chunnels express. To bootstrap a connection to
start with, Chunnels that can appear at the bottom of the stack need to be
able to transform a connection with no implementation (represented by a unit
type in Rust).

We show an example of using a Chunnel stack in an application in Listing 2.
Note that we implemented Bertha in Rust (we discuss this in §6), and we thus
describe our interfaces using Rust code. However, the core ideas and
abstractions we describe here are general and are readily expressible in other
languages. On line $2$, we show an example of a Chunnel, the KernelUdpChunnel,
that transforms the unit type into a UDP connection that can send and receive
byte arrays. On line $3$, we compose this with a SerializeChunnel that
transforms this connection into one that exposes an object interface. Then, on
line $4$, we again compose this Chunnel with a sharding Chunnel. This produces
a connection that can evaluate a sharding function and rewrite a message’s
destination address to the appropriate shard. Finally, on line $5$ we make a
Chunnel stack from these three Chunnels, and create a connection with all the
specified features for the application.

⬇

// A Chunnel control path.

pub struct AChunnel { /*…*/ }

// The ChunnelTransformer<R> trait implements connection establishment logic.

impl<R> ChunnelTransformer<R> for AChunnel

where R: ChunnelDatapath</*…*/>> {

// Specify that AChunnelDP is the datapath used

type Connection = AChunnelDP<R>;

// Chunnel composition interface: compose AChunnel with inner.

fn connect_wrap(&mut self, inner: R) ->

Self::Connection {/*…*/}

// Specify relative compat. with other impls.

type Capability = /*..*/;

fn capabilities() -> Self::Capability { /*…*/ }

}

impl AChunnel {

// Create a new AChunnel

pub fn new(/*…*/) -> AChunnel { /* … */}

}

// The AChunnel datapath

pub struct AChunnelDP<R>{/*…*/}

impl<R> ChunnelDatapath for AChunnelDP<R>

where R: /* input data type requirements */ {

type Data = /*..*/;

fn send(&self, ms: impl Iterator<Item=Self::Data>) { /*..*/ }

fn recv(&self, buf: &mut [Option<Self::Data>]) { /*..*/ }

}

Figure 4: An overview of the Chunnel interface. Implementors of
ChunnelDatapath are connection types, and implementors of ChunnelTransformer
allow us to create connection types either from scratch (i.e., R = ()) or from
an input ChunnelDatapath.

###  3.2 Implementing a Chunnel

To expose their library’s functionality as a Chunnel in Bertha, library
developers provide a function that implements the transformer functionality as
well as a returned connection type. We show an example of how a library
developer would implement these in Listing 4. Bertha exposes the transformer
functionality via the ChunnelTransformer trait (on line 4), and connection-
level functionality via the ChunnelDatapath trait (on line 21). The
connection-level ChunnelDatapath trait is straightforward: it defines its data
type (line 23) and functions to implement send() and recv(). send() accepts a
batch of messages to send, and recv() waits for incoming messages and writes
them into a caller-provided array.

Meanwhile, the ChunnelTransformer trait’s function, connect_wrap (line 9)
composes Chunnels in a connection. Figure 3 shows this process. If Chunnel A
appears above (i.e., is pushed after) Chunnel B in a connection’s stack,
Bertha calls Chunnel A’s connect_wrap function with an inner argument
corresponding to Chunnel B’s ChunnelDatapath return type. Chunnel A’s
connect_wrap function then returns a type that also implements
ChunnelDatapath, AChunnelDP, whose data is first processed with Chunnel A’s
functionality and then passed to Chunnel B (by calling ChunnelDatapath methods
on its inner R). Each Chunnel’s connect_wrap function is also responsible for
performing connection initialization tasks, including initializing connection
state, beginning connection handshake, etc. Bertha calls connect_wrap
recursively (starting with the bottom) on all Chunnels in the connection’s
stack, thus initializing all Chunnels used by the connection.

Observe that the ChunnelTransformer and ChunnelDatapath take the inner
connection’s type as a generic type parameter (R). This allows the compiler to
enumerate all possible connection types and optimize them at compile time,
thus avoiding most dynamic dispatch overheads.We discuss how Bertha
encapsulates reconfigurable Chunnels in §6.

Developers can port existing libraries to Bertha by encapsulating their
functionality in Chunnels. We did this for our implementation, and as we
discuss in §6, the largest Chunnel we implemented is a DPDK Chunnel (1,700
lines of Rust code). Most communication libraries are already designed to be
composed with application logic and functionality from other libraries;
Bertha’s Chunnel abstraction merely enforces a unified interface that these
libraries must implement. This unified interface both simplifies composition
and enables reconfiguration. We have implemented several Chunnels and Bertha
applications that we describe in §7, demonstrating the generality of this
abstraction.

Chunnel Datapath Type Safety Chunnel stacks are type-safe: we use the Rust
type system to ensure that when assembling a connection from a Chunnel stack,
the data types of the resulting ChunnelDatapath will be compatible at each
layer of encapsulation. This is because Chunnel implementors specify their
data type (type Data on line 23) as well as type restrictions on the
connection type their ChunnelTransformer can accept (restrictions on R on line
5, restrictions elided). Bertha imposes type restrictions on the input Chunnel
stack (implemented as part of the connect function in Listing 2 line 5). So,
if one of the Chunnels in a Chunnel stack does not match the others’ data
types, the application will fail to compile. For example, a Chunnel that
implements sharding needs access to a key that it can use to identify the
correct shard. Such a tunnel can specify an input type such as (String,
Vec<u8>) to indicate that it requires a string key to be passed along with the
data. Naturally, datapath type safety does not extend across the network; this
is why negotiation (§5) is necessary.

##  4 Reconfiguring Chunnels

⬇

// Server-side code to "select" a peer-compatible sharding impl. and between
DPDK datapaths.

let ser = SerializeChunnel::new(/*…*/);

let shrd = select!(MboxShard::new(/*..*/),

ClientShard::new(/*..*/));

let dp = select!(DpdkThread::new(/*…*/),

DpdkInline::new(/*…*/));

let handle = dp.handle();

// conns: a stream of incoming connections.

let conns = bertha::make_stack!(ser, shrd, dp).listen(addr);

/* application logic… */

// trigger a re-configuration to the ‘DpdkInline‘ chunnel.

handle.reconfigure(DpdkInline);

Figure 5: Bertha applications use select in their Chunnel stack to specify
options for reconfiguration.

The uniform interface Chunnels provide enables reconfiguration in Bertha.
Recall that each Chunnel in Bertha is associated with a datapath structure
that implements the ChunnelDatapath trait. A Bertha connection that uses the
Chunnel instantiates an instance of its datapath structure; the Chunnel’s
connection state is stored in this structure. Reconfiguring, i.e., replacing
one Chunnel with another, thus merely requires transferring connection state
from the old Chunnel’s datapath structure to the new one, and swapping it into
the connection structure. We detail our reconfiguration API and mechanism
below.

###  4.1 Reconfiguration API

Listing 5 shows how an application might implement a function that
reconfigures connections. Developers can express a choice between Chunnels
using Bertha’s select type, shown on lines 3 and 5. The select type
encapsulates both Chunnels and uses one of them (according to a developer-
specified preference order) when instantiating a connection. select can also
encapsulate two Chunnel _stacks_ , which can in turn contain additional
instances of select. When instantiating or reconfiguring a connection, Bertha
considers all stacks in the resulting tree of possibilities. Once a connection
is established, the application can use the returned handle (line 7) to call
reconfigure and change this decision.

When implementing the reconfigure function, Bertha must take into account the
type of Chunnel being reconfigured. All reconfiguration requires changing the
Chunnel implementation used by the host; we refer to this step as _unilateral_
reconfiguration (§4.2). Reconfiguring some Chunnels not only requires changing
implementations but also requires agreement between all communicating hosts.
We refer to this step as _multilateral_ reconfiguration and reuse Bertha’s
reconfiguration (§5) to ensure safety in this case. The DPDK Chunnel (line
5–6) is an example of a Chunnel that only requires unilateral reconfiguration,
while the sharding Chunnel (line 3–4) requires both unilateral and
multilateral reconfiguration. Chunnel implementors indicate whether
multilateral reconfiguration is necessary by implementing a capabilities()
function (Listing 4 lines 12-13).

We show an example of reconfiguration in Figure 6, where we replace one
Chunnel, a Kernel UDP socket Chunnel, with a DPDK Chunnel in the connection.
Note that multiple Chunnels can be reconfigured at a time, in which case
Bertha amortizes the cost of each step. First, Bertha checks compatibility
between DPDK’s implementation and implementations used by other endpoints in
the connection. In this case, the Chunnel is unilateral, so the change is
trivially compatible. Then, Bertha calls DPDK’s connect_wrap function to
initialize it (recall that DPDK’s connect_wrap function bootstraps connections
from the unit type). Note that changing this specific Chunnel implementation
requires reconfiguring all the connections on the host, since using DPDK often
requires exclusive access to the NIC. This is one reason why the lock shown in
Figure 6’s step \raisebox{-.9pt} {1}⃝ is necessary; we discuss this further in
§4.2. Then, Bertha helps the new DPDK connection type translate the connection
state from the kernel’s Chunnel implementation (\raisebox{-.9pt} {2}⃝) For
these Chunnels, the state is simply a list of active connections and the ports
they are listening on, but other Chunnels may include other useful state such
as sequence numbers. Finally, Bertha switches over to the new implementation
(\raisebox{-.9pt} {3}⃝).

Figure 6: Bertha reconfigures connections using Select, which can safely
change its encapsulated Chunnel implementation.

###  4.2 Reconfiguration Mechanism

Here we explain how we switch implementations on a single host. The core
challenge lies in ensuring that Bertha preserves state as it switches Chunnel
implementations. Individual Chunnels may have state (e.g., the DPDK example
from before relies on host-wide NIC configuration), so when re-configuring a
connection Bertha allows the new Chunnel to initialize the corresponding
state. Thus, for safety, we need to ensure that there is a “switch point”
after which no connection on any thread uses the old implementation, or uses
state held by the old implementation. Bertha must ensure that the state is
copied to the new implementation before this switch point.

We can implement reconfiguration by protecting the encapsulated connection
with a lock. The reconfigure() call simply initializes the new encapsulated
connection, locks the connection state, and updates it to use the newly
initialized connection. In this implementation, the switching point in this
implementation is when reconfigure() releases the connection lock. We describe
a lock-free optimization in §6.

When re-configuring a multilateral Chunnel, we must ensure that all peers
agree on when to switch to a new Chunnel stack. Therefore, the negotiation
protocol occurs while reconfigure() holds the connection lock. Bertha listens
for negotiation protocol packets concurrently with the application. If it
receives one, it takes the connection lock (this is the switching point)
before sending any negotiation response that would result in a reconfiguration
to ensure there is no race condition between the negotiation response and the
subsequent application data packet from the application. During
reconfiguration, Bertha uses a two-phase commit to ensure that all endpoints
agree on when to switch. If any peer refuses to switch (or times out),
reconfiguration fails.333Because all peers must accept the transition for it
to commit, a faulty peer cannot force other connection participants to switch
Chunnel stacks. We demonstrate this process in §7.2.

##  5 Negotiation

Bertha’s negotiation protocol ensures compatibility between the Chunnel stacks
used by hosts communicating using a connection and is used when connections
are established or reconfigured. We say two Chunnels are _compatible_ if data
sent by one can be processed by the other. For example, two serialization
Chunnels are compatible if data serialized by one can be deserialized by the
other. Checking compatibility requires knowledge of the Chunnel stack being
used at all endpoints and thus requires communication. In this section, we
first describe our negotiation protocol for connections that have two
endpoints, i.e., point-to-point connections. Then in §5.3 we describe a more
general protocol that supports connections with more than two endpoints. The
larger number of endpoints brings additional challenges and complexity, and
hence, we use both protocols, choosing between them depending on the type of
connection.

###  5.1 Negotiation Protocol

To begin negotiation, Bertha needs a channel over which it can communicate
with its peer. Therefore, we require that the bottom layer of the Chunnel
stack always be able to establish a compatible initial connection (by
transforming the unit type ()). We refer to this initial connection as _base
connection_ , and only require that it provide best-effort delivery. Bertha
implements a simple reliability and ordering protocol that it uses for
negotiation.

Using the base connection, the client sends the server a message describing
its Chunnel stack (represented as a set of Chunnel options). On receiving this
message, the server checks whether its connection has a Chunnel stack (making
a choice for each instance of select) that is compatible with the client’s
options (§5.2). If it finds a compatible stack, it sends it to the client;
otherwise, it returns an error to the client and terminates the connection.
Both the server and the client then use recursive connect_wrap calls (§3.1) to
initialize the selected stack.

Each Bertha connection is associated with a single address, which is used
during negotiation. Therefore, Bertha does not support connection-less
sockets. This restriction also holds for multi-party connections (§5.3), where
the address might represent a pub/sub topic or a multicast IP address. In
cases where a Chunnel, e.g., a sharding Chunnel, can send data to multiple
endpoints, we require either multi-party negotiation (§5.3) or that all
endpoints support the same set of Chunnel implementations. This ensures that
negotiating with one endpoint is sufficient to ensure compatibility with the
rest. Thus, after negotiation, Bertha returns a nonce that encodes which
select branches to use; a Chunnel can use this nonce to inform endpoints about
the stack they should use with the given client. We adopted this approach for
the load-balancing Chunnel in §7.3.

###  5.2 Checking Compatibility

The negotiation protocol requires Bertha to check compatibility between pairs
of Chunnels. A tempting approach to do so would be to use static analysis to
check compatibility between Chunnel implementations. However, Bertha does not
constrain how developers write Chunnels, and checking compatibility is at
least as hard as checking equivalence, and well-known results in logic show
that checking equivalence between programs is undecidable [40]. Therefore, in
Bertha, we adopt a simpler approach where Chunnels provide runtime type
information that Bertha uses to check compatibility. However, we allow for a
more general version of compatibility and allow Chunnels with different types
to be compatible. To allow for this, Bertha requires developers to provide
capabilities (opaque to Bertha) that implement a comparison function for
checking compatibility between Chunnels (Listing 4 lines 12-13).

Chunnel developers can either define a new capability type (or reuse an
existing one to indicate compatibility with that Chunnel), and return an
instance of this type when Bertha calls the Chunnel’s capability function. A
capability type needs to implement the Capability interface, which requires
defining a comparison function, and must be serializable. We do not assume
that capabilities are standardized; instead, we only require that developers
writing networked programs that communicate with each other (e.g., a client
and a server) use the same capability types. Thus, capabilities allow Chunnel
developers to indicate _relative_ compatibility between different
implementations. For example, if the ProtoBuf developers provide a software
reference implementation of their serialization library, a later compatible
implementation based on ProtoACC [23] can reuse the same capability types to
indicate compatibility, as can any future implementations.

In practice, we found that we can encode Chunnel capabilities as a set of
labels and that checking compatibility between them requires either checking
that the sets are equal – we refer to this as “exact-match” – or that there is
a non-empty intersection between sets – we refer to this as “composition”. So,
to check Chunnel stack compatibility, Bertha ensures that “exact-match”
capabilities (such as serialization) are present in both compared stacks, and
that “compositional” capabilities (such as sharding) are present in at least
one. If there are multiple possibilities, Bertha uses the developer’s
specified preference order (i.e., the branch listed first).

###  5.3 Multi-Party Negotiation

Bertha supports connections with an arbitrarily large number of endpoints
communicating over a single connection (e.g., multicast or publish-subscribe).
Before designing a negotiation protocol for this scenario, we need to first
determine how negotiation works in a multi-endpoint setting. A point-to-point
connection only exists for as long as the client and server are communicating
and does not need to consider cases where endpoints join after the negotiation
step has finished, but with multiple endpoints we can neither assume that all
endpoints are known when a connection is first established nor can we require
that all endpoints connect at the same time. As a result, multi-party
negotiation must allow (a) endpoints to recover a connection’s datapath stack
even if they did not participate in its negotiation, and (b) when necessary,
allow endpoints to trigger another round of negotiation to transition to a
different datapath stack.

We implement a “rendezvous-based” negotiation protocol for the multi-endpoint
case. We implement negotiation using a key-value store that can be accessed by
all endpoints. The key-value store is also responsible for recording the
negotiated datapath stack, thus allowing endpoints to recover the connection’s
datapath stack even when they do not participate in the negotiation. We
require that the key-value store support serializable multi-key transactions.
However, we do not impose other requirements, and we allow the use of either
single-node key-value stores (e.g., Redis) or replicated consensus-based
stores (e.g., etcd). While a failure of this key-value store would prevent new
endpoints from joining a multi-endpoint connection, it would not impact
endpoints that have already joined. Furthermore, the negotiation state is not
shared across connections, and thus the key-value store can be easily sharded
for scalability. While using an external-key value store makes establishing a
multi-endpoint connection take longer, any algorithm for multi-endpoint
negotiation requires agreement and would thus impose similar performance
costs.

A peer starts multi-party negotiation by connecting to the key-value store and
proposing a Chunnel implementation stack, using compare-and-swap (e.g., via a
transaction) to check for an existing stack. If the compare-and-swap operation
succeeds, the peer can safely use the Chunnel implementation stack it
proposed. Otherwise, the key-value store returns the Chunnel implementation
stack already in place amongst the existing connection participants along with
the number of participants in the connection. The new peer then uses the same
stack comparison procedure described in §5.2 to determine its local stack’s
compatibility. The peer can then use this stack to create its connection or
propose a reconfiguration as per §4.2.

##  6 Bertha Extensions

We implemented Bertha in Rust. While our implementation uses several Rust
features, including traits, async444 We implement most Chunnel operations,
including connect_wrap(), send(), and recv(), as async functions. We elide
this (as well as Rust’s lifetime annotations) in code listings for clarity. We
use the popular Tokio [26] async runtime for scheduling the resulting
coroutines. , and the type system, we believe our ideas are more general and
can be implemented using similar features and metaprogramming support in other
languages. For example, we believe C++ templates, Racket macros, and Go’s
generate tool would allow us to implement all of the features we have
described in those languages.

We implement Chunnel stacks with a structure similar to Lisp’s cons. We
represent select simply as a struct that contains both branches. Only
unilateral selects implement connect_wrap; Bertha uses a Rust enum to
represent the two choices and returns an instance of this enum. This enum
represents a limited form of dynamic dispatch; at runtime, the connection
dispatches to the appropriate implementations of the send() and recv()
functions based on the enum’s variant. Multilateral selects cannot directly
implement connect_wrap, since they must use negotiation to pick an
implementation branch. Once the negotiation process returns a choice, the
datapath for these multilateral selects is the same as the unilateral case. We
implemented Bertha’s core libraries in ~$5,000$ lines of Rust, including the
negotiation protocol (of those, ~$4,000$ lines).

For the remainder of this section, we describe two optimizations we
implemented in Bertha: zero-rtt negotiation (§6.1), and lock-free single-host
reconfiguration (§6.2).

###  6.1 Zero-RTT Negotiation

As we note in §5, runtime-reconfigurability requires agreement, and hence
endpoints must communicate before they can transfer data. The point-to-point
protocol described in §5 completes in one RTT. To address this, the Bertha
implements an additional optimization that allows clients to reestablish
connections without additional negotiation, thus reducing overheads for
applications that use many short-lived connections. Our approach to doing so
is inspired by QUIC’s zero-RTT [25] connection establishment.

Zero-RTT negotiation requires the client to remember the datapath stack used
by previous connections and re-use it when reconnecting. The client sends the
server a zero-RTT negotiation message when it knows of a previously negotiated
stack and then immediately instantiates that stack; the client can then send
data. When the server receives a zero-RTT negotiation message, it checks if
the previously negotiated stack can still be used. If so, the server re-
initiates the stack and uses it to process all subsequent data (including any
in-flight data from the client). If, however, the server cannot use the stack,
it sends the client a message indicating that the negotiation has failed and
proposing a new stack. If Bertha on the client receives such a message
indicating negotiation failure, it tears down the existing stack and
instantiates the stack included in the failure message instead.

###  6.2 Faster Reconfiguration

Changing one or more Chunnel implementations during reconfiguration requires
coordinating with any thread that might be using the connection. This
coordination is necessary for safety. The approach we described in §4.2 used
locks to implement this coordination. While using locks ensures safety, as we
show in §8.3, this adds overheads in the common case when no reconfiguration
occurs.

We thus also implement a different approach with lower fast-path overheads:
during reconfiguration, we wait for all application threads to arrive at a
barrier. Once the barrier resolves, threads other than the one performing the
reconfiguration wait on a channel before performing any further operations on
the encapsulated connection. Thus, we know it is safe to modify the
encapsulated connection by simply writing to the appropriate memory555We use
Rust’s UnsafeCell to do this., since we know there will be no concurrent
accesses to the connection object. The switching point in this case is the
barrier’s resolution: after this, no connection will use the old connection
objects. Note that in the case of multilateral Chunnels, the barrier must
protect the negotiation phase because the negotiation protocol uses the
connection to communicate. This approach is similar to that used by stop-the-
world garbage collectors. The only overhead on the fast path is that when
sending or receiving data, each thread needs to read a boolean value
indicating whether it needs to stop at a barrier. We evaluate reconfiguration
mechanisms in §8.3.

##  7 Application Benefits Evaluation

In this section, we discuss implementing an ETL (extract, transform, and load)
application in Bertha, and use it to demonstrate how Bertha can lower the
operating cost of applications deployed in the cloud and improve their
throughput and latency. We delay an evaluation of Bertha’s abstractions to §8.

ETLApp is a Bertha based application that processes streaming logs in near-
realtime to produce summary statistics and metrics that can be used by users
or displayed on a dashboard. ETLApp consists of four microservices: (1)
_producers_ , which run at the data source (i.e., where logs are generated)
and send logs and other telemetry information to (2) _ingesters_ , which
perform light-weight analysis on these logs to determine how they should be
ordered and then publish them to an appropriate topic in a publish-subscribe
service. (3) _Parsers_ subscribe to these topics, parse log entries published
by the ingesters, compute summary statistics and forward them to (4)
_consumers_ , which store them and allow users or applications to query and
retrieve computed statistics.

ETLApp uses a variety of communication functions and Chunnels: the producers
use QUIC or TLS (and PSP [43], where available) to protect their communication
with the ingester (§7.1) and a load balancer Chunnel to decide what ingester
the producer should send data to (§7.3; ingesters and parsers use a Kafka [24]
or cloud provider publish-subscribe Chunnel (§7.2; and the parser uses a local
communication Chunnel that minimizes encryption and transport costs (using
techniques inspired by Slim [46]) when the parser and consumer microservices
are colocated on the same server.

ETLApp consists of ~2,300 lines of Rust code. The Chunnels we used, which
include ones implementing QUIC, TCP, UDP (via the kernel, Shenango [34], and a
custom DPDK datapath), TLS, ordering, four publish-subscribe services,
reliability, routing, serialization, and sharding, consist of ~15,000 lines of
Rust code. Individual Chunnels range from ~400 lines for TLS and QUIC Chunnels
to ~5,000 lines for a DPDK Chunnel.

Figure 7: The ingesters and producers negotiate whether to use Kafka or GCP
Pub/Sub. Here, we force the connection to take either choice, respectively, to
demonstrate the performance tradeoffs involved.

In the remainder of this section, we first evaluate ETLApp’s end-to-end
performance and demonstrate reconfiguration’s performance benefits. After
that, we describe reconfiguration in more detail for three cases: QUIC and TLS
(§7.1), publish-subscribe message queues (§7.2 and load balancing (§7.3).

End-to-End Performance We use processing latency as the main metric for
evaluating ETLApp’s performance. We define processing latency as the time
taken between when log messages are available to the producer and when the
messages are incorporated in the consumer’s summary statistics. Our
performance evaluation uses a fixed offered load, so processing latency
accounts for both computation time and queueing. Figure 7 shows ETLApp’s
processing latency in two configurations: one with a Kafka [24] instance
available in the local cluster and one without. Deciding whether to run a
local Kafka instance or use a managed publish-subscribe service is a common
choice application operators must make: running a self-hosted Kafka instance
requires constant hardware resources but can be closer to other application
components and tuned for the application’s workload, offering better
performance. Meanwhile, managed services such as GCP Pub/Sub offer a per-
message cost model (see §7.2) but can have lower performance. Bertha allows a
single ETLApp implementation to work in both cases, since programmers Select
between these two alternatives when establishing connections at the ingester
and parser. We ran this evaluation on Cloudlab m510 machines using Linux
kernel 5.15 with Microk8s version 1.27.5 as a container runtime and Kafka
version 3.1.0, and used 8 ingesters and 4 parsers. Every interarrival
duration, our workload adds a batch of 16 150 byte records to the producer’s
log for processing. Our results (Figure 7) show that using a Kafka Chunnel
enables lower processing latency for ETLApp at higher offered loads (or lower
interarrival times). However, at lower offered loads using the GCP Pub/Sub
Chunnel both matches the Kafka configuration’s performance and has a lower
cost. As a result, no one configuration is optimal for ETLApp across all
scenarios we evaluate. Indeed, we find similar tradeoffs exist in the other
ETLApp components, and we discuss them in the rest of this section.

###  7.1 Encryption Options

Following standard guidance, we want to ensure that communications between
ETLApp’s producer and ingester are encrypted. Application developers have
several options available to ensure this: they can use HTTP/2 with TLS and
TCP, HTTP/3 [10] which uses QUIC [25] and UDP, academic proposals such as
TCPLS [36], or cloud-specific hardware encryption such as Google’s PSP [43]
and Microsoft’s hardware accelerated TLS [30]. Each of these carries different
overheads (e.g., PSP and other hardware accelerated variants require nearly no
CPU cycles, while other options must use CPU cycles for encryption), imposes
different deployment limitations (e.g., requiring deployment in Google’s
cloud), and also affects what services an application can access (one cannot
use QUIC to access a TLS-only service). While there have been ad-hoc efforts
to resolve some of these problems, e.g., QUIC servers can advertise QUIC
support to TLS clients using an Alt-Svc HTTP header, and TCPLS clients use a
TLS extension to similar effect, these efforts are not widely accessible.

Bertha offers an alternative that avoids deployment and connectivity concerns
while using applications to use the most efficient encryption protocol for
each connection. This is because Bertha uses negotiation to agree on what
Chunnel to use, thus ensuring compatibility and portability. In ETLApp, we
express these options simply as follows: Select(EnsurePSP, Select(QUIC,
make_stack!(TLS,TCP))).connect(..).

###  7.2 Publish-Subscribe Message Queues

ETLApp uses ordered publish-subscribe (pub/sub) services for communication
between ingesters and parsers to ease scaling up ingesters and parsers and
simplify fault tolerance. Pub/sub services are widely available both as
managed cloud services, e.g., AWS SQS [2], GCP PubSub [16], Azure Storage
Queues [28], and Azure Service Bus [29]), and as open-source software, e.g.,
Kafka, that applications can deploy and manage. We demonstrated above (Figure
7) that Kafka, when deployed by an application, offers ETLApp better
performance at a high load but can incur increased costs. In this subsection,
we dive deeper into how the choice of pub/sub implementation can affect
ordering and delivery semantics and costs and why workloads and requirements
might change which implementation an application uses.

Ordering and delivery semantics can vary significantly across pub/sub
services: AWS SQS and GCP PubSub support in-order message delivery, Azure
Storage Queues does not guarantee any message ordering, and Kafka always
guarantees message ordering. Similarly, in terms of delivery semantics GCP
PubSub and Kafka allow application developers to create broadcast topics or
subscriptions where multiple receivers can receive the same message, but AWS
SQS requires the use of a separate service (AWS SNS [1]), and Azure Queues
does not natively support multicast. Services also exhibit significant pricing
variance: in 2023, AWS SQS provides $1$ million free 64KB messages per month
and charges per million subsequent unordered ($0.40) and ordered ($$0.50$)
messages. GCP PubSub provides $10$GB of free messages per month with a minimum
message billing size of $1,000$ bytes, and then $$40$/TB. Azure Storage Queues
[28] charges $$0.004$/$10,000$ messages.

Reconfiguring Ordering Implementations As a result, the best option for ETLApp
depends on its input workload, since this determines the complexity of
application logic required to reorder messages. Specifically, we found that
when a single parser is in use, using SQS’s best-effort queueing setting and
ordering messages at the parser yields 63% lower median latency than using a
service-ordered message queue. However, this does not work when multiple
parsers are in use, since ordering would require coordination across all
parsers. Bertha allows ETLApp to switch between both modes: we specify that
when using SQS, ETLApp should use ordered queues only when multiple receivers
are active, and we use Bertha’s multi-party negotiation to dynamically
reconfigure the pub/sub connection.

Figure 8: Runtime reconfiguration allows switching datapath stacks. Note that
the first few requests after the second receiver arrives still use the
original stack while the transition commits.

We demonstrate this performance benefit with a microbenchmark: in Figure 8, we
send $100$ messages with an inter-arrival time of $25$ms, then start a second
receiver and send another $100$ messages. We use $5$ ordering groups and AWS
SQS. We observe that the application uses receive-side ordering and observes
lower message latencies when a single receiver is present and then safely
transitions to the higher-latency and more costly service-provided ordering
when the second receiver arrives.

Figure 9: Client sharding scales better with lower latency, and can take
advantage of more concurrency at the client without burdening the server
implementation with more connections.

###  7.3 Load Balancing

We use load balancers to spread load between active ingesters in an ETLApp
deployment. This allows us to handle cases where the rate of log updates
varies across producers and cases where ingesters might straggle. Modern
applications commonly employ load balancing for this purpose, and a variety of
different load-balancers are available: both cloud managed versions, e.g.,
Amazon’s “Application Load Balancing” (ALB), Azure’s “Application Gateway”, or
GCP’s “Internal HTTP(S) Load Balancing”, open-source versions, and even ones
that are incorporated within the application [39]. The choice of load
balancers has an impact on cost and performance, and the appropriateness of
each varies by workload.

Evaluating the benefit of reconfigurable load-balancing using ETLApp is
challenging since external factors (e.g., where the load balancer is deployed)
often determine performance. Therefore, we instead demonstrate this benefit
using a key-value store implemented with Bertha. We also used two load-
balancing Chunnels that provide us with greater control and an easier way to
reason about performance: one in which a remote server evaluates a hash
function on message keys and then forwards messages to the appropriate server,
and one where the client evaluates the hash function and sends messages that
bypass the load balancer. We evaluated these options using a YCSB-based
benchmark. In each experiment, we first run a warm-up phase and loading phase
that issues $12,000$ PUT requests, then we use 3 clients, each running on its
own server and splitting its requests among $8$ connections, to issue requests
according to the “Workload B” request distribution. In this workload, each
message is $132$B in size. Each client issues 1,214,020 requests, and we
control the request rate (using Poisson arrivals) to meet a target offered
load. To ensure that the offered load remains consistent, we terminate the
experiment after any connection sends its last request. We report results from
evaluation run on xl170 nodes in Cloudlab, which uses 25 Gbit/s Mellanox CX-4
Lx NICs, with one server machine and two client machines. All machines used
Linux kernel 5.4.0 and Mellanox OFED driver version 5.6-2.0.9.0. We used a
custom DPDK-based UDP Chunnel with this application, which uses DPDK version
20.11.0.

Our results in Figure 9 show that client-side load balancing offers better
performance: when using server-side sharding, the key-value store cannot meet
a $50\mu$s bound for p$95$ latency beyond 40,000 requests per second, but with
client-side sharding can meet the latency bound up to 800,000 requests per
second. However, client-side load balancing makes application management more
complex since changing the set of backend servers requires updating all
clients. Bertha allows application deployers to switch between these options:
using the client-side approach in cases where the set of backend servers is
relatively stable and using server-side load balancing otherwise.

##  8 Bertha Overheads Evaluation

We next evaluate Bertha’s overheads. We focus on cases that require the most
processing, and consequently, we report worse-case overheads for Bertha. Our
evaluation focuses on three aspects of Bertha’s performance: (a) the impact
Chunnels have on application throughput (§8.1); (b) how does Bertha impact
application latency (§8.2); and (c) how long Bertha takes to reconfigure a
connection (§8.3).

###  8.1 Throughput

(a) GBit/s

(b) MPackets/s

Figure 10: The overhead of including a marginal Chunnel in a DPDK-based UDP
connection is between 0-27%, and becomes negligible with larger packet sizes.
Columns show p50 and errorbars show p25 and p75 across 10 trials. Figure 11:
Overhead of a marginal Chunnel for Linux TCP-based connections.

We evaluate Bertha’s impact on application throughput using a file transfer
benchmark where a client uses $10$ threads, each of which connects to a
server, sends a short request receives a $5$GB file in response (thus
transfering a total of $50$GB. We implemented two versions of this benchmark:
a UDP version that uses a UDP Chunnel over a DPDK Chunnel that implements
logic for multiplexing and demultiplexing connections, and a TCP version that
uses a TCP Chunnel implemented using sockets. We refer to these two base
implementations as ‘No Chunnel’ implementation, since each only includes code
that any such client would require. As a baseline, we compare this version’s
throughput to that of a version of the application written without DPDK, which
our figures refer to as the ‘Inlined (Baseline)’ configuration. We also
compare the throughput of these ‘No Chunnel’ versions to the throughput
achieved by an extension of the program that adds between one and five no-op
Chunnels 666We used Rust’s std::hint::black_box in the no-op Chunnel, forcing
the compiler to assume that the caller can use any part of the data. (labelled
‘1 Chunnel’, etc.), that merely reads the data and forwards it to the next
Chunnel.

We measure goodput (i.e., rate of file transfer) for both versions of the
application, while running them on XL710 nodes connected by $25$Gbit/s links
in CloudLab (the same setup as in §7.3). For the UDP applications, we vary
packet sizes between $64$ and $1,460$ bytes, while for the TCP applications,
we vary the size of the buffer provided to the send call to be between $1,024$
to $8,192$ bytes.

We show the UDP results in Figure 10 and the TCP results in Figure 11. For the
UDP configuration, we find that in the worst case (the $64$Byte packets), the
median achieved goodput and packets transmitted are $27\%$ lower with 5
Chunnels and $18\%$ lower with 1 Chunnel. However, even a small increase in
packet size to $128$ Bytes decreases this overhead to $4\%$ with 5 Chunnels,
and further increases in packet size cause this to decrease even further. For
the TCP configuration, trends are less clear because of the inherent
variability in using the kernel networking stack, but as a general trend, the
overhead decreases as the write size increases.

Figure 12: “Full” shows the KV-store application using a multi-threaded DPDK
datapath. “No Ch.” removes the use of Chunnels and negotiation, and “No Ch.-No
Cn.” further removes connection multiplexing and demultiplexing.

###  8.2 Key-Value Store Operation Latency

Next, we use the key-value store benchmark described in §7.3 to evaluate
Bertha’s impact on latency. In this case we use three functional Chunnels
(serialization, sharding, and reliability) and compare Bertha’s latency
(referred to as ‘Full’) in our graphs to two baselines: one that removes
negotiation and Chunnels from the application and inlines the remaining
implementation, but retains connection multiplexing and demultiplexing
features in the datapath (referred to as ‘No Ch.’); and a second that also
removes connection multiplexing (‘No Ch.-No Cn.’).

Figure 12 shows response latencies as we vary the offered load. Compared to
‘No Ch.-No Cn.’, Bertha (‘Full’) exhibits $28\%$ higher median latency at an
offered load of $80,000$ requests per second, and this $64\%$ higher median
latency at $800,000$ requests per second. We analyzed this result further and
found that this difference is because of time spent in connection multiplexing
and demultiplexing. Consequently, when we compare Bertha’s latency to ‘No
Ch.’, we observe that Bertha’s median latency is within $6\%$ at both $80,000$
and $800,000$ requests per second of offered load. Connection multiplexing and
demultiplexing are common requirements for most applications, and therefore,
our results show that Bertha imposes minimal latency overheads while providing
additional functionality in the form of reconfiguration.

Figure 13: Bertha can swap a connection’s datapath at runtime without
disrupting application state. In this example, we change a server from a
multi-threaded DPDK datapath to one that spin-polls only on one thread.

###  8.3 Reconfiguration

Finally, we evaluate the effect of dynamic reconfiguration on application
latency. Recall from §6.2 that Bertha offers two mechanisms for dynamic
reconfiguration, one using a lock and another which is lock-free; we evaluate
both implementations. We implement a microbenchmark that switches between two
implementations of a DPDK datapath: “DPDK Single Thread”, which uses only one
spin-polling thread to perform packet operations (similar to Shenango [34] or
Snap [27]), and “DPDK Multi Thread” uses spin-polling coroutines on all
threads to perform packet operations.

We show the results of switching from “DPDK-Inline” to “DPDK-Thread” in Figure
13. The echo benchmark triggers a Chunnel switch at the server after 3,000
messages, and we display percentile boxplots (p5, p25, p50, p75, p95) of the
achieved latency for the two configurations within each 100ms measurement
epoch. Since the lock-full reconfiguration implementation must acquire a lock
for every send and receive, the lock-free version achieves lower latencies
with the multi-thread datapath: 13$\mu$s median latency vs 17$\mu$s for lock-
ful. However, the lock-free version incurs a larger reconfiguration time: we
observe a 68$\mu$s median latency in the 10 seconds following the
reconfiguration vs. 18$\mu$s when using a lock. Note that reconfiguration in
this case requires reconfiguring the NIC, and this results in a short duration
when the application can neither send nor receive network traffic.

##  9 Discussion and Conclusion

Porting Existing Libraries to Bertha To use Bertha, application developers
must find Chunnels that implement the functionality they require. At present,
this might require them to port existing libraries to Bertha. Porting most
libraries requires minimal effort (as discussed in §6): the developer calls
the appropriate library functions from the Chunnel’s implementation. However,
building Chunnels is more complicated in two cases: the first is when trying
to encapsulate systems, e.g., Shenango [34, 14] and Shinjuku [22], that
provide specialized runtimes rather than composable libraries; the second is
when dealing with libraries like OpenSSL [33] which limit how they can be
composed (OpenSSL uses sockets internally). While we found that we could
always encapsulate these runtimes, doing so required taking approaches such as
running them in a different thread or process and communicating with them
using IPC or memory buffers, adding to Bertha’s overhead.

Deploying Bertha Because Bertha introduces a negotiation protocol, all parties
in a connection must use it to be able to communicate. This negotiation
protocol can replace existing ad-hoc methods of feature discovery, such as the
QUIC discovery convention described in §7.1. Of course, to use Bertha’s
negotiation an endpoint must be aware of it; past work has described various
mechanisms for advertising protocol availability such as via DNS, and any such
mechanism would work with Bertha. While we acknowledge that using Bertha
between arbitrary Internet hosts may be challenging, there are common cases
where a single entity controls all connection participants, such as in service
proxies, a cloud provider’s SDK, or for internal communication between
components of a distributed system.

Conclusion Bertha provides abstractions that allow the network stack to be
reconfigured at runtime. These abstractions, which provide a uniform
interface, not only enable reconfiguration but also make it easy to extend
Bertha with new functionality and allow the Bertha runtime to ensure check
compatibility between communicating hosts, thus improving reliability. We will
open-source our implementations and experiment scripts.

## References

  * [1] Amazon.  Amazon SNS.  https://aws.amazon.com/sns/.
  * [2] Amazon.  Amazon SQS Pricing.  https://aws.amazon.com/sqs/pricing/.
  * [3] Apache.  Thrift.  https://thrift.apache.org/.
  * [4] J. Arnold and M. F. Kaashoek.  Ksplice: Automatic Rebootless Kernel Updates.  In EuroSys, 2009.
  * [5] Axboe, Jens.  Efficient IO with io_uring.  https://kernel.dk/io_uring.pdf, 2019.
  * [6] N. T. Bhatti and R. D. Schlichting.  A System for Constructing Configurable High-Level Protocols.  In SIGCOMM, 1995.
  * [7] E. Biagioni.  A Structured TCP in Standard ML.  In SIGCOMM, 1994.
  * [8] K. Birman and T. Joseph.  Exploiting Virtual Synchrony in Distributed Systems.  In SOSP, 1987.
  * [9] A. D. Birrell and B. J. Nelson.  Implementing Remote Procedure Calls.  ACM TOCS, 2(1):39–59, 1984.
  * [10] M. Bishop.  HTTP/3.  RFC 9114, 2022.
  * [11] I. Choi, E. Michael, Y. Li, D. R. K. Ports, and J. Li.  Hydra: Serialization-Free Network Ordering for Strongly Consistent Distributed Applications.  In NSDI, 2023.
  * [12] DPDK Authors.  DPDK.  https://www.dpdk.org/.
  * [13] Envoy Project Authors.  Envoy Proxy.  https://www.envoyproxy.io/.
  * [14] J. Fried, Z. Ruan, A. Ousterhout, and A. Belay.  Caladan: Mitigating Interference at Microsecond Timescales.  In OSDI, 2020.
  * [15] C. Giuffrida, A. Kuijsten, and A. S. Tanenbaum.  Safe and Automatic Live Update for Operating Systems.  In ASPLOS, 2013.
  * [16] Google.  GCP PubSub Pricing.  https://cloud.google.com/pubsub/pricing.
  * [17] Google.  gRPC.  https://grpc.io/.
  * [18] Hashicorp.  Terraform.  https://www.terraform.io/.
  * [19] H. Huang, C. Xiang, L. Zhong, and Y. Zhou.  $\\{$PYLIVE$\\}$:$\\{$On-the-Fly$\\}$ Code Change for Python-based Online Services.  In USENIX ATC, 2021.
  * [20] N. C. Hutchinson and L. L. Peterson.  The X-Kernel: An Architecture for Implementing Network Protocols.  IEEE Trans. Softw. Eng., 17(1), 1991.
  * [21] Istio.  Istio.  https://www.istio.io/.
  * [22] K. Kaffes, T. Chong, J. T. Humphries, A. Belay, D. Mazières, and C. Kozyrakis.  Shinjuku: Preemptive Scheduling for µsecond-Scale Tail Latency.  In NSDI, 2019.
  * [23] S. Karandikar, C. Leary, C. Kennelly, J. Zhao, D. Parimi, B. Nikolic, K. Asanovic, and P. Ranganathan.  A Hardware Accelerator for Protocol Buffers.  In MICRO, 2021.
  * [24] J. Kreps, N. Narkhede, J. Rao, et al.  Kafka: A Distributed Messaging System for Log Processing.  In NetDB, volume 11, pages 1–7, 2011.
  * [25] A. Langley, A. Riddoch, A. Wilk, A. Vicente, C. Krasic, D. Zhang, F. Yang, F. Kouranov, I. Swett, J. Iyengar, J. Bailey, J. Dorfman, J. Roskind, J. Kulik, P. Westin, R. Tenneti, R. Shade, R. Hamilton, V. Vasiliev, W.-T. Chang, and Z. Shi.  The QUIC Transport Protocol: Design and Internet-Scale Deployment.  In SIGCOMM, 2017.
  * [26] Lerche, Carl and others.  Tokio.  https://www.tokio.rs.
  * [27] M. Marty, M. de Kruijf, J. Adriaens, C. Alfeld, S. Bauer, C. Contavalli, M. Dalton, N. Dukkipati, W. C. Evans, S. Gribble, N. Kidd, R. Kononov, G. Kumar, C. Mauer, E. Musick, L. Olson, E. Rubow, M. Ryan, K. Springborn, P. Turner, V. Valancius, X. Wang, and A. Vahdat.  Snap: A Microkernel Approach to Host Networking.  In SOSP, 2019.
  * [28] Microsoft.  Azure Queues Storage Pricing.  https://azure.microsoft.com/en-us/pricing/details/storage/queues/.
  * [29] Microsoft.  Azure Service Bus Pricing.  https://azure.microsoft.com/en-us/pricing/details/service-bus/.
  * [30] Microsoft.  Encryption for data-in-transit.  https://learn.microsoft.com/en-us/compliance/assurance/assurance-encryption-in-transit, 2023\.
  * [31] A. Narayan, A. Panda, M. Alizadeh, H. Balakrishnan, A. Krishnamurthy, and S. Shenker.  Bertha: Tunneling through the Network API.  In HotNets, 2020.
  * [32] Z. Niu, H. Xu, D. Han, P. Cheng, Y. Xiong, G. Chen, and K. Winstein.  Network Stack as a Service in the Cloud.  In HotNets, 2017.
  * [33] OpenSSL Software Foundation.  OpenSSL.  https://www.openssl.org/.
  * [34] A. Ousterhout, J. Fried, J. Behrens, A. Belay, and H. Balakrishnan.  Shenango: Achieving High CPU Efficiency for Latency-sensitive Datacenter Workloads.  In NSDI, 2019.
  * [35] L. Popa, A. Ghodsi, and I. Stoica.  HTTP as the Narrow Waist of the Future Internet.  In HotNets, 2010.
  * [36] F. Rochet, E. Assogba, M. Piraux, K. Edeline, B. Donnet, and O. Bonaventure.  TCPLS: Modern Transport Services with TCP and TLS.  In SOSP, 2021.
  * [37] F. Rommel, C. Dietrich, D. Friesel, M. Köppen, C. Borchert, M. Müller, O. Spinczyk, and D. Lohmann.  From Global to Local Quiescence: Wait-Free Code Patching of Multi-Threaded Processes.  In OSDI, 2020.
  * [38] M. Russinovich, N. Govindaraju, M. Raghuraman, D. Hepkin, J. Schwartz, and A. Kishan.  Virtual Machine Preserving Host Updates for Zero Day Patching in Public Cloud.  In EuroSys ’21, 2021.
  * [39] H. Saokar, S. Demetriou, N. Magerko, M. Kontorovich, J. Kirstein, M. Leibold, D. Skarlatos, H. Khandelwal, and C. Tang.  ServiceRouter: Hyperscale and Minimal Cost Service Mesh at Meta.  In OSDI, 2023.
  * [40] M. Sipser.  Introduction to the Theory of Computation.  Course Technology, Third edition, 2013.
  * [41] M. M. Swift, M. Annamalai, B. N. Bershad, and H. M. Levy.  Recovering Device Drivers.  ACM ToCS, 24(4), 2006.
  * [42] M. M. Swift, B. N. Bershad, and H. M. Levy.  Improving the Reliability of Commodity Operating Systems.  ACM ToCS, 23(1), 2005.
  * [43] Vahdat, Amin and Soheil Hassas Yeganeh.  Announcing PSP’s cryptographic hardware offload at scale is now open source.  https://cloud.google.com/blog/products/identity-security/announcing-psp-security-protocol-is-now-open-source.
  * [44] M. Welzl, S. Islam, M. Gundersen, and A. Fischer.  Transport Services: A Modern API for an Adaptive Internet Transport Layer.  IEEE Communications Magazine, 59(4), 2021.
  * [45] I. Zhang, A. Raybuck, P. Patel, K. Olynyk, J. Nelson, O. S. N. Leija, A. Martinez, J. Liu, A. K. Simpson, S. Jayakar, P. H. Penna, M. Demoulin, P. Choudhury, and A. Badam.  The Demikernel Datapath OS Architecture for Microsecond-Scale Datacenter Systems.  In SOSP, 2021.
  * [46] D. Zhuo, K. Zhang, Y. Zhu, H. H. Liu, M. Rockett, A. Krishnamurthy, and T. Anderson.  Slim: OS Kernel Support for a Low-Overhead Container Overlay Network.  In NSDI, 2019.