# Vertical Federated Learning over Cloud-RAN: Convergence Analysis and System
Optimization

Yuanming Shi, Shuhao Xia, Yong Zhou, Yijie Mao, Chunxiao Jiang, and Meixia Tao
Y. Shi, S. Xia, Y. Zhou, and Y. Mao are with the School of Information Science
and Technology, ShanghaiTech University, Shanghai 201210, China (e-mail:
{shiym, xiashh, zhouyong, maoyj}@shanghaitech.edu.cn). _(Corresponding author:
Yong Zhou.)_ C. Jiang is with the Tsinghua Space Center and the Beijing
National Research Center for Information Science and Technology, Tsinghua
University, Beijing 100084, China (e-mail: jchx@tsinghua.edu.cn). M. Tao is
with Department of Electronic Engineering, Shanghai Jiao Tong University,
Shanghai 201210, China (e-mail:mxtao@sjtu.edu.cn).

###### Abstract

Vertical federated learning (FL) is a collaborative machine learning framework
that enables devices to learn a global model from the feature-partition
datasets without sharing local raw data. However, as the number of the local
intermediate outputs is proportional to the training samples, it is critical
to develop communication-efficient techniques for wireless vertical FL to
support high-dimensional model aggregation with full device participation. In
this paper, we propose a novel cloud radio access network (Cloud-RAN) based
vertical FL system to enable fast and accurate model aggregation by leveraging
over-the-air computation (AirComp) and alleviating communication straggler
issue with cooperative model aggregation among geographically distributed edge
servers. However, the model aggregation error caused by AirComp and
quantization errors caused by the limited fronthaul capacity degrade the
learning performance for vertical FL. To address these issues, we characterize
the convergence behavior of the vertical FL algorithm considering both uplink
and downlink transmissions. To improve the learning performance, we establish
a system optimization framework by joint transceiver and fronthaul
quantization design, for which successive convex approximation and alternate
convex search based system optimization algorithms are developed. We conduct
extensive simulations to demonstrate the effectiveness of the proposed system
architecture and optimization framework for vertical FL.

###### Index Terms:

Vertical federated learning, cloud radio access network, over-the-air
computation.

##  I Introduction

Federated learning (FL), as an emerging distributed learning paradigm, has
recently attracted lots of attention by enabling devices to collaboratively
learn a global machine learning (ML) model without local data exchanging to
protect data privacy [1]. Based on the partition of datasets, FL is typically
categorized into horizontal FL and vertical FL [2]. Specifically, horizontal
FL is adopted in the sample-partition scenario where different devices share
the same feature space but have different samples [3], as shown in Fig. 1(a).
Vertical FL, on the other hand, enables devices to collaboratively learn a
global model from the feature-partition datasets that share the same sample
space but different feature spaces [4], as shown in Fig. 1(b). Vertical FL has
wide applications across e-commerce, smart healthcare, and Internet of Things
(IoT). For example, in IoT networks, data collected by different types of
sensors (e.g., video cameras, GPS, and inertial measurement units) need to be
smartly fused for planning and decision-making [5, 6]. As each participant
owns partial model in vertical FL, the messages exchanged between participants
are based on the intermediate outputs of the sub-model and its local data,
rather than local updates as in horizontal FL. The number of intermediate
outputs is proportional to the number of training samples. This leads to high-
dimensional data transmission with a large volume of training samples [7]. In
addition, since vertical FL needs to aggregate all local intermediate outputs
to obtain the final prediction, it requires all devices to participate in the
training process in vertical FL. Therefore, the high-dimensional data
transmission with full device participation in vertical FL brings a unique
challenge for communication-efficient system design. Existing works on device
selections in horizontal FL cannot address this issue in wireless vertical FL.

To design communication-efficient wireless vertical FL, it is critical to
develop novel wireless techniques to transmit intermediate outputs over shared
wireless medium with limited radio resources [8]. Conventional orthogonal
multiple access schemes aim to decode individual information, which however
ignores the communication task for data transmission. Instead, over-the-air
computation (AirComp) serves as a promising approach to enable efficient
aggregation for FL [9, 10, 11, 12]. By exploiting waveform superposition of
multiple access channel, AirComp can directly aggregate the local updates via
concurrent uncoded transmission [13]. As a result, AirComp can reduce the
required radio resources and the access latency compared with orthogonal
multiple access schemes [14]. To further improve the communication and
learning efficiency of wireless FL with AirComp, the intrinsic properties of
the local updates are exploited, e.g., gradient sparsity [15], temporal
correlation [16], spatial correlation [17], and gradient statistics [18].
However, due to the heterogeneity of channel conditions across different
devices, AirComp based FL inevitably suffers from the communication
stragglers, i.e., the devices with the weak channel conditions [19, 20].
Specifically, the communication straggler devices enforcing the magnitude
alignment for AirComp may result in high model aggregation error, which
degrades the learning performance. Although it can be alleviated by excluding
the stragglers from the model aggregation process [21, 22], partial device
participation reduces the size of the FL training datasets, which deteriorates
the performance for FL models. Besides, these works mainly focus on horizontal
FL with partial device selection and are inapplicable in the vertical FL
scenario. The reason is that vertical FL requires all the devices to
participate the training process since each device owns one sub-model of the
global model. It is thus critical to design a novel AirComp-based transmission
scheme for wireless vertical FL with full device participation.

(a) Horizontal FL

(b) Vertical FL

Figure 1: Sample and feature spaces of horizontal and vertical FL.

To enable all devices including the straggler devices to participate in the
training process, one approach is to improve the wireless channel quality
through advanced communication technologies such as unmanned aerial vehicles
[23] and reconfigurable intelligent surface [24, 25, 26]. However, these
wireless techniques may become inapplicable in the IoT scenarios with massive
geographically distributed devices. An alternative approach is to effectively
extend the wireless transmission coverage by deploying multiple edge servers,
thereby cooperatively assisting data exchanges between the devices and the
edge servers [27, 28, 29]. This can shorten the communication distances
between the devices and the edge servers, which enhances the transmission
reliability and reduces the latency for the straggler devices. A growing body
of recent work has demonstrated the effectiveness of deploying multiple edge
servers for FL, including semi-decentralized FL [30] and hierarchical FL [31,
32]. To mitigate inter-cluster interference, the authors in [30, 31] proposed
to allocate orthogonal radio resource blocks for each device for model
aggregation in FL. However, this orthogonal transmission scheme is inefficient
due to the limited spectrum resources. In addition, many studies have
considered distributed edge computing systems based on Cloud-RAN architecture
with various communication metrics, such as energy consumption [33] and
latency [34], for resource management.

In this paper, we propose to leverage cloud radio access network (Cloud-RAN)
to enable cooperative model aggregation across densely deployed multiple edge
servers over large geographic areas. This is achieved by moving baseband
processing from remote radio heads (RRHs) (i.e., edge servers) to a baseband
unit (BBU) pool (i.e., central server), where digital fronthaul links connect
the central server and the edge servers [35]. Therefore, the learning
performance and communication efficiency can be significantly improved through
cooperative model aggregation and centralized signal processing for large-
scale wireless vertical FL system. Besides, since the baseband signal
processing and model aggregation are moved to the central server in the BBU
pool, edge servers only need to support basic signal transmission
functionality, which further reduces their energy consumption and deployment
cost [35, 36].

In addition, we propose a fast and reliable transmission scheme to support
vertical FL with geographically distributed devices over wireless networks.
Different from the existing works on vertical FL over wireless networks [37],
we shall implement AirComp-based vertical FL over Cloud-RAN to support
heterogeneous devices. However, the fronthaul links between the edge servers
and the central server have limited capacity, which causes additional
quantization error. Furthermore, the quantization error along with AirComp
model aggregation error severely degrade the learning performance of vertical
FL. The main purpose of this paper is to reveal the impact of limited
fronthaul capacity and AirComp aggregation error on the learning performance
and then optimize the Cloud-RAN enabled vertical FL system by jointly
considering the learning performance and communication efficiency. The main
contributions are summarized as follows:

  * •

We propose a Cloud-RAN based network architecture for communication-efficient
vertical FL to alleviate the communication straggler issue by leveraging the
distributed edge servers with shortened communication distance between devices
and edge servers. This is achieved by transmitting the intermediate outputs at
each device to multiple edge servers (i.e., RRHs) via AirComp. The edge
servers then compress the received aggregation signals and forward the
quantized signals to the central server via digital fronthaul links for
centralized global model aggregation in vertical FL.

  * •

We characterize the convergence behaviors of the gradient-based vertical FL
algorithm under AirComp model aggregation error and limited fronthaul capacity
in both uplink and downlink transmission processes. We further propose a
system optimization framework with joint transceiver and quantization design
to minimize the learning optimality gap. The resulting system optimization
problem is solved based on the successive convex approximation and alternate
convex search approaches with convergence guarantees.

  * •

We conduct extensive simulations to demonstrate the effectiveness of the
proposed system optimization framework and system architecture. We demonstrate
that the proposed framework can effectively combat the effects of
communication limitations on the learning performance of vertical FL in terms
of the optimality gap. The superiority of the distributed antenna system in
Cloud-RAN for vertical FL is also demonstrated.

The remainder of this paper is constructed as follows. Section II introduces
the learning framework of vertical FL and the communication model. Section III
provides the convergence analysis of the vertical FL algorithm, and Section IV
formulates the system optimization problem for resource allocation. Extensive
numerical results of the proposed scheme are presented in Section V, followed
by the conclusion in Section VI.

Throughout this paper, we denote the complex number set and the real number
set by $\mathbb{C}$ and $\mathbb{R}$. We denote scalars, vectors, and matrices
by regular letters, bold small letters, and bold capital letters,
respectively. The operations of transpose and conjugate transpose are
respectively denoted by the superscripts $(\cdot)^{\mathsf{T}}$ and
$(\cdot)^{\mathsf{H}}$. $\mathbb{E}[\cdot]$ denotes the expectation operator.
$\mathcal{N}(\bm{x};\bm{\mu},\bm{\Sigma})$ and
$\mathcal{CN}(\bm{x};\bm{\mu},\bm{\Sigma})$ denote that the random variable
$\bm{x}$ follows Gaussian distribution and complex Gaussian distribution with
mean $\bm{\mu}$ and covariance $\bm{\Sigma}$, respectively. We use $\bm{I}$
and $\operatorname{diag}\\{\bm{x}\\}$ to respectively denote the identity
matrix and the diagonal matrix with diagonal entries specified by $\bm{x}$. We
denote real and imaginary components of $x$ by $\Re(x)$ and $\Im(x)$,
respectively.

##  II System Model

###  II-A Vertical Federated Learning

Figure 2: Aggregation of local prediction results for the $i$-th training
sample in vertical FL.

Consider a vertical FL system consisting of $K$ IoT devices and one central
server, as shown in Fig. 2. Let
$\mathcal{D}=\\{(\bm{x}_{i,1},\ldots,\bm{x}_{i,k}),y_{i}\\}_{i=1}^{L}$ denote
the whole training dataset of $L$ samples, where
$\bm{x}_{i,k}\in\mathbb{R}^{d_{k}}$ denotes the partial non-overlapped
features with dimension $d_{k}$ of sample $i$ located at device $k$, and
$y_{i}$ denotes the corresponding label only available at the central server
[37]. We use the concatenated vector
$\bm{x}_{i}=[(\bm{x}_{i,1})^{\mathsf{T}},\ldots,(\bm{x}_{i,k})^{\mathsf{T}}]^{\mathsf{T}}\in\mathbb{R}^{d}$
to denote the whole feature vector of sample $i$ with dimension
$d=\sum_{k=1}^{K}d_{k}$. The goal of vertical FL is to collaboratively learn a
global model $\bm{w}$ that maps an input $\bm{x}_{i}$ to the corresponding
prediction through a continuously differentiable function
$h(\bm{w},\bm{x}_{i})$. We assume that device $k$ maps the local feature
$\bm{x}_{i,k}$ to the local prediction result $g_{k}(\bm{w}_{k},\bm{x}_{i,k})$
using sub-model $\bm{w}_{k}$, where $g_{k}(\cdot)$ is defined as the local
prediction function. The dimension of each local prediction result is the
same, which is determined by the number of classes for the classification
task. For example, for a binary classification task, the local prediction
result $g_{k}(\bm{w}_{k},\bm{x}_{i,k})$ is a scalar.

The input of the final prediction function $h(\bm{w},\bm{x}_{i})$ is based on
the aggregation of the local prediction results
$\\{g_{k}(\bm{w}_{k},\bm{x}_{i,k})\\}$. We assume the final prediction is
obtained by aggregating the local prediction results111For a wide range of
models such as logistic classification and support vector machines the
intermediate outputs of these models are additive, i.e., the final inner
product can be split into the sum of multiple inner products., followed by a
non-linear transformation $\sigma(\cdot)$ (e.g., a softmax function) as
follows:

$\displaystyle
h(\bm{w},\bm{x}_{i})=\sigma\left(\sum_{k=1}^{K}g_{k}(\bm{w}_{k},\bm{x}_{i,k})\right),$
(1)

where $\bm{w}$ is the global model concatenated by the sub-models
$\\{\bm{w}_{k}\\}$, i.e.,
$\bm{w}=\left[(\bm{w}^{\mathsf{T}}_{1},\cdots,\bm{w}^{\mathsf{T}}_{K})\right]^{\mathsf{T}}$.
Based on (1), the central server can aggregate local prediction results
without directly accessing local features and local model.

To learn the global model $\bm{w}$, we focus on the following empirical risk
minimization problem:

$\displaystyle\min_{\bm{w}\in\mathbb{R}^{d}}F(\bm{w})$
$\displaystyle=\frac{1}{L}\sum_{i=1}^{L}f_{i}(\bm{w})+\lambda\sum_{k=1}^{K}r_{k}(\bm{w}),$
(2)

where $f_{i}(\cdot)$ is the sample-wise loss function indicating the loss
between the prediction $h(\bm{w},\bm{x}_{i})$ and the true label $y_{i}$ for
sample $i$, $r_{k}(\cdot)$ is the regularization for the sub-model
$\bm{w}_{k}$ on device $k$, and $\lambda$ is a hyperparameter.

In this paper, we consider using the full-batch gradient descent (GD)
algorithm to solve (2). Let $\nabla F(\bm{w})$ denote the gradient of
$F(\cdot)$ with respect to $\bm{w}$, then we obtain

$\nabla
F(\bm{w})=\frac{1}{L}\sum_{i=1}^{L}\begin{bmatrix}\nabla_{1}f_{i}(\bm{w})\\\
\vdots\\\ \nabla_{K}f_{i}(\bm{w})\end{bmatrix}+\lambda\begin{bmatrix}\nabla
r_{1}(\bm{w})\\\ \vdots\\\ \nabla r_{K}(\bm{w})\end{bmatrix},$ (3)

where $\nabla_{k}f_{i}(\bm{w})=\frac{\partial
f_{i}(\bm{w})}{\partial\bm{w}_{k}}$ is the partial gradient of the loss
function $f_{i}(\cdot)$ with respect to the sub-model $\bm{w}_{k}$. Based on
the chain rule in calculus, the partial gradient of $\nabla_{k}f_{i}(\bm{w})$
can be rewritten as

$\displaystyle\nabla_{k}f_{i}(\bm{w})$
$\displaystyle=f^{\prime}_{i}\left(\sigma\left(\sum_{k=1}^{K}g_{k}(\bm{w}_{k},\bm{x}_{i,k})\right)\right)\sigma^{\prime}\left(\sum_{k=1}^{K}g_{k}(\bm{w}_{k},\bm{x}_{i,k})\right)\nabla
g_{k,i}(\bm{w}_{k})$ (4)
$\displaystyle=G_{i}\left(\sum_{k=1}^{K}g_{k}(\bm{w}_{k},\bm{x}_{i,k})\right)\nabla
g_{k,i}(\bm{w}_{k}),$

where $f^{\prime}_{i}(\cdot)$ and $\sigma^{\prime}(\cdot)$ are the derivatives
of $f_{i}(\cdot)$ and $\sigma(\cdot)$, $\nabla
g_{k,i}(\bm{w}_{k})=\frac{\partial
g_{k}(\bm{w}_{k},\bm{x}_{i,k})}{\partial\bm{w}_{k}}$ is the partial gradient
of the local prediction function $g_{k}(\cdot)$ with respect to $\bm{w}_{k}$,
and $G_{i}(\cdot)$ is an auxiliary function with respect to the aggregation of
local prediction results for sample $i$.

In vertical FL, the features are distributed at local devices and the labels
are known by the central server, so the partial gradients can be calculated
separately. Specifically, device $k$ computes and uploads the local prediction
result $g_{k}(\bm{w}_{k},\bm{x}_{i,k})$ to the central server. Then, the
central server aggregates the local prediction results to obtain
$\sum_{k=1}^{K}g_{k}(\bm{w}_{k},\bm{x}_{i,k})$ and broadcast
$G_{i}\left(\sum_{k=1}^{K}g_{k}(\bm{w}_{k},\bm{x}_{i,k})\right)$ back to the
devices. Consequently, the devices obtain the partial gradient
$\nabla_{k}f_{i}(\bm{w})$ through Eq. (4) since $\nabla g_{k,i}(\bm{w}_{k})$
can be calculated locally. Each device then updates its local model by taking
one step of gradient decent with learning rate $\mu$, i.e.,

$\displaystyle\bm{w}^{(t+1)}_{k}=\bm{w}^{(t)}_{k}-\mu\left(\frac{1}{L}\sum_{i=1}^{L}\nabla_{k}f_{i}(\bm{w}^{(t)})+\lambda\nabla
r_{k}(\bm{w}^{(t)})\right).$ (5)

Algorithm 1 summarizes the above procedure. Since the central server only
receives the aggregation of local prediction results, i.e., neither local
features nor local models are uploaded to the central server, the privacy of
local data can be guaranteed. Nevertheless, the number of local prediction
results at each device is proportional to the number of training samples,
which leads to high-dimensional data transmission. In addition, all devices
are required to participate the training process in vertical FL since each
device owns a sub-model $\bm{w}_{k}$. Hence, it is essential to design
communication efficient technologies for vertical FL in wireless networks to
tame the challenge brought by the high-dimensional data transmission with full
device participation.

Require : learning rate $\mu$, maximum communication rounds $T$,
initialization $\\{\bm{w}_{k}^{(0)}\\}$.

Output :
$\bm{w}^{(T)}=\left[(\bm{w}_{1}^{(T)})^{\mathsf{T}},\ldots,(\bm{w}_{K}^{(T)})^{\mathsf{T}}\right]^{\mathsf{T}}$.

1 for  _$t=0,1,\ldots,T$_ do

2  _each device $k\in[K]$_ do in parallel

3 if  _$t=0$_ then

4 Send the local prediction results
$\left\\{g_{k}(\bm{w}^{(0)}_{k},\bm{x}_{i,k})\right\\}^{L}_{i=1}$ to the
central server;

5 end if

6 else

7 Compute $\\{\nabla_{k}f(\bm{w}^{(t)})\\}_{i=1}^{L}$ via (4), and update the
local model $\bm{w}^{(t+1)}_{k}$ via (5);

8 Send the local prediction results
$\left\\{g_{k}(\bm{w}^{(t+1)}_{k},\bm{x}_{i,k})\right\\}^{L}_{i=1}$ to the
central server;

9 end if

10

11 end

12  _the central server_ do

13 Receive and aggregate the local prediction results
$\left\\{\sum_{k=1}^{K}g_{k}(\bm{w}^{(t)}_{k},\bm{x}_{i,k})\right\\}^{L}_{i=1}$;

14 Compute
$\left\\{G_{i}\left(\sum_{k=1}^{K}g_{k}(\bm{w}^{(t)}_{k},\bm{x}_{i,k})\right)\right\\}^{L}_{i=1}$
and broadcast it back to all the devices;

15 end

16

17 end for

Algorithm 1 Error-free gradient-based algorithm for vertical FL

###  II-B Over-the-Air Computation based Cloud Radio Access Network

To enable fast aggregation with full device participation in vertical FL, we
shall propose an AirComp based Cloud-RAN to support vertical FL over ultra
dense wireless networks, where multiple RRHs serving as edge servers are
coordinated by a centralized BBU pool serving as a central server, as shown in
Fig. 3. In particular, the central server coordinates $N$ edge servers with
limited-capacity digital fronthaul links to serve $K$ single-antenna devices.
Each edge server is equipped with $M$ antennas. Let $C_{n}$ denote the
fronthaul capacity of the link between edge server $n$ and the central server.
The fronthaul capacities need to satisfy an overall capacity constraint, i.e.,
$\sum_{n=1}^{N}C_{n}\leq C$. We assume perfect synchronization222 For the
AirComp-based system, we can implement synchronization by sharing a reference-
clock among the devices [38], or using the timing advance technique commonly
adopted in 4G Long Term Evolution and 5G New Radio [39]. between among devices
and perfect channel state information (CSI) between the edge servers to the
devices is available to the central server and the devices333The RRHs
estimates the channel state information (CSI) by receiving pilot signals sent
from devices [40].. The notations and parameters used in the system model are
summarized in Table I. In the following, we introduce the communication models
for the uplink and downlink transmission to vertically train the FL model,
respectively.

TABLE I: Important Notations Notation | Definition
---|---
$K$ | Number of edge devices
$N$ | Number of edge servers
$M$ | Number of antennas for each edge server
$C_{n}$ | Fronthaul capacity of edge server $n$
$L$ | Number of time slots (samples)
$\bm{h}^{(t)}_{\mathrm{UL},k,n}$ | Uplink channel between device $k$ and server $n$
$b^{(t)}_{\mathrm{UL},k}$ | Uplink transmit scalar for device $k$ in the $t$-th round
$s^{(t)}_{k}(i)$ | Uplink transmit signal for device $k$ in the $t$-th round
$z^{(t)}_{\mathrm{UL},n}(i)$ | Uplink channel noise for RRH $n$ in the $t$-th round
$P_{\mathrm{UL}}$ | Maximum uplink transmit power
$\bm{q}^{(t)}_{\mathrm{UL},n}(i)$ | Uplink quantization noise for RRH $n$ in the $t$-th round
$\bm{Q}^{(t)}_{\mathrm{UL},n}$ | Covariance matrix of $\bm{q}^{(t)}_{\mathrm{UL},n}(i)$
$n^{(t)}_{\mathrm{UL}}(i)$ | Effective uplink noise in the $t$-th round
$\bm{h}^{(t)}_{\mathrm{DL},k,n}$ | Downlink channel between device $k$ and server $n$
$z^{(t)}_{\mathrm{DL},n}(i)$ | Downlink channel noise for RRH $n$ in the $t$-th round
$P_{\mathrm{DL}}$ | Maximum downlink transmit power
$\bm{q}^{(t)}_{\mathrm{DL},n}(i)$ | Downlink quantization noise for RRH $n$ in the $t$-th round
$\bm{Q}^{(t)}_{\mathrm{DL},n}$ | Covariance matrix of $\bm{q}^{(t)}_{\mathrm{DL},n}(i)$
$n^{(t)}_{\mathrm{DL}}(i)$ | Effective downlink noise in the $t$-th round
Figure 3: AirComp-based Cloud-RAN network for vertical FL.

####  II-B1 Uplink Transmission Model

In the uplink transmission, we assume the devices communicate with the edge
servers over a shared wireless multiple access channel via AirComp. In this
case, the edge servers first receive the AirComp results of the local
prediction results transmitted by the devices, and then forward the aggregated
results to the central server. By exploiting the signal superposition property
of a wireless multiple access channel, the central server directly processes
the aggregated version of analog modulated local information simultaneously
transmitted by the devices, which significantly reduces transmission latency.

The channels between the devices and the edge servers are assumed to be quasi-
static flat-fading [41, 42], where the channel coefficients remain the same
during one communication block. Each block is assumed to contain $L$ time
slots, so that each device is able to transmit an $L$-dimensional signal
vector encoding local prediction results of $L$ samples within one block444
For large data sample size, the local prediction results can be transmitted
within multiple consecutive coherence blocks, which will be left in the future
work.. We assume the devices complete the $t$-th communication round for model
training in the $t$-th transmission block. Then, the received signal at edge
server $n$ during the $i$-th uplink transmission time slot in the $t$-th
communication round is given by

$\displaystyle\bm{y}^{(t)}_{\mathrm{UL},n}(i)=\sum_{k=1}^{K}\bm{h}^{(t)}_{\mathrm{UL},k,n}b^{(t)}_{\mathrm{UL},k}s^{(t)}_{k}(i)+\bm{z}_{\mathrm{UL},n}^{(t)}(i),$
(6)

where $\bm{h}^{(t)}_{\mathrm{UL},k,n}\in\mathbb{C}^{M}$ is the uplink channel
between device $k$ and edge server $n$ in the $t$-th communication round,
$b^{(t)}_{\mathrm{UL},k}$ denotes the transmit scalar,
$s^{(t)}_{k}(i)=g_{k}(\bm{w}^{(t)}_{k},\bm{x}_{i,k})$ denotes the transmit
signal, and
$\bm{z}_{\mathrm{UL},n}^{(t)}(i)\sim\mathcal{CN}(0,\sigma^{2}_{z})$ denotes
the additive white Gaussian noise (AWGN) for edge server $n$. Without loss of
generality, we assume that the transmit signals $\\{s^{(t)}_{k}(i)\\}$ follow
the standard Gaussian distribution555Each neuron in a neural network receives
multiple input signals that vary randomly during both forward and backward
propagation. Due to the central limit theorem, the sum of these random
variables tends to be Gaussian distributed., i.e.,
$s^{(t)}_{k}(i)\sim\mathcal{N}(0,1)$. Hence, the transmit power constraint of
each device is written as

$\displaystyle\mathbb{E}\left[|b^{(t)}_{\mathrm{UL},k}s^{(t)}_{k}(i)|^{2}\right]$
$\displaystyle=|b^{(t)}_{\mathrm{UL},k}|^{2}\mathbb{E}[(s^{(t)}_{k}(i))^{2}]\leq\left|b^{(t)}_{\mathrm{UL},k}\right|^{2}\leq
P_{\mathrm{UL}},\quad\forall k\in[K],$ (7)

where $P_{\mathrm{UL}}$ is the maximum transmit power for each device.

To forward the received signals to the central server through the limited
fronthaul links, edge server $n$ quantizes the received signal
$\bm{y}^{(t)}_{\mathrm{UL},n}(i)$ and then sends the quantized signals to the
central server. In this paper, we use compression-based strategies to transmit
the information between the central server and edge servers via fronthaul
links. By adopting Gaussian quantization test channel666To model the cost for
transmitting high fidelity version of $\bm{y}^{(t)}_{\mathrm{UL},n}(i)$, we
study a theoretical quantization model by viewing Eq. (8) as a test channel
based on the rate-distortion [43]. The results can be extended to the other
quantization schemes, i.e., uniform scalar quantization schemes., the
quantized signal $\tilde{\bm{y}}^{(t)}_{\mathrm{UL},n}(i)$ received at the
central server from edge server $n$ can be modeled as [43]

$\displaystyle\tilde{\bm{y}}^{(t)}_{\mathrm{UL},n}(i)=\bm{y}^{(t)}_{\mathrm{UL},n}(i)+\bm{q}^{(t)}_{\mathrm{UL},n}(i),$
(8)

where
$\bm{q}^{(t)}_{\mathrm{UL},n}(i)\in\mathbb{C}^{M}\sim\mathcal{CN}\left(\bm{0},\bm{Q}^{(t)}_{\mathrm{UL},n}\right)$
denotes the quantization noise and $\bm{Q}^{(t)}_{\mathrm{UL},n}$ is the
diagonal covariance matrix of the quantization noise for edge server $n$. The
quantization noise level directly provides an indication of the accuracy of
$\tilde{\bm{y}}^{(t)}_{\mathrm{UL},n}(i)$. On the other hand, the level of the
quantization noise indicates the amount of fronthaul capacity required for
compression. In general, higher fronthaul capacity results in better
compression resolution and less quantization noise.

Based on the rate-distortion theory, the fronthaul rates of $N$ edge servers
at the $i$-th time slot in the $t$-th communication round should satisfy [44,
Ch. 3]

$\displaystyle\sum_{n=1}^{N}C^{(t)}_{\mathrm{UL},n}=\sum_{n=1}^{N}I\left(\bm{y}^{(t)}_{\mathrm{UL},n}(i);\tilde{\bm{y}}^{(t)}_{\mathrm{UL},n}(i)\right)$
$\displaystyle=$
$\displaystyle\sum_{n=1}^{N}\log\frac{\left|\sum_{k=1}^{K}|b^{(t)}_{\mathrm{UL},k}|^{2}\bm{h}^{(t)}_{\mathrm{UL},k,n}(\bm{h}^{(t)}_{\mathrm{UL},k,n})^{\mathsf{H}}+\sigma^{2}_{z}\bm{I}+\bm{Q}^{(t)}_{\mathrm{UL},n}\right|}{\left|\bm{Q}^{(t)}_{\mathrm{UL},n}\right|}$
$\displaystyle\leq$
$\displaystyle\log\frac{\left|P_{\mathrm{UL}}\sum_{k=1}^{K}\bm{h}^{(t)}_{\mathrm{UL},k}(\bm{h}^{(t)}_{\mathrm{UL},k})^{\mathsf{H}}+\sigma^{2}_{z}\bm{I}+\bm{Q}^{(t)}_{\mathrm{UL}}\right|}{\left|\bm{Q}^{(t)}_{\mathrm{UL}}\right|}\leq
C,$ (9)

where $I(X;Y)$ denotes the mutual information between input $X$ and output
$Y$, $\bm{h}^{(t)}_{\mathrm{UL},k}$ is the concatenated channel vector of
$\\{\bm{h}_{\mathrm{UL},k,1}^{(t)}\\}$, and $\bm{Q}^{(t)}_{\mathrm{UL}}$ is
the uplink covariance matrix defined by
$\bm{Q}^{(t)}_{\mathrm{UL}}=\text{diag}\left\\{\bm{Q}^{(t)}_{\mathrm{UL},1},\ldots,\bm{Q}^{(t)}_{\mathrm{UL},N}\right\\}$.

Stacking the quantized signals $\\{\tilde{\bm{y}}^{(t)}_{\mathrm{UL}}(i)\\}$
transmitted from all edge servers yields

$\displaystyle\tilde{\bm{y}}^{(t)}_{\mathrm{UL}}(i)=\sum_{k=1}^{K}\bm{h}^{(t)}_{\mathrm{UL},k}b^{(t)}_{\mathrm{UL},k}s^{(t)}_{k}(i)+\bm{z}_{\mathrm{UL}}^{(t)}(i)+\bm{q}^{(t)}_{\mathrm{UL}}(i),$

where

$\displaystyle\bm{z}_{\mathrm{UL}}^{(t)}(i)=\left[\left(\bm{z}_{\mathrm{UL},1}^{(t)}(i)\right)^{\mathsf{H}},\ldots,\left(\bm{z}_{\mathrm{UL},N}^{(t)}(i)\right)^{\mathsf{H}}\right]^{\mathsf{H}},\bm{q}^{(t)}_{\mathrm{UL}}(i)=\left[\left(\bm{q}^{(t)}_{\mathrm{UL},1}(i)\right)^{\mathsf{H}},\ldots,\left(\bm{q}^{(t)}_{\mathrm{UL},N}(i)\right)^{\mathsf{H}}\right]^{\mathsf{H}}.$

We design receive beamforming vectors at the central server for
$\tilde{\bm{y}}^{(t)}_{\mathrm{UL}}(i)$, and take the real part to estimate
the aggregation of local predictions
$s^{(t)}(i)=\sum_{k=1}^{K}g_{k}(\bm{x}_{i,k};\bm{w}^{(t)}_{k})$, through the
following procedure

$\displaystyle\hat{s}^{(t)}(i)$
$\displaystyle=\frac{1}{\sqrt{\eta^{(t)}}}\Re((\bm{m}^{(t)})^{\mathsf{H}}\tilde{\bm{y}}^{(t)}_{\mathrm{UL}}(i))=\frac{1}{\sqrt{\eta^{(t)}}}\Re\left((\bm{m}^{(t)})^{\mathsf{H}}\sum_{k=1}^{K}\bm{h}^{(t)}_{\mathrm{UL},k}b^{(t)}_{\mathrm{UL},k}s^{(t)}_{k}(i)\right)+n^{(t)}_{\mathrm{UL}}(i),$
(10)

where
$n^{(t)}_{\mathrm{UL}}(i)=\frac{1}{\sqrt{\eta^{(t)}}}\Re\left((\bm{m}^{(t)})^{\mathsf{H}}\left(\bm{z}_{\mathrm{UL}}^{(t)}(i)+\bm{q}^{(t)}_{\mathrm{UL}}\right)\right)$
is the effective uplink noise, and
$\bm{m}^{(t)}=[(\bm{m}^{(t)}_{1})^{\mathsf{H}},\ldots,(\bm{m}^{(t)}_{N})^{\mathsf{H}}]^{\mathsf{H}}\in\mathbb{C}^{NM}$
is the receive beamforming vector and $\eta^{(t)}$ is the power control
factor. Given $\bm{m}^{(t)}$ and $\eta^{(t)}$, the effective uplink noise is
distributed as
$n^{(t)}_{\mathrm{UL}}(i)\sim\mathcal{N}(0,\sigma^{2}_{\mathrm{UL}}(t))$ with
the variance

$\displaystyle\sigma^{2}_{\mathrm{UL}}(t)=\frac{1}{2\eta^{(t)}}(\bm{m}^{(t)})^{\mathsf{H}}\left(\sigma^{2}_{z}\bm{I}+\bm{Q}^{(t)}_{\mathrm{UL}}\right)\left(\bm{m}^{(t)}\right).$
(11)

####  II-B2 Downlink Transmission Model

After obtaining the estimate $\hat{s}^{(t)}(i)$, the central server computes
$G_{i}\left(\hat{s}^{(t)}(i)\right)$ based on the noisy aggregation
$\hat{s}^{(t)}(i)$ and then broadcasts the result back to the devices. The
central server first computes the beamformed signals to encode
$G_{i}\left(\hat{s}^{(t)}(i)\right)$, and then send them to the edge servers
over the limited fronthaul links in the $t$-th communication round. The edge
servers directly broadcast the beamformed signals to the devices over the
wireless fading channels. The resulting transmit signal at the central server
for edge server $n$ is given as
$\tilde{\bm{r}}^{(t)}_{\mathrm{DL},n}(i)=\bm{u}^{(t)}_{n}G_{i}\left(\hat{s}^{(t)}(i)\right),$
where $\bm{u}^{(t)}_{n}\in\mathbb{C}^{M}$ is the transmit beamforming vector
at edge server $n$.

Similar to the uplink transmission, these signals are then compressed and sent
to the edge servers over the finite-capacity fronthaul link. The compression
noise is modeled as

$\displaystyle\bm{r}^{(t)}_{\mathrm{DL},n}(i)=\tilde{\bm{r}}^{(t)}_{\mathrm{DL},n}(i)+\bm{q}^{(t)}_{\mathrm{DL},n}(i),\quad\forall
i\in[L],$ (12)

where $\bm{r}^{(t)}_{\mathrm{DL},n}(i)$ is the reconstructed signal that edge
server $n$ actually broadcasts to the devices, and
$\bm{q}_{\mathrm{DL},n}^{(t)}(i)\in\mathbb{C}^{M}\sim\mathcal{CN}(\bm{0},\bm{Q}^{(t)}_{\mathrm{DL},n})$
denotes the downlink quantization noise with the covariance matrix
$\bm{Q}^{(t)}_{\mathrm{DL},n}$.

Without loss of generality, we assume that the transmit signal follows the
standard Gaussian distribution, i.e.,
$G_{i}\left(\hat{s}^{(t)}(i)\right)\sim\mathcal{N}(0,1)$, the transmit power
for all edge servers should satisfy

$\displaystyle\sum_{n=1}^{N}\mathbb{E}\left[\|\bm{r}^{(t)}_{\mathrm{DL},n}(i)\|^{2}\right]$
$\displaystyle=\sum_{n=1}^{N}\mathbb{E}\left[\|\bm{u}^{(t)}_{n}G_{i}\left(\hat{s}^{(t)}(i)\right)\|^{2}\right]+\sum_{n=1}^{N}{\mathrm{Tr}}(\bm{Q}^{(t)}_{\mathrm{DL},n})$
$\displaystyle=\|\bm{u}^{(t)}\|^{2}+{\mathrm{Tr}}(\bm{Q}^{(t)}_{\mathrm{DL}})\leq
P_{\mathrm{DL}},$ (13)

where $P_{\mathrm{DL}}$ is the maximum total transmit power for all edge
servers,
$\bm{u}^{(t)}=[(\bm{u}^{(t)}_{1})^{\mathsf{H}},\ldots,(\bm{u}^{(t)}_{N})^{\mathsf{H}}]^{\mathsf{H}}$
is the concatenated transmit beamforming vector, and
$\bm{Q}^{(t)}_{\mathrm{DL}}$ is the downlink covariance matrix defined by
$\bm{Q}^{(t)}_{\mathrm{DL}}=\text{diag}\\{\bm{Q}^{(t)}_{\mathrm{DL},1},\ldots,\bm{Q}^{(t)}_{\mathrm{DL},N}\\}$.
Similarly, the fronthaul capacity required for independent compression should
satisfy

$\displaystyle\sum_{n=1}^{N}C^{(t)}_{\mathrm{DL},n}=\sum_{n=1}^{N}I(\bm{r}^{(t)}_{\mathrm{DL},n}(i);\tilde{\bm{r}}^{(t)}_{\mathrm{DL},n}(i))=\log\frac{\left|\bm{u}^{(t)}(\bm{u}^{(t)})^{\mathsf{H}}+\bm{Q}^{(t)}_{\mathrm{DL}}\right|}{\left|\bm{Q}^{(t)}_{\mathrm{DL}}\right|}\leq
C.$ (14)

The received signal at device $k$ under the compression strategy can be
expressed as

$\displaystyle
y^{(t)}_{\mathrm{DL},k}(i)=\sum_{n=1}^{N}\left(\bm{h}^{(t)}_{\mathrm{DL},k,n}\right)^{\mathsf{H}}\bm{r}^{(t)}_{\mathrm{DL},n}(i)+z^{(t)}_{\mathrm{DL},k}(i)$
$\displaystyle=G_{i}\left(\hat{s}^{(t)}(i)\right)(\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{u}^{(t)}+(\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{q}^{(t)}_{\mathrm{DL}}+z^{(t)}_{\mathrm{DL},k}(i),$

where

$\displaystyle\bm{Q}^{(t)}_{\mathrm{DL}}=\left[(\bm{q}^{(t)}_{\mathrm{DL},1})^{\mathsf{H}},\ldots,(\bm{q}^{(t)}_{\mathrm{DL},N})^{\mathsf{H}}\right]^{\mathsf{H}},\bm{h}^{(t)}_{\mathrm{DL},k}=\left[(\bm{h}^{(t)}_{\mathrm{DL},k,1})^{\mathsf{H}},\ldots,(\bm{h}^{(t)}_{\mathrm{DL},k,N})^{\mathsf{H}}\right]^{\mathsf{H}}.$

After receiving signals, device $k$ estimates
$G_{i}\left(\hat{s}^{(t)}(i)\right)$ by scaling the received signals with the
receive scalar $b^{(t)}_{\mathrm{DL},k}$ and taking the real part, i.e.,

$\displaystyle\hat{G}_{i,k}\left(\hat{s}^{(t)}(i)\right)=\Re\left(b^{(t)}_{\mathrm{DL},k}y^{(t)}_{\mathrm{DL},k}(i)\right)=\Re\left(b^{(t)}_{\mathrm{DL},k}G_{i}\left(\hat{s}^{(t)}(i)\right)(\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{u}^{(t)}\right)+n^{(t)}_{\mathrm{DL},k}(i),$
(15)

where $\hat{G}_{i,k}\left(\hat{s}^{(t)}(i)\right)$ is the estimate of
$G_{i}\left(\hat{s}^{(t)}(i)\right)$ at device $k$, and

$\displaystyle
n^{(t)}_{\mathrm{DL},k}(i)=\Re\left(b^{(t)}_{\mathrm{DL},k}((\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{q}^{(t)}_{\mathrm{DL}}+z^{(t)}_{\mathrm{DL},k}(i))\right)$
(16)

denotes the effective downlink noise for device $k$ distributed as
$n^{(t)}_{\mathrm{DL},k}(i)\sim\mathcal{N}(0,\sigma^{2}_{\mathrm{DL},k}(t))$.
The variance is given by

$\displaystyle\sigma^{2}_{\mathrm{DL},k}(t)=\frac{|b^{(t)}_{\mathrm{DL},k}|^{2}}{2}\left(\sigma_{z}^{2}+(\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{Q}^{(t)}_{\mathrm{DL}}\bm{h}^{(t)}_{\mathrm{DL},k}\right).$
(17)

The proposed framework performs centralized signal processing via AirComp by
exploiting the cooperation among edge servers, thereby achieving reliable and
low-latency model aggregation. Note that we assume all the RRHs are active
during FL training, since vertical FL require all the devices to participate
in FL training. A key challenge of the Cloud-RAN system is that the
quantization errors caused by the limited fronthaul capacity and the
aggregation errors caused by AirComp will severely degrade the learning
performance of vertical FL. In this paper, our goal is to understand the
impact of limited fronthaul capacity and AirComp aggregation error on the
learning performance for vertical FL, followed by designing efficient Cloud-
RAN system optimization framework to improve the vertical FL performance. To
this end, we analyze the convergence of Algorithm 1 under the AirComp based
Cloud-RAN, and characterize the convergence with respect to the transmission
parameters (e.g., transmit/receive beamforming vectors and quantization
covariance matrices) in the next section.

##  III Convergence Analysis

In this section, we characterize the convergence behavior of Algorithm 1 under
the AirComp based Cloud-RAN with limited fronthaul capacity. Specifically, we
first derive the closed-form expression of the vertical FL convergence in
terms of optimality gap between objective values and the optimal value
$F(\bm{w}^{*})$, i.e., $\mathbb{E}[F(\bm{w}^{(T)})]-F(\bm{w}^{*})$, where the
expectation is taken over the communication noise.

###  III-A Zero-forcing Precoding

In this subsection, we first design the transmit and receiver scalars of
devices to compensate the channel fading based on perfect CSI. To achieve
channel inversion, the transmit scalar $b^{(t)}_{\mathrm{UL},k}$ at device $k$
is designed as

$\displaystyle
b^{(t)}_{\mathrm{UL},k}=\frac{\sqrt{\eta^{(t)}}((\bm{m}^{(t)})^{\mathsf{H}}\bm{h}^{(t)}_{\mathrm{UL},k})^{\mathsf{H}}}{|(\bm{m}^{(t)})^{\mathsf{H}}\bm{h}^{(t)}_{\mathrm{UL},k}|^{2}},\quad\forall
k\in[K].$ (18)

Here, we assume that $\bm{m}^{(t)}$ and $\eta^{(t)}$ are known at devices
through the feedback process before transmission. As a result, the received
signal at the central server becomes the desired aggregation of the local
prediction results with noises. Recall (10), the estimate $\hat{s}^{(t)}(i)$
at the central server can be written as

$\displaystyle\hat{s}^{(t)}(i)=\sum_{k=1}^{K}g_{k}(\bm{w}^{(t)}_{k},\bm{x}_{i,k})+n^{(t)}_{\mathrm{UL}}(i)=s^{(t)}(i)+n^{(t)}_{\mathrm{UL}}(i),$
(19)

for all $i\in[L]$. Since the uplink noise $n^{(t)}_{\mathrm{UL}}(i)$ is zero-
mean and independent to $s^{(t)}(i)$, the estimate $\hat{s}^{(t)}(i)$ is an
unbiased estimation of $s^{(t)}(i)$, i.e.,
$\mathbb{E}[\hat{s}^{(t)}(i)]=s^{(t)}(i)$, with the expectation taken over the
uplink noise. Considering the transmit power constraint of each device, i.e.,
$\|b^{(t)}_{\mathrm{UL},k}\|^{2}\leq P_{\mathrm{UL}}$, the power control
factor is given by
$\eta^{(t)}=P_{\mathrm{UL}}\min_{k}\left|(\bm{m}^{(t)})^{\mathsf{H}}\bm{h}^{(t)}_{\mathrm{UL},k}\right|^{2}$.

Similarly, to perfectly compensate the channel fading of downlink
transmission, the receive scalar at device $k$ can be designed as

$\displaystyle
b^{(t)}_{\mathrm{DL},k}=\frac{((\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{u}^{(t)})^{\mathsf{H}}}{\left|(\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{u}^{(t)}\right|^{2}},\quad\forall
k\in[K].$ (20)

According to (15), the estimate $\hat{G}_{i,k}\left(\hat{s}^{(t)}(i)\right)$
at device $k$ can be rewritten as

$\displaystyle\hat{G}_{i,k}\left(\hat{s}^{(t)}(i)\right)=G_{i}\left(\hat{s}^{(t)}(i)\right)+n^{(t)}_{\mathrm{DL},k}(i)=G_{i}\left(s^{(t)}(i)+n^{(t)}_{\mathrm{UL}}(i)\right)+n^{(t)}_{\mathrm{DL},k}(i).$
(21)

Different from (19), the uplink noise is embedded in function $G(\cdot)$. In
order to characterize the impact of the effective noise of uplink and
downlink, we have the following lemma:

######  Lemma 1.

Suppose that the amplitude of uplink noise is small enough, the estimate
$\hat{G}_{i}\left(\hat{s}^{(t)}(i)\right)$ becomes an unbiased estimation of
$G_{i}\left(s^{(t)}(i)\right)$.

To verify Lemma 1, we can write the estimate
$\hat{G}_{i,k}\left(\hat{s}^{(t)}(i)\right)$ as its first-order Taylor
expansion as follows

$\displaystyle\hat{G}_{i,k}\left(\hat{s}^{(t)}(i)\right)=$ $\displaystyle
G_{i}(s^{(t)}(i)+n^{(t)}_{\mathrm{UL}})+n^{(t)}_{\mathrm{DL},k}(i)$
$\displaystyle=$ $\displaystyle
G_{i}\left(s^{(t)}(i)\right)+G^{\prime}_{i}(s^{(t)}(i))n^{(t)}_{\mathrm{UL}}(i)+\mathcal{O}(|n^{(t)}_{\mathrm{UL}}(i)|^{2})+n^{(t)}_{\mathrm{DL},k}(i)$
$\displaystyle\approx$ $\displaystyle
G_{i}(s^{(t)}(i))+G^{\prime}_{i}(s^{(t)}(i))n^{(t)}_{\mathrm{UL}}(i)+n^{(t)}_{\mathrm{DL},k}(i),$
(22)

where $G_{i}^{\prime}(\cdot)$ is the first derivative of $G_{i}(\cdot)$.
Assuming that the amplitude of uplink noise is small, the term
$\mathcal{O}(|n^{(t)}_{\mathrm{UL}}(i)|^{2})$ is neglected, which implies the
last approximated equality in (III-A) [45]. Actually, we can perform
beamforming design and capacity allocation to control the amplitude of the
effective noise of uplink and downlink, which is introduced in Section IV.

######  Remark 1.

Through zero-forcing precoding, we can perfectly compensate the channel fading
and thus minimize the impact caused by the channel distortion, which gives the
closed-form solution with respect to $\eta^{(t)}$. As a sequence, the
communication noises $n^{(t)}_{\mathrm{UL}}(i)$ and $n^{(t)}_{\mathrm{DL}}(i)$
can equivalently represent the impact caused by uplink and downlink
communications, i.e., (19) and (21), respectively. As discussed later in this
section, the communication noises significantly degrade the convergence
performance of the vertical FL algorithm in terms of the optimality gap over
the AirComp-based Cloud-RAN. It is noteworthy that a similar type of
aggregation noise was also considered in [46]. However, they addressed the
aggregation noise from the algorithmic perspective, while this paper aims to
reduce the noise impact through the design of the communication system. In
addition, the effect of aggregation noise on vertical FL algorithms is
different from that on horizontal FL algorithms.

###  III-B Convergence Result

In the following, we first provide the convergence analysis of Algorithm 1
under AirComp based Cloud-RAN with a non-zero optimality gap from the optimal
solution. We first present some assumptions which are widely adopted in FL
literature [47, 48].

######  Assumption 1 ($\alpha$-strongly convexity).

The function $F(\cdot)$ is assumed to be $\alpha$-strongly convex on
$\mathbb{R}^{d}$ with constant $\alpha$, namely, satisfying the following
inequality

$\displaystyle F(\bm{y})\geq F(\bm{x})+\nabla
F(\bm{x})^{\mathsf{T}}(\bm{y}-\bm{x})+\frac{\alpha}{2}\|\bm{y}-\bm{x}\|^{2},$
(23)

for all $\bm{x},\bm{y}\in\mathbb{R}^{d}$.

######  Assumption 2 ($\beta$-smoothness).

The function $F(\cdot)$ is assumed to be $\beta$-smooth on $\mathbb{R}^{d}$
with constant $\beta$, namely, satisfying the following inequality

$\displaystyle F(\bm{y})\leq F(\bm{x})+\nabla
F(\bm{x})^{\mathsf{T}}(\bm{y}-\bm{x})+\frac{\beta}{2}\|\bm{y}-\bm{x}\|^{2},$
(24)

for all $\bm{x},\bm{y}\in\mathbb{R}^{d}$.

Based on Assumptions 1-2, we analyze the performance of Algorithm 1 under the
AirComp based Cloud-RAN communication system proposed in the previous section.
Specifically, the optimality gap from the optimal objective value
$F(\bm{w}^{*})$ with respect to the system parameters
$\\{\bm{m}^{(t)},\bm{u}^{(t)},\bm{Q}^{(t)}_{\mathrm{UL}},\bm{Q}^{(t)}_{\mathrm{DL}}\\}$
is established. The details are summarized in the following theorem.

######  Theorem 1 (Convergence of Algorithm 1 under AirComp based Cloud-RAN).

Consider Assumptions 1-2 with constant learning rate $\mu^{(t)}=1/\beta$, and
also assuming that estimate $\hat{G}_{i}\left(\hat{s}^{(t)}(i)\right)$ is an
unbiased estimation of $G_{i}\left(s^{(t)}(i)\right)$ according to Lemma 1.
The expected optimality gap for Algorithm 1 under AirComp based Cloud-RAN is
upper bounded as

$\displaystyle\mathbb{E}\left[F(\bm{w}^{(T)})-F(\bm{w}^{*})\right]$
$\displaystyle\leq\rho^{T}\mathbb{E}\left[F(\bm{w}^{(0)})-F(\bm{w}^{*})\right]+\frac{3}{2L^{2}\beta}B(T),$
(25)

where $\rho=1-\frac{\alpha}{\beta}$ is the contraction rate and the optimality
gap $B(T)$ in the $T$-th communication round is given by

$\displaystyle
B(T)=\sum_{t=0}^{T-1}\rho^{T-t-1}\sum_{k=1}^{K}(\Phi_{1,k}\sigma^{2}_{\mathrm{UL}}(t)+\Phi_{2,k}\sigma^{2}_{\mathrm{DL},k}(t)),$
(26)

where

$\displaystyle\Phi_{1,k}(t)=\sum_{i=1}^{L}G^{\prime}_{i}\left(s^{(t)}(i)\right)^{2}\left\|\bm{x}_{i,k}\right\|^{2},\Phi_{2,k}(t)=\sum_{i=1}^{L}\left\|\bm{x}_{i,k}\right\|^{2}.$
(27a)

###### Proof.

Please refer to Appendix A. ∎

######  Remark 2.

Comparing Theorem 1 to the convergence results of the vanilla GD algorithm
[49] with error-free communication, we observe that Algorithm 1 under AirComp
based Cloud-RAN achieves the same convergence rate, but with a non-zero
optimality gap $B(T)$ depending on the communication noise variances
$\\{\sigma^{2}_{\mathrm{UL}}(t)\\}^{T}_{t=0}$ and
$\\{\sigma^{2}_{\mathrm{DL},t}\\}^{T}_{t=0}$. Hence, the non-zero gap $B(T)$
reveals the impact of the communication noises on the convergence performance.
In particular, a smaller $B(T)$ leads to a smaller gap between
$F(\bm{w}^{(T)})$ and $F(\bm{w}^{*})$, which motivates us to treat the
optimality gap $B(T)$ as the metric of vertical FL performance over the Cloud-
RAN. Based on (11) and (17), we can control the noise variances by jointly
designing the Cloud-RAN system parameters
$\\{\bm{m}^{(t)},\bm{u}^{(t)},\bm{Q}^{(t)}_{\mathrm{UL}},\bm{Q}^{(t)}_{\mathrm{DL}}\\}$.

##  IV System Optimization

This section presents the system optimization for vertical FL over the
AirComp-based Cloud-RAN. We first formulate the resource allocation problem by
minimizing the optimality gap under the system constraints including the
limited fronthaul capacity and the transmit power constraints. Then, we
decompose the resulting optimization problem into two sub-problems, i.e.,
uplink and downlink optimization problems which are solved by the successive
convex approximation (SCA) and alternate convex search (ACS) approaches.

###  IV-A Problem Formulation

In this section, we optimize the Cloud-RAN assisted vertical FL system by
minimizing the global loss $F(\bm{w})$ based on the convergence results in the
previous section. Specifically, we minimize the optimality gap in terms of
$B(T)$ in Theorem 1 while satisfying the communication constraints (e.g., (7),
(II-B1), (II-B2), (14)). Then, the corresponding optimization problem can be
formulated as

$\begin{array}[]{cl}\underset{\\{\bm{m}^{(t)},\bm{u}^{(t)},\bm{Q}^{(t)}_{\mathrm{UL}},\bm{Q}^{(t)}_{\mathrm{DL}}\\}}{\operatorname{min}}&\sum_{t=0}^{T-1}\rho^{T-t-1}\sum_{k=1}^{K}(\Phi_{1,k}\sigma^{2}_{\mathrm{UL}}(t)+\Phi_{2,k}\sigma^{2}_{\mathrm{DL},k}(t))\\\
\mathrm{s.t.}&\sum_{n=1}^{N}C^{(t)}_{\mathrm{UL},n}\leq
C,\sum_{n=1}^{N}C^{(t)}_{\mathrm{DL},n}\leq C,\forall t\in[T]\\\
&\bm{Q}^{(t)}_{\mathrm{UL}}(i,i)>0,\bm{Q}^{(t)}_{\mathrm{UL}}(i,j)=0,\forall
i\neq j,\forall t\in[T]\\\
&\bm{Q}^{(t)}_{\mathrm{DL}}(i,i)>0,\bm{Q}^{(t)}_{\mathrm{DL}}(i,j)=0,\forall
i\neq j,\forall t\in[T]\\\
&\|\bm{u}^{(t)}\|^{2}+{\mathrm{Tr}}(\bm{Q}^{(t)}_{\mathrm{DL}})\leq
P_{\mathrm{DL}},\forall t\in[T].\end{array}$ (28)

Since the communication constraints are independent during $T$ rounds, we can
decompose the above problem into $T$ independent sub-problems. Additionally,
note that both the objective function and the constraints can be divided into
uplink and downlink transmissions, which are independent of each other. As a
consequence, we solve problem (28) by solving the following two sub-problems
in parallel at each communication round,

$\begin{array}[]{cl}\underset{\bm{m}^{(t)},\bm{Q}^{(t)}_{\mathrm{UL}}}{\operatorname{min}}&\sum_{k=1}^{K}\Phi_{1,k}(t)\sigma^{2}_{\mathrm{UL}}(t)\\\
\mathrm{s.t.}&\sum_{n=1}^{N}C^{(t)}_{\mathrm{UL},n}\leq C,\\\
&\bm{Q}^{(t)}_{\mathrm{UL}}(i,i)>0,\bm{Q}^{(t)}_{\mathrm{UL}}(i,j)=0,\forall
i\neq j,\end{array}$ (29)

and

$\begin{array}[]{cl}\underset{\bm{u}^{(t)},\bm{Q}^{(t)}_{\mathrm{DL}}}{\operatorname{min}}&\sum_{k=1}^{K}\Phi_{2,k}(t)\sigma^{2}_{\mathrm{DL},k}(t)\\\
\mathrm{s.t.}&\sum_{n=1}^{N}C^{(t)}_{\mathrm{DL},n}\leq C,\\\
&\bm{Q}^{(t)}_{\mathrm{DL}}(i,i)>0,\bm{Q}^{(t)}_{\mathrm{DL}}(i,j)=0,\forall
i\neq j,\\\
&\|\bm{u}^{(t)}\|^{2}+{\mathrm{Tr}}(\bm{Q}^{(t)}_{\mathrm{DL}})\leq
P_{\mathrm{DL}}.\end{array}$ (30)

###  IV-B Optimization Framework

In the following, we specify the optimization framework for solving the uplink
and downlink optimization problems, respectively. To simplify the notations,
we omit the time index of the optimized variables in the following as we focus
on the system optimization problem at one communication round.

####  IV-B1 Uplink Optimization

By substituting (II-B1) and (11) into problem (29), we then obtain the
following optimization problem in the uplink transmission:

$\displaystyle\min_{\bm{m},\bm{Q}_{\mathrm{UL}}}$
$\displaystyle\quad\frac{1}{2\eta}\bm{m}^{\mathsf{H}}\left(\sigma^{2}_{z}\bm{I}+\bm{Q}_{\mathrm{UL}}\right)\bm{m}$
(31) s.t.
$\displaystyle\log\frac{\left|P_{\mathrm{UL}}\sum_{k=1}^{K}\bm{h}_{\mathrm{UL},k}(\bm{h}_{\mathrm{UL},k})^{\mathsf{H}}+\sigma^{2}_{z}\bm{I}+\bm{Q}_{\mathrm{UL}}\right|}{\left|\bm{Q}_{\mathrm{UL}}\right|}\leq
C,$
$\displaystyle\bm{Q}_{\mathrm{UL}}(i,i)>0,\bm{Q}_{\mathrm{UL}}(i,j)=0,\quad\forall
i\neq j,$

where
$\eta=P_{\mathrm{UL}}\min_{k}|(\bm{m})^{\mathsf{H}}\bm{h}_{\mathrm{UL},k}|^{2}$.

We propose to adopt the alternating optimization technique to solve the uplink
optimization problem (31). Specifically, we first fix the covariance matrix
$\bm{Q}_{\mathrm{UL}}$ in problem (31) and optimize the receive beamforming
vector $\bm{m}$ by solving the following min-max problem:

$\displaystyle\qquad\min_{\bm{m}}\max_{k}\quad\frac{\bm{m}^{\mathsf{H}}\tilde{\bm{Q}}\bm{m}}{{|\bm{m}^{\mathsf{H}}\bm{h}_{\mathrm{ul,k}}|^{2}}},\quad\text{s.t.}\quad\tilde{\bm{Q}}=\sigma^{2}_{z}\bm{I}+\bm{Q}_{\mathrm{UL}},$
(32)

which is equivalent to

$\min_{\bm{m}}\max_{k}\quad-|\bm{m}^{\mathsf{H}}\bm{h}_{\mathrm{ul,k}}|^{2},\quad\mathrm{s.t.}\quad\bm{m}^{\mathsf{H}}\tilde{\bm{Q}}\bm{m}=1.$
(33)

Problem (33) is a quadratically constrained quadratic programming (QCQP)
problem with a non-convex constraint. To tackle the non-convex constraint in
problem (33), we relax the equality constraint to an inequality constraint,
i.e., $\bm{m}^{\mathsf{H}}\tilde{\bm{Q}}\bm{m}\leq 1$, which yields the
following relaxed problem:

$\min_{\bm{m}}\max_{k}\quad-|\bm{m}^{\mathsf{H}}\bm{h}_{\mathrm{ul,k}}|^{2},\quad\mathrm{s.t.}\quad\bm{m}^{\mathsf{H}}\tilde{\bm{Q}}\bm{m}\leq
1.$ (34)

We adopt the successive convex approximation (SCA) technique to solve problem
(34) with good performance and low computational complexity.

To develop the SCA approach, we convert problem (34) from the complex domain
to the real domain with the following variables:

$\displaystyle\tilde{\bm{m}}$
$\displaystyle=\left[\Re(\bm{m})^{\mathsf{T}}\Im(\bm{m})^{\mathsf{T}}\right]^{\mathsf{T}},$
(35a) $\displaystyle\tilde{\bm{H}}_{k}$
$\displaystyle=-\begin{bmatrix}\Re(\bm{h}_{\mathrm{UL},k}\bm{h}^{\mathsf{H}}_{\mathrm{UL},k})&-\Im(\bm{h}_{\mathrm{UL},k}\bm{h}^{\mathsf{H}}_{\mathrm{UL},k})\\\
\Im(\bm{h}_{\mathrm{UL},k}\bm{h}^{\mathsf{H}}_{\mathrm{UL},k})&\Re(\bm{h}_{\mathrm{UL},k}\bm{h}^{\mathsf{H}}_{\mathrm{UL},k})\end{bmatrix},\forall
k\in[K],$ (35b) $\displaystyle\tilde{\bm{Q}^{\prime}}$
$\displaystyle=\begin{bmatrix}\Re(\tilde{\bm{Q}})&-\Im(\tilde{\bm{Q}})\\\
\Im(\tilde{\bm{Q}})&\Re(\tilde{\bm{Q}})\end{bmatrix}.$ (35c)

Then, problem (34) can be reformulated as follows:

$\min_{\tilde{\bm{m}}}\max_{k}\quad\tilde{\bm{m}}^{\mathsf{T}}\tilde{\bm{H}}_{k}\tilde{\bm{m}},\quad\mathrm{s.t.}\quad\tilde{\bm{m}}^{\mathsf{T}}\tilde{\bm{Q}^{\prime}}\tilde{\bm{m}}\leq
1.$ (36)

For simplicity, we define
$u_{k}(\tilde{\bm{m}})=\tilde{\bm{m}}^{\mathsf{T}}\tilde{\bm{H}}_{k}\tilde{\bm{m}}$.
Since $u_{k}(\tilde{\bm{m}})$ is a concave function, we can upper bound
$u_{k}(\tilde{\bm{m}})$ by the tangent of the $l$-th point $\bm{m}_{(l)}$ as
follows:

$\displaystyle u_{k}(\tilde{\bm{m}})$ $\displaystyle\leq\nabla
u_{k}(\tilde{\bm{m}}_{(l)})^{\mathsf{T}}(\tilde{\bm{m}}-\tilde{\bm{m}}_{(l)})+u_{k}(\tilde{\bm{m}}_{(l)})$
$\displaystyle=\left(2\tilde{\bm{H}}_{k}\tilde{\bm{m}}_{(l)}\right)^{\mathsf{T}}\tilde{\bm{m}}-(\tilde{\bm{m}}_{(l)})^{\mathsf{T}}\tilde{\bm{H}}_{k}\tilde{\bm{m}}_{(l)}.$

The sub-problem for each SCA iteration can be reformulated as follows:

$\displaystyle\min_{\tilde{\bm{m}}}\max_{k}$
$\displaystyle\quad\left(2\tilde{\bm{H}}_{k}\tilde{\bm{m}}_{(l)}\right)^{\mathsf{T}}\tilde{\bm{m}}-(\tilde{\bm{m}}_{(l)})^{\mathsf{T}}\tilde{\bm{H}}_{k}\tilde{\bm{m}}_{(l)}$
(37) $\displaystyle\mathrm{s.t.}$
$\displaystyle\quad\tilde{\bm{m}}^{\mathsf{T}}\tilde{\bm{Q}^{\prime}}\tilde{\bm{m}}\leq
1.$

Starting from an initial point $\tilde{\bm{m}}_{(0)}$, a sequence of solutions
$\\{\tilde{\bm{m}}_{(l)}\\}$ is generated by solving a series of the above
sub-problems.

When fixing the receive beamforming vector $\bm{m}$, problem (31) is reduced
to the following problem:

$\displaystyle\min_{\bm{Q}_{\mathrm{UL}}}$
$\displaystyle\quad\bm{m}^{\mathsf{H}}\left(\sigma^{2}_{z}\bm{I}+\bm{Q}_{\mathrm{UL}}\right)\bm{m}$
(38) $\displaystyle\mathrm{s.t.}$
$\displaystyle\quad\log\frac{\left|P_{\mathrm{UL}}\sum_{k=1}^{K}\bm{h}_{\mathrm{UL},k}(\bm{h}_{\mathrm{UL},k})^{\mathsf{H}}+\sigma^{2}_{z}\bm{I}+\bm{Q}_{\mathrm{UL}}\right|}{\left|\bm{Q}_{\mathrm{UL}}\right|}\leq
C,$
$\displaystyle\quad\bm{Q}_{\mathrm{UL}}(i,i)>0,\bm{Q}_{\mathrm{UL}}(i,j)=0,\quad\forall
i\neq j.$

It is easy to verify that problem (38) is convex with respect to
$\bm{Q}_{\mathrm{UL}}$, which can be solved efficiently with polynomial
complexity by using CVX, a package for specifying and solving convex programs
[50].

The proposed algorithm for uplink optimization is summarized in Algorithm 2.
The convergence results are summarized as follows:

  * •

In the inner loop (Steps 4-7), the values of objective function in problem
(36) achieved by sequence $\\{\tilde{\bm{m}}^{(i)}_{(l)}\\}_{l=0}^{\infty}$
converge monotonically.

  * •

In the outer loop (Steps 2-11), the values of objective function in problem
(33) achieved by sequence
$\\{\tilde{\bm{m}}^{(i)}_{(0)},\bm{Q}_{\mathrm{UL}}^{(i)}\\}_{i=0}^{\infty}$
converge monotonically.

Problem (37) can be equivalently formulated as a convex QCQP problem and then
be solved by using the interior-point method, and the computational complexity
is $\mathcal{O}((MN+K)^{3.5})$. Similarly, the complexity of solving problem
(38) is $\mathcal{O}((MN)^{3.5})$.

Input : Initial points $\tilde{\bm{m}}^{(0)}_{(0)},\bm{Q}^{(0)}_{\mathrm{UL}}$
and threshold $\epsilon$;

1 Set $i=0$;

2 repeat

3 Set $l=0$;

4 repeat

5 Update $\tilde{\bm{m}}^{(i)}_{(l+1)}$ by solving problem (37);

6 $l\leftarrow l+1$;

7

8 until
_$\left\|\tilde{\bm{m}}^{(i)}_{(l+1)}-\tilde{\bm{m}}^{(i)}_{(l)}\right\|
<\epsilon$_;

9 Set $\tilde{\bm{m}}^{(i+1)}_{(0)}\leftarrow\tilde{\bm{m}}^{(i)}_{(l)}$;

10 Update $\bm{Q}_{\mathrm{UL}}^{(i+1)}$ by solving problem (38);

11 Set $i\leftarrow i+1$;

12

13until
_$\left\|\tilde{\bm{m}}^{(i+1)}_{(0)}-\tilde{\bm{m}}^{(i)}_{(0)}\right\|+\left\|\bm{Q}_{\mathrm{UL}}^{(i+1)}-\bm{Q}_{\mathrm{UL}}^{(i)}\right\|
<\epsilon$_;

Algorithm 2 Proposed Algorithm for Uplink Optimization (31)

####  IV-B2 Downlink Optimization

By substituting (17) into problem (30), we solve the following problem:

$\displaystyle\min_{\bm{u},\bm{Q}_{\mathrm{DL}}}$
$\displaystyle\quad\sum_{k=1}^{K}\frac{\Phi_{2,k}}{|(\bm{h}^{(t)}_{\mathrm{DL},k})^{\mathsf{H}}\bm{u}|^{2}}\left(\sigma_{z}^{2}+(\bm{h}_{\mathrm{DL},k})^{\mathsf{H}}\bm{Q}_{\mathrm{DL}}\bm{h}_{\mathrm{DL},k}\right)$
(39) s.t.
$\displaystyle\quad\log\frac{\left|\bm{u}\bm{u}^{\mathsf{H}}+\bm{Q}_{\mathrm{DL}}\right|}{\left|\bm{Q}_{\mathrm{DL}}\right|}\leq
C,$
$\displaystyle\quad\bm{Q}_{\mathrm{DL}}(i,i)>0,\bm{Q}_{\mathrm{DL}}(i,j)=0,\forall
i\neq j,$
$\displaystyle\quad\|\bm{u}\|^{2}+{\mathrm{Tr}}(\bm{Q}_{\mathrm{DL}})\leq
P_{\mathrm{DL}}.$

Although the objective function in problem (39) is convex with respect to each
optimization variable when other optimization variables are fixed, the
capacity constraint is non-convex with respect to $\bm{u}^{(t)}$. Hence, we
approximate problem (39) to a convex optimization by linearizing the limited
fronthaul capacity constraint. By solving the approximated optimization
problem, we successively approximate the optimal solution of the original
problem.

Before reformulating the downlink optimization problem, we first present the
following lemma for convex approximation, which is also adopted in [51].

######  Lemma 2.

For positive definite Hermitian matrices
$\bm{\Omega},\bm{\Sigma}\in\mathbb{C}^{N\times N}$, we have

$\displaystyle\log|\bm{\Omega}|\leq\log|\bm{\Sigma}|+{\mathrm{Tr}}\left(\bm{\Sigma}^{-1}\bm{\Omega}\right)-N,$
(40)

with equality if and only if $\bm{\Omega}=\bm{\Sigma}$.

By applying Lemma 1 to the capacity constraint in problem (39) and setting
$\bm{\Omega}=\bm{u}\bm{u}^{\mathsf{H}}+\bm{Q}_{\mathrm{DL}}$, we can
approximate the capacity constraint in problem (39) with the following convex
constraint:

$\displaystyle\log|\bm{\Sigma}|+{\mathrm{Tr}}\left(\bm{\Sigma}^{-1}\left(\bm{u}\bm{u}^{\mathsf{H}}+\bm{Q}_{\mathrm{DL}}\right)\right)-\log|\bm{Q}_{\mathrm{DL}}|\leq
C+MN.$ (41)

Note that the capacity constraint in problem (39) is always feasible when the
convex constraint (41) is feasible. The two constraints are equivalent when

$\displaystyle\bm{\Sigma}^{*}=\bm{u}\bm{u}^{\mathsf{H}}+\bm{Q}_{\mathrm{DL}}.$
(42)

By using the convex constraint (41) to replace the capacity constraint in
(39), we reformulate the downlink optimization problem to the following
equivalent problem:

$\displaystyle\min_{\bm{u},\bm{Q}_{\mathrm{DL}},\bm{\Sigma}}$
$\displaystyle\quad{\mathrm{Tr}}\left(\bm{Q}_{\mathrm{DL}}\sum_{k=1}^{K}\frac{\Phi_{2,k}}{\bm{u}^{\mathsf{H}}\bm{H}_{\mathrm{DL},k}\bm{u}}\bm{H}_{\mathrm{DL},k}\right)$
(43) s.t.
$\displaystyle\quad\log|\bm{\Sigma}|+{\mathrm{Tr}}\left(\bm{\Sigma}^{-1}\bm{\Omega}\right)-\log|\bm{Q}_{\mathrm{DL}}|\leq
C+MN,$
$\displaystyle\quad\bm{Q}_{\mathrm{DL}}(i,i)>0,\bm{Q}_{\mathrm{DL}}(i,j)=0,\forall
i\neq j,$
$\displaystyle\quad\|\bm{u}\|^{2}+{\mathrm{Tr}}(\bm{Q}_{\mathrm{DL}})\leq
P_{\mathrm{DL}},$

where $\bm{\Omega}=\bm{u}\bm{u}^{\mathsf{H}}+\bm{Q}_{\mathrm{DL}}$ and
$\bm{H}_{\mathrm{DL},k}=\bm{h}_{\mathrm{DL},k}\bm{h}_{\mathrm{DL},k}^{\mathsf{H}}$.

In this paper, we propose to alternatively optimize problem (43). When
$\\{\bm{u},\bm{Q}_{\mathrm{DL}}\\}$ are fixed, the optimal value of
$\bm{\Sigma}$ is given by (42). When $\bm{\Sigma}$ is fixed, the solutions of
$\\{\bm{u},\bm{Q}_{\mathrm{DL}}\\}$ are given by solving the following
problem:

$\displaystyle\min_{\bm{u},\bm{Q}_{\mathrm{DL}}}$
$\displaystyle\quad{\mathrm{Tr}}\left(\bm{Q}_{\mathrm{DL}}\sum_{k=1}^{K}\frac{\Phi_{2,k}}{\bm{u}^{\mathsf{H}}\bm{H}_{\mathrm{DL},k}\bm{u}}\bm{H}_{\mathrm{DL},k}\right)$
(44) s.t.
$\displaystyle\quad{\mathrm{Tr}}\left(\bm{\Sigma}^{-1}\bm{\Omega}\right)-\log|\bm{Q}_{\mathrm{DL}}|\leq
C+MN-\log|\bm{\Sigma}|,$
$\displaystyle\quad\bm{Q}_{\mathrm{DL}}(i,i)>0,\bm{Q}_{\mathrm{DL}}(i,j)=0,\forall
i\neq j,$
$\displaystyle\quad\|\bm{u}\|^{2}+{\mathrm{Tr}}(\bm{Q}_{\mathrm{DL}})\leq
P_{\mathrm{DL}}.$

The objective function in problem (44) is a bi-convex problem for
$\\{\bm{u},\bm{Q}_{\mathrm{DL}}\\}$, e.g., by fixing $\bm{u}$ the function is
convex for $\bm{Q}_{\mathrm{DL}}$, and by fixing $\bm{Q}_{\mathrm{DL}}$ the
function is convex for $\bm{u}$. By exploiting the bi-convex structure of the
problem, we adopt the ACS approach to solve problem (44). Specifically,
problem (44) is transformed into the following two sub-problems:

$\displaystyle\min_{\bm{Q}_{\mathrm{DL}}}$
$\displaystyle\quad{\mathrm{Tr}}\left(\bm{Q}_{\mathrm{DL}}\sum_{k=1}^{K}\frac{\Phi_{2,k}}{\bm{u}^{\mathsf{H}}\bm{H}_{\mathrm{DL},k}\bm{u}}\bm{H}_{\mathrm{DL},k}\right)$
(45) s.t.
$\displaystyle\quad{\mathrm{Tr}}\left(\bm{\Sigma}^{-1}\bm{\Omega}\right)-\log|\bm{Q}_{\mathrm{DL}}|\leq
C+MN-\log|\bm{\Sigma}|,$
$\displaystyle\quad\bm{Q}_{\mathrm{DL}}(i,i)>0,\bm{Q}_{\mathrm{DL}}(i,j)=0,\forall
i\neq j,$
$\displaystyle\quad\|\bm{u}\|^{2}+{\mathrm{Tr}}(\bm{Q}_{\mathrm{DL}})\leq
P_{\mathrm{DL}},$

and

$\displaystyle\min_{\bm{u}}$
$\displaystyle\quad{\mathrm{Tr}}\left(\bm{Q}_{\mathrm{DL}}\sum_{k=1}^{K}\frac{\Phi_{2,k}}{\bm{u}^{\mathsf{H}}\bm{H}_{\mathrm{DL},k}\bm{u}}\bm{H}_{\mathrm{DL},k}\right)$
(46) s.t.
$\displaystyle\quad\|\bm{u}\|^{2}+{\mathrm{Tr}}(\bm{Q}_{\mathrm{DL}})\leq
P_{\mathrm{DL}}.$

The overall algorithm for downlink optimization is summarized in Algorithm 3.
The sub-problems (45) and (46) are convex and can be solved by using the
interior-point method within $\mathcal{O}((MN)^{3.5})$ iterations. The
convergence of ACS has already been intensively studied in [52].

Input : Initial points $\bm{\Sigma}^{(0)}$ and threshold $\epsilon$;

1 Set $i=0$;

2 repeat

3 Update $\bm{Q}_{\mathrm{DL}}^{(i+1)}$ by solving problem (45);

4 Update $\bm{u}^{(i+1)}$ by solving problem (46);

5 Update $\bm{\Sigma}^{(i+1)}$ by using (42);

6 Set $i\leftarrow i+1$;

7

8until
_$\left\|\bm{u}^{(i+1)}-\bm{u}^{(i)}\right\|+\left\|\bm{Q}_{\mathrm{DL}}^{(i+1)}-\bm{Q}_{\mathrm{DL}}^{(i)}\right\|+\left\|\bm{\Sigma}^{(i+1)}-\bm{\Sigma}^{(i)}\right\|
<\epsilon$_;

Algorithm 3 Proposed Algorithm for Downlink Optimization (39)

##  V Numerical Results

In this section, we conduct extensive numerical experiments to evaluate the
performance of the proposed system optimization algorithm for the AirComp
based Cloud-RAN for wireless vertical FL.

###  V-A Simulation Settings

####  V-A1 Vertical FL Setting

We consider a vertical FL setting where $K=49$ devices cooperatively train a
regularized logistic regression model. We simulate the image classification
task on Fashion-MNIST dataset. The local training data are independent and
identically distributed (i.i.d.) drawn from $L=50,000$ images, and the test
dataset contains $10,000$ different images. For each training image of 784
features, each device is assigned with $784/49=16$ non-overlapped features.
The learning rate $\mu^{(t)}$ is set to 0.01. We use the optimality gap to
characterize the convergence of the proposed algorithm. Furthermore, the
learning performance of wireless vertical FL is evaluated by the training loss
for training and the test accuracy for testing.

####  V-A2 Communication Setting

We consider a Cloud-RAN assisted vertical FL system with $N$ edge servers and
$K=49$ devices randomly located in a circle area of radius $500$ m. The edge
servers are randomly distributed in the circular area. Each edge server is
equipped with $M$ antennas, and each device is equipped with single antenna.
The channel coefficients are given by the small-scale fading coefficients
multiplied by the square root of the path loss, i.e., $30.6+36.7\log_{10}(d)$
dB [53], where $d$ (in meter) is the distance between the device and the edge
server. The small-scale fading coefficients follow the standard i.i.d.
Gaussian distribution. The transmit power constraint for all device is set to
$P_{\mathrm{UL}}=23$ dBm, the power spectral density of the background noise
at each edge server is assumed to be $-169$ dBm/Hz [42], and the noise figure
is $7$ dB. All numerical results are averaged over $50$ trials.

###  V-B Convergence of the Proposed Algorithm

Figure 4: Optimality gap versus communication round for different downlink
power constraints.

In this subsection, we examine the convergence performance of the proposed
algorithm for image classification task. We consider Algorithm 1 as
Performance Upper Bound, where the channels are noiseless (i.e.,
$\sigma^{2}_{z}$ = 0) and the fronthaul capacity is infinite (i.e.,
$C=+\infty$), which characterizes the best possible learning performance for
vertical FL.

In Fig. 4, we plot the optimality gap in $1000$ communication rounds for
different downlink power constraints with the capacity constraint $C=200$
Mbps. The number of edge servers and the number of antennas at each edge
server are set to $N=8$ and $M=10$, respectively. It is observed that the
proposed algorithms under two considered downlink power constraints are both
able to linearly converge. This implies that the proposed algorithm can
effectively compensate for the distortion of channels and noises, thereby
speeding up the convergence and reducing optimality gap. Furthermore, the case
with higher downlink power constraint achieves better performance gains
(smaller optimality gap), since the signal-to-noise ratio (SNR) in this case
is higher.

###  V-C Impact of Key System Parameters

In this subsection, we show the performance gain of joint optimization for
wireless and fronthaul resource allocation, and investigate the impact of
various key system parameters. For clarity, we refer the proposed algorithm to
jointly optimize beamforming vectors and covariance matrices as Joint
Optimization. We also consider different schemes for designing beamforming
vectors and covariance matrices as baselines:

  * •

Baseline 1: Uniform quantization with uniform beamforming. The transmit
beamforming $\bm{u}^{(t)}$ and the receive beamforming $\bm{m}^{(t)}$ are
uniformly designed, i.e., $\bm{u}^{(t)}=\kappa\mathbf{1}$ and
$\bm{m}^{(t)}=\sqrt{1/MN}\mathbf{1}$, where the scalar $\kappa$ is chosen to
satisfy the maximum power constraint (II-B2). The central server performs
uniform quantization noise levels across the antennas for all edge servers,
i.e., setting $\bm{Q}_{\mathrm{UL}}^{(t)}=\lambda_{\mathrm{UL}}\mathbf{I}$ and
$\bm{Q}_{\mathrm{DL}}^{(t)}=\lambda_{\mathrm{DL}}\mathbf{I}$, where the
scalars $\lambda_{\mathrm{UL}}$ and $\lambda_{\mathrm{DL}}$ are chosen to
satisfy the fronthaul capacity constraints (II-B1) and (14).

  * •

Baseline 2: Uniform quantization with optimized beamforming. This baseline is
similar to Baseline 1 except that the receive beamforming vector
$\\{\bm{m}^{(t)}\\}$ and the transmit beamforming vector $\\{\bm{u}^{(t)}\\}$
are optimized by Algorithms 2 and 3 with the uniformly designed covariance
matrices $\bm{Q}_{\mathrm{UL}}^{(t)}=\lambda_{\mathrm{UL}}\mathbf{I}$ and
$\bm{Q}_{\mathrm{DL}}^{(t)}=\lambda_{\mathrm{DL}}\mathbf{I}$.

  * •

Baseline 3: Optimized quantization with uniform beamforming. This baseline is
similar to Baseline 1 except that the covariance matrices
$\\{\bm{Q}^{(t)}_{\mathrm{UL}},\bm{Q}^{(t)}_{\mathrm{DL}}\\}$ are optimized by
Algorithms 2 and 3 with the uniformly designed beamforming vectors
$\bm{u}^{(t)}=\kappa\mathbf{1}$ and $\bm{m}^{(t)}=\sqrt{1/MN}\mathbf{1}$.

In order to record the performance of each scheme, each numerical experiment
runs $500$ communication rounds, and the performance of the last round is
recorded. Unless specified otherwise, the number of edge servers is set to
$8$, i.e., $N=8$, the number of antennas at each edge server is set to $2$,
i.e., $M=2$, and the downlink power constraint is set to $P_{\mathrm{DL}}=100$
dBm.

Fig. 5 shows the training loss and test accuracy achieved by different schemes
versus the fronthaul capacity $C$. It is observed that the proposed joint
optimization for wireless and fronthaul resource allocation achieves higher
test accuracy and less training loss over Baselines 1-3, and approaches the
performance upper bound when the fronthaul capacity is greater than or equal
to 180 Mbps. Furthermore, Fig. 5 shows that in the region of small the
fronthaul capacity ($C<200$ Mbps), Baseline 3 performs closer to the proposed
joint optimization algorithm compared with Baselines 1 and 2 using uniform
quantization. We can therefore conclude that the performance gain of joint
optimization is mainly obtained from the quantization optimization at edge
servers. On the other hand, as the fronthaul capacity increases, it is
observed that Baseline 2 outperforms Baseline 3. This is due to the impact of
quantization error on the performance of vertical FL becomes negligible, and
thus the scheme with optimized beamforming can achieve better performance.

Figure 5: Training loss and test accuracy versus fronthaul capacity $C$ with
$N=8$ and $M=2$.

Next, we investigate the impact of the antennas at each edge server on the
performance of vertical FL, where the fronthaul capacity is set to $C=200$
Mbps. Fig. 6 shows that the joint optimization for beamforming and
quantization allocation still outperforms Baselines 1-3, and achieves the
performance upper bound with a few antennas (i.e., $M=10$). In addition, we
note that when $M$ is small (i.e., $M\leq 10$), Baseline 2 has better
performance compared with Baseline 3, which is due to the performance gain of
beamforming optimization dominates in this stage. However, as the number of
antennas $M$ increases, the performance gap between Baseline 2 and Baseline 3
vanishes. The uniform quantization scheme used in Baseline 2 leads to severe
quantization error since the capacity allocated to each antenna is very
limited, which counteracts the performance gain of optimized beamforming.

Figure 6: Training loss and test accuracy versus number of antennas of at each
edge server $M$ with $N=8$ and $C=200$ Mbps.

###  V-D Cloud-RAN versus Massive MIMO

In this subsection, we compare two promising techniques among the antenna
configuration proposed for 5G wireless networks, i.e., Cloud-RAN and Massive
MIMO. The former is a distributed antenna system, while the latter is a
centralized antenna system. For comparison fairness, we assume that there are
a total of $\tilde{M}$ antennas to serve $K$ devices. In particular, for the
massive MIMO network, we assume that there is only one BS equipped with all
$\tilde{M}$ antennas, while for the Cloud-RAN network, we assume that there
are $N$ edge servers each equipped with $\tilde{M}/N$ antennas. In order to
compare the two configurations, we eliminate the limited fronthaul constraint
in the Cloud-RAN network by setting $C=+\infty$, which leads to zero
quantization errors when aggregating. The specific simulation settings are as
follows: $K=20$ devices are randomly located in a circular area of $800$ m.
There are a total number of antennas $\tilde{M}=40$. For the massive MIMO
network, the BS is located in the center of the circular area; while for the
Cloud-RAN network, the edge servers are randomly located in the circular area.
Furthermore, the beamforming vectors $\\{\bm{m}^{(t)},\bm{u}^{(t)}\\}$ for the
considered networks are obtained by Algorithm 2 and Algorithm 3 with
$C=+\infty$. In order to record the performance of each network, each
numerical experiment runs $500$ communication rounds, and the performance of
the last round is recorded.

Fig. 7 shows the learning performance comparison between Cloud-RAN and massive
MIMO with different number of edge servers. It is observed that when there is
only one edge server, the performance of the Cloud-RAN network is slightly
worse than that of the massive MIMO network. However, when the number of edge
servers $N\geq 2$, the performance over the Cloud-RAN network outperforms that
over the massive MIMO network, which is due to that the gain of distributed
antenna system dominates the performance of Cloud-RAN network. Specifically,
by deploying more edge servers in the network, each device can be served by
the nearest edge server with strong channel conditions, which significantly
mitigates communication straggler issues. A larger number of RRHs results in
better channel condition due to dense deployment of RRHs. However, the
capacity allocated on each server will be small, resulting in a large
quantization error on the learning performance, vice versa. This leads to a
trade-off between the channel conditions and quantization error in multi-
antenna Cloud-RAN, which allows an optimization of the number of RRHs for
maximizing the learning performance. As the total capacity increases, we
should allocate more edge servers to exploit better channels for devices to
improve performance.

Figure 7: Training loss and test accuracy versus number of RRHs $N$.

##  VI Conclusion

In this paper, we proposed a novel framework based on AirComp and Cloud-RAN to
support communication-efficient vertical FL over wireless networks. To reveal
the impact of limited fronthaul capacity and AirComp aggregation error on the
learning performance, we characterized the convergence behavior in terms of
resource allocation. Based on the derived optimization gap for the wireless
vertical FL algorithm, we further established a system optimization framework
to minimize optimization gap for the wireless vertical FL algorithm under
various system constraints. To solve this problem, we proposed to decompose
the original system optimization problem into two sub-problems, i.e., uplink
and downlink optimization problems, which can be efficiently solved by
successive convex approximation and alternate convex search approaches.
Extensive numerical results have shown that the proposed system optimization
schemes can achieve high learning performance and the effectiveness of the
proposed Cloud-RAN network architecture for vertical FL was also verified.

##  Appendix A Proof of Theorem 1

According to (4) and Lemma 1, the partial gradient estimate
$\hat{\nabla}_{k}F(\bm{w}^{(t)})$ is given by

$\displaystyle\hat{\nabla}_{k}F(\bm{w}^{(t)})=$
$\displaystyle\frac{1}{L}\sum_{i=1}^{L}\hat{G}_{i}\left(\hat{s}^{(t)}(i)\right)\nabla
g_{k,i}(\bm{w}_{k})+\lambda\nabla r_{k}(\bm{w})$ $\displaystyle=$
$\displaystyle\frac{1}{L}\sum_{i=1}^{L}\left(G_{i}\left(s^{(t)}(i)\right)+G^{\prime}_{i}\left(s^{(t)}(i)\right)n^{(t)}_{\mathrm{UL}}(i)+n^{(t)}_{\mathrm{DL},k}(i)\right)\nabla
g_{k,i}(\bm{w}_{k})+\lambda\nabla r_{k}(\bm{w})$ $\displaystyle=$
$\displaystyle\nabla_{k}F(\bm{w}^{(t)})+\bm{e}^{(t)}_{k},$ (47)

where $\bm{e}^{(t)}_{k}$ denotes the effective communication noise for device
$k$ in the $t$-th communication round that is given by

$\displaystyle\bm{e}^{(t)}_{k}=\frac{1}{L}\sum_{i=1}^{L}\left(G^{\prime}_{i}\left(s^{(t)}(i)\right)n^{(t)}_{\mathrm{UL}}(i)+n^{(t)}_{\mathrm{DL},k}(i)\right)\nabla
g_{k,i}(\bm{w}_{k}).$ (48)

In addition, the expected norm of $\bm{e}^{(t)}$ over the noises
$n^{(t)}_{\mathrm{UL}}(i)$ and $n^{(t)}_{\mathrm{DL},k}(i)$ can be bounded as
follows

$\displaystyle\mathbb{E}[\|\bm{e}^{(t)}_{k}\|^{2}]=$
$\displaystyle\frac{1}{L^{2}}\mathbb{E}\left[\left\|\sum_{i=1}^{L}\left(G^{\prime}_{i}\left(s^{(t)}(i)\right)n^{(t)}_{\mathrm{UL}}(i)+n^{(t)}_{\mathrm{DL},k}(i)\right)\nabla
g_{k,i}(\bm{w}_{k})\right\|^{2}\right]$
$\displaystyle\overset{(\mathrm{i})}{=}$
$\displaystyle\frac{1}{L^{2}}\sum_{i=1}^{L}\mathbb{E}\left[\left(G^{\prime}_{i}\left(s^{(t)}(i)\right)n^{(t)}_{\mathrm{UL}}(i)+n^{(t)}_{\mathrm{DL},k}(i)\right)^{2}\right]\left\|\nabla
g_{k,i}(\bm{w}_{k})\right\|^{2}$ $\displaystyle=$
$\displaystyle\frac{1}{L^{2}}(\Phi_{1,k}\sigma^{2}_{\mathrm{UL}}(t)+\Phi_{2,k}\sigma^{2}_{\mathrm{DL},k}(t)),$
(49)

where

$\displaystyle\Phi_{1,k}=\sum_{i=1}^{L}G^{\prime}_{i}\left(s^{(t)}(i)\right)^{2}\left\|\nabla
g_{k,i}(\bm{w}_{k})\right\|^{2},\quad\Phi_{2,k}=\sum_{i=1}^{L}\left\|\nabla
g_{k,i}(\bm{w}_{k})\right\|^{2}.$ (50)

Let
$\bm{w}^{(t+1)}=[(\bm{w}^{(t+1)}_{1})^{\mathsf{T}},\ldots,(\bm{w}^{(t+1)}_{k})^{\mathsf{T}}]^{\mathsf{T}}$,
and use the definition of $\bm{w}^{(t+1)}_{k}$ to obtain

$\displaystyle\bm{w}^{(t+1)}=\left[\left(\bm{w}^{(t)}_{1}-\mu^{(t)}\hat{\nabla}_{1}F(\bm{w}^{(t)})\right)^{\mathsf{T}},\ldots,\left(\bm{w}^{(t)}_{K}-\mu^{(t)}\hat{\nabla}_{k}F(\bm{w}^{(t)})\right)^{\mathsf{T}}\right]^{\mathsf{T}}=\bm{w}^{(t)}-\mu^{(t)}\hat{\nabla}F(\bm{w}^{(t)}),$
(51)

where

$\displaystyle\hat{\nabla}F(\bm{w}^{(t)})=\underbrace{\left[\left(\nabla_{1}F(\bm{w}^{(t)})\right)^{\mathsf{T}},\ldots,\left(\nabla_{K}F(\bm{w}^{(t)})\right)^{\mathsf{T}}\right]^{\mathsf{T}}}_{=\nabla
F(\bm{w}^{(t)})}+\underbrace{\left[\left(\bm{e}^{(t)}_{1}\right)^{\mathsf{T}},\ldots,\left(\bm{e}^{(t)}_{K}\right)^{\mathsf{T}}\right]^{\mathsf{T}}}_{=\bm{e}^{(t)}}.$
(52)

Here, the equality (i) comes from (A).

Under Assumption 2, we have

$\displaystyle F(\bm{w}^{(t+1)})$ $\displaystyle\leq F(\bm{w}^{(t)})+\nabla
F(\bm{w}^{(t)})^{\mathsf{T}}(\bm{w}^{(t+1)}-\bm{w}^{(t)})+\frac{\beta}{2}\|\bm{w}^{(t+1)}-\bm{w}^{(t)}\|^{2}$
$\displaystyle=F(\bm{w}^{(t)})-\mu^{(t)}\nabla
F(\bm{w}^{(t)})^{\mathsf{T}}\hat{\nabla}F(\bm{w}^{(t)})+\frac{(\mu^{(t)})^{2}\beta}{2}\left\|\hat{\nabla}F(\bm{w}^{(t)})\right\|^{2}$
$\displaystyle=F(\bm{w}^{(t)})-\frac{1}{2\beta}\left\|\nabla
F(\bm{w}^{(t)})\right\|^{2}+\frac{1}{2\beta}\|\bm{e}^{(t)}\|^{2},$ (53)

where the learning rate $\mu^{(t)}=1/\beta$ implies the last equality.

In order to derive a lower bound for $\left\|\nabla
F(\bm{w}^{(t)})\right\|^{2}$, we use Assumption 1 and minimize both sides of
(23) to obtain

$\displaystyle F(\bm{w}^{*})\geq F(\bm{w})-\frac{1}{\alpha}\nabla
F(\bm{w})^{\mathsf{T}}\nabla F(\bm{w})+\frac{1}{2\alpha}\nabla
F(\bm{w})^{\mathsf{T}}\nabla
F(\bm{w})=F(\bm{w})-\frac{1}{2\alpha}\left\|\nabla F(\bm{w})\right\|^{2}.$
(54)

Rearranging the terms, we have

$\displaystyle\left\|\nabla F(\bm{w}^{(t)})\right\|^{2}\geq
2\alpha\left(F(\bm{w}^{(t)})-F(\bm{w}^{*})\right).$ (55)

Subtract $F(\bm{w}^{*})$ from both sides and substitute (55) into (A) to get

$\displaystyle F(\bm{w}^{(t+1)})-F(\bm{w}^{*})\leq$ $\displaystyle
F(\bm{w}^{(t)})-F(\bm{w}^{*})-\frac{\alpha}{\beta}\left(F(\bm{w}^{(t)})-F(\bm{w}^{*})\right)+\frac{1}{2\beta}\|\bm{e}^{(t)}\|^{2}$
$\displaystyle=$
$\displaystyle\left(1-\frac{\alpha}{\beta}\right)\left(F(\bm{w}^{(t)})-F(\bm{w}^{*})\right)+\frac{1}{2\beta}\|\bm{e}^{(t)}\|^{2}$
(56)

Applying this recursively, we obtain

$\displaystyle F(\bm{w}^{T})-F(\bm{w}^{*})\leq$
$\displaystyle\rho^{T}\left(F(\bm{w}^{0})-F(\bm{w}^{*})\right)+\frac{1}{2\beta}\sum_{t=0}^{T-1}\rho^{T-t-1}\|\bm{e}^{(t)}\|^{2},$
(57)

where $\rho=1-\alpha/\beta$ is the contraction rate.

Taking expectations of (57) over the noises $n^{(t)}_{\mathrm{UL}}(i)$ and
$\\{n^{(t)}_{\mathrm{DL},k}(i)\\}_{k=1}^{K}$, we obtain

$\displaystyle\mathbb{E}\left[F(\bm{w}^{T})-F(\bm{w}^{*})\right]\leq$
$\displaystyle\rho^{T}\mathbb{E}\left[F(\bm{w}^{0})-F(\bm{w}^{*})\right]+\frac{1}{2\beta}\sum_{t=0}^{T-1}\rho^{T-t-1}\mathbb{E}[\|\bm{e}^{(t)}\|^{2}]$
$\displaystyle\leq$
$\displaystyle\rho^{T}\mathbb{E}\left[F(\bm{w}^{0})-F(\bm{w}^{*})\right]+\frac{1}{2L^{2}\beta}B(T),$
(58)

where the optimality gap $B(T)$ is given by

$\displaystyle
B(T)=\sum_{t=0}^{T-1}\rho^{T-t-1}\sum_{k=1}^{K}(\Phi_{1,k}\sigma^{2}_{\mathrm{UL}}(t)+\Phi_{2,k}\sigma^{2}_{\mathrm{DL},k}(t)),$
(59)

which completes the proof of Theorem 1.

## References

  * [1] K. B. Letaief, Y. Shi, J. Lu, and J. Lu, “Edge artificial intelligence for 6G: Vision, enabling technologies, and applications,” _IEEE J. Sel. Areas Commun._ , vol. 40, no. 1, pp. 5–36, Jan. 2022.
  * [2] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” _ACM Trans. Intell. Syst. Technol._ , vol. 10, no. 2, pp. 12:1–12:19, Mar. 2019.
  * [3] W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated learning in mobile edge networks: A comprehensive survey,” _IEEE Commun. Surv. Tutorials_ , vol. 22, no. 3, pp. 2031–2063, Thirdquarter 2020.
  * [4] K. Yang, T. Fan, T. Chen, Y. Shi, and Q. Yang, “A quasi-newton method based vertical federated learning framework for logistic regression,” in _Proc. Neural Inf. Process. Syst. (NeurIPS)_ , Dec. 2019.
  * [5] L. Liang, H. Ye, and G. Y. Li, “Toward intelligent vehicular networks: A machine learning framework,” _IEEE Internet Things J._ , vol. 6, no. 1, pp. 124–135, Feb. 2019.
  * [6] P. Liu, G. Zhu, W. Jiang, W. Luo, J. Xu, and S. Cui, “Vertical federated edge learning with distributed integrated sensing and communication,” _IEEE Commun. Lett._ , vol. 26, no. 9, pp. 2091–2095, 2022.
  * [7] K. Wei, J. Li, C. Ma, M. Ding, S. Wei, F. Wu, G. Chen, and T. Ranbaduge, “Vertical federated learning: Challenges, methodologies and experiments,” _CoRR_ , vol. abs/2202.04309, 2022. [Online]. Available: https://arxiv.org/abs/2202.04309
  * [8] M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor, “Distributed learning in wireless networks: Recent progress and future challenges,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 12, pp. 3579–3605, Dec. 2021.
  * [9] K. B. Letaief, W. Chen, Y. Shi, J. Zhang, and Y. A. Zhang, “The roadmap to 6G: AI empowered wireless networks,” _IEEE Commun. Mag._ , vol. 57, no. 8, pp. 84–90, Aug. 2019.
  * [10] K. Yang, T. Jiang, Y. Shi, and Z. Ding, “Federated learning via over-the-air computation,” _IEEE Trans. Wireless Commun._ , vol. 19, no. 3, pp. 2022–2035, Mar. 2020.
  * [11] Y. Zou, Z. Wang, X. Chen, H. Zhou, and Y. Zhou, “Knowledge-guided learning for transceiver design in over-the-air federated learning,” _IEEE Trans. Wireless Commun.,_Early Access__ , pp. 1–1, 2022, doi: 10.1109/TWC.2022.3192550.
  * [12] Y. Shi, Y. Zhou, D. Wen, Y. Wu, C. Jiang, and K. Letaief, “Task-oriented communications for 6g: Vision, principles, and technologies,” _IEEE Wiress Commun. Mag._ , to appear.
  * [13] G. Zhu, J. Xu, K. Huang, and S. Cui, “Over-the-air computing for wireless data aggregation in massive IoT,” _IEEE Wireless Commun._ , vol. 28, no. 4, pp. 57–65, Aug. 2021.
  * [14] Y. Shi, K. Yang, T. Jiang, J. Zhang, and K. B. Letaief, “Communication-efficient edge AI: algorithms and systems,” _IEEE Commun. Surv. Tutorials_ , vol. 22, no. 4, pp. 2167–2191, Fourthquarter 2020.
  * [15] M. M. Amiri and D. Gündüz, “Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air,” _IEEE Trans. Signal Process._ , vol. 68, pp. 2155–2169, Mar. 2020.
  * [16] D. Fan, X. Yuan, and Y. A. Zhang, “Temporal-structure-assisted gradient aggregation for over-the-air federated edge learning,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 12, pp. 3757–3771, Dec. 2021.
  * [17] C. Zhong, H. Yang, and X. Yuan, “Over-the-air multi-task federated learning over MIMO interference channel,” _CoRR_ , vol. abs/2112.13603, 2021. [Online]. Available: https://arxiv.org/abs/2112.13603
  * [18] N. Zhang and M. Tao, “Gradient statistics aware power control for over-the-air federated learning,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 8, pp. 5115–5128, Aug. 2021.
  * [19] H. Guo, A. Liu, and V. K. N. Lau, “Analog gradient aggregation for federated learning over wireless networks: Customized design and convergence analysis,” _IEEE Internet Things J._ , vol. 8, no. 1, pp. 197–210, Aug. 2021.
  * [20] S. Xia, J. Zhu, Y. Yang, Y. Zhou, Y. Shi, and W. Chen, “Fast convergence algorithm for analog federated learning,” in _Proc. IEEE Int. Conf. Commun. (ICC)_ , Jun. 2021, pp. 1–6.
  * [21] G. Zhu, Y. Wang, and K. Huang, “Broadband analog aggregation for low-latency federated edge learning,” _IEEE Trans. Wireless Commun._ , vol. 19, no. 1, pp. 491–506, Jan. 2020.
  * [22] H. Liu, X. Yuan, and Y. A. Zhang, “Reconfigurable intelligent surface enabled federated learning: A unified communication-learning design approach,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 11, pp. 7595–7609, Nov. 2021\.
  * [23] H. Yang, J. Zhao, Z. Xiong, K. Lam, S. Sun, and L. Xiao, “Privacy-preserving federated learning for UAV-enabled networks: Learning-based joint scheduling and resource management,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 10, pp. 3144–3159, Oct. 2021.
  * [24] Z. Wang, J. Qiu, Y. Zhou, Y. Shi, L. Fu, W. Chen, and K. B. Letaief, “Federated learning via intelligent reflecting surface,” _IEEE Trans. Wireless Commun._ , vol. 21, no. 2, pp. 808–822, Feb. 2022.
  * [25] H. Liu, Z. Lin, X. Yuan, and Y. A. Zhang, “Reconfigurable intelligent surface empowered over-the-air federated edge learning,” _IEEE Wireless Commun.,_Early Access__ , 2022, doi: 10.1109/MWC.007.2200101.
  * [26] K. Yang, Y. Shi, Y. Zhou, Z. Yang, L. Fu, and W. Chen, “Federated machine learning for intelligent iot via reconfigurable intelligent surface,” _IEEE Netw._ , vol. 34, no. 5, pp. 16–22, Sept. 2020.
  * [27] T. T. Vu, D. T. Ngo, N. H. Tran, H. Q. Ngo, M. N. Dao, and R. H. Middleton, “Cell-free massive MIMO for wireless federated learning,” _IEEE Trans. Wireless Commun._ , vol. 19, no. 10, pp. 6377–6392, Oct. 2020.
  * [28] Z. Lin, H. Liu, and Y. A. Zhang, “Relay-assisted cooperative federated learning,” _IEEE Trans. Wireless Commun.,_Early Access__ , 2022, doi: 10.1109/TWC.2022.3155596.
  * [29] Z. Wang, Y. Zhou, Y. Shi, and W. Zhuang, “Interference management for over-the-air federated learning in multi-cell wireless networks,” _IEEE J. Sel. Areas Commun._ , vol. 40, no. 8, pp. 2361–2377, Aug. 2022\.
  * [30] F. P. Lin, S. Hosseinalipour, S. S. Azam, C. G. Brinton, and N. Michelusi, “Semi-decentralized federated learning with cooperative D2D local model aggregations,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 12, pp. 3851–3869, Dec. 2021.
  * [31] W. Y. B. Lim, J. S. Ng, Z. Xiong, D. Niyato, C. Miao, and D. I. Kim, “Dynamic edge association and resource allocation in self-organizing hierarchical federated learning networks,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 12, pp. 3640–3653, Dec. 2021.
  * [32] W. Zhang, D. Yang, W. Wu, H. Peng, N. Zhang, H. Zhang, and X. Shen, “Optimizing federated learning in distributed industrial IoT: A multi-agent approach,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 12, pp. 3688–3703, Dec. 2021.
  * [33] J. Chen, Z. Chang, X. Guo, R. Li, Z. Han, and T. Hämäläinen, “Resource allocation and computation offloading for multi-access edge computing with fronthaul and backhaul constraints,” _IEEE Trans. Veh. Technol._ , vol. 70, no. 8, pp. 8037–8049, 2021.
  * [34] S. Park, S. Jeong, J. Na, O. Simeone, and S. Shamai, “Collaborative cloud and edge mobile computing in C-RAN systems with minimal end-to-end latency,” _IEEE Trans. Signal Inf. Process. over Networks_ , vol. 7, pp. 259–274, 2021.
  * [35] Y. Shi, J. Zhang, K. B. Letaief, B. Bai, and W. Chen, “Large-scale convex optimization for ultra-dense Cloud-RAN,” _IEEE Wireless Commun._ , vol. 22, no. 3, pp. 84–91, Jun. 2015.
  * [36] M. Tao, E. Chen, H. Zhou, and W. Yu, “Content-centric sparse multicast beamforming for cache-enabled cloud RAN,” _IEEE Trans. Wireless Commun._ , vol. 15, no. 9, pp. 6118–6131, Sept. 2016.
  * [37] L. Su and V. K. N. Lau, “Hierarchical federated learning for hybrid data partitioning across multitype sensors,” _IEEE Internet Things J._ , vol. 8, no. 13, pp. 10 922–10 939, Jul. 2021.
  * [38] O. Abari, H. Rahul, D. Katabi, and M. Pant, “Airshare: Distributed coherent transmission made seamless,” in _Proc. IEEE Conf. Comput. Commun. (INFOCOM),_ , Apr. 2015, pp. 1742–1750.
  * [39] A. Mahmood, M. I. Ashraf, M. Gidlund, J. Torsner, and J. Sachs, “Time synchronization in 5g wireless edge: Requirements and solutions for critical-mtc,” _IEEE Commun. Mag._ , vol. 57, no. 12, pp. 45–51, Dec. 2019\.
  * [40] R. G. Stephen and R. Zhang, “Uplink channel estimation and data transmission in millimeter-wave CRAN with lens antenna arrays,” _IEEE Trans. Commun._ , vol. 66, no. 12, pp. 6542–6555, Nov. 2018.
  * [41] Y. Zhou and W. Yu, “Fronthaul compression and transmit beamforming optimization for multi-antenna uplink C-RAN,” _IEEE Trans. Signal Process._ , vol. 64, no. 16, pp. 4138–4151, Aug. 2016.
  * [42] L. Liu and R. Zhang, “Optimized uplink transmission in multi-antenna C-RAN with spatial compression and forward,” _IEEE Trans. Signal Process._ , vol. 63, no. 19, pp. 5083–5095, Oct. 2015.
  * [43] S. Park, O. Simeone, O. Sahin, and S. S. (Shitz), “Fronthaul compression for cloud radio access networks: Signal processing advances inspired by network information theory,” _IEEE Signal Process. Mag._ , vol. 31, no. 6, pp. 69–79, Nov. 2014.
  * [44] A. E. Gamal and Y. Kim, _Network Information Theory_. Cambridge University Press, 2011.
  * [45] C. M. Bishop, “Training with noise is equivalent to tikhonov regularization,” _Neural Comput._ , vol. 7, no. 1, pp. 108–116, 1995.
  * [46] F. Ang, L. Chen, N. Zhao, Y. Chen, W. Wang, and F. R. Yu, “Robust federated learning with noisy communication,” _IEEE Trans. Commun._ , vol. 68, no. 6, pp. 3452–3464, 2020.
  * [47] X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” in _Proc. Int. Conf. Learn. Represent. (ICLR)_ , 2020.
  * [48] M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 1, pp. 269–283, Jan. 2021\.
  * [49] M. P. Friedlander and M. Schmidt, “Hybrid deterministic-stochastic methods for data fitting,” _SIAM J. Sci. Comput._ , vol. 34, no. 3, 2012.
  * [50] M. Grant and S. Boyd, “CVX: Matlab software for disciplined convex programming, version 2.1,” http://cvxr.com/cvx, Mar. 2014.
  * [51] Y. Zhou and W. Yu, “Optimized backhaul compression for uplink cloud radio access network,” _IEEE J. Sel. Areas Commun._ , vol. 32, no. 6, pp. 1295–1307, Jun. 2014.
  * [52] J. Gorski, F. Pfeuffer, and K. Klamroth, “Biconvex sets and optimization with biconvex functions: a survey and extensions,” _Math. Methods Oper. Res._ , vol. 66, no. 3, pp. 373–407, 2007.
  * [53] E. Access, “Further advancements for e-utra physical layer aspects,” _3GPP Technical Specification TR_ , vol. 36, p. V2, 2010.