# A Resource-Adaptive Approach for Federated Learning under Resource-
Constrained Environments

Ruirui Zhang
Shandong University
Shandong, China
<EMAIL_ADDRESS>
&Xingze Wu
Shandong University
Shandong, China
<EMAIL_ADDRESS>
Yifei Zou
Shandong University
Shandong, China
<EMAIL_ADDRESS>
&Zhenzhen Xie
Shandong University
Shandong, China
<EMAIL_ADDRESS>
&Peng Li
the University of Aizu
Japan
<EMAIL_ADDRESS>
&Xiuzhen Cheng
Shandong University
Shandong, China
<EMAIL_ADDRESS>
&Dongxiao Yu
Shandong University
Shandong, China
<EMAIL_ADDRESS>

###### Abstract

The paper studies a fundamental federated learning (FL) problem involving
multiple clients with heterogeneous constrained resources. Compared with the
numerous training parameters, the computing and communication resources of
clients are insufficient for fast local training and real-time knowledge
sharing. Besides, training on clients with heterogeneous resources may result
in the straggler problem. To address these issues, we propose Fed-RAA: a
Resource-Adaptive Asynchronous Federated learning algorithm. Different from
vanilla FL methods, where all parameters are trained by each participating
client regardless of resource diversity, Fed-RAA adaptively allocates
fragments of the global model to clients based on their computing and
communication capabilities. Each client then individually trains its assigned
model fragment and asynchronously uploads the updated result. Theoretical
analysis confirms the convergence of our approach. Additionally, we design an
online greedy-based algorithm for fragment allocation in Fed-RAA, achieving
fairness comparable to an offline strategy. We present numerical results on
MNIST, CIFAR-10, and CIFAR-100, along with necessary comparisons and ablation
studies, demonstrating the advantages of our work. To the best of our
knowledge, this paper represents the first resource-adaptive asynchronous
method for fragment-based FL with guaranteed theoretical convergence.

##  1 Introduction

Trillions of parameters enable the large-scale machine learning model to find
the complex relationship hidden behind the inputs and outputs, and achieve a
high performance on addressing the complex tasks in computer vision (CV) and
natural language processing (NLP) [1, 2, 3]. Whereas, it also poses great
challenges in the training process, especially for those devices with limited
computing and data resources [4, 5]. As an efficient distributed learning
framework proposed by Google [6], federated learning (FL) can effectively
leverage the scattered resources among multiple clients in the learning
process and maintain user privacy, as only the model information is exchanged,
but not the data itself [7, 8]. Thus, it has been regarded as a classical
approach for those resource-constrained clients to cooperatively obtain a
comprehensive large-scale ML model [6].

In the past decades, a series of relevant works have been proposed to train a
large-scale ML model among resource-constrained clients under FL framework [9,
10, 11, 12, 13, 5, 14, 15], which generally fall into two categories. The
works in the first branch [12, 13, 15] focus on clients with limited
computational resources. By splitting the whole ML model into multiple
submodels, each client only trains one of the submodels locally according to
its computation ability. In the model aggregation step, those submodels will
be aggregated into a global one and disseminated to the clients for the next
round of training. In the second branch, the works mainly address the
communication efficiency problem in FL since clients may have limited
bandwidth. The techniques such as model compression, fine-tuning, and model
scaling have been adopted in [10], [5, 14], and [11] separately, which
mitigate the communication overhead for clients in local model updating and
global model disseminating steps. Additionally, in [9], the computing and
network resources constraints are simultaneously considered in their model
splitting process. Whereas, all of the works mentioned above are executed
under the synchronous aggregation mode, i.e. the model aggregation in each
training round starts only when all the clients have their local models
updated. Thus, the efficiency of the FL process will be delayed by the slowest
client, which is often the client with the smallest computing and
communication resources. The summary of the previous works is listed in Table
1. To the best of our knowledge, few of the works consider the FL problem
under the constrained computing and communication resources simultaneously
under the asynchronous setting.

Table 1: Comparison of existing works Related works | Computing Resource | Communication Resource | Aggregation mode
---|---|---|---
[12], [13], [15] | Limited | Sufficient | Synchronous
[14], [16] | Sufficient | Limited | Synchronous
[9], [17] | Limited | Limited | Synchronous
Our work | Limited | Limited | Asynchronous
Figure 1: Overview of Fed-RAA

To address the federated learning problem among multiple clients with
heterogeneous constrained computing and communication resources, we propose a
Resource-Adaptive Asynchronous Federated learning algorithm, named Fed-RAA.
Since the local training, downloading, and uploading of an ML model with
trillions of parameters are time-consuming or even not affordable for clients
with limited resources, our method only designates a tailored submodel from
the full model to each client for local training. For each client, once it
receives a submodel from the parameter server, it trains the submodel based on
its dataset. Considering that the synchronous FL process will be delayed by
the slowest client in each training round, the asynchronous model aggregation
is adopted in our Fed-RAA, i.e. once a client uploads its submodel, the
parameters of the submodel will immediately updated to the global model.
Theoretical analysis is presented to prove the convergence of our algorithm.
Additionally, an online scheduling algorithm is designed as the designation
strategy in our Fed-RAA, which considers the fairness of clients with respect
to their computing and communication abilities and the fairness of parameters
with respect to their opportunities to be updated. We also prove that our
online algorithm has optimal performance on the submodel designation.

In summary, this paper introduces a Resource-Adaptive Asynchronous Federated
learning algorithm, named Fed-RAA, to address the asynchronous FL problem
among clients with heterogeneous constrained resources. Theoretical analysis
is given to prove the convergence of Fed-RAA. Numerical results with necessary
comparisons and ablation studies are presented, showing that our Fed-RAA has
promising performance in Accuracy and Time cost. Additionally, an online
scheduling algorithm is designed to designate the submodels to clients for
local training, with a fairness guarantee.

##  2 Related Work

The relevant works considering the FL problem of resource constrained clients
can be divided into two categories. The works in the first category address
the challenge of constrained computing resources. Yuan et al. in [15]
decompose the original neural network into a collection of disjoint
subnetworks, which are locally trained by clients and iteratively reassembled
on the parameter server. SplitFed [12] splits the global model into the
client-side model and server-side model. Each client just needs to train the
client-side model, thus reducing its computing cost. In [13], the submodel
structures of clients are assigned according to their local computing
resources. The works in the second category focus on the issue of limited
communication resources in FL. Yang et al. [16] introduce online model
compression technique, to store model parameters in a compressed format and
decompress them only when necessary. Moreover, DGC [18] combines gradient
sparsity and multiple optimization technologies to reduce communication costs
with comparable accuracy. There are also some works considering both computing
and communication resource constraints. For instance, HeteroFL [9] decomposes
the large model into multiple subnetworks, which are trained by the clients
synchronously in the FL process. Simulations are presented in [9] to validate
the performance of HeteroFL. Moreover, PruneFL [17] adapts the model size
during FL to reduce both communication and computation overhead and minimize
the overall training time.

However, the aforementioned works are built upon a synchronous FL framework.
In this setup, the efficiency of FL in each training round is constrained by
the speed of the slowest client, often referred to as the straggler. This
synchronization may result in prolonged durations for each global training
round, further leading to inefficient training. Different from the existing
works, this paper investigates the asynchronous FL problem among multiple
clients with heterogeneous constrained computing and communication resources,
which is the novelty of our problem.

##  3 Preliminary

We consider an FL system that consists of $n$ clients and a parameter server
$S$, with reliable communication connections between the server and each
client. Each client $n$ ($n=1,\ldots,N$) has its own local dataset $D^{n}$
with size $\|D^{n}\|$ and limited computing resource $cmp_{n}$. All clients
train a global model in FL, the parameter set of which is denoted by $\theta$.

The existing works [9, 19, 15] have shown how to divide a global model into
multiple submodels, each of which contains a fraction of the parameter
$\theta$. Thus, this paper no longer discusses the model division but starts
from the assumption that the global model has already been divided into $M$
submodels, denoted by $\\{\theta^{1},\theta^{2},\ldots,\theta^{M}\\}$.

The goal is to optimize the empirical risk minimization like traditional
federated learning setting:

$\min_{\theta\in\mathbb{R}^{d}}{f(\theta)}:=\frac{1}{N}\sum_{n=1}^{N}{f_{n}(\theta)}$
(1)

where $f_{n}(\theta):=\mathbb{E}_{\xi_{n}\sim D_{n}}[f_{n}(\theta,\xi_{n})]$
is the local loss function of client $n$ on dataset $D_{n}$.

##  4 Methodology

Figure 2: The pipeline of Fed-RAA

In this section, we design the resource-adaptive asynchronous FL algorithm,
named Fed-RAA, in which the submodels
$\\{\theta^{1},\theta^{2},\ldots,\theta^{M}\\}$ are locally trained by the
clients and asynchronously aggregated.

###  4.1 Algorithm Design

The asynchronous training process takes $t$ global epochs. In each epoch
$\tau$, the server assigns idle client $n$ submodel $\theta^{j}_{q(j)}$ for
training, where $j=h_{\tau}(n)$ is the index assigned to worker $n$ in epoch
$\tau$ and $q(j)$ denotes the number of update for submodel $\theta^{j}$. Then
in the $t^{th}$ epoch, the server receives a locally trained model
$\theta_{new}^{j}$ from any arbitrary worker, and updates the corresponding
submodel $\theta^{j}$ by weighted averaging:
$\theta^{j}_{q(j)}\leftarrow(1-\alpha_{t})\theta^{j}_{q(j)-1}+\alpha_{t}*\theta_{new}^{j}$,
where $\alpha\in(0,1)$ is the mixing hyperparameter.

On an arbitrary device client $n$, after receiving a submodel
$\theta^{j}_{q(j)}$ (potentially stale) from the server, we locally solve the
following regularized optimization problem using SGD for several iterations:

$\min_{\theta^{j}\in\mathbb{R}^{d}}{\mathbb{E}_{z^{n}\sim
D^{n}}{f_{n}(\theta^{j};z^{n})+\frac{\rho}{2}{\|\theta^{j}-\theta^{j}_{q(j)}\|}}}$
(2)

For convenience, we define
$g_{\theta^{\prime}}(\theta;z)=f(\theta;z)+\frac{\rho}{2}{\|\theta-\theta^{\prime}\|^{2}}$.

The server and clients conduct updates asynchronously, i.e., the server will
immediately update the global model whenever it receives a locally trained
submodel. Moreover, the communications between the server and clients are non-
blocking. Thus, the server and clients can update the submodels at any time
without synchronization, which is favorable when the devices have
heterogeneous resources.

The detailed algorithm is shown in Algorithm 1. The submodel parameter
$\theta^{j}_{\tau,i}$ is updated in the $i$th local iteration after receiving
$\theta^{j}_{\tau}$ on device client $n$. The data sample $z^{n}_{\tau,i}$ is
randomly drawn from local dataset $D^{n}$ and $I^{n}_{t}$ is the number of
local iterations after receiving $\theta^{j}_{\tau}$ on device client $n$.
Moreover, $\gamma$ is the learning rate, and $t$ is the total number of global
epochs.

As the pipeline is shown in Figure. 2, for each client, the process begins
with the server assigning an appropriate submodel via the scheduler module
(Step 1). Upon receiving the submodel, each client initiates local training
(Step 2). Once a client completes their local training, they upload their
submodel to the server. The server then integrates the submodel into the
global model using an updater (Step 3), thereby completing that round of
updates. This continues for each client until the server signifies a halt.

Input: Local Data $D^{n}$ of client $n$ on $N$ clients.

Progress  _Server $(\alpha\in(0,1))$_:

Initialize
$\theta_{0}=\\{\theta_{0}^{1},\theta_{0}^{2},\ldots,\theta_{0}^{M}\\}$,
$\alpha_{t}\leftarrow\alpha$, $\forall t\in[T]$;

Run Scheduler() thread and Updater() thread asynchronously in parallel;

Thread  _Scheduler()_ :

Periodically trigger training tasks on some idle workers, and send
$\theta_{q(h_{t}(n))}^{h_{t}(n)}$ to worker $n$ with time stamp;

Where $h_{t}(n)$ is the index assigned to worker $n$ in round $t$;

Thread  _Updater()_ :

for  _epoch $t\in[T]$_ do

Receive the pair $(\theta_{new}^{h_{\tau}(n)},\tau)$ from any worker;

Optional: $\alpha_{t}\leftarrow\alpha*s(t-\tau)$, where $s(\cdot)$ is a
function of the staleness;

$q(h_{\tau}(n))\leftarrow q(h_{\tau}(n))+1$;

$\theta^{h_{\tau}(n)}_{q(h_{\tau}(n))}\leftarrow(1-\alpha_{t})\theta^{h_{\tau}(n)}_{q(h_{\tau}(n))-1}+\alpha_{t}*\theta_{new}^{h_{\tau}(n)}$;

end for

Progress  _Worker()_ :

for  _$n\in[N]$ in parallel_ do

if  _triggered by the scheduler_ then

Receive the pair of the global model and its time stamp
$(\theta_{q(h_{t}(n))}^{h_{t}(n)},t)$ from the server;

$j\leftarrow h_{t}(n)$ which is the index of model region;

$\tau\leftarrow q(j)$;

$\theta_{\tau,0}^{j}\leftarrow\theta_{q(j)}^{j}$;

Define
$g_{\theta_{q(j)}^{j}}(\theta^{j};z)=f_{n}(\theta^{j};z)+\frac{\rho}{2}\|\theta^{j}-\theta^{j}_{q(j)}\|^{2}$,
where $\rho>\mu$;

for  _local iteration $i\in[I^{n}_{t}]$_ do

Randomly sample $z^{n}_{\tau,i}\sim D^{n}$;

Update $\theta_{\tau,i}^{j}\leftarrow\theta_{\tau,i-1}^{j}-\gamma\nabla
g_{\theta_{q(j)}^{j}}(\theta_{\tau,i-1}^{j};z^{n}_{\tau,i})$;

end for

Push $(\theta_{\tau,I^{n}_{t}}^{j},t)$ to the server;

end if

end for

Algorithm 1 Fed-RAA

###  4.2 Convergence Analysis

######  Definition 1.

(Smoothness) A differentiable function $f$ is $L$-smooth if for $\forall
x,y,f(y)-f(x)\leq\langle\nabla f(x),y-x\rangle+\frac{L}{2}\|y-x\|^{2}$, where
$L>0$.

######  Definition 2.

(Weak convexity) A differentiable function $f$ is $\mu$-weakly convex if the
function $g$ with $g(\theta)=f(\theta)+\frac{\mu}{2}\|\theta\|^{2}$ is convex,
where $\mu\geq 0$. $f$ is convex if $\mu=0$, and non-convex if $\mu>0$.

######  Theorem 1.

Assume that every function $F_{n}$ is $L$-smooth and $\mu$-weakly convex, and
each worker executes at least $I_{\text{min}}$ and at most $I_{\text{max}}$
local updates before pushing models to the server. We assume bounded delay
$t-\tau\leq K$. The imbalance ratio of local updates is
$\delta=\frac{I_{\text{max}}}{I_{\text{min}}}$. Furthermore, we assume that
for $\forall\theta\in\mathbb{R}^{d},n\in[N]$, and $\forall z\sim D^{n}$, we
have $\|\nabla f(\theta;z)\|^{2}\leq V_{1}$ and $\|\nabla
g_{\theta^{\prime}}(\theta;z)\|^{2}\leq V_{2},\forall\theta^{\prime}$. For any
small constant $\epsilon>0$, taking $\rho$ large enough such that $\rho>\mu$
and
$-(1+2\rho+\epsilon)V_{2}+\rho^{2}\|\theta_{\tau,i-1}^{j}-\theta_{\tau}^{j}\|^{2}-\frac{\rho}{2}\|\theta_{\tau,i-1}^{j}-\theta_{\tau}^{j}\|^{2}\geq
0,\forall\theta_{\tau,i-1}^{j},\theta_{\tau}^{j},j\in[M],$ and
$\gamma<\frac{1}{L}$, after $Q$ updates for every submodel $\theta^{j}$,
Algorithm (1) converges to a critical point:

$\min_{q=0}^{T-1}\mathbb{E}\|\nabla
F(\theta_{q})\|^{2}\leq\frac{\mathbb{E}[F(\theta_{0})-F(\theta_{T})]}{\alpha\gamma\epsilon
TI_{\text{min}}}+\mathcal{O}\left(\frac{\gamma I_{\text{max}}^{3}+\alpha
KI_{\text{max}}}{\epsilon
I_{\text{min}}}\right)+\mathcal{O}\left(\frac{\alpha^{2}\gamma
K^{2}I_{\text{max}}^{2}+\gamma K^{2}I_{\text{max}}^{2}}{\epsilon
I_{\text{min}}}\right).$

Where $T=Q*M$

Theorem1 shows the convergence rate of algorithm Fed-RAA by giving the upper
bound on the gradient of all clients for all trained submodels. It is worth
noting that in the convergence result, we ensure that all submodels can be
updated uniformly, that is, $T=Q*M$.

######  Remark 1.

Impact of the training delay bound $K$. Our convergence result shows that the
smaller the upper bound $K$ of staleness $(t-\tau)$ would lead to a faster
convergence rate and better performance in our federated learning process.

######  Corollary 1.

Let all assumptions hold. Using $\delta=\frac{I_{max}}{I_{min}}$, and taking
$\alpha=\frac{1}{\sqrt{I_{min}}}$, $\gamma=\frac{1}{\sqrt{T}}$,
$I_{min}=T^{\frac{1}{5}}$, we have the convergence rate as follows:

$\displaystyle\min_{q=0}^{T-1}\mathbb{E}[\|\nabla F(\theta_{q})\|^{2}]$
$\displaystyle\leq\mathcal{O}\left(\frac{1}{T^{\frac{3}{5}}}\right)+\mathcal{O}\left(\frac{\delta^{3}}{T^{\frac{1}{10}}}\right)+\mathcal{O}\left(\frac{K\delta}{T^{\frac{1}{10}}}\right)+\mathcal{O}\left(\frac{K^{2}\delta^{2}}{T^{\frac{1}{2}}}\right)+\mathcal{O}\left(\frac{K^{2}\delta^{2}}{T^{\frac{3}{10}}}\right).$

Corollary 1 indicates that the term
$\mathcal{O}\left(\frac{\delta^{3}+K\delta}{T^{\frac{1}{10}}}+\frac{K^{2}\delta^{2}}{T^{\frac{3}{10}}}\right)$
will dominate the convergence rate.

The detailed theoretical proof of Theorem1 is provided in Supplement.

##  5 Online Submodel Assignment Algorithm

As mentioned in Algorithm 1, we rely on a submodel assignment algorithm to
ensure an upper bound on the training delay, thereby facilitating the
convergence analysis for the whole model. The implementation of this algorithm
can be either learning-based or scheduling-based. In this section, we propose
an online Greedy-based Resource Adaptive submodel Assignment algorithm, Gre-
RAA, which is scheduling-based and proved as optimal as an offline solution on
the upper bound of training delay $K$.

###  5.1 Algorithm Design for Submodel Assignment

Considering the varying communication resources between clients, we define
$com^{up}_{i,j}$ as the number of global epochs for client $i$ to update
submodel $\theta^{j}$ and $com^{d}_{i,j}$ as the download time. Assuming each
client updates the same local iterations when receiving one region submodel,
we define
$c_{i,j}=\frac{\|\theta^{j}\|\|D^{n}\|}{cmp_{i}}+com^{up}_{i,j}+com^{d}_{i,j}$
as the global epochs consumed by client $i$ to train submodel $\theta^{j}$.

Fairness. Considering the limited heterogeneous computational and
communication resources of users, we ensure the fairness of clients with
respect to their computing and communication abilities and the fairness of
parameters with respect to their opportunities to be updated. Specifically, we
strive to ensure that (1) if two clients receive their submodels at a similar
time, the completion time for both should also be close, meaning that each
client receives an adaptive submodel tailored to their resources, and (2)
during the training process, each submodel has roughly equal opportunities or
frequency of being updated.

To ensure such fairness, when a submodel is assigned by the server to the
client $n$ at the epoch $t$, it must satisfy the following requirements: (1)
the training delay of it for client $n$ does not exceed the upper bound $K$
and (2) it has been updated the fewest number of times so far.

If multiple submodels meet the two aforementioned conditions, we randomly
select one for assignment. Additionally, to ensure that every user has at
least one submodel that satisfies the conditions for training, we can adjust
the value of the hyperparameter $K$. Specifically, we assume that the chosen
$K$ is sufficiently large so that the set of submodels meeting the first
condition is not empty.

The details are shown in Algorithm 2, which can be found in the Appendix.

###  5.2 Algorithm Analysis

######  Theorem 2.

Given the asynchronous federated learning system proposed in Alg. 1, there is
an online scheduling-based model assignment algorithm, shown as Algorithm 2,
reaching the same upper bound of training delay as that conducted by an
optimal offline model assignment strategy.

The detailed theoretical proof of Theorem2 is provided in Supplement.

##  6 Experiment

To evaluate the performance of Fed-RAA and compare it with other algorithms,
our experiments selected three common benchmark datasets in the field of image
classification: CIFAR-10 [20], CIFAR-100 [20], and MNIST [21]. In simulation,
the convergence rate of the global model and its maximum achievable accuracy
are observed. More details are provided in the Appendix.

###  6.1 Comparisons with Baselines

To substantiate the superior performance of Fed-RAA, we benchmark it against
state-of-the-art methods in FL, namely, FedAvg [22], FedSync [23], FedProx
[24], FedPrun [19], SplitFed [12], and RAM-Fed [13]. FedAvg represents a
classical synchronous full-model FL framework, while FedSync exemplifies the
classical asynchronous approach. FedProx, SplitFed, and RAM-Fed focus
primarily on the local computational limitations of users; in FedProx, users
train the full model, whereas, in SplitFed and RAM-Fed, users employ a
submodel for synchronous FL. FedPrun considers both computational and
communication resources but operates within a synchronous FL framework.

Table 2: Comparison of the time and epochs required to achieve specified
accuracy.
<MNIST,0.93>, <CIFAR-10,0.45>, and <CIFAR-100,0.19> are the tuple of <Dataset, Accuracy>. Time (s) | <MNIST,0.93> | <CIFAR-10,0.45> | <CIFAR-100,0.19>
---|---|---|---
Fed-Avg | 406.35 | 390.83 | 435.51
Fed-Sync | 242.46 | 253.11 | 1123.25
Fed-Prox | 866.40 | 870.44 | 910.98
Fed-Prun | 776.82 | 975.48 | 3158.55
Split-FL | 840.15 | 769.88 | 2181.59
Ram-Fed | 808.96 | 783.06 | 918.11
Fed-RAA | 91.06 | 89.85 | 101.86

We conducted experiments across various datasets with our algorithm and
baseline algorithms and found that our approach converges more rapidly. As
demonstrated in Table 2 and Fig. 3, on the MNIST dataset, our algorithm
achieved a 0.93% accuracy in just 91.06 seconds. This faster convergence can
be attributed to more frequent updates within shorter intervals due to its
asynchronous nature. Similar effects were observed on other datasets as well.
This highlights our algorithm’s superiority in rapidly updating and converging
compared to traditional synchronous methods and other asynchronous or submodel
training approaches.

(a) MNIST

(b) CIFAR-10

(c) CIFAR-100

Figure 3: Detailed results of comparison among different algorithms

In the Appendix, we have included an additional table detailing accuracy and
time costs. The findings are similar: Our Fed-RAA algorithm exhibits
exceptional time efficiency across multiple datasets while sustaining
competitive accuracy, making it an ideal option for federated learning
scenarios that require swift model updates and deployments. It substantially
surpasses traditional synchronous and several asynchronous methods,
underscoring its utility in contexts where time efficiency and rapid
convergence are paramount.

###  6.2 Ablation Study

In addressing the model assignment problem within Fed-RAA, we designed a
scheduler-based online algorithm, named Gre-RAA, as outlined in Algorithm 2.
Although it has been theoretically affirmed that Gre-RAA achieves the same
upper bound of training delay as that achieved by the optimal offline model
assignment strategy, it remains imperative to conduct ablation studies to
validate the contributions of Gre-RAA toward enhancing both the convergence
speed and the maximum accuracy within Fed-RAA. To this end, we instituted two
variance algorithms, encompassing: (1) Random Fed-RAA: random assignment, and
(2) Minimum Priority Fed-RAA(MP-RAA): consistently allocating the model
segment that has undergone the least amount of training. Furthermore, to
validate the effectiveness of the asynchronous aggregation approach utilized
by Fed-RAA within this framework, we designed a variance algorithm for an
ablation study: Sync Fed-RAA. This variant employs a synchronous aggregation
method for FL, with all other aspects remaining identical to Fed-RAA. We
evaluate the impact of these variance algorithms on Fed-RAA’s performance
under the conditions of three distinct dataset scenarios. The accuracy results
from this comparative analysis are depicted in Figure 4 and Table 3.

Table 3: Comparison of algorithms on MNIST, CIFAR-10, and CIFAR-100 datasets | MNIST | CIFAR-10 | CIFAR-100
---|---|---|---
Baseline | Accuracy (%) | Time (s) | Accuracy (%) | Time (s) | Accuracy (%) | Time (s)
Gre-RAA | 93.95 | 69.46 | 47.43 | 68.58 | 20.74 | 75.96
Random | 93.76 | 78.54 | 47.43 | 84.63 | 18.77 | 60.10
MP-RAA | 93.88 | 80.73 | 47.39 | 84.77 | 20.75 | 78.28
Sync | 93.94 | 162.71 | 47.84 | 162.82 | 21.46 | 179.44

In our ablation study, we observed distinct performance characteristics among
the variants: Random Fed-RAA, Minimum Priority Fed-RAA (MP-RAA), and Sync Fed-
RAA. The results demonstrate Fed-RAA’s efficiency on the MNIST dataset with a
high accuracy of 93.95% and a low training time of 23.15 seconds. However, on
the more complex CIFAR-10 and CIFAR-100 datasets, although Sync Fed-RAA
occasionally achieved higher accuracies (47.84% and 21.46%, respectively), it
consistently demanded significantly more training time, indicating a trade-off
between accuracy and computational efficiency. The Random and MP-RAA variants
showed competitive performance across all datasets, suggesting their potential
utility in scenarios where a balance between accuracy and training time is
critical.

(a) MNIST

(b) CIFAR-10

(c) CIFAR-100

Figure 4: The comparison of ablation methods on different datasets

This analysis elucidates the nuanced trade-offs involved in different Fed-RAA
configurations. While Sync Fed-RAA may offer accuracy advantages, its higher
time consumption could limit its applicability in resource-constrained
settings. On the other hand, Random and MP-RAA present more balanced
solutions, potentially optimizing federated learning outcomes in diverse
environments. These insights will guide future developments in federated
learning strategies, aiming to adaptively balance accuracy, efficiency, and
resource usage across varying conditions.

###  6.3 Scalability of Our Approach

In our experiments, we conducted a detailed investigation into the performance
of the Federated Random Assignment Algorithm (Fed-RAA), with particular
emphasis on the impact of two critical hyperparameters: the proportion of
extremely low-capability users, denoted as $\beta$, and the total number of
sub-models post-segmentation, denoted as $M$, on the time-accuracy trade-off
inherent to the algorithm. For ease of description, in this experiment, we
categorize the user groups into two based on their capability levels:
extremely low-capability users (capability of 1) and low-capability users
(capability of 3). The proportion of extremely low-capability users, denoted
by $\beta$, illustrates the heterogeneity in user capabilities; for instance,
$\beta=0.1$ indicates that 10% of the users are extremely low-capability,
while the remaining 90% are low-capability users.

(a) MNIST

(b) CIFAR-10

(c) CIFAR-100

Figure 5: The influence of $\beta$

The analysis in Figure 5 shows that increasing the proportion of extremely
low-capability clients ($\beta$) severely affects the training time and
accuracy of federated learning models, particularly in complex datasets like
CIFAR-100 where training time increases considerably. As $\beta$ increases,
training slows significantly due to the higher number of extremely low-
capability clients, and accuracy declines more noticeably in complex datasets.
This emphasizes the substantial impact of client capability distribution on
federated learning performance, especially under challenging conditions. These
findings indicate the necessity for adaptive strategies and careful management
of client heterogeneity to improve both efficiency and effectiveness in
federated learning across diverse datasets.

(a) MNIST

(b) CIFAR-10

(c) CIFAR-100

Figure 6: The influence of $M$

The results from Figure 6 highlight the influence of increasing submodel
numbers $M$ on training times, particularly underlining scalability issues in
complex datasets like CIFAR-100, where more submodels significantly extend
training durations without enhancing accuracy. This suggests a diminishing
return on training efficiency as the number of submodels increases, especially
in complex settings. While the MNIST and CIFAR-10 datasets demonstrate only
moderate time increases, the accuracy gains do not justify the additional
submodels, indicating a crucial trade-off in asynchronous federated learning
like Fed-RAA between submodel count and training efficiency. These findings
emphasize the need to optimize the balance between model fragmentation and
overall training efficacy, especially in environments with varied data
complexities.

Moreover, we provide 12 detailed result figures for comparison of $\beta$ and
$M$, which show similar conclusions as above.

##  7 Conclusion

This paper studies the federated learning problem among clients with
heterogeneous limited resources and proposes a resource-adaptive asynchronous
FL algorithm, named Fed-RAA, as the solution. Fed-RAA uniquely assigns model
fragments based on individual client capabilities, enabling asynchronous
updates and reducing delays from slower clients. This method is validated
through robust theoretical analysis and empirical tests on the MNIST,
CIFAR-10, and CIFAR-100 datasets, demonstrating substantial gains in learning
speed and accuracy. We also employ an online greedy-based algorithm to
optimize resource allocation and enhance fairness in local model training.
Future work will aim to refine this algorithm and extend its application in
federated learning, with a focus on adapting it for specific large model
architecture, such as large language model with Mixture of Experts or Lora.

## References

  * [1] Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, XiVictoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, Giri Anantharaman, Xian Li, Shuohui Chen, Halil Akin, Mandeep Baines, Louis Martin, Xing Zhou, PunitSingh Koura, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Mona Diab, Zornitsa Kozareva, and Ves Stoyanov.  Efficient large scale language modeling with mixtures of experts.
  * Fedus et al. [2021] William Fedus, Barret Zoph, and Noam Shazeer.  Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.  _arXiv: Learning,arXiv: Learning_ , Jan 2021.
  * Lepikhin et al. [2020] Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen.  Gshard: Scaling giant models with conditional computation and automatic sharding.  _Cornell University - arXiv,Cornell University - arXiv_ , Jun 2020.
  * Liu and Zhao [2024] Chang Liu and Jun Zhao.  Resource allocation in large language model integrated 6g vehicular networks.  _CoRR_ , abs/2403.19016, 2024.  doi: 10.48550/ARXIV.2403.19016.  URL https://doi.org/10.48550/arXiv.2403.19016.
  * Woisetschläger et al. [2023] Herbert Woisetschläger, Alexander Isenko, Shiqiang Wang, Ruben Mayer, and Hans-Arno Jacobsen.  Federated fine-tuning of llms on the very edge: The good, the bad, the ugly, 2023.
  * McMahan et al. [2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas.  Communication-efficient learning of deep networks from decentralized data.  In _AISTATS 2017_ , volume 54 of _Proceedings of Machine Learning Research_ , pages 1273–1282. PMLR, 2017.
  * Nguyen et al. [2024] Quan Nguyen, Hieu H. Pham, Kok-Seng Wong, Phi Le Nguyen, Truong Thao Nguyen, and Minh N. Do.  Feddct: Federated learning of large convolutional neural networks on resource-constrained devices using divide and collaborative training.  _IEEE Trans. Netw. Serv. Manag._ , 21(1):418–436, 2024.  doi: 10.1109/TNSM.2023.3314066.  URL https://doi.org/10.1109/TNSM.2023.3314066.
  * Qin et al. [2023] Jiangcheng Qin, Xueyuan Zhang, Baisong Liu, and Jiangbo Qian.  A split-federated learning and edge-cloud based efficient and privacy-preserving large-scale item recommendation model.  _J. Cloud Comput._ , 12(1):57, 2023.  doi: 10.1186/S13677-023-00435-5.  URL https://doi.org/10.1186/s13677-023-00435-5.
  * Diao et al. [2020] Enmao Diao, Jie Ding, and Vahid Tarokh.  Heterofl: Computation and communication efficient federated learning for heterogeneous clients.  _arXiv: Learning,arXiv: Learning_ , Oct 2020.
  * Guo et al. [2021] Peizhen Guo, Bo Hu, and Wenjun Hu.  Mistify: Automating DNN model porting for On-Device inference at the edge.  In _18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21)_ , pages 705–719. USENIX Association, April 2021.  ISBN 978-1-939133-21-2.  URL https://www.usenix.org/conference/nsdi21/presentation/guo.
  * Ogden et al. [2021] Samuel S. Ogden, Guin R. Gilman, Robert J. Walls, and Tian Guo.  Many models at the edge: Scaling deep inference via model-level caching.  In _2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)_ , pages 51–60, 2021.  doi: 10.1109/ACSOS52086.2021.00027.
  * Thapa et al. [2022] Chandra Thapa, Mahawaga Arachchige Pathum Chamikara, Seyit Camtepe, and Lichao Sun.  Splitfed: When federated learning meets split learning.  In _Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022_ , pages 8485–8493. AAAI Press, 2022.  doi: 10.1609/AAAI.V36I8.20825.  URL https://doi.org/10.1609/aaai.v36i8.20825.
  * Wang et al. [2023] Yangyang Wang, Xiao Zhang, Mingyi Li, Tian Lan, Huashan Chen, Hui Xiong, Xiuzhen Cheng, and Dongxiao Yu.  Theoretical convergence guaranteed resource-adaptive federated learning with mixed heterogeneity.  In Ambuj K. Singh, Yizhou Sun, Leman Akoglu, Dimitrios Gunopulos, Xifeng Yan, Ravi Kumar, Fatma Ozcan, and Jieping Ye, editors, _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, USA, August 6-10, 2023_ , pages 2444–2455. ACM, 2023.  doi: 10.1145/3580305.3599521.  URL https://doi.org/10.1145/3580305.3599521.
  * Xu et al. [2023] Mengwei Xu, Yaozong Wu, Dongqi Cai, Xiang Li, and Shangguang Wang.  Federated fine-tuning of billion-sized language models across mobile devices.  _CoRR_ , abs/2308.13894, 2023.  doi: 10.48550/ARXIV.2308.13894.  URL https://doi.org/10.48550/arXiv.2308.13894.
  * Yuan et al. [2022] Binhang Yuan, Cameron R. Wolfe, Chen Dun, Yuxin Tang, Anastasios Kyrillidis, and Chris Jermaine.  Distributed learning of fully connected neural networks using independent subnet training.  _Proceedings of the VLDB Endowment_ , page 1581–1590, Apr 2022.  doi: 10.14778/3529337.3529343.  URL http://dx.doi.org/10.14778/3529337.3529343.
  * Yang et al. [2023] Tien-Ju Yang, Yonghui Xiao, Giovanni Motta, Françoise Beaufays, Rajiv Mathews, and Mingqing Chen.  Online model compression for federated learning with large models.  In _IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023_ , pages 1–5. IEEE, 2023.  doi: 10.1109/ICASSP49357.2023.10097124.  URL https://doi.org/10.1109/ICASSP49357.2023.10097124.
  * Jiang et al. [2022a] Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K. Leung, and Leandros Tassiulas.  Model pruning enables efficient federated learning on edge devices.  _IEEE Transactions on Neural Networks and Learning Systems_ , page 1–13, Jan 2022a.  doi: 10.1109/tnnls.2022.3166101.  URL http://dx.doi.org/10.1109/tnnls.2022.3166101.
  * Lin et al. [2017] Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally.  Deep gradient compression: Reducing the communication bandwidth for distributed training.  _arXiv preprint arXiv:1712.01887_ , 2017.
  * Jiang et al. [2022b] Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K Leung, and Leandros Tassiulas.  Model pruning enables efficient federated learning on edge devices.  _IEEE Transactions on Neural Networks and Learning Systems_ , 2022b.
  * Krizhevsky et al. [2009] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * Deng [2012] Li Deng.  The mnist database of handwritten digit images for machine learning research.  _IEEE Signal Processing Magazine_ , 29(6):141–142, 2012.
  * McMahan et al. [2016] H. Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Agüera y Arcas.  Federated learning of deep networks using model averaging.  _CoRR_ , abs/1602.05629, 2016.  URL http://arxiv.org/abs/1602.05629.
  * Xie et al. [2019] Cong Xie, Sanmi Koyejo, and Indranil Gupta.  Asynchronous federated optimization.  _arXiv preprint arXiv:1903.03934_ , 2019.
  * Li et al. [2020] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith.  Federated optimization in heterogeneous networks.  _Proceedings of Machine learning and systems_ , 2:429–450, 2020.
  * Zhang et al. [2024a] Zuyuan Zhang, Hanhan Zhou, Mahdi Imani, Taeyoung Lee, and Tian Lan.  Collaborative ai teaming in unknown environments via active goal deduction.  _arXiv preprint arXiv:2403.15341_ , 2024a.
  * Zou et al. [2024] Yifei Zou, Zuyuan Zhang, Congwei Zhang, Yanwei Zheng, Dongxiao Yu, and Jiguo Yu.  A distributed abstract mac layer for cooperative learning on internet of vehicles.  _IEEE Transactions on Intelligent Transportation Systems_ , 2024.
  * Zhang et al. [2024b] Zuyuan Zhang, Mahdi Imani, and Tian Lan.  Modeling other players with bayesian beliefs for games with incomplete information.  _arXiv preprint arXiv:2405.14122_ , 2024b.
  * Gao et al. [2024] Mengtong Gao, Yifei Zou, Zuyuan Zhang, Xiuzhen Cheng, and Dongxiao Yu.  Cooperative backdoor attack in decentralized reinforcement learning with theoretical guarantee.  _arXiv preprint arXiv:2405.15245_ , 2024.

##  Appendix A Convergence Analysis of Fed-RAA

Let us start the convergence proof of Fed-RAA:

Without loss of generality, we assume that in the $q^{th}$ epoch of model
$\theta^{j}$ for each $j\in[M]$, the server receives the model
$\theta^{j}_{new}$, with the time stamp $\tau$. We assume that
$\theta^{j}_{new}$ is the result of applying $I_{min}^{n}\leq I^{n}\leq
I_{max}^{n}$ local updates to $\theta_{\tau}^{j}$ on the $n$th device. For
convenience, we ignore $n$ in all variances above and also in
$z^{n}_{\tau,i}$. Thus, using smoothness and strong convexity, conditional on
$\theta_{\tau,i-1}$, for $\forall i\in[I]$ we have

$\displaystyle\mathbb{E}[F(\theta^{j}_{\tau,i})-F(\theta_{*})]$
$\displaystyle\leq\mathbb{E}[G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i})-F(\theta_{*})]$
$\displaystyle\leq
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1})-F(\theta_{*})-\gamma\mathbb{E}[\langle\nabla
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1}),\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})\rangle]$
$\displaystyle\ \ \ \
+\frac{L\gamma^{2}}{2}\mathbb{E}[\|g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})\|^{2}]$
$\displaystyle\leq
F(\theta^{j}_{\tau,i-1})-F(\theta_{*})+\frac{\rho}{2}\|\theta^{j}_{\tau,i-1}-\theta^{j}_{\tau}\|^{2}$
$\displaystyle\ \ \ \ -\gamma\mathbb{E}[\langle\nabla
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1}),\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})\rangle]+\frac{L\gamma^{2}}{2}\mathbb{E}[\|g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})\|^{2}]$
$\displaystyle\leq
F(\theta^{j}_{\tau,i-1})-F(\theta_{*})-\gamma\mathbb{E}[\langle\nabla
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1}),\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})\rangle]$
$\displaystyle\ \ \ \ +\frac{L\gamma^{2}}{2}V_{2}+\frac{\rho
I_{max}^{2}\gamma^{2}}{2}V_{2}$ $\displaystyle\leq
F(\theta^{j}_{\tau,i-1})-F(\theta_{*})-\gamma\mathbb{E}[\langle\nabla
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1}),\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})\rangle]$
$\displaystyle\ \ \ \ +\gamma^{2}\mathcal{O}(\rho I_{max}^{2}V_{2})$

Taking $\rho$ large enough such that
$-(1+2\rho+\epsilon)V_{1}+\rho^{2}\|\theta^{j}_{\tau,i}-\theta^{j}_{\tau}\|^{2}-\frac{\rho}{2}\|\theta^{j}_{\tau,i}-\theta^{j}_{\tau}\|^{2}\geq
0,\forall\theta^{j}_{\tau,i},\theta^{j}_{\tau}$, and write $\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})$ as $\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1})$ for convenience, we have

$\displaystyle\langle\nabla
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1}),\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1})\rangle$
$\displaystyle-\epsilon\|\nabla F(\theta^{j}_{\tau,i-1})\|^{2}$
$\displaystyle=\langle\nabla
F(\theta^{j}_{\tau,i-1})+\rho(\theta^{j}_{\tau,i-1}-\theta^{j}_{\tau}),\nabla
f(\theta^{j}_{\tau,i-1})+\rho(\theta^{j}_{\tau,i-1}-\theta^{j}_{\tau})\rangle$
$\displaystyle\quad-\epsilon\|\nabla F(\theta^{j}_{\tau,i-1})\|^{2}$
$\displaystyle=\langle\nabla F(\theta^{j}_{\tau,i-1}),\nabla
f(\theta^{j}_{\tau,i-1})\rangle$ $\displaystyle\quad+\rho\langle\nabla
F(\theta^{j}_{\tau,i-1})+\nabla
f(\theta^{j}_{\tau,i-1}),(\theta^{j}_{\tau,i-1}-\theta^{j}_{\tau})\rangle$
$\displaystyle\quad+\rho^{2}\|\theta^{j}_{\tau,i-1}-\theta^{j}_{\tau}\|^{2}-\epsilon\|\nabla
F(\theta^{j}_{\tau,i-1})\|^{2}$ $\displaystyle\geq-\frac{1}{2}\|\nabla
F(\theta^{j}_{\tau,i-1})\|^{2}-\frac{1}{2}\|\nabla
f(\theta^{j}_{\tau,i-1})\|^{2}$ $\displaystyle\quad-\frac{\rho}{2}\|\nabla
F(\theta^{j}_{\tau,i-1})+\nabla f(\theta^{j}_{\tau,i-1})\|^{2}$
$\displaystyle\quad-\frac{\rho}{2}\|\theta^{j}_{\tau,i-1}-\theta^{j}_{\tau}\|^{2}+\rho^{2}\|\theta^{j}_{\tau,i-1}-\theta^{j}_{\tau}\|^{2}-\epsilon\|\nabla
F(\theta^{j}_{\tau,i-1})\|^{2}$
$\displaystyle\geq-(1+2\rho+\epsilon)V_{1}+\rho^{2}\|\theta^{j}_{\tau,i}-\theta^{j}_{\tau}\|^{2}-\frac{\rho}{2}\|\theta^{j}_{\tau,i}-\theta^{j}_{\tau}\|^{2}$
$\displaystyle=a\rho^{2}+b\rho+c\geq 0,$

where $a=\|\theta^{j}_{\tau,i}-\theta^{j}_{\tau}\|^{2}>0$,
$b=-2V_{1}-\frac{1}{2}\|\theta^{j}_{\tau,i}-\theta^{j}_{\tau}\|^{2}$,
$c=-(1+\epsilon)V_{1}$. Thus, we have $-\gamma\langle\nabla
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1}),\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1})\rangle\leq-\gamma\epsilon\|\nabla
F(\theta^{j}_{\tau,i-1})\|^{2}$.

Using $\tau-(q-1)\leq K$, we have
$\|\theta_{\tau}-\theta_{q-1}\|^{2}$$\leq\|(\theta_{\tau}-\theta_{\tau+1})+\ldots+(\theta_{q-1}-\theta_{q-1})\|^{2}\leq
K\|\theta_{\tau}-\theta_{\tau+1}\|^{2}+\ldots+K\|\theta_{q-1}-\theta_{q-1}\|^{2}$$\leq\alpha^{2}\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2})$.

Also, we have
$\|\theta_{\tau}-\theta_{q-1}\|$$\leq\|(\theta_{\tau}-\theta_{\tau+1})+\ldots+(\theta_{q-1}-\theta_{q-1})\|\leq\|\theta_{\tau}-\theta_{\tau+1}\|^{2}+\ldots+\|\theta_{q-1}-\theta_{q-1}\|^{2}$$\leq\alpha\gamma
KI_{max}\mathcal{O}(\sqrt{V_{2}})$.

Thus, we have

$\displaystyle\mathbb{E}[F(\theta^{j}_{\tau,i})-F(\theta_{*})]$
$\displaystyle\leq F(\theta^{j}_{\tau,i-1})-F(\theta_{*})$
$\displaystyle-\gamma\mathbb{E}[\langle\nabla
G_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1}),\nabla
g_{\theta^{j}_{\tau}}(\theta^{j}_{\tau,i-1};z_{\tau,i})\rangle]+\gamma^{2}\mathcal{O}(\rho
I_{max}^{2}V^{2})$ $\displaystyle\leq F(\theta^{j}_{\tau,i-1})-F(\theta_{*})$
$\displaystyle-\gamma\epsilon\|\nabla
F(\theta^{j}_{\tau,i-1})\|^{2}+\gamma^{2}\mathcal{O}(\rho I_{max}^{2}V_{2})$

where $\theta^{j}_{\tau}$ can be also written as $\theta_{\tau}$.

By rearranging the terms and telescoping, we have

$\displaystyle\mathbb{E}[F(\theta^{j}_{\tau,I})-F(\theta_{\tau})]$
$\displaystyle\leq-\gamma\epsilon\sum_{i=0}^{I-1}{\mathbb{E}\|\nabla
F(\theta^{j}_{\tau,i})\|^{2}}+\gamma^{2}\mathcal{O}(\rho I_{max}^{3}V_{2}).$

Then, we have

$\displaystyle\mathbb{E}[F(\theta_{q})-F(\theta_{q-1})]$
$\displaystyle\leq\mathbb{E}[G_{\theta_{q-1}}(\theta_{q})-F(\theta_{q-1})]$
$\displaystyle\leq\mathbb{E}[(1-\alpha)G_{\theta_{q-1}}(\theta_{q-1})+\alpha
G_{\theta_{q-1}}(\theta^{j}_{\tau,I})-F(\theta_{q-1})]$
$\displaystyle\leq\mathbb{E}[\alpha(F(\theta^{j}_{\tau,I})-F(\theta_{q-1}))+\frac{\alpha\rho}{2}\|\theta^{j}_{\tau,I}-\theta_{q-1}\|^{2}]$
$\displaystyle\leq\alpha\mathbb{E}[F(\theta^{j}_{\tau,I})-F(\theta_{q-1})]+\alpha\rho\|\theta^{j}_{\tau,I}-\theta_{\tau}\|^{2}+\alpha\rho\|\theta_{\tau}-\theta_{q-1}\|^{2}$
$\displaystyle\leq\alpha\mathbb{E}[F(\theta^{j}_{\tau,I})-F(\theta_{q-1})]+\alpha\rho(\gamma^{2}I_{max}^{2}\mathcal{O}(V_{2})+\alpha^{2}\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2}))$
$\displaystyle\leq\alpha\mathbb{E}[F(\theta^{j}_{\tau,I})-F(\theta_{q-1})]+\alpha\rho(\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2}))$
$\displaystyle\leq\alpha\mathbb{E}[F(\theta^{j}_{\tau,I})-F(\theta_{\tau})+F(\theta_{\tau})-F(\theta_{q-1})]+\alpha\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2})$

Using $L$-smoothness, we have

$\displaystyle F(\theta_{\tau})-F(\theta_{q-1})$
$\displaystyle\leq\langle\nabla
F(\theta_{q-1}),\theta_{\tau}-\theta_{q-1}\rangle+\frac{L}{2}\|\theta_{\tau}-\theta_{q-1}\|^{2}$
$\displaystyle\leq\|\nabla
F(\theta_{q-1})\|\|\theta_{\tau}-\theta_{q-1}\|+\frac{L}{2}\|\theta_{\tau}-\theta_{q-1}\|^{2}$
$\displaystyle\leq\sqrt{V_{1}}\alpha\gamma
KI_{max}\mathcal{O}(\sqrt{V_{2}})+\frac{L}{2}\alpha^{2}\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2})$
$\displaystyle\leq\alpha\gamma
KI_{max}\mathcal{O}(\sqrt{V_{1}V_{2}})+\alpha^{2}\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2})$

Thus, we have

$\displaystyle\mathbb{E}[F(\theta_{q})-F(\theta_{q-1})]\leq$
$\displaystyle-\alpha\gamma\epsilon\sum_{i=1}^{I-1}{\mathbb{E}\|\nabla
F(\theta^{j}_{\tau,i})\|^{2}}$ $\displaystyle+\alpha\gamma^{2}\mathcal{O}(\rho
I_{max}^{3}V_{2})$ $\displaystyle+\alpha^{2}\gamma
KI_{max}\mathcal{O}(\sqrt{V_{1}V_{2}})$
$\displaystyle+\alpha^{3}\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2})$
$\displaystyle+\alpha\gamma^{2}K^{2}I_{max}^{2}\mathcal{O}(V_{2})$

By rearranging the terms, we have

$\displaystyle\sum_{i=1}^{I_{q}^{\prime}-1}{\mathbb{E}\|\nabla
F(\theta^{j}_{\tau,i})\|^{2}}\leq$
$\displaystyle\frac{\mathbb{E}[F(\theta_{q-1})-F(\theta_{q})]}{\alpha\gamma\epsilon}$
$\displaystyle+\frac{\gamma I_{max}^{3}}{\epsilon}\mathcal{O}(V_{2})$
$\displaystyle+\frac{\alpha KI_{max}}{\epsilon}\mathcal{O}(\sqrt{V_{1}V_{2}})$
$\displaystyle+\frac{\alpha^{2}\gamma
K^{2}I_{max}^{2}}{\epsilon}\mathcal{O}(V_{2})$ $\displaystyle+\frac{\gamma
K^{2}I_{max}^{2}}{\epsilon}\mathcal{O}(V_{2}),$

where $I_{q}^{\prime}$ is the number of local iterations applied in the $q$th
iteration.

By telescoping and taking total expectation, after $T=Q*M$ global epochs which
means $Q$ global epochs on each submodel $\theta^{j}$, we have

$\displaystyle\min_{q=0}^{T-1}\mathbb{E}[\|\nabla F(\theta_{q})\|^{2}]$
$\displaystyle\leq\frac{1}{\sum_{q=1}^{T}{I_{q}^{\prime}}}\sum_{q=1}^{T}\sum_{i=0}^{I_{q}^{\prime}-1}\|\nabla
F(\theta_{\tau,i})\|^{2}$
$\displaystyle\leq\frac{\mathbb{E}[F(\theta_{0})-F(\theta_{T})]}{\alpha\gamma\epsilon
TI_{min}}+\frac{\gamma TI_{max}^{3}}{\epsilon
TI_{min}}\mathcal{O}(V_{2})+\frac{\alpha KTI_{max}}{\epsilon
TI_{min}}\mathcal{O}(\sqrt{V_{1}V_{2}})$
$\displaystyle\quad+\frac{\alpha^{2}\gamma K^{2}TI_{max}^{2}}{\epsilon
TI_{min}}\mathcal{O}(V_{2})+\frac{\gamma K^{2}TI_{max}^{2}}{\epsilon
TI_{min}}\mathcal{O}(V_{2})$
$\displaystyle\leq\frac{\mathbb{E}[F(\theta_{0})-F(\theta_{T})]}{\alpha\gamma\epsilon
TI_{min}}+\mathcal{O}(\frac{\gamma I_{max}^{3}}{\epsilon
I_{min}})+\mathcal{O}(\frac{\alpha KI_{max}}{\epsilon
I_{min}})+\mathcal{O}(\frac{\alpha^{2}\gamma K^{2}I_{max}^{2}}{\epsilon
I_{min}})$ $\displaystyle\quad+\mathcal{O}(\frac{\gamma
K^{2}I_{max}^{2}}{\epsilon I_{min}})$

Using $\delta=\frac{I_{max}}{I_{min}}$, and taking
$\alpha=\frac{1}{\sqrt{I_{min}}}$, $\gamma=\frac{1}{\sqrt{T}}$,
$T=I_{min}^{5}$, we have

$\displaystyle\min_{q=0}^{T-1}\mathbb{E}[\|\nabla F(\theta_{q})\|^{2}]$
$\displaystyle\leq\mathcal{O}\left(\frac{1}{\epsilon
I_{min}^{3}}\right)+\mathcal{O}\left(\frac{\delta^{3}}{\epsilon\sqrt{I_{min}}}\right)+\mathcal{O}\left(\frac{K\delta}{\epsilon\sqrt{I_{min}}}\right)$
$\displaystyle\quad+\mathcal{O}\left(\frac{K^{2}\delta^{2}}{\epsilon\sqrt{I_{min}^{5}}}\right)+\mathcal{O}\left(\frac{K^{2}\delta^{2}}{\epsilon\sqrt{I_{min}^{3}}}\right).$

##  Appendix B Details of Gre-RAA

Input: $n$ clients, model $\theta$ with $M$ regions
$\\{\theta^{1},\ldots,\theta^{M}\\}$, training delay bound $K$, global epoch
$t\in[T]$

Output: $h_{t}(n)$

for  _each idle client $n\in[N]$ in $t$_ do

Initialize $c_{n,j}$, $\forall j\in[M]$ ;

Find one submodel set $S=\\{\theta^{j}\|c_{n,j}\leq K\\}$;

Set $h_{t}(n)\leftarrow j$ where $\theta^{j}$ is randomly chosen from $S$;

for  _each submodel $\theta^{j^{\prime}}\in S$_ do

$q(j^{\prime})$ is the number of times $\theta^{j^{\prime}}$ has been globally
updated;

if  _$q(j^{\prime}) <q(h_{t}(n))$_ then

$h_{t}(n)\leftarrow j^{\prime}$

end if

end for

end for

Algorithm 2 Gre-RAA

Proof of Theorem 2.  This part shows the detailed proof of Theorem 2:

Let $K^{\prime}$ be the upper bound of training delay which follows an offline
submodel assignment strategy in the same environment. The whole proof process
is as follows: We figure out what $K$ is like following an offline model
assignment strategy in our proposed FL system. We then show that given $K$,
the online model assignment algorithm, Algorithm 2, always has a feasible
solution.

We first introduce the offline model assignment strategy. Given $N$ clients
and $M$ regions, the offline model assignment strategy assigns the submodel
with a fixed index to each user. If $N<M$, we can simply apply the idea of
time division multiplexing for the strategy where each client is cyclically
assigned to submodels within a fixed set. Then $K$ can be simply computed in
the same way as that in situation $N\geq M$. So here we consider the situation
where $N\geq M$. To begin with, we rank $N$ clients according to their
computational resources and get a sorted client list where $C_{i}\geq
C_{i+1},\forall i\in[N-1]$ and $C_{i}$ is the computational ability of client
$i$. Similarly, we sort the submodels according to the model size and get a
submodel list where $\|\theta^{j}\|\geq\|\theta^{j+1}\|,\forall j\in[M-1]$. In
the offline model assignment strategy, for each submodel $\theta^{j}$, we
assign it to the client set
$\\{(j-1)*\lfloor\frac{N}{M}\rfloor+1,\ldots,\min{(j*\lfloor\frac{N}{M}\rfloor,N)}\\}$.
Then the training delay bound $K$ can be computed as
$K=\max{(\frac{\theta^{1}}{C_{1}},\frac{\theta^{1}}{C_{1}},\ldots,\frac{\theta^{1}}{C_{\lfloor\frac{N}{M}\rfloor}},\ldots,\frac{\theta^{M}}{C_{N}})}$.

We can see that there would not exist one $K^{\prime}$ that is smaller than
$K$. If we swap any two elements in the sorted client list, the new
$K^{\prime}$ would be
$\max(K,\frac{\theta^{j}}{C_{i^{\prime}}},\frac{\theta^{j^{\prime}}}{C_{i}})$
and obviously $K^{\prime}\geq K$.

Then, we show that given $K$ which is figured out in the offline scenario, the
proposed online model assignment algorithm always has a feasible solution.
That is, Algorithm 2 always can find one non-empty submodel set $S$. It is
intuitive that there always exists element $\theta^{\lceil\frac{nM}{N}\rceil}$
in the set $S$ when considering idle client $n$. Therefore, it always can find
one feasible submodel assigned to each client.

So the proposed model assignment algorithm, shown in Algorithm 2, reaches the
same upper bound of training delay as that conducted by the optimal offline
model assignment strategy.

##  Appendix C Experiment Details

###  C.1 Experimental Setup

Dataset.  The CIFAR-10 [20] dataset consists of 60,000 32x32 color images
spanning 10 classes, with 50,000 for training and 10,000 for testing, ensuring
equal representation across classes. CIFAR-100 [20] expands upon CIFAR-10 with
the same image resolution but includes 100 classes categorized into 20
superclasses, each class providing 500 training and 100 test images labeled
with both class and superclass identifiers. MNIST [21] remains a cornerstone
in the field, featuring 70,000 normalized 28x28 grayscale images of
handwritten digits, with a training set of 60,000 and a test set of 10,000,
sourced from a diverse population for reliable model evaluation.

Hardware and Software. We implement the algorithmic simulation via PyTorch
employing multi-threading, wherein each thread sustains a local client model
in shared memory for individual training, while the master program oversees
the global model. During the algorithm’s initialization phase, the simulation
introduces random latency based on clients’ computational capabilities to
emulate the feedback delay of slow devices. The simulation software operates
on a Linux server equipped with an AMD EPYC 9654 96-Core Processor clocked at
3.463GHz, an NVIDIA RTX 4090 GPU with 24GB VRAM, and 60GB of system memory.

Models.  Taking into account the varying complexity of datasets, our
simulation experiments evaluate the performance of both MLP and CNN
architectures as global models. For MLP, the number of hidden neurons is
configured as 200 for MNIST, 2048 for CIFAR-10, and 4096 for CIFAR-100.

Model Splitting Strategy.  In the Fed-RAA framework, sub-models are formed by
a subset of neurons taken from the global model, enabling clients to train a
reduced set of parameters to enhance the convergence speed. This is realized
by implementing a partitioning strategy on the global model, which is governed
by predefined hyperparameters and results in the model being divided into
distinct segments. Each client, guided by its computational performance,
selects an appropriate sub-model for its training endeavors. We have devised
four distinct partition strategies, where the global model’s designated layers
are segmented into 2, 3, 4, or 5 parts respectively, with the detailed
allocation of these partitions outlined in Table 4. To ensure compatibility
across a wide range of client capabilities, the partitioning strategy employs
a non-uniform approach: weaker devices are assigned smaller sub-models to
minimize total training time, whereas more capable clients receive larger sub-
models to optimally leverage their superior processing power.

Table 4: model partition strategy Number of partitions | Partition ratio
---|---
2 | 40%, 60%
3 | 20%, 30%, 50%
4 | 10%, 20%, 30%, 40%
5 | 5% , 10%, 20%, 30%, 35%

Other Settings.  In the controlled experiments comparing Fed-RAA with these
baselines, we employ the momentum SGD optimizer for training across 10
clients, utilizing a batch size of 128 for MNIST and CIFAR-10 datasets, and 64
for CIFAR-100, with a momentum parameter set to 0.5. Each client is tasked
with 5 local epochs per global iteration, and the learning rate is fixed at
0.005. To ensure a fair comparison, identical data partitioning schemes, and
hardware configurations are maintained across all methodologies.

###  C.2 Baselines Comparison Analysis

Upon meticulous scrutiny of the experimental data encapsulated in Tables 5 and
2, our Fed-RAA algorithm emerges as conspicuously advantageous in terms of
time efficiency relative to conventional federated learning benchmarks. The
empirical evidence illustrates that Fed-RAA considerably diminishes the wall-
clock time to reach benchmark accuracies on the MNIST, CIFAR-10, and CIFAR-100
datasets. It notably completes training in 91.06 seconds for MNIST, 89.85
seconds for CIFAR-10, and 101.86 seconds for CIFAR-100, thereby outperforming
the closest competitor, FedSync, by over a twofold improvement in speed.

Although Fed-RAA displays slightly reduced accuracy in comparison to top-
performing algorithms like RamFed, the marginal accuracy trade-off is
counterbalanced by the dramatic reductions in training time, making Fed-RAA an
appealing option in time-critical circumstances. This delicate balance of time
and accuracy underscores Fed-RAA’s practical relevance for rapid model updates
and accelerates deployment, a significant contribution to the domain of
federated learning that warrants emphasis in our experimental analysis.

It is noteworthy, however, that Fed-RAA’s accelerated processing did
necessitate a higher number of epochs, with 98399 for MNIST, 98499 for
CIFAR-10, and 98599 for CIFAR-100, surpassing the epoch requirements of other
baselines. This observation, notwithstanding the reduced time per epoch,
implies that Fed-RAA capitalizes on computational efficiency, permitting swift
iterative processes even when individual epochs are of minimal computational
cost. This attribute makes Fed-RAA particularly valuable in contexts where
communication latency poses significant challenges, highlighting the
algorithm’s capacity to enable rapid convergence and frequent model updates
within distributed ML frameworks. Consequently, this dual analysis of time and
epoch dynamics bolsters the case for Fed-RAA’s adoption in environments where
both expedited learning and resource optimization are paramount.

The experiment details shown in Figure 3 demonstrate the efficacy of the Fed-
RAA algorithm in federated learning environments, particularly under
asynchronous operations. In the MNIST and CIFAR-10 datasets, Fed-RAA shows
significant advantages in balancing accuracy and computational time,
outperforming other algorithms such as Fed-Avg, Fed-Sync, and Fed-Prox, which
either sacrifice accuracy for lower time costs or vice versa. For the more
complex CIFAR-100 dataset, Fed-RAA remarkably excels by achieving higher
accuracies more rapidly compared to its counterparts, which face sharp
increases in computational time with minimal accuracy gains. This consistent
performance underscores the benefits of asynchronous learning and training
submodels, suggesting that Fed-RAA’s approach to handling partial model
training can substantially enhance the efficiency and effectiveness of
federated learning systems.

Table 5: Comparison of accuracy and time cost across different baselines Baseline | MNIST | CIFAR-10 | CIFAR-100
---|---|---|---
| Accuracy (%) | Time (s) | Accuracy (%) | Time (s) | Accuracy (%) | Time (s)
FedAvg | 96.35 | 1219.05 | 52.88 | 1172.49 | 25.22 | 1306.53
FedSync | 96.50 | 727.38 | 53.01 | 759.33 | 25.41 | 3369.75
FedProx | 96.40 | 2599.20 | 52.89 | 2611.32 | 25.87 | 2732.94
FedPrun | 94.13 | 2330.46 | 53.60 | 2926.44 | 24.97 | 9475.65
SplitFL | 95.30 | 2520.45 | 51.30 | 2309.64 | 23.68 | 6544.77
Ram-Fed | 96.55 | 2426.88 | 54.22 | 2349.18 | 27.04 | 2754.33
Fed-RAA | 93.86 | 273.18 | 47.38 | 269.54 | 20.55 | 305.58

###  C.3 Detailed Results of Comparison for $beta$ and $M$

(a) $M=2$

(b) $M=3$

(c) $M=4$

(d) $M=5$

Figure 7: Results of the MNIST dataset.

(e) $M=2$

(f) $M=3$

(g) $M=4$

(h) $M=5$

Figure 8: Results of the CIFAR-10 dataset.

(a) $M=2$

(b) $M=3$

(c) $M=4$

(d) $M=5$

Figure 9: Results of the CIFAR-100 dataset. Figure 10: Detailed results of
comparison for $\beta$ of different submodel numbers $M$ on dataset MNIST,
CIFAR-10, and CIFAR-100.

In the experimental analyses depicted in Figure 5, we observed that increasing
the proportion of weak-capability clients ($\beta$) significantly impacts both
the training time and accuracy of federated learning models across various
datasets. Specifically, as $\beta$ was raised, particularly to 0.9, a clear
increase in training time was noted across all datasets, with the most
pronounced effects in the CIFAR-100 dataset due to its higher complexity. This
indicates that a higher number of less capable clients markedly slows down the
learning process. Accuracy trends also varied with dataset complexity; the
MNIST dataset maintained relative stability in accuracy until a sharp decline
at $\beta=0.9$, whereas both CIFAR datasets exhibited a more consistent and
sensitive decline in accuracy as $\beta$ increased. These observations
underscore the significant influence of client capability distribution on the
performance of federated learning systems, particularly in complex tasks. The
findings highlight the crucial need for advanced mechanisms to manage client
heterogeneity, suggesting that adaptive strategies and careful management of
client capabilities are essential for enhancing both the efficiency and
effectiveness of federated learning outcomes in diverse and challenging data
environments.

Results shown in Figure 6 reveal significant insights into the impact of model
fragmentation on training dynamics. Notably, increasing the number of
submodels from 2 to 5 consistently led to prolonged training times, with the
effect being most pronounced in the CIFAR-100 dataset. This dataset,
characterized by its complexity, showed steep increases in training duration
without corresponding improvements in accuracy, suggesting a scalability
challenge with Fed-RAA when managing numerous small submodels in complex
environments. Moreover, while the MNIST and CIFAR-10 datasets exhibited more
moderate increases in training time, the benefit in accuracy did not
substantially improve with more submodels, particularly in CIFAR-10 where
training times converged across submodel configurations as accuracy increased.
These observations indicate that there exists a critical trade-off in the
asynchronous federated learning context of Fed-RAA between the number of
submodels and the efficiency of training. Smaller submodels, although
potentially beneficial for individual client training cycles, may introduce
significant overhead in managing multiple updates, especially in datasets with
higher complexity. This underscores the necessity for optimizing the balance
between model fragmentation and training efficiency to enhance the
effectiveness of federated learning systems in diverse data conditions.

##  Appendix D Discussion

Limitations. Although our proposed algorithm, Fed-RAA, serves as a
foundational approach that is adaptable to any model partitioning strategy, it
is important to recognize that its effectiveness could be significantly
enhanced by tailoring the partitioning strategy to account for the
heterogeneous and constrained resources at the client level. This
consideration emerges as a limitation in the current design of Fed-RAA, where
the generic approach to model fragmentation does not explicitly address
variations in client capabilities. By integrating a partitioning mechanism
that dynamically adjusts to the resource constraints and computational
capabilities of individual clients, the performance of Fed-RAA could be
optimized, leading to improved efficiency and effectiveness in practical
federated learning scenarios. This potential refinement highlights a valuable
direction for future enhancements and underscores the importance of context-
aware strategies in the development of federated learning algorithms. In
addition, we refer to the following articles [25, 26, 27, 28]

Impacts.  The algorithm introduced in this paper, Fed-RAA, has meaningful
positive impacts on the field of federated learning. It empowers resource-
constrained edge network devices to collaboratively train and deploy large
models, thereby democratizing access to sophisticated machine learning
capabilities which would be impractical with traditional centralized training
methods. Moreover, as a foundational and neutral algorithm, Fed-RAA avoids
inherent biases and specific adverse societal impacts. This characteristic
makes it an adaptable framework that can be tailored by researchers and
practitioners to align with ethical standards and societal needs, ensuring a
conscientious deployment of AI technologies that mitigates risks associated
with more specialized algorithms potentially amplifying biases or fostering
inequality.


# Optimal shapes for positivity preserving111Research partially funded by the
Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - EI 1155/1.

Sascha Eichmann
Mathematisch-Naturwissenschaftliche Fakultät,
Eberhard Karls Universität Tübingen,
Auf der Morgenstelle 10,
D-72076 Tübingen, Germany
E-Mail<EMAIL_ADDRESS>The author thanks Prof. Reiner
Schätzle for fruitful discussions.

###### Abstract

We are looking for an optimal convex domain on which the boundary value
problem

$\left\\{\begin{array}[]{cc}(-\Delta)^{2}u_{\gamma}-\gamma\Delta
u_{\gamma}=f,&\mbox{ in }\Omega\\\
u_{\gamma}=\partial_{\nu}u_{\gamma}=0,&\mbox{ on
}\partial\Omega\end{array}\right.$

admits a nonnegative solution for the most $\gamma$, if $f$ is a given
nonnegative function.

Keywords. Positivity preserving, Boggio-Hadamard conjecture, shape
optimisation
MSC. 35B09, 35B30, 35G15, 49Q10

##  1 Introduction

The Boggio-Hadamard conjecture states, that for any convex domain
$\Omega\subseteq\mathbb{R}^{n}$ any solution to

$\left\\{\begin{array}[]{cc}(-\Delta)^{2}u=f,&\mbox{ in }\Omega\\\
u=\partial_{\nu}u=0,&\mbox{ on }\partial\Omega\end{array}\right.$

satisfies $u\geq 0$, if $f\geq 0$. Here $\nu$ is the outer unit normal of
$\Omega$. This differential equation is also called the clamped plate equation
(see e.g. [11] for more context). In it $\Omega$ is a thin plate clamped at
the boundary and subject to an orthogonal force $f$. Then $u$ is the
displacement of $\Omega$. Hence the conjecture can also be restated as:

Does upward pushing imply upward bending?

This conjecture was substantiated by Boggio by some explicit formulas for the
Greens function in [4] in case $\Omega$ is a ball (see also the letters by
Hadamard [16] and [17]). First counterexamples to this conjecture were given
by Duffin in [7] and a little later by Garabedian in [10]. Today we even have
counterexamples with constant $f$, see [14] and [15]. On the other hand by
e.g. conformal invariance one can use the ball to construct other domains with
positivity preserving, see [6]. In general it is not known, what the driving
force reagrding $\Omega$ behind positivity preserving or a lack thereof is,
see e.g. the discussion in [11, § 5, § 6] and the references therein.
For general elliptic differential operators of higher order positivity
preserving cannot be expected, see [13].

In [5] a related problem to (1.1) has been discussed, i.e. whether for given
$f\geq 0$ the boundary value problem

$\left\\{\begin{array}[]{cc}(-\Delta)^{2}u_{\gamma}-\gamma\Delta
u_{\gamma}=f,&\mbox{ in }\Omega\\\
u_{\gamma}=\partial_{\nu}u_{\gamma}=0,&\mbox{ on
}\partial\Omega\end{array}\right.$ (1.1)

has a positive solution $u_{\gamma}$, if $\gamma$ is big enough. $\gamma$ is
called tension and this kind of equation was first used by Bickley in [2] to
model the membrane in microphones. Here the motivation differs: If $\gamma$
becomes bigger, the influence of the laplace term $-\Delta u_{\gamma}$ should
increase. Since $-\Delta$ satisfies a maximum principle, i.e. also positivity
preserving, the chances, that $u_{\gamma}$ is nonnegative increases if
$\gamma$ increases. This statement was proven rigorously in all dimensions in
[8] and is cited in appendix A for the readers convenience (see also [12] for
positivity preserving for all $\gamma\geq 0$ in dimension one).
Here we like to search for convex domains in which we have the ’most’
positivity preserving, i.e. for the largest range of $\gamma\in\mathbb{R}$.
Let us make the notion of our agenda more precise by letting $R>0$, $f\in
L^{\infty}(B_{R}(0))$, $c_{1}>0$, $f\geq 0$, $n=2,3$. Now we consider the
boundary value problem

$\left\\{\begin{array}[]{cc}(-\Delta)^{2}u_{\gamma}-\gamma\Delta
u_{\gamma}=f,&\mbox{ in }\Omega\\\
u_{\gamma}=\partial_{\nu}u_{\gamma}=0,&\mbox{ on
}\partial\Omega\end{array}\right.$ (1.2)

for any domain $\Omega\subseteq B_{R}(0)\subseteq\mathbb{R}^{n}$. Since we
like to solve this boundary value problem even for nonsmooth $\Omega$ we
define $u_{\gamma}$ to be a (weak) solution of (1.2), if and only if
$u_{\gamma}\in W^{2,2}_{0}(\Omega)$ and for all $\varphi\in
C^{\infty}_{0}(\Omega)$ we have

$\int_{\Omega}\Delta u_{\gamma}\Delta\varphi+\gamma\nabla
u_{\gamma}\cdot\nabla\varphi-f\varphi=0.$ (1.3)

By $\mu_{1}(\Omega)$ we will denote the buckling load, a kind of first
eigenvalue, given by

$\mu_{1}(\Omega):=\inf_{u\in
W^{2,2}_{0}(\Omega)-\\{0\\}}\frac{\int_{\Omega}|\Delta
u|^{2}}{\int_{\Omega}|\nabla u|^{2}}.$ (1.4)

$\mu_{1}(\Omega)$ acts as a lower bound for $\gamma$, because upon reaching
it, we would have nonuniqueness or nonexistence of the boundary value problem
(1.2). See e.g. [11, § 3.2] for more details regarding the buckling load. Next
we introduce the set in which we will minimise:

$M:=\\{\Omega\subseteq B_{R}(0)|\ \Omega\mbox{ is a convex domain and
}\mathcal{L}^{n}(\Omega)=c_{1}\\}.$ (1.5)

We define an energy on $M$ by

$\gamma_{f}(\Omega):=\inf\\{\gamma>-\mu_{1}(\Omega)|\
\forall\tilde{\gamma}\geq\gamma\mbox{ the weak solution to \eqref{eq:1_1}
satisfies }u_{\tilde{\gamma}}\geq 0\mbox{ a.e.}\\}$ (1.6)

The result [8, Thm. 1.1] (see also the appendix A) guarantees, that
$\gamma_{f}$ is finite for bounded domains $\Omega$ with boundary in $C^{4}$.
This note is dedicated to proving the following theorem:

######  Theorem 1.1.

There exists an $\Omega_{f}\in M$, such that for all $\Omega\in M$ we have

$\gamma_{f}(\Omega_{f})\leq\gamma_{f}(\Omega).$

This $\Omega_{f}$ is an optimal domain for positivity preserving, since the
range for $\gamma\in\mathbb{R}$ is as large as possible. The strategy of the
proof is as follows: We use the direct variational method. Compactness for a
sequence of domains will be achieved w.r.t. Hausdorff distance, see section 2.
Lower-semicontinuity is at the heart of the proof, in which the convexity of
the domains will play a crucial role, see Thm. 2.2. The proof itself is given
in section 3. Finally we discuss some open questions in section 4.

##  2 Hausdorff distance and convexity

Here we collect some important results concerning the Hausdorff distance of
sets in $\mathbb{R}^{n}$. Some of them are focused on convex sets. The
Hausdorff distance is defined as

$d_{H}(A,B):=\inf\\{\varepsilon>0|\ A\subseteq B_{\varepsilon}\mbox{ and
}B\subseteq A_{\varepsilon}\\}.$ (2.1)

Here

$A_{\varepsilon}:=\bigcup_{x\in A}B_{\varepsilon}(x)=\\{x\in\mathbb{R}^{n}|\
d(x,A)<\varepsilon\\}.$

For compact sets $d_{H}$ is a metric. Furthermore for all compact sets
$K\subset\mathbb{R}^{n}$ the set

$M_{K}:=\\{\tilde{K}\subseteq K|\ \tilde{K}\mbox{ compact}\\}$

is compact with respect to $d_{H}$ (see e.g. [9, § 2.10.21]). Furthermore we
have the following result for convex sets, which is also part of the famous
Blaschke selection Theorem (see also [1] for a different approach concerning
the convergence of Lebesgue measure and again [9, § 2.10.21] for the convexity
of the limit). We define convergence of convex domains, i.e. for open convex
sets $\Omega_{m},\Omega\subseteq\mathbb{R}^{n}$ as

$d_{H}(\overline{\Omega}_{m},\overline{\Omega})\rightarrow 0,\
m\rightarrow\infty.$ (2.2)

The result is then as follows:

######  Lemma 2.1 ([3] p.61 & p.62).

Let $\Omega_{m}\subseteq\mathbb{R}^{n}$ be a sequence of convex domains
converging w.r.t. to $d_{H}$ to $\Omega\subseteq\mathbb{R}^{n}$. Then

$\mathcal{L}^{n}(\Omega_{m})\rightarrow\mathcal{L}^{n}(\Omega)$

and $\Omega$ is convex.

The next Theorem will greatly help in showing that the limit of some sequences
considered in the proof of the main result will satisfy a differential
equation weakly. The proof is inspired by [21, Prop. 10].

######  Theorem 2.2.

Let $\Omega_{m}\subseteq\mathbb{R}^{n}$ be a sequence of convex open domains
converging w.r.t. (2.2) to an open convex $\Omega\subseteq\mathbb{R}^{n}$. Let
$K\subset\Omega$ be compact in $\mathbb{R}^{n}$. Then there exists an
$m_{0}\in\mathbb{N}$, such that for all $m\geq m_{0}$ we have

$K\subset\Omega_{m}.$

###### Proof.

We proceed by contradiction and assume after possibly choosing a subsequence,
that

$\forall m\in\mathbb{N}\ \exists x_{m}\in K:\ x_{m}\notin\Omega_{m}.$

Since $K$ is compact, there exists an $r>0$ independent of $m$, such that

$B_{r}(x_{m})\subseteq\Omega\mbox{ for all }m\in\mathbb{N}.$

Since $\Omega_{m}$ is convex, the Hahn-Banach seperation theorem, see e.g.
[20, Thm 3.4], yields $L_{m}\in\partial B_{1}(0)$, $\alpha_{m}\in\mathbb{R}$,
such that

$\Omega_{m}\subseteq\\{y\in\mathbb{R}^{n}|\ \langle
L_{m},y\rangle\geq\alpha_{m}\\}\mbox{ and }\langle
L_{m},x_{m}\rangle\leq\alpha_{m}.$

Here $\langle\cdot,\cdot,\rangle$ denotes the euclidean scalar product. By
e.g. setting $y_{m}:=x_{m}-\frac{r}{2}L_{m}$ (see Figure 2.1), we obtain an
$y_{m}\in B_{r}(x_{m})\subseteq\Omega$ with

$d(y_{m},\Omega_{m})\geq\frac{r}{2}.$

Figure 1: Seperating $x_{m}$ from $\Omega_{m}$ by Hahn-Banach.

Therefore

$y_{m}\notin(\Omega_{m})_{\frac{r}{4}}.$

Since $y_{m}\in\Omega$, we have by the definition of Hausdorff distance

$d_{H}(\Omega,\Omega_{m})\geq\frac{r}{4}.$

This implies that $\Omega_{m}\nrightarrow\Omega$ w.r.t. Hausdorff distance,
which is a contradiction. ∎

######  Remark 2.3.

Theorem 2.2 fails without convexity. Let $\Omega$ be any open set and let
$x\in\Omega$. Then we define the sequence

$\Omega_{R}:=\Omega-B_{R}(x).$

Then we have (see also Figure 2)

$\Omega_{R}\rightarrow\Omega\mbox{ w.r.t. }d_{H}\mbox{ for }R\downarrow 0.$

Figure 2: Counterexample for Thm. 2.2 without convexity.

If we now choose any compact set $K\subset\Omega$ containing $x$, the result
of Theorem 2.2 cannot be expected.

Let us also remark, that the boundary of a convex set is Lipschitz and
therefore of Lebesgue measure zero:

######  Lemma 2.4.

Let $C\subseteq\mathbb{R}^{n}$ be convex. Then $\partial C$ is a Lipschitz
boundary and therefore

$\mathcal{L}^{n}(\partial C)=0.$

###### Proof.

Since $C$ is convex, $\partial C$ is locally a graph of a convex function $f$.
$f$ is therefore also locally Lipschitz. By covering the boundary with at most
countable many balls we then obtain $\mathcal{L}^{n}(\partial C)=0$. ∎

##  3 Proof of the main Theorem

###### Proof.

Let $\Omega_{m}\in M$ be a minimising sequence for $\gamma_{f}$, i.e.

$\gamma_{f}(\Omega_{m})\rightarrow\inf_{\Omega\in M}\gamma_{f}(\Omega),\mbox{
for }m\rightarrow\infty.$

Then $\overline{\Omega}_{m}\subseteq\overline{B_{R}(0)}$ and after relabeling
$\overline{\Omega}_{m}$ converges w.r.t. the Hausdorff distance $d_{H}$ to a
compact set $K\subseteq\overline{B_{R}(0)}$. By Lemma 2.1 $K$ is convex and
$\mathcal{L}^{n}(K)=c_{1}>0$. Hence by Lemma 2.4

$\mathcal{L}^{n}(\partial K)=0$

and we set $\Omega_{f}:=\dot{K}=K-\partial K$, i.e. the topological inner part
of $K$. Since the measure of it is positive, it is not empty. Furthermore it
is still convex and satisfies

$\mathcal{L}^{n}(\Omega_{f})=c_{1},$

hence $\Omega_{f}\in M$. Now we need to show a lower semi-continuity estimate.
For this let

$\gamma>\inf_{\Omega\in M}\gamma_{f}(\Omega)$

be fixated. For $m$ big enough we have

$\gamma>\gamma_{f}(\Omega_{m})\geq-\mu_{1}(\Omega_{m}),$

hence we have unique weak solutions $u_{m}\in W^{2,2}_{0}(\Omega_{m})$ to
(1.2) on $\Omega_{m}$. We extend these by $0$ to the whole of $B_{R}(0)$.
Therefore we have

$u_{m}\in W^{2,2}_{0}(B_{R}(0)).$

We find an $\alpha>0$ such that after choosing a subsequence and relabeling,
we obtain

$\gamma-\alpha>\gamma_{f}(\Omega_{m})\geq-\mu_{1}(\Omega_{m}).$

This yields

$\gamma-\alpha+\mu_{1}(\Omega_{m})>0\mbox{ for all }m\in\mathbb{N}.$

If we now choose $\delta,\varepsilon^{\prime}>0$ small enough, we also have

$(1-\delta)\mu_{1}(\Omega_{m})+\gamma-\varepsilon^{\prime}>0.$

Since $C^{\infty}_{0}(\Omega_{m})$ is dense in $W^{2,2}_{0}(\Omega_{m})$, the
solution $u_{m}$ is an admissible test function. Hence Poincarés and Youngs
inequality yield

$\displaystyle\int_{\Omega_{m}}(\Delta u_{m})^{2}+\gamma|\nabla u_{m}|^{2}=$
$\displaystyle\int_{\Omega_{m}}fu_{m}$ $\displaystyle\leq$ $\displaystyle
C(\varepsilon,\operatorname{diam}(\Omega_{m}))\int_{\Omega_{m}}f^{2}+\varepsilon\int_{\Omega}|\nabla
u_{m}|^{2}$

for every $\varepsilon>0$. Rearranging yields with the above choosen
$\delta>0$

$\delta\int_{\Omega_{m}}(\Delta u_{m})^{2}+(1-\delta)\int_{\Omega_{m}}(\Delta
u_{m})^{2}+(\gamma-\varepsilon)\int_{\Omega_{m}}|\nabla u_{m}|^{2}\leq
C\int_{\Omega_{m}}f^{2}.$

By the definition of $\mu_{1}(\Omega_{m})$, see (1.4), we obtain

$\delta\int_{\Omega_{m}}(\Delta
u_{m})^{2}+((1-\delta)\mu_{1}(\Omega_{m})+(\gamma-\varepsilon))\int_{\Omega_{m}}|\nabla
u_{m}|^{2}\leq
C(\varepsilon,\operatorname{diam}(\Omega_{m}))\int_{\Omega_{m}}f^{2}.$

Choosing $\varepsilon<\varepsilon^{\prime}$ yields a constant
$C=C(\varepsilon,R,f)>0$ such that

$\|u_{m}\|_{W^{2,2}_{0}(B_{R}(0))}\leq C.$

After choosing a subsequence and relabeling, we can assume there exists a
$u_{0}\in W^{2,2}_{0}(B_{R}(0))$, such that

$\displaystyle u_{m}\rightarrow u_{0}$ $\displaystyle\mbox{ weakly in
}W^{2,2}_{0}(B_{R}(0))$ $\displaystyle u_{m}\rightarrow u_{0}$
$\displaystyle\mbox{ strongly in }W^{1,2}_{0}(B_{R}(0))$ $\displaystyle
u_{m}\rightarrow u_{0}$ $\displaystyle\mbox{ uniformely on }B_{R}(0).$

Here $n=2,3$ and the Sobolev embedding has been applied. Since $u_{m}\geq 0$,
the pointwise convergence yields $u_{0}\geq 0$ almost everywhere. Now we show,
that $u_{0}$ satisfies (1.3) on $\Omega_{f}$:
Let $\varphi\in C^{\infty}_{0}(\Omega_{f})$ be arbitrary. Then

$\operatorname{spt}(\varphi)\subset\Omega_{f}$

is a compact set. By Theorem 2.2 we find an $m_{0}\in\mathbb{N}$, such that
for all $m\geq m_{0}$ we have

$\operatorname{spt}(\varphi)\subset\Omega_{m}.$

Since the $u_{m}$ are solutions to the equation, we therefore have for these
$m$

$0=\int_{B_{R}(0)}\Delta u_{m}\Delta\varphi+\gamma\nabla
u_{m}\nabla\varphi-f\varphi.$

Then the weak convergence in $W^{2,2}_{0}(B_{R}(0))$ yields

$\displaystyle 0=$ $\displaystyle\int_{B_{R}(0)}\Delta
u_{0}\Delta\varphi+\gamma\nabla u_{0}\nabla\varphi-f\varphi$ $\displaystyle=$
$\displaystyle\int_{\Omega_{f}}\Delta u_{0}\Delta\varphi+\gamma\nabla
u_{0}\nabla\varphi-f\varphi,$

hence $u_{0}$ satisfies (1.3) on $\Omega_{f}$. Now we have to show that
$u_{0}$ also satisfies the boundary values, i.e. $u_{0}\in
W^{2,2}_{0}(\Omega_{f})$. In a first step we show

$\operatorname{spt}(u_{0})\subset\overline{\Omega_{f}}.$

Let $x\in B_{R}(0)$ such that $u_{0}(x)>0$. Then we find an
$m_{0}(x)\in\mathbb{N}$ such that for all $m\geq m_{0}$ we have $u_{m}(x)>0$.
In this argument the uniform convergence in the form of pointwise convergence
everywhere is central. Hence $x\in\Omega_{m}$. The Hausdorff convergence
yields for all $\varepsilon>0$ a $z_{m}\in\overline{\Omega_{f}}$ such that

$x\in B_{\varepsilon}(z_{m})$

for $m$ big enough. Since $\overline{\Omega_{f}}$ is compact, we find a
subsequence, such that $z_{m}\rightarrow
z_{\varepsilon}\in\overline{\Omega_{f}}$. Furthermore

$x\in\overline{B_{\varepsilon}(z_{\varepsilon})}.$

If we choose $\varepsilon:=\frac{1}{k}$ with $k\in\mathbb{N}$, we can again
employ the compactness of $\overline{\Omega_{f}}$, which yields after choosing
a subsequence $z_{\frac{1}{k}}\rightarrow z_{0}\in\overline{\Omega_{f}}$. By
$x\in\overline{B_{\frac{1}{k}}(z_{\frac{1}{k}})}$ we have

$x=z_{0}\in\overline{\Omega_{f}}.$

Since $x$ was choosen arbitrary, we have

$\operatorname{spt}(u_{0})\subseteq\overline{\Omega_{f}}.$

Now we can show $u_{0}\in W^{2,2}_{0}(\Omega_{f})$. Without loss of generality
we assume $0\in\Omega_{f}$. We define

$v_{\lambda}(x):=u_{0}(\lambda\cdot x).$

Since $\Omega_{f}$ is convex, we have for all $\lambda>1$

$\operatorname{spt}(v_{\lambda})\subsetneq\Omega_{f}.$

This yields $v_{\lambda}\in W^{2,2}_{0}(\Omega_{f})$. Furthermore

$\|v_{\lambda}\|_{W^{2,2}_{0}(\Omega_{f})}\leq\lambda^{2}\|u_{0}\|_{W^{2,2}_{0}(B_{R}(0))}.$

Hence for $\lambda\downarrow 1$ we find a weakly convergent subsequence, such
that

$v_{\lambda_{k}}\rightarrow g\in W^{2,2}_{0}(\Omega_{f}).$

Since the dimensions satisfy $n=2,3$, we can employ the Sobolev embedding
theorem and by continuity of $u_{0}$ obtain

$v_{\lambda_{k}}\rightarrow u_{0}\mbox{ pointwise everywhere.}$

Without a restriction to the dimension, this step could also be proven by
inner regularity theory, see e.g. [11, § 2.5]. Either way the limit $g$ is
always $u_{0}$, irrespective of the choice of subsequence. This yields

$v_{\lambda}\rightarrow u_{0}\mbox{ weakly in }W_{0}^{2,2}(\Omega_{f})\mbox{
for }\lambda\downarrow 1\mbox{ and }u_{0}\in W^{2,2}_{0}(\Omega_{f}).$

Hence $u_{0}\in W^{2,2}_{0}(\Omega_{f})$ is a weak solution of (1.2) w.r.t.
our choosen $\gamma$ and $u_{0}\geq 0$. Therefore
$\gamma_{f}(\Omega_{f})\leq\gamma.$ Since $\gamma>\inf_{\Omega\in
M}\gamma_{f}(\Omega)$ arbitrary, we have

$\gamma_{f}(\Omega_{f})\leq\inf_{\Omega\in
M}\gamma_{f}(\Omega)=\lim_{m\rightarrow\infty}\gamma_{f}(\Omega_{m}).$

Hence $\Omega_{f}$ is a minimiser in $M$.

As an addendum we can also show, that $\inf_{\Omega\in
M}\gamma_{f}(\Omega)\geq-\mu_{1}(\Omega_{f})$. Let $v_{1}\in
W^{2,2}_{0}(\Omega_{f})$ be such that

$\int_{\Omega_{f}}|\nabla v_{1}|\,=1\mbox{ and
}\mu_{1}(\Omega_{f})=\int_{\Omega_{f}}|\Delta v_{1}|^{2}\,.$

Then there exists a sequence $\varphi_{k}\in C^{\infty}_{0}(\Omega_{f})$, such
that

$\varphi_{k}\rightarrow v_{1}\mbox{ in }W^{2,2}_{0}(\Omega_{f}).$

Since the support of $\varphi_{k}$ is compact and in $\Omega_{f}$ we can apply
Theorem 2.2 and find for every $k\in\mathbb{N}$ an $m_{0}(k)\in\mathbb{N}$,
such that for all $m\geq m_{0}$

$\varphi_{k}\in C^{\infty}_{0}(\Omega_{m}).$

Hence

$\mu_{1}(\Omega_{f})=\lim_{k\rightarrow\infty}\frac{\int_{B_{R}(0)}|\Delta\varphi_{k}|^{2}}{\int_{B_{R}(0)}|\nabla\varphi_{k}|^{2}}\geq\limsup_{m\rightarrow\infty}\mu_{1}(\Omega_{m}).$

Therefore

$-\mu_{1}(\Omega_{f})\leq-\limsup_{m\rightarrow\infty}\mu_{1}(\Omega_{m})\leq\limsup_{m\rightarrow\infty}\gamma_{f}(\Omega_{m})=\inf_{\Omega\in
M}\gamma_{f}(\Omega)$

and the result follows. ∎

##  4 Open problems

In this section we discuss some open problems and possible further lines of
inquiry:

  * •

Numerics: The proof of [8, Thm. 1.1] is indirect, i.e. it yields no direct
control on $\gamma_{f}$. Hence providing a sound numerical ansatz for
calculating $\gamma_{f}$ seems out of reach at the moment. Finding such an
ansatz would be highly beneficial for further research though.

  * •

Interpolation: One possible way of attacking the numerical problem, would be
to show the following interpolation property: Given
$\Omega\subseteq\mathbb{R}^{n}$, $f>0$ and $\gamma_{1}<\gamma_{2}$ such that
the solution to (1.2) $u_{\gamma_{i}}$ is nonnegative ($i=1,2$). Then prove or
disprove that for all $\gamma\in]\gamma_{1},\gamma_{2}[$ the solution
$u_{\gamma}$ to (1.2) is also nonnegative.

  * •

Regularity: Completely open is the question of higher regularity of the
minimiser $\Omega_{f}$ of Thm. 1.1. Since $\Omega_{f}$ is convex, the boundary
is rectifiable but it is unclear if it is smooth. The answer will probably
depend on properties of $f$. The hard part is actually finding a good sense of
Euler-Lagrange equation for $\Omega_{f}$, if such a thing even exists. One
could possibly try to perform a shape derivative, but since it is unclear how
$\Omega\rightarrow\gamma_{f}(\Omega)$ behaves under small perturbations of
$\Omega$, this is not trivial. As a first pointer one may want to look at
small smooth perturbations of the ball, see e.g. [11, § 6.1.1] and references
therein.

  * •

Examples: It should be quite useful to be able to actually calculate some
explicit values for $\gamma_{f}$ for example for an ellipsoid and constant
$f$. In dimension one this has been done in [12]. In higher dimension there is
a useful result for the ball in [19, Thm. 1.4], although it is not readily
applicable, because the assumptions $c_{i}\leq 0$ in [19, Eq. 1.12] may be a
problem. Similarly [18, Prop. 2.1] is not directly applicable.

  * •

Symmetry breaking: The Boggio-Hadamards conjecture was substantiated by the
ball. If $f$ is radially symmetric, or even constant, one may wonder, if this
symmetry transfers over to a minimiser $\Omega_{f}$. If one chooses $f=0$,
then always $\gamma_{f}(\Omega)=-\mu_{1}(\Omega)$. One would therefore look
for a maximiser of the buckling load, but it is conjectured, that the
minimiser is the ball, see e.g. [11, §3.2]. Therefore the present author
expects a symmetry breaking phenomenon.

  * •

Convexity: The proof of Theorem 1.1 heavily relies on the domains $\Omega\in
M$ being convex. If we drop this assumption, it is unclear whether such a
minimiser still exists. On the other hand by [8, Thm 1.1]
$\Omega\rightarrow\gamma_{f}(\Omega)$ is still well defined, if
$\partial\Omega\in C^{4}$, even without $\Omega$ being convex. Here the hard
part seems to be to actually obtain a suitable control in some suitable
topology for a minimising sequence, which allows for a similar result like
Theorem 2.2.

##  Appendix A Positivity for large tension

By the following theorem $\gamma_{f}$ is well defined under some regularity
conditions.

######  Theorem A.1 (see Thm. 1.1 in [8]).

Let $\Omega\subset\subset\mathbb{R}^{n}$ be connected and open with
$\partial\Omega\in C^{4}$. Furthermore let $\tau>0$. Then there exists a
$\gamma_{0}=\gamma_{0}(\Omega,\tau)>0$ such that for all $f\in
L^{\infty}(\Omega)$ with $f\geq 0$ and

$\int f\,d\mathcal{L}^{n}\geq\tau\|f\|_{L^{\infty}(\Omega)}>0,$

we have that the solution $u_{\gamma}$ of

$\left\\{\begin{array}[]{cc}(-\Delta)^{2}u_{\gamma}-\gamma\Delta
u_{\gamma}=f,&\mbox{ in }\Omega\\\ u=\partial_{\nu}u=0,&\mbox{ on
}\partial\Omega\end{array}\right.$

satisfies

$u_{\gamma}>0\mbox{ in }\Omega\mbox{ for all }\gamma_{0}\leq\gamma<\infty.$

## References

  * [1] G. A. Beer.  Hausdorff metric and convergence in measure.  Michigan Math. J., 21(1):63–64, 1974.  https://doi.org/10.1307/mmj/1029001209.
  * [2] W. G. Bickley.  Deflexions and vibrations of a circular elastic plate under tension.  Phil. Mag., (7) 15:776–797, 1933.  https://doi.org/10.1080/14786443309462222.
  * [3] W. Blaschke.  Kreis und Kugel.  Leipzig:: Verlag von Veit & Comp., 1916.  https://doi.org/10.1515/9783111506937.
  * [4] T. Boggio.  Sulle funzioni di green d’ordine.  m. Rend. Circ. Mat. Palermo, 20:97–135, 1905.  https://doi.org/10.1007/BF03014033.
  * [5] D. Cassani and A. Tarsia.  Maximum principle for higher order operators in general domains.  Advances in Nonlinear Analysis, 11(1):655–671, 2021.  https://doi.org/10.1515/anona-2021-0210.
  * [6] A. Dall’Acqua and G. Sweers.  On domains for which the clamped plate system is positivity preserving.  Differential Equations and Inverse Problems, ed. by Carlos Conca, Raul Manasevich, Gunter Uhlmann and Michael Vogelius, AMS, 2004, 2004\.  https://doi.org/10.1090/conm/362/06609.
  * [7] R. J. Duffin.  On a question of Hadamard concerning super-biharmonic functions.  Journal of Mathematics and Physics, 27:253–258, 1948.  https://doi.org/10.1002/sapm1948271253.
  * [8] S. Eichmann and R. Schätzle.  Positivity for the clamped plate equation under high tension.  Annali di Matematica, 201:2001–2020, 2022.  https://doi.org/10.1007/s10231-022-01188-9.
  * [9] Herbert Federer.  Geometric Measure Theory.  Springer-Verlag Berlin, 1969.  https://doi.org/10.1007/978-3-642-62010-2.
  * [10] P. R. Garabedian.  A partial differential equation arising in conformal mapping.  Pacific J. Math, 1:485–524, 1951.
  * [11] F. Gazzola, H.-Chr. Grunau, and G. Sweers.  Polyharmonic Boundary Value Problems.  Springer, 2010.  Lecture Notes in Mathematics 1991 https://doi.org/10.1007/978-3-642-12245-3.
  * [12] H.-Chr. Grunau.  Positivity, change of sign and buckling eigenvalues in a one-dimensional fourth order model problem.  Adv. Differential Equations, 7(2):177–196, 2002.  https://doi.org/10.57262/ade/1356651850.
  * [13] H.-Chr. Grunau, G. Romani, and G. Sweers.  Differences between fundamental solutions of general higher order elliptic operators and of products of second order operators.  Math. Ann., 381:1031–1084, 2020.  https://doi.org/10.1007/s00208-020-02015-3.
  * [14] H.-Chr. Grunau and G. Sweers.  A clamped plate with a uniform weight may change sign.  Discrete & Continuous Dynamical Systems - S, 7(4):761–166, 2014\.  https://doi.org/10.3934/dcdss.2014.7.761.
  * [15] H.-Chr. Grunau and G. Sweers.  In any dimension a ”clamped plate” with a uniform weight may change sign.  Nonlinear Analysis A: T.M.A, 97:119–124, 2014.  https://doi.org/10.1016/j.na.2013.11.017.
  * [16] J. Hadamard.  Mémoire sur le problème d’analyse relatif à l’équibilibre des plaques élastiques encastrées.  Œuvres de Jaques Hadamard, Tome II, CNRS Paris, pages 515–641, 1968.  Reprint of: Mémoire présentés par divers savant a l’Académie des Sciences (2), 33:1-128, 1908.
  * [17] J. Hadamard.  Sur certains cas intéressants du problème biharmonique.  Œuvres de Jaques Hadamard, Tome III, CNRS Paris, pages 1297–1299, 1968.  Reprint of: Atti IV Congr. Intern. Mat. Rome 12-14, 1908.
  * [18] Hanen Ben Omrane and Saïma Khenissy.  Positivity preserving results for a biharmonic equation under Dirichlet boundary conditions.  Opuscula Math., 34(3):601–608, 2014.  http://dx.doi.org/10.7494/OpMath.2014.34.3.601.
  * [19] P. Laurençot and C. Walker.  Sign-preserving property for some fourth-order elliptic operators in one dimension or in radial symmetry.  JAMA, 127:69–89, 2015.  https://doi.org/10.1007/s11854-015-0024-2.
  * [20] W. Rudin.  Functional Analysis.  Mc Graw Hill, 1973.
  * [21] M.D. Willis.  Hausdorff Distance and Convex Sets.  Journal of Convex Analysis, 14(1):109–117, 2007.