# Evaluating the Smooth Control of Attribute Intensity
in Text Generation with LLMs

Shang Zhou∗, Feng Yao∗, Chengyu Dong$\dagger$, Zihan Wang, Jingbo
Shang$\dagger$
Department of Computer Science and Engineering,
University of California San Diego
<EMAIL_ADDRESS>
###### Abstract

Controlling the attribute intensity of text generation is crucial across
scenarios (e.g., writing conciseness, chatting emotion, and explanation
clarity). The remarkable capabilities of large language models (LLMs) have
revolutionized text generation, prompting us to explore such _smooth control_
of LLM generation. Specifically, we propose metrics to assess the range,
calibration, and consistency of the generated text’s attribute intensity in
response to varying control values, as well as its relevance to the intended
context. To quantify the attribute intensity and context relevance, we propose
an effective evaluation framework leveraging the Elo rating system and GPT4,
both renowned for their robust alignment with human judgment. We look into two
viable training-free methods for achieving smooth control of LLMs: (1)
Prompting with semantic shifters, and (2) Modifying internal model
representations. The evaluations of these two methods are conducted on $5$
different attributes with various models. Our code and dataset can be obtained
from https://github.com/ShangDataLab/Smooth-Control.

Evaluating the Smooth Control of Attribute Intensity
in Text Generation with LLMs

Shang Zhou∗, Feng Yao∗, Chengyu Dong$\dagger$, Zihan Wang, Jingbo
Shang$\dagger$ Department of Computer Science and Engineering, University of
California San Diego<EMAIL_ADDRESS>

$*$$*$footnotetext: Equal contribution. Listing order is
random.$\dagger$$\dagger$footnotetext: Corresponding authors.

##  1 Introduction

Figure 1: A demonstration for the _smooth control_ of the understandability
attribute in the concept explanation scenario, where the control values enable
the continuous adjustment of response professionalism, highlighting the
nuanced customization of communication.

Controllable text generation (CTG) for meeting certain constraints imposed by
the target applications and users is an important topic in natural language
generation. For example, it is often required to control sentiment Song et al.
(2019) or politeness Niu and Bansal (2018) in the task of dialogue response
generation. Controllable text generation becomes even more crucial as the
modern natural language generation system is becoming increasingly tailored to
individual preferences. For example, a dialogue response generator may need to
compose its answer to a question in a completely different way based on the
backgrounds of the user Wolf et al. (2019); Zheng et al. (2019); Liu et al.
(2020a); Song et al. (2021); Huang et al. (2022). Such personalized systems
can cultivate more engaging and efficient user interactions among a diverse
array of digital platforms and services.

In this paper, we aim to meet more fine-grained application requirements and
user preferences by focusing on a more refined controllable generation task,
dubbed _smoothly controllable text generation_ (SCTG). While a CTG task is to
ensure that the generated text satisfies desired attributes such as emotion or
writing style, an SCTG task targets at further ensure the intensity of such an
attribute can be modulated into multiple degrees per user’s preference. A
typical example is that while writing an email, one would adjust the degree of
formality according to the purpose and specific recipient of the email.
Another example is that when explaining a scientific concept, one would vary
the level of detail based on the knowledge background of the audience. In the
rest of the paper, we use _smooth control_ to denote a SCTG task for
simplicity.

Successful smooth control requires a response that not only contains proper
attribute intensity, but also adequately addresses the query regardless of the
attribute intensity it contains. We propose a framework with curated metrics
to evaluate the smooth control performance from both aspects. First, to
evaluate whether the attribute intensity is proper, we quantify the following
$2$ factors, including (1) calibration, namely the consistency between the
attribute intensity and the control value; and (2) variance, namely the
difference of the attribute intensity across different queries given the same
control value. Second, to evaluate whether the response is meaningful, we
quantify the relevance between the query and the generated response.

To conduct the above evaluation without humans in the loop, a prerequisite is
an automatic pipeline that can accurately estimate the intensity of an
attribute in the response. To this end, we leverage the state-of-the-art LLM
as a surrogate for humans, and the Elo rating system to ensure the LLM
evaluation is well aligned with human assessment. Specifically, among multiple
responses containing different intensities of one attribute, we select pairs
of two responses and query GPT-4 OpenAI (2023) to select the more intense one
in each pair. We then use an Elo rating algorithm to convert these comparative
results to absolute scores, which represent the attribute intensities of the
corresponding responses. To reduce the cost, we further renovate this pipeline
properly to ensure we can achieve accurate scores without the need to
exhaustively compare all pairs of responses.

Finally, as LLMs become increasingly popular as text generators in various
applications, we apply such an evaluation pipeline to explore their capability
of achieving smooth control. We investigate two approaches to achieve smooth
control with LLMs, including (1) prompting with semantic shifters that are
carefully curated for each attribute; and (2) representation engineering
(RepE) Zou et al. (2023), which locates and interpolates a $1$-dimensional
subspace corresponding to a specific attribute in LLM’s intermediate
representation. The latter approach requires access to the inference internals
of LLMs, but can potentially achieve much more fine-grained control of the
attribute intensity.

We conduct our evaluation on a wide variety of tasks, including (1)
controlling the intensity of emotions in casual chatting; (2) controlling the
degree of conciseness and formality in writing; and (3) controlling the amount
of details in concept explanation. We find that (1) Model sizes may negatively
affect the smooth performance. (2) Prompting is almost as good as, if
relatively better than repE.

Our contributions can be summarized as follows: (1) We formally define the
task of smooth control and propose a novel evaluation benchmark, consisting of
an accurate and efficient Elo-based rating system and a large-scale benchmark
dataset. (2) We comprehensively evaluate the smooth control capabilities of
prevailing LLMs through two training-free methods. The dataset we construct
and source code we use in the paper are publicly
released111https://github.com/ShangDataLab/Smooth-Control to facilitate the
research in this field.

##  2 Related Work

###  2.1 Controllable Text Generation

Our smooth control is based on attributed-based controlled text generation.
The goal of attribute-based CTG is to craft sentences that adhere to specific
characteristics, such as topic, sentiment, and keywords. Effectively managing
these sentence attributes is crucial for sophisticated writing tasks. By
manipulating multiple attributes simultaneously, it’s theoretically possible
to generate coherent and adjustable paragraphs or articles, making this an
area of keen interest in text generation research. Strategies to achieve CTG
include prompting, fine-tuning, retraining, or post-processing pre-trained
language models (PLMs) to create models tailored for CTG. Fine-tuning PLMs is
among the most straightforward methods for CTG, and one often only needs to
fine-tune specific model modules (Zeldes et al., 2020; Ribeiro et al., 2021;
Madotto et al., 2020) or model parameters (Li and Liang, 2021; Lester et al.,
2021; Yang et al., 2022). Reinforcement learning has also been widely employed
in CTG to explicitly learn from the signal of the existence of desired
attributes in the text (Ziegler et al., 2019; Liu et al., 2020b; Tambwekar et
al., 2018; Ribeiro et al., 2023). Another line of methods attempt to train a
conditional language model from scratch to further ensure the quality of CTG
(Khalifa et al., 2020; Zhang et al., 2020). Finally, with the increasing model
scale of PLMs, it is possible to achieve CTG without fine-tuning or
retraining. PPLM (Dathathri et al., 2019) trains an attribute discriminator
and then employs its gradient to drive the PLM to generate text leaning
towards the desired attribute. MEGATRON-CNTR (Xu et al., 2020) retrieves
relevant sentences from external knowledge bases as context to control PLM to
generate desired text. Attribute discriminators have also been used to control
the decoding process alone to increase the probability of tokens with desired
attributes (Krause et al., 2020). In this work, we focus on prompting and RepE
for smooth control as they require no training or fine-tuning of the model,
which is more feasible for downstream applications considering the scale of
LLMs.

###  2.2 Text Style Transfer

Smooth control is also related to text style transfer (TST) in text
generation. TST aims to automatically control the text style attributes while
preserving its content. Standard sequence-to-sequence modeling can be directly
applied to TST if parallel data in different styles are available (Rao and
Tetreault, 2018). For more realistic cases where such parallel data are not
available, it is possible to disentangle text into content and attribute in
the latent space, followed by generative modeling to generate text with
desired attributes (Hu et al., 2017; Shen et al., 2017). Other approaches
include prototype editing, which extracts a sentence template and manipulates
its attribute markers to generate the text with desired attributes (Li et al.,
2018), and pseudo-parallel corpus construction, which locates parallel
sentence pairs from two text corpora with different styles (Zhang et al.,
2018; Jin et al., 2019). TST is extensively utilized in downstream
applications such as persona-based dialog generation (Niu and Bansal, 2018;
Huang et al., 2018), stylistic summarization (Jin et al., 2020) and online
text debiasing (Pryzant et al., 2019; Ma et al., 2020).

##  3 Problem Formulation

In this section, we formally define _smooth control_ of the LLM-generated
text’s intensity of a certain attribute, and introduce the benchmark data we
construct for the evaluation of this task.

###  3.1 Definition of Smooth Control

Given an open-ended query, the objective of _smooth control_ is to achieve
refined manipulations over the intensity of a specified attribute in LLM-
generated text. Such control should extend to varying degrees, enabling
precise adjustments for aligning with specific requirements or preferences.

As depicted in Figure 1, for a given query $\mathcal{Q}$ that has non-fixed
answers, smooth control requires specifications on a particular attribute
$\mathcal{A}$ as well as a quantitative control value $cv$ to control a model
$M$ to generate a customized response $\mathcal{R}$. Ideally, the observed
intensity of $\mathcal{A}$ in $\mathcal{R}$ should correlate to $cv$. It can
be formally described as follows.

$\displaystyle\mathcal{R}=M(\mathcal{Q},\mathcal{A},cv),s.t.,\text{Intensity}(\mathcal{R},\mathcal{A})\propto
cv$

Based on the definition above, we emphasize three critical aspects for
investigating smooth control below. (1) Control Value. Control value $cv$
preferably assumes real values. But, the multitude of potential responses,
each with varying intensities of a specific $\mathcal{A}$, renders the
evaluation impossible. Besides, extremely nuanced preferences are uncommon.
Hence, we adopt 10 discrete degrees (0-9) to emulate ideal smooth control. (2)
Intensity Measurement. There is no standard for measuring the absolute
intensity of a certain attribute in the response, which is the key challenge
to evaluate smooth control. (3) Intensity-$\bf{cv}$ Correlation The
correlation between control value $cv$ and intensity of $\mathcal{A}$ in
$\mathcal{R}$ directly reflects the smooth control capability of a certain
method with a specific model.

To this end, we propose a novel automatic evaluation framework based on
pairwise comparison and calibration of attribute intensity. We provide a
detailed discussion on it in Section 4.

###  3.2 Benchmark Data Construction

Further to the definition of smooth control, query $\mathcal{Q}$, attribute
$\mathcal{A}$, and control value $cv$ are three key components of this task.
As mentioned above, the control value $cv$ has been finalized to 10 discrete
values. In this section, we introduce the selections of $\mathcal{Q}$ and
$\mathcal{A}$ for benchmark data construction.

As $\mathcal{Q}$ should be open-ended and meaningful when combined with a
given attribute $\mathcal{A}$, we begin with determining the attributes first.

#### Attribute Selection.

To the best of our knowledge and observations, attributes of the text in
common applications mainly encompass the following categories. (1) Sentiment:
It refers to the overall emotional tone conveyed by the text, such as anger
and happiness, which is valuable for human communication. (2) Style: This
covers various aspects of writing. The most common two are formality and
understandability (clarity) which are crucial to communication effectiveness.
(3) Linguistic Property: It reflects the structural and grammatical features
of the text. The most characteristic one is conciseness which ensures
efficiency in conveying information. We select the most common and practical
attributes for the evaluation, denoting them as Anger, Happiness, Formality,
Understandability, and Conciseness for easier reference.

#### Query Generation.

For the evaluation for smooth control, it is essential to ensure that the
selected queries can be validly responded to in various ways, particularly
when constrained by the given attribute. Given that the control value $cv$ has
10 possible discrete values, each query should elicit at least 10 different
answers, each with varying intensities of the given attribute. This can be
challenging for humans to manage effectively and efficiently. Therefore, for
each of the 5 attributes $\mathcal{A}$, we utilize GPT-4-turbo OpenAI (2023)
to generate $300$ queries, each could be answered by 10 possible responses
with different intensities in $\mathcal{A}$. The constructed dataset contains
$1,500$ queries in total, of which each has 14 tokens on average. The specific
prompt we use for GPT-4-turbo to generate such queries is provided in Appendix
A.1.

Finally, our constructed benchmark dataset for smooth control consists of
1,500 query sentences covering 5 different attributes. The evaluation aims to
be conducted based on the responses elicited by these queries, which we
discuss in Section 4.

Bin ID | Rating |  Example Sentence
---|---|---
0 | 860 |  Let’s work on this issue together.
1 | 1011 |  I’m neither for nor against the idea.
2 | 1162 |  We need to look at the bigger picture.
3 | 1289 |  I respect your opinion.
4 | 1431 |  Let’s take a step back and reassess.
5 | 1572 |  I’m quite upset about this.
6 | 1710 |  We’re not on the same page.
7 | 1858 |  I can’t agree with this at all.
8 | 1994 |  I’ve had enough of this nonsense!
9 | 2134 |  I won’t tolerate this madness!
Table 1: We bin sentences by their Elo rating using GPT-4 to annotate the
pairwise comparisons on the Anger attribute. For each bin of range 140 rating,
we calculate the average rating of the sentences in the bin, and present a
sentence near that rating in the bin. We present short examples here due to
the layout constraint. Longer examples can be found in Table 7.

##  4 Evaluating Smooth Control

We start with the introduction of our automatic rating system and then
introduce the metrics we design to measure the smooth control.

###  4.1 Rating System

We need an automatic way to estimate the degree of a sentence on a certain
attribute222Apart from the attribute Conciseness, since it can be easily
defined as the number of words in the sentence.. To achieve this, we leverage
an Elo rating system which was used in recent benchmarks Zheng et al. (2023).
In a nutshell, Elo models the ratings to reflect a probability of one instance
being preferred over the other, in our case, the probability of one sentence
having a higher degree than the other on an attribute. The ratings can be
calculated given pairwise comparison results of the sentences, such that for
any two sentences, the probability of preference would depend solely on the
absolute difference of the ratings. In our case, a rating difference of $100$
resembles a probability of preference of $0.64$, calculated according to the
definition of Elo rating333https://en.wikipedia.org/wiki/Elo_rating_system.

To automate the rating calculation, we leverage GPT-4 to annotate the sentence
pairs. The prompt template can be found in Appendix A.2.

###  4.2 Human evaluation of the rating system

We validate how well ratings calculated from GPT-4 annotations match with
human beliefs, by performing a qualitative study and a quantitative study.

For the qualitative study, we group sentences into bins based on the
calculated ratings, and present some sampled responses for Anger in Table 1.
For simplicity, the longer responses are shown in Table 7. We observe that
these bins correspond to different degrees of anger quite well.

For the quantitative study, we randomly sample sentence pairs (of difference
of ratings at a granularity level of 100 rating difference) and ask different
human annotators to label the preference (i.e., which sentence is of higher
intensity). We plot two curves in Figure 2, one indicating the percentage of
human preferences of the higher rated sentence at different rating
differences, and the other the Elo algorithm indicated win probability based
on the rating difference. We can observe that the two curves match closely
throughout a wide range of rating differences. As a comparison, a weaker LLM
annotator, gpt-3.5-turbo, would make mistakes during the annotations,
reflecting a worse-aligned curve to the Elo probabilities.

Figure 2: In our quantitative study, we determine the percentage of human
preference for pairs of sentences with varying Elo ratings, as assessed
through annotations by GPT-4 or GPT-3.5. Additionally, we present the
theoretical win probability as defined by the Elo rating algorithm.

###  4.3 Speed-up of Elo Calculations

Our study suggests that, for any group of sentences, we can use GPT-4 as a
reliable pair-wise annotator to obtain the corresponding Elo ratings. Usually,
one would need many pairwise comparisons per instance to estimate its rating
with good confidence. Here, we introduce the tricks we adopt to speed up the
calculation of the ratings.

  * •

We first construct a “library” of $300$ sentences sampled from the group. We
can spend an arbitrary calculation here, since it is only a small number of
sentences.

  * •

For other sentences, we calculate the ratings through _closest match_
comparisons on the library–pick pairwise comparisons of similar ratings to
annotate. This is contrary to a _random match_ of opponents by usual Elo
rating algorithms.

We compare the choice of this strategy by a synthetic experiment, where we
generate a uniform random list of ratings, and experiment with different
strategies to (re-)calculate their ratings:

  * •

No library, pair opponents with _random match_.

  * •

No library, pair opponents with _closest match_.

  * •

With library, pair opponents with _random match_.

  * •

With library, pair opponents with _closest match_.

As shown in Figure 3, we visualize the error rate on the ratings for the four
strategies as the number of comparisons per instance increases. For a fair
comparison, we ignored the accuracy of the library instances in calculating
the rating estimation errors. The results indicate that our proposed strategy
could require as few as one-third of the number of comparisons needed by other
methods to reach a similar error rate. Creating a library also makes it easy
to calculate ratings for new sentences.

Figure 3: Comparison of convergence speeds of four different strategies on
calculating the Elo ratings.

###  4.4 Metrics

We measure the quality of a method’s control on a certain attribute by using
the method to answer several questions conditioned on different control
values. We present 3 metrics based on the sentences generated by the method,
and their ratings calculated by our rating system.

Mean-MAE is a measurement of the error of the sentence ratings on the control
values. It is used to quantify the rating difference of the generated
sentences to an optimally controlled hypothetical. Through our human
inspection of different control values in the library, we have a range of
ratings that we wish to be controlled. This range is pre-defined for each
attribute prior to our evaluation. The expected rating for each control value
is therefore characterized by a linear interpolation of the minimum rating and
maximum rating of the value. The error is defined by the absolute difference
between the average rating of the sentences and the expected rating, then
averaged over all the control values. For a given list of $n$ average ratings
$r_{0},\ldots,r_{n-1}$ of sentences for each control value $c$, and the
maximum and minimum range $R_{\text{max}},R_{\text{min}},$ the Mean-MAE metric
can be written as

$\begin{split}\text{Mean-MAE}&=\sum_{c=0}^{n-1}|r_{c}-r^{*}_{c}|,\\\
\text{where,
}r^{*}_{c}&=R_{\text{min}}+\frac{c}{n-1}\times(R_{\text{max}}-R_{\text{min}}).\end{split}$

Mean-STD measures the variation of the sentence ratings on the control values.
A good smooth control method should be able to generate sentences of similar
ratings. As the name suggests, this metric is calculated by averaging the
standard deviations of ratings across different control values.

Relevance quantifies the utility of the responses in answering the questions.
A perfect smooth control method should not sacrifice the utility for a smaller
error or variation. Here, we employ GPT-4-turbo to judge the relevance between
a question and a response. The specific prompt we use is provided in Appendix
A.3.

##  5 Experiments Setup

Mean-MAE ($\downarrow$) Mean-STD ($\downarrow$) Relevance ($\uparrow$) Method
Attr. mistral llama llama llama gpt gpt mistral llama llama llama gpt gpt
mistral llama-7b llama llama gpt gpt 7b 7b 13b 70b 3.5 4 7b 7b 13b 70b 3.5 4
7b 7b 13b 70b 3.5 4 Prompt A $0.87$ $0.63$ $0.72$ $1.50$ $0.63$ $0.51$ $0.83$
$0.97$ $0.68$ $0.87$ $0.93$ $1.03$ $0.79$ $0.79$ $0.91$ $0.83$ $1.00$ $0.99$ H
$1.07$ $0.42$ $0.32$ $0.66$ $0.58$ $0.45$ $1.34$ $1.03$ $0.82$ $0.92$ $1.02$
$1.33$ $0.83$ $0.90$ $0.92$ $0.90$ $1.00$ $0.99$ F $1.33$ $1.08$ $1.24$ $1.21$
$1.14$ $0.73$ $1.47$ $1.19$ $0.90$ $1.10$ $1.06$ $1.07$ $0.75$ $0.97$ $1.00$
$0.93$ $0.96$ $0.99$ U $3.70$ $1.55$ $1.63$ $0.70$ $1.89$ $0.69$ $1.73$ $1.52$
$1.47$ $2.34$ $2.77$ $1.06$ $0.72$ $0.84$ $0.91$ $0.88$ $0.95$ $0.99$ C $1.30$
$1.11$ $1.30$ $0.73$ $2.36$ $0.79$ $3.59$ $3.62$ $2.68$ $4.94$ $1.62$ $3.41$
$0.76$ $0.95$ $0.93$ $0.90$ $0.98$ $1.00$ RepE A $1.94$ $1.33$ $1.32$ - - -
$0.87$ $1.22$ $1.13$ - - - $0.81$ $0.93$ $0.95$ - - - H $1.18$ $1.52$ $1.68$ -
- - $2.14$ $2.10$ $1.92$ - - - $0.80$ $0.87$ $0.93$ - - - F $1.86$ $2.06$
$1.90$ - - - $1.44$ $1.50$ $1.59$ - - - $0.93$ $0.82$ $0.94$ - - - U $2.08$
$2.29$ $0.58$ - - - $1.75$ $2.02$ $2.07$ - - - $0.97$ $0.88$ $0.72$ - - - C
$1.26$ $1.40$ $1.39$ - - - $0.92$ $0.76$ $0.90$ - - - $0.95$ $0.97$ $0.67$ - -
-

Table 2: Evaluation results after parameter selection for each model and
attribute. Here, ‘Attr.’ is short for ‘Attribute’, Mean-MAE denotes the
calibration error, standard deviation indicates the robustness of smooth
control, and relevance suggests if the generated response aligns with the
topic. Some values are marked as ‘-’ due to the constraints of accessing the
model parameters and the coefficient range.

In this section, we apply our proposed evaluation framework, along with the
constructed benchmark dataset, to assess the smooth control capability of
various modern LLMs through two viable training-free methods: (1) Prompting
with semantic shifters, and (2) Editing the internal model representations. We
first introduce the experiment settings and then present the results and
analyses.

###  5.1 Baseline Methods For Smooth Control

Prompting LLMs. The most straightforward method to smoothly control the LLM to
generate according to an attribute is to provide it with instruction on the
degree level required. To achieve this, we need one description
$\mathcal{D}_{\mathcal{A},cv}$ for each degree $cv$ of the attribute
$\mathcal{A}$:

$\displaystyle\mathcal{R}=M_{\text{prompt}}(\mathcal{Q},\mathcal{D}_{\mathcal{A},cv}),$

We call this prompting method parameterized by the descriptions we choose. We
consider two types of degree descriptions, first a list of semantic shifters
that can describe the intensity paired with the adjective of the attribute
(e.g., “a little bit angry” or “very angry”), and the second, a crafted list
of phrases that not necessary sticks with a format (e.g., “slightly relaxed”
or “extremely enraged”). The advantage of the first type is that they are
seemingly easy to apply directly to different attributes, while for the second
type, there is more flexibility in the descriptions. The exact descriptions we
use for each attribute and the prompt templates to use these descriptions are
in Appendix A.4 and A.7.

#### Representation Engineering (RepE).

Different from prompting, RepE Zou et al. (2023) is a top-down approach to
post-processing pretrained models via manipulating their internal
representations for understanding and controlling neural networks.

Specifically, it involves two distinct steps in particular. (1) Reading:
localizing the functional representations for a specific concept, which is
generally achieved by analyzing the neural activities after stimulating the
model with certain input prompts. The original stimulus prompts are manually
written by humans, which have limited scope and lack generalizability to
unseen concepts. In our experiments, we employ GPT-4 OpenAI (2023) to generate
those stimuli automatically and the prompt template is in Appendix A.6. (2)
Controlling. The extracted representations from the reading step are then
utilized as high-dimensional vectors to perturb the original model
representations to different extents indicated by a control strength, which
perfectly aligns with the concept of control value in our task. Therefore, we
specify the control strength for each control value of the attribute A. Such
manipulation of the internal representations is also parameterized by the
strength we indicate.

$\displaystyle\mathcal{R}=M_{\text{RepE}}(\mathcal{Q},\text{Strength}_{\mathcal{A},cv}),$

###  5.2 Parameter Selection

It is not immediately clear whether the human-interpreted degree descriptions
for prompting or the human-selected degree intensities in RepE transfer to a
smooth degree control for the LLM. Therefore, we consider a “parameter
selection” process for these two methods for calibration of the degrees.
Specifically, we proactively consider a larger number of degree parameters
(descriptions for prompting or strength for RepE), and obtain generations of
the LLM based on the parameter through a held-out set of questions. Then, we
select the sequence of parameters that leads to the best overall metric, which
is defined and calculated as:

$\displaystyle\text{Metric}=\frac{\text{Mean-MAE}+\text{Mean-
STD}}{(R_{\text{max}}-R_{\text{min}})*\text{Relevance}}$

This metric is designed to determine a better set of generations from a
specific smooth control method. The breakdown and intuitive explanations of
this formula are as follows. (1) The nominator is the sum of the two
aforementioned rating errors. A high Mean-MAE indicates misalignment with
rating scales, while a high Mean-STD indicates unstable, varied ratings. To
keep both values reasonable, we add rather than multiply them, as they share
the same scale. Empirical evaluation shows that an unweighted average performs
nearly best based on human inspection. The corresponding statistics are
exhibited in Appendix B. (2) The denominator is the multiplication of the
normalization term and the relevance penalty factor. A low relevance score is
undesirable, so we use its reciprocal to heavily penalize low-relevance
generations.

The selection can be done efficiently by brute-force enumeration when the
number of the total considered parameters is not too large and the specific
number in our case is 20.

###  5.3 Experiment Settings

The evaluations are conducted on diverse LLMs for the smooth control of
specific attributes. As such, we present the models, attributes, and datasets
that are utilized in the experiments here.

#### Models.

We employ both open-source and closed-source LLMs for our experiments.
Specifically, we adopt Mistral (Jiang et al., 2023) and LLaMA2 (Touvron et
al., 2023) at different scales for the experiments of editing the internal
model representations, as it requires access to the model parameters. For
prompting with semantic shifters, we further utilize GPT-3.5 (OpenAI, 2022)
and GPT-4 (OpenAI, 2023) models.

#### Attribute.

As explained in Section 3.2, we select Anger, Happiness, Formality,
Understandability, and Conciseness as the attributes to evaluate. In
particular, the intensity of Conciseness is measured differently than other
attributes by directly counting the number of words in the responses.

#### Dataset.

We adopt the constructed benchmark dataset introduced in Section 3.2, which
consists of $1,500$ query sentences in total, with $500$ for each of the
aforementioned 5 attributes.

#### Metric.

According to our evaluation framework introduced in Section 4, we adopt mean-
MAE, standard deviation, and relevance as the main metrics.

##  6 Experiment results

Figure 4: Comparisons between prompting with universal and selected semantic
shifters. The Y axis is the attribute intensity. The black dashed lines are
the ideal correlation between the control value and the attribute intensity.

###  6.1 Main Results

Table 2 shows the smooth control performance achieved by different models with
different methods, on several attributes. One can observe that GPT-4 is
significantly better than other models for all attributes, especially in terms
of Mean-MAE, namely the consistency between the control values and the
obtained attribute intensities. GPT-4 is also significantly better than other
models in terms of the relevance between the model’s response and the query,
despite the potential cause being that GPT-4 is also used to evaluate such
relevance.

Interestingly, we observe that model sizes may negatively affect the smooth
performance. A relatively fair test bed for this is the Llama family, where
one can observe that for most attributes, Mean-MAE decreases constantly as the
model size increases from 7B, 13B, to 70B.

Finally, we also observe that prompting is almost as good as, if relatively
better than repE. This implies that prompting is preferred in realistic
applications of smooth control since it requires no access to the internal
model representations and thus can be potentially applied to more LLMs.

Mean-MAE ($\downarrow$) Mean-STD ($\downarrow$) Relevance ($\uparrow$) mistral
llama llama llama gpt gpt mistral llama llama llama gpt gpt mistral llama-7b
llama llama gpt gpt 7b 7b 13b 70b 3.5 4 7b 7b 13b 70b 3.5 4 7b 7b 13b 70b 3.5
4 A $1.94$ $1.08$ $0.84$ $1.49$ $0.86$ $0.85$ $1.17$ $1.10$ $1.20$ $1.67$
$1.23$ $1.47$ $0.78$ $0.75$ $0.86$ $0.72$ $0.92$ $0.99$ H $1.26$ $0.51$ $0.50$
$0.96$ $0.77$ $1.25$ $1.48$ $1.25$ $1.23$ $1.23$ $1.42$ $1.60$ $0.81$ $0.87$
$0.94$ $0.87$ $1.00$ $0.99$ F $1.46$ $1.48$ $1.54$ $1.50$ $1.36$ $1.29$ $1.19$
$1.23$ $1.03$ $1.23$ $1.16$ $1.12$ $0.75$ $0.95$ $0.99$ $0.91$ $0.96$ $0.99$ U
$3.35$ $1.65$ $1.70$ $2.68$ $4.03$ $2.30$ $1.67$ $1.56$ $1.63$ $2.08$ $2.09$
$1.12$ $0.45$ $0.81$ $0.83$ $0.91$ $0.86$ $0.93$ C $1.60$ $1.08$ $1.30$ $0.90$
$3.19$ $1.63$ $2.66$ $2.71$ $1.98$ $2.43$ $1.36$ $2.13$ $0.77$ $0.92$ $0.93$
$0.91$ $0.89$ $1.00$

Table 3:  Prompting with intensity descriptors that are not specific to
models.

###  6.2 Specificity of Parameter Selection in Intensity Calibration

In this section, we wish to explore whether the intensity descriptors we
selected for each attribute and each model are specific to the model or the
attribute. Here we mainly conduct the investigation based on prompting since
prompting is more preferred than RepE based on the above results.

extremely not | a little bit
---|---
very not | somewhat
moderately not | moderately
somewhat not | very
a little bit not | extremely
Table 4: Universal Semantic Shifters

#### Should intensity descriptors be specific to attribute?

We validate whether it is possible to use a universal set of descriptors to
control the intensities of all attributes listed. If possible, such a set can
greatly ease the implementation of smooth control of LLMs and reduce the
inference cost for selecting specific descriptors of each attribute.

To this end, we experiment with using a set of fixed semantic shifters to
modulate the intensity of an attribute in prompting. In specific, we prompt
GPT-4 multiple times to generate $30$ different adverbs of degrees that are
commonly used. We then select $10$ that appear the most frequently in
responses, which are shown in Table 4.

Figure 4 and Table 5 show the results of using fixed semantic shifters for
prompting LLMs. We observe that such fixed semantic shifters achieve
significantly worse performance in smooth control, especially in terms of
Mean-MAE. This means fixed semantic shifters cannot properly control the
attribute to match the desired intensities.

#### Are intensity descriptors specific to model?

Among our experiment results, we found that the intensity descriptors selected
to achieve the best smooth control performance vary significantly across
models. According to our observation of the results for attribute “Formality”,
to achieve an intensity level of $3$, GPT-4 would prefer “Highly
Inappropriate” in its prompt. In contrast, Llama2-70b would prefer “Neural” to
achieve the same intensity level. Further, we found that the intensity
descriptors preferred by different models may not even be consistent in terms
of order.

Mean-MAE Mean-STD Relevance Attri gpt-4 llama-70b gpt-4 llama-70b gpt-4
llama-70B A $1.00$ $2.02$ $1.27$ $1.45$ $0.98$ $0.77$ H $1.76$ $2.29$ $1.31$
$1.18$ $0.97$ $0.83$ F $1.75$ $2.3$ $1.03$ $1.06$ $0.99$ $0.95$ U $2.01$
$1.84$ $1.64$ $2.07$ $1.00$ $0.83$ C $7.17$ $3.07$ $3.81$ $4.61$ $1.00$ $0.86$

Table 5: Baseline with fixed semantic shifters.

Since the best intensity descriptors are not specific to the model, one cannot
simply transfer the intensity descriptors selected for one model to another
model. We conduct an additional experiment to demonstrate this. In Table 3,
for each model, we use all models except this model to select the intensity
descriptors. One may observe that these intensity descriptors transferred from
other models cause significantly worse smooth control performance, especially
in terms of Mean-MAE. This shows that it is necessary to select intensity
descriptors specific to each model to properly control the attribute to match
the desired intensity.

##  7 Conclusions and Future Work

This work studies smoothly controllable text generation for large language
models. We created an evaluation system on five different attributes to
evaluate smooth control methods on different intensity levels for three
metrics: error, variation of the generated sentence’s intensities and the
relevance to the generation questions. The system is implemented based on Elo
ratings, automatically evaluating using LLMs, and designed to be efficient in
evaluation. We evaluate two representative methods, prompting and
representation engineering. We find that (1) Model sizes may negatively affect
the smooth performance. (2) Prompting is almost as good as, if relatively
better than repE.

## Limitations

Our work presents an evaluation of smooth control methods for LLM generations.
There are several limitations that we have considered:

  * •

We used GPT-4 as an automatic evaluator in building our evaluation system,
mainly for reducing human effort. While we have verified its closeness with
human preference on all the 5 attributes we considered, we admit that our
system will suffer from the same limitations of using LLM as annotators, such
as not being robust to certain (manually crafted) sentences, and not being a
free service to use, especially that we find not as competent LLMs (e.g.,
GPT-3.5) do not have a similar strong annotation power.

  * •

We mainly evaluated two training-free methods, Prompting and Representation
Engineering for their soft control ability, due to their simplicity and
representativeness. Other soft control methods, including some that require
model fine-tuning, could be evaluated in future work.

## Acknowledgement

We thank the anonymous reviewers for their insightful comments. Our work is
sponsored in part by NSF CAREER Award 2239440, NSF Proto-OKN Award 2333790, as
well as generous gifts from Google, Adobe, and Teradata. Any opinions,
findings, conclusions, or recommendations expressed herein are those of the
authors and should not be interpreted as necessarily representing the views,
either expressed or implied, of the U.S. Government. The U.S. Government is
authorized to reproduce and distribute reprints for government purposes
notwithstanding any copyright annotation hereon.

## References

  * Dathathri et al. (2019) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019.  Plug and play language models: A simple approach to controlled text generation.  _ArXiv_ , abs/1912.02164.
  * Hu et al. (2017) Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017.  Toward controlled generation of text.  In _International Conference on Machine Learning_.
  * Huang et al. (2018) Chenyang Huang, Osmar R Zaiane, Amine Trabelsi, and Nouha Dziri. 2018.  Automatic dialogue generation with expressed emotions.  In _North American Chapter of the Association for Computational Linguistics_.
  * Huang et al. (2022) Qiushi Huang, Yu Zhang, Tom Ko, Xubo Liu, Boyong Wu, Wenwu Wang, and Lilian Tang. 2022.  Personalized dialogue generation with persona-adaptive attention.  _ArXiv_ , abs/2210.15088.
  * Jiang et al. (2023) Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023.  Mistral 7b.  _ArXiv preprint_ , abs/2310.06825.
  * Jin et al. (2020) Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Peter Szolovits. 2020.  Hooks in the headline: Learning to generate headlines with controlled styles.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Jin et al. (2019) Zhijing Jin, Di Jin, Jonas W. Mueller, Nicholas Matthews, and Enrico Santus. 2019.  Imat: Unsupervised text attribute transfer via iterative matching and translation.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Khalifa et al. (2020) Muhammad Khalifa, Hady ElSahar, and Marc Dymetman. 2020.  A distributional approach to controlled text generation.  _ArXiv_ , abs/2012.11635.
  * Krause et al. (2020) Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq R. Joty, Richard Socher, and Nazneen Rajani. 2020.  Gedi: Generative discriminator guided sequence generation.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.  The power of scale for parameter-efficient prompt tuning.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Li et al. (2018) Juncen Li, Robin Jia, He He, and Percy Liang. 2018.  Delete, retrieve, generate: a simple approach to sentiment and style transfer.  In _North American Chapter of the Association for Computational Linguistics_.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , abs/2101.00190.
  * Liu et al. (2020a) Qian Liu, Yihong Chen, B. Chen, Jian-Guang Lou, Zixuan Chen, Bin Zhou, and Dongmei Zhang. 2020a.  You impress me: Dialogue generation via mutual persona perception.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Liu et al. (2020b) Ruibo Liu, Guangxuan Xu, Chenyan Jia, Weicheng Ma, Lili Wang, and Soroush Vosoughi. 2020b.  Data boost: Text data augmentation through reinforcement learning guided conditional generation.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Ma et al. (2020) Xinyao Ma, Maarten Sap, Hannah Rashkin, and Yejin Choi. 2020.  Powertransformer: Unsupervised controllable revision for biased language correction.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Madotto et al. (2020) Andrea Madotto, Zhaojiang Lin, Yejin Bang, and Pascale Fung. 2020.  The adapter-bot: All-in-one controllable conversational model.  In _AAAI Conference on Artificial Intelligence_.
  * Niu and Bansal (2018) Tong Niu and Mohit Bansal. 2018.  Polite dialogue generation without parallel data.  _Transactions of the Association for Computational Linguistics_ , 6:373–389.
  * OpenAI (2022) OpenAI. 2022.  Introducing chatgpt.  https://openai.com/blog/chatgpt.
  * OpenAI (2023) OpenAI. 2023.  Gpt-4 technical report.
  * Pryzant et al. (2019) Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2019.  Automatically neutralizing subjective bias in text.  _ArXiv_ , abs/1911.09709.
  * Rao and Tetreault (2018) Sudha Rao and Joel R. Tetreault. 2018.  Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer.  In _North American Chapter of the Association for Computational Linguistics_.
  * Ribeiro et al. (2023) Leonardo F. R. Ribeiro, Mohit Bansal, and Markus Dreyer. 2023.  Generating summaries with controllable readability levels.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Ribeiro et al. (2021) Leonardo F. R. Ribeiro, Yue Zhang, and Iryna Gurevych. 2021.  Structural adapters in pretrained language models for amr-to-text generation.  _ArXiv_ , abs/2103.09120.
  * Shen et al. (2017) Tianxiao Shen, Tao Lei, Regina Barzilay, and T. Jaakkola. 2017.  Style transfer from non-parallel text by cross-alignment.  _ArXiv_ , abs/1705.09655.
  * Song et al. (2021) Haoyu Song, Yan Wang, Kaiyan Zhang, Weinan Zhang, and Ting Liu. 2021.  Bob: Bert over bert for training persona-based dialogue models from limited personalized data.  _ArXiv_ , abs/2106.06169.
  * Song et al. (2019) Zhenqiao Song, Xiaoqing Zheng, Lu Liu, Mu Xu, and Xuanjing Huang. 2019.  Generating responses with a specific emotion in dialog.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Tambwekar et al. (2018) Pradyumna Tambwekar, Murtaza Dhuliawala, Lara J. Martin, Animesh Mehta, Brent Harrison, and Mark O. Riedl. 2018.  Controllable neural story plot generation via reward shaping.  In _International Joint Conference on Artificial Intelligence_.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _ArXiv preprint_ , abs/2307.09288.
  * Wolf et al. (2019) Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019.  Transfertransfo: A transfer learning approach for neural network based conversational agents.  _ArXiv_ , abs/1901.08149.
  * Xu et al. (2020) Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, and Bryan Catanzaro. 2020.  Controllable story generation with external knowledge using large-scale language models.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Yang et al. (2022) Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, and Jun Xie. 2022.  Tailor: A prompt-based approach to attribute-based controlled text generation.  _ArXiv_ , abs/2204.13362.
  * Zeldes et al. (2020) Yoel Zeldes, Dan Padnos, Or Sharir, and Barak Peleg. 2020.  Technical report: Auxiliary tuning and its application to conditional text generation.  _ArXiv_ , abs/2006.16823.
  * Zhang et al. (2020) Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, and Bill Dolan. 2020.  Pointer: Constrained progressive text generation via insertion-based generative pre-training.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Zhang et al. (2018) Zhirui Zhang, Shuo Ren, Shujie Liu, Jianyong Wang, Peng Chen, Mu Li, Ming Zhou, and Enhong Chen. 2018.  Style transfer as unsupervised machine translation.  _ArXiv_ , abs/1808.07894.
  * Zheng et al. (2023) Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023.  Judging llm-as-a-judge with mt-bench and chatbot arena.
  * Zheng et al. (2019) Yinhe Zheng, Rongsheng Zhang, Xiaoxi Mao, and Minlie Huang. 2019.  A pre-training based personalized dialogue generation model with persona-sparse data.  In _AAAI Conference on Artificial Intelligence_.
  * Ziegler et al. (2019) Daniel M. Ziegler, Nisan Stiennon, Jeff Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019.  Fine-tuning language models from human preferences.  _ArXiv_ , abs/1909.08593.
  * Zou et al. (2023) Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. 2023.  Representation engineering: A top-down approach to ai transparency.  _arXiv preprint arXiv:2310.01405_.

## Appendices

##  Appendix A Prompt Templates

We list all prompt templates we used in this paper.

###  A.1 Question Generation

Our dataset consists of questions that potentially can be answered with
different degrees of attributes. The template to generate the questions is

⬇

Generate 10 prompts that can be answered with varying degrees of <concept>.

###  A.2 Pairwise Annotation

This template is used to compare two responses to decide which shows a greater
degree of the concept.

⬇

For each pair of responses, identify which response expresses more <concept>.
Write the pair number followed by ’1’ if the first response is more <concept>,
or ’2’ if the second response is more <concept>. Format your response like
this: ’1. 1’, ’2. 2’, etc.

###  A.3 Relevance Annotation

This template is used to Judge if a response is relevant (1) or not (0) to the
query.

⬇

Given the following query and response, please assess whether the response is
relevant to the query. Answer with ’1’ if the response is relevant, and ’0’ if
it is not relevant.

###  A.4 Prompting with Degree Descriptions

This template is used to respond to queries with a specified emotional tone or
style.

⬇

Please respond to {{queries[i]}} with a paragraph in a [tone | style] that is {{semantic shifter}}. The response should be three sentences long.

###  A.5 Generating Degree Descriptions

This template is used to identify words or phrases that can shift the meaning
of a concept, either intensifying or diminishing its strength.

⬇

Describing <concept> levels on a scale from -9 to 10 using phrases.

###  A.6 Stimulus Prompts Generation.

⬇

Generate 10 prompts that can stimulate <concept>.\end{lstlisting}

###  A.7 Candidates for Semantic Shifters

##  Appendix B Parameter Selection Analysis

We considered different sets of $\alpha$ (from 0 to 1) for the weighted
average of Mean-MAE and Mean-STD to calculate the overall metric.

$\alpha\times\text{Mean-MAE}+(1-\alpha)\times\text{Mean-STD}$ (1)

For each weight factor $\alpha$, we considered pairs of error bar plots of the
average and standard deviation values, and asked humans to judge which plot is
better as shown in Figure 5. We compare the human evaluation result with the
result that our metric provides, and record the percentage of alignment. As
shown in Figure 6, alignment follows a bell curve, peaking at 0.87 when
$\alpha$ is 0.5-ish. Therefore, we directly adopt the vanilla average of these
two rating errors rather than the weighted ones.

Figure 5: Examples for human evaluation. Figure 6: Alignment with humans for
different weight factors.

##  Appendix C Generated Data Examples

Attribute |  Query
---|---
Anger |  Why did you use my personal items without asking for permission?
|  What exactly is causing the continuous delays on the construction project
you assured would be finished by now?
|  What possessed you to share our private conversations with others?
|  What makes you think it’s okay to leave such a mess in the kitchen every
time you cook?
|  Why do you consistently interrupt me when I’m trying to have a serious
conversation?
Happiness |  Can you share a story of a time when you overcame a challenge and felt great contentment?
|  What activity do you enjoy doing that always lifts your spirits?
|  What is your favorite memory from childhood that still makes you smile?
|  What was your reaction when you received the gift you’ve always wanted?
|  Imagine your perfect day; what does it look like and how does it make you
feel?
Formality |  Could you please tell me about the latest developments in artificial intelligence?
|  Describe the process of photosynthesis in plants.
|  What steps should I take to prepare for a job interview?
|  How does one go about making a traditional Italian pizza from scratch?
|  Describe the impact of social media on interpersonal communication in
today’s society.
Understandability |  Illustrate the impact of climate change on polar bear habitats.
|  How does a computer process information?
|  Assess the philosophical implications of artificial intelligence in
society.
|  Outline the basic tenets of existentialist philosophy.
|  What are black holes, and can they affect our planet?
Conciseness |  What steps would one take to secure their home Wi-Fi network?
|  Elucidate the reasons behind the diversity of languages spoken around the
world.
|  Can you provide the steps involved in solving a Rubik’s Cube?
|  Describe the roles enzymes play in the human digestive system.
|  Explain the theory of relativity and its implications for physics.
Table 6: Benchmark data examples generated by GPT-4 Bin ID | Rating |  Example Sentence
---|---|---
0 | 871 |  I hold a deep belief in the natural flow of truth, as it always finds a way into the open, much like a gentle stream that carves its path through the landscape. In taking action, my intention was to nurture understanding without disruption, trusting that, with time, our paths would align in clarity and harmony. I welcome this moment for reflection and connection, serene in the knowledge that openness will guide us toward mutual tranquility.
1 | 1027 |  I appreciate your patience and understanding, as time often seems to flow like a gentle stream, unfettered by our human constructs, leading me to drift subtly off schedule. Each moment unfolds with its own unique rhythm and sometimes that melody harmonizes with the clock differently than intended. Rest assured, my intentions are to honor our appointments just as I honor each serene pulse of the present, and I am taking measures to align my peace with punctuality.
2 | 1147 |  I deeply appreciate your patience and understanding as we address the timing concerns. My tardiness has been primarily a result of unexpected complications that arise despite my initial planning and efforts to arrive on time, which I’m earnestly working to overcome. Rest assured, I value our meetings and am implementing new strategies to ensure that I honor our appointments punctually moving forward.
3 | 1302 |  Honestly, I didn’t realize I had left the kitchen as such a disaster; clearly, cooking got ahead of me this time. Nevertheless, it’s frustrating to hear that my oversight has caused inconvenience since I usually tidy up after myself. I’ll address the mess immediately, as I certainly didn’t mean to add any stress to our day.
4 | 1441 |  I assure you that I am fully aware of the importance of taking responsibility for my own actions. Your implication that I habitually push blame onto others is neither fair nor accurate. However, I’ll reflect on this feedback and commit to being more mindful of how I address issues in the future.
5 | 1567 |  Frankly, I can’t fathom what was going through the mind of the person who brazenly spoiled the movie’s ending for everyone. It’s basic movie-watching etiquette to keep plot twists to oneself, especially in a communal space where the anticipation is part of the communal experience. Some courtesy would be appreciated, to not ruin the suspense we’ve all been patiently waiting to enjoy together.
6 | 1694 |  I am growing increasingly frustrated by the lack of updates regarding the refund that was due two weeks ago. Your inability to process it in the promised timeframe is both inconvenient and unacceptable. I need a clear explanation for this delay and an immediate resolution to ensure I receive my refund forthwith.
7 | 1873 |  Seriously, the nerve of some people cutting in line as if the concept of waiting their turn simply evaporates when it comes to them! It’s a blatant disregard for common courtesy and the unspoken social contract we all agree to when joining a queue. Their sense of entitlement is astounding and a slap in the face to everyone who respects the order of things.
8 | 1995 |  I’ve had enough of constantly being painted as the one who avoids accountability! Frankly, it’s exhausting and hypocritical for you to suggest I haven’t faced my own faults when you’ve hardly glanced at your own missteps. It’s high time for a reality check on both ends because I refuse to be the scapegoat for problems I haven’t caused alone!
9 | 2168 |  Absolutely unbelievable, isn’t it? I’m constantly stuck picking up the pieces after your careless blunders, pouring my energy into fixing what should never have been an issue in the first place! I won’t stand for this any longer; it’s about time you step up and take responsibility for your own actions rather than expecting me to clean up your incessant disasters!
Table 7: Responses with different intensities in the attribute of anger. Anger |  Happiness |  Formality |  Understandability |  Conciseness
---|---|---|---|---
serenely peaceful |  despair |  extremely disrespectful |  completely unintelligible |  extremely redundant
deeply relaxed |  miserable |  highly inappropriate |  extremely confusing |  highly redundant
very calm |  very unhappy |  very casual |  very hard to understand |  very redundant
quite tranquil |  unhappy |  informal |  quite difficult to understand |  quite redundant
mildly peaceful |  slightly unhappy |  casual |  challenging to understand |  moderately redundant
slightly relaxed |  neutral/negative |  slightly casual |  hard to follow |  slightly redundant
neutral, neither calm nor angry |  neutral |  neutral |  somewhat unclear |  marginally wordy
slightly irritated |  neutral/positive |  slightly formal |  slightly unclear |  somewhat wordy
mildly annoyed |  slightly happy |  moderately formal |  almost clear |  mildly wordy
neutral, balanced emotion |  content |  formal |  neutral |  neutral
slightly upset |  satisfied |  very formal |  fairly easy to understand |  mildly concise
moderately annoyed |  cheerful |  highly formal |  clear |  somewhat concise
fairly irritated |  happy |  ceremonial |  very clear |  moderately concise
quite angry |  very happy |  old-fashioned formal |  extremely clear |  quite concise
very angry |  joyful |  courtly |  crystal clear |  very concise
intensely furious |  elated |  aristocratic |  intuitively understandable |  highly concise
extremely enraged |  overjoyed |  regal |  effortlessly understandable |  extremely concise
seething with rage |  ecstatic |  imperial |  instantly understandable |  terse
nearly uncontrollable anger |  blissful |  divine |  universally understandable |  overly terse
utterly livid, maximum anger |  nirvana |  transcendent |  absolute clarity |  cryptic
Table 8: Candidates for Semantic Shifters


#  Federated Learning for Chronic Obstructive Pulmonary Disease Classification
with Partial Personalized Attention Mechanism ††thanks: *Corresponding Author.
Email<EMAIL_ADDRESS>
Yiqing Shen1,2, Baiyun Liu2, Ruize Yu2, Yudong Wang2, Shaokang Wang2, Jiangfen
Wu2, Weidao Chen2* 1Johns Hopkins University, MD, USA 2Beijing Infervision
Technology Co., Ltd., Beijing, China

###### Abstract

Chronic Obstructive Pulmonary Disease (COPD) is the fourth leading cause of
death worldwide. Yet, COPD diagnosis heavily relies on spirometric examination
as well as functional airway limitation, which may cause a considerable
portion of COPD patients underdiagnosed especially at the early stage. Recent
advance in deep learning (DL) has shown their promising potential in COPD
identification from CT images. However, with heterogeneous syndromes and
distinct phenotypes, DL models trained with CTs from one data center fail to
generalize on images from another center. Due to privacy regularizations, a
collaboration of distributed CT images into one centralized center is not
feasible. Federated learning (FL) approaches enable us to train with
distributed private data. Yet, routine FL solutions suffer from performance
degradation in the case where COPD CTs are not independent and identically
distributed (Non-IID). To address this issue, we propose a novel personalized
federated learning (PFL) method based on vision transformer (ViT) for
distributed and heterogeneous COPD CTs. To be more specific, we partially
personalize some heads in multiheaded self-attention layers to learn the
personalized attention for local data and retain the other heads shared to
extract the common attention. To the best of our knowledge, this is the first
proposal of a PFL framework specifically for ViT to identify COPD. Our
evaluation of a dataset set curated from six medical centers shows our method
outperforms the PFL approaches for convolutional neural networks.

###### Index Terms:

Personalized federated learning, Vision transformer, Chronic obstructive
pulmonary disease

##  I Introduction

Chronic Obstructive Pulmonary Disease (COPD) is a prevalent chronic pulmonary
condition that remains a major cause of morbidity and mortality worldwide [7,
21]. Present-day spirometric criteria often result in under-diagnosis and
under-report in routine exams as COPD patients can be functional intact at
their early state [19]. Previous studies have demonstrated the successful
application of deep learning (DL) models in the identification of COPD using
computed tomography (CT) images as inputs, particularly the convolutional
neural networks (CNNs) [8, 14]. These models were mostly trained and validated
on the large public database and the performance of which in the real-world
testing varied depending on the patient source and distribution. Therefore,
there is a clear need to generate a universally effective approach for COPD
diagnosis in real clinical settings.

It is well recognized that COPD is a heterogeneous syndrome and adopts
distinct phenotypes such as emphysema and airway loss, depending on the
genetic variants and inflammatory subtypes [5]. Thus, a rich source of data
covering these phenotypes is critical for understanding the underlying
mechanism of COPD heterogeneity. Yet, current privacy regulations e.g. [1, 20]
restrict a collaboration of the distributed data into a centralized center,
because the availability of such confidential data is not permitted. To narrow
this gap and mitigate the risk of privacy leakage, federated learning (FL)
[15], as a machine learning paradigm, is proposed, where deep learning models
are trained locally at the involved client without explicit data
communication. Specifically, the clients can keep their data private, and only
share weights and gradients for the model update. In this scenario, FL has
already shown empirical success in various tasks of medical imaging [17], e.g.
COVID-19 CT detection [16, 23], histology tissue subtype classification [11].

Data heterogeneity in COPD CTs, characterized by varying geographical
locations, patient distribution, image scanners, and diagnosis protocols,
usually hinders the successful deployment of the FL model. Concretely, vanilla
federate learning approaches are prone to suffer from performance degradation
with non-IID (not independently and identically distributed) data, primarily
caused by the weight divergence [24]. Subsequently, personalized federated
learning (PFL) is introduced to enable local models to learn local
personalized weights [12], where the global shared model is personalized for
each local client as an intermediate paradigm between pure local training and
FL. Model weights personalization has received notable attention [2, 13]. For
example, FedPer decouples a neural network architecture into global base
layers to be trained by FedAvg, and the top personalized layers to address the
label distribution skew [10]. LG-FedAvg is proposed by personalizing the
bottom layer to tackle the feature distribution skew [13]. Yet, these PFL
approaches are primarily restricted to convolutional neural network (CNN)
structures, without any specific design for Vision Transformer (ViT). In this
paper, we first develop a PFL model for ViT [6] because of its potential to
tackle the heterogeneity in medical images [4]. Innovatively, a partial
personalization scheme is proposed for the Multiheaded Self-Attention (MSA)
layers in ViT.

The major contributions are four-fold. First, we propose an FL model based on
ViT for COPD CT image classification. Second, to address the data
heterogeneity, a novel partial personalization for the multiheaded self-
attention layer in ViT is developed. To the best of our knowledge, this is the
first attempt at a PFL scheme for ViT. Third, to facilitate the PFL training,
we propose a consistent regularization between the global shared and local
private attention. Finally, experimental results on COPD data collected from
multiple medical centers yield the effectiveness of our approach in comparison
with the CNN-based PFL methods.

Figure 1: The semantic illustration of partial personalization scheme for
multiheaded self-attention layers in ViT. In MSA layers, we partially
personalize some heads to learn the attentions for local personalized
features, and the rest are remained shared globally to learn common
attentions. The personalized weights reside locally, and the shared weights
are aggregated globally.

##  II Methods

In this section, we elaborate on the personalized federated multiple-instance
learning framework based on ViT for COPD classification.

ViT Backbone. We decompose the three dimensional CT volume input
$\mathbf{x}=[x_{1},\cdots,x_{n}]\in\mathbb{R}^{H\times W\times C}$, where
$(H,W)$ is the image resolution, into $C$ number of 2D slices i.e.
$x_{i}\in\mathbb{R}^{H\times W}$ ($i=1,\cdots,n$) to perform image flatting.
Then, we denote the patch input as $x_{i}^{p}\in\mathbb{R}^{N\times P^{2}}$
for slice $x_{i}$, where $(P,P)$ is the patch resolution and $N=HW/P^{2}$ is
the number of patches in one slice. Afterwards, a learnable embedding is
employed to vectorize the patch inputs into vector embedding $\mathbf{z}_{0}$
[6]. Rather than the convolutional layers, Vision Transformer (ViT) [6] adopts
Multi-head Self-Attention (MSA) and Multi-Layer Perceptron (MLP) operators on
the embeddings. To be more specific, the latent output of $l$th layer, for
$l=1,\cdots,L$, $\mathbf{z}_{l}$ is computed as:

$\displaystyle\mathbf{z}_{l}^{\prime}$
$\displaystyle=\operatorname{MSA}_{l}\left(\operatorname{LN}_{l,0}(\mathbf{z}_{l-1})\right)+\mathbf{z}_{l-1},$
(1) $\displaystyle\mathbf{z}_{l}$
$\displaystyle=\operatorname{MLP}_{l}\left(\operatorname{LN}_{l,1}(\mathbf{z}_{l-1}^{\prime})\right)+\mathbf{z}_{l-1}^{\prime}\,,$

where we use the superscript $l$ to denote the weights associated with the
$l$th layer, $\operatorname{LN}(\cdot)$ represents the layernorm [3].

Partial Personalization Scheme for MSA Layer in FL. We propose a model
personalization scheme for ViT in FL by keeping a designed percentage of
weights in the attention layers residing locally during FL training, while the
rest are shared across all clients. Specifically, a proportion of MSAs is
personalized in each layer to learn local personalized attention
representations from heterogeneous CT data, while the rest parameters retains
globally shared to extract the common attention representations. We write the
$l$th MSA layer as

$\displaystyle=\mathbf{z}_{l-1}\mathbf{U}_{\mathbf{qkv},l,k},~{}~{}\text{for}~{}~{}k=1,\cdots,K_{l};$
(2) $\displaystyle SA_{l,k}(\mathbf{z}_{l-1})$
$\displaystyle=\operatorname{softmax}(\mathbf{q}_{l,k}\mathbf{k}_{l,k}^{T}/\sqrt{D_{h}})\mathbf{v}_{l,k}$
$\displaystyle MSA_{l}(\mathbf{z}_{l})$
$\displaystyle=[SA_{l,1}(\mathbf{z}_{l-1}),\cdots,SA_{l,K_{l}}(\mathbf{z}_{l-1})]\mathbf{U}_{\mathbf{msa},l}$

where $D_{h}$ is the size of vectors
$\mathbf{q}_{l,k},\mathbf{k}_{l,k},\mathbf{v}_{l,k}$, $K_{l}$ is the total
number of self-attention heads of
$\mathbf{U}_{\mathbf{msa},l}=[\mathbf{U}_{\mathbf{msa},l,1},\cdots\mathbf{U}_{\mathbf{msa},l,K_{l}}]^{T}$.
We follow the same terminology and definitions of $\mathbf{U}$ as in [6]. To
enable the ViT to capture the local personalized attention for heterogeneous
data, we personalize a number of $P_{l}<K_{l}$ heads in the MSA layers.
Specifically, the collection of personalized weights in $l$th MSA comprises
two parts i.e.

$\displaystyle\mathbf{U}_{\mathbf{qkv},l}^{p}$
$\displaystyle=[\mathbf{U}_{\mathbf{qkv},l,1},\cdots,\mathbf{U}_{\mathbf{qkv},l,P_{l}}]$
(3) $\displaystyle\mathbf{U}_{\mathbf{msa},l}^{p}$
$\displaystyle=[\mathbf{U}_{\mathbf{msa},l,1},\cdots,\mathbf{U}_{\mathbf{msa},l,P_{l}}],$
(4)

where superscript $p$ denotes the personalized weights. On the other hand, the
global shared weights are made up of
$\mathbf{U}_{\mathbf{qkv},l}^{g}=[\mathbf{U}_{\mathbf{qkv},l,P_{l}+1},\cdots,\mathbf{U}_{\mathbf{qkv},l,K_{l}}]$
and
$\mathbf{U}_{\mathbf{msa},l}^{g}=[\mathbf{U}_{\mathbf{msa},l,P_{l}+1},\cdots,\mathbf{U}_{\mathbf{msa},P_{l}+1,K_{l}}]$,
where superscript $p$ denotes the personalized weights. The global shared
weights will be uploaded to the central server and aggregated by FedAvg in the
training stage [15] to extract the global shared attentions. For model
simplicity, we use a uniform personalization ratio for all MSAs i.e.
$\frac{P_{l}}{K_{l}}=p\in(0,1)$ for $l=1,\cdots L$. Then, the value of $p$
determines the strength of model personalization. For example, $p=1$ degrades
the method to be a local training way i.e., no shared parameters via the
central server except the MLP layers; while $p=0$ becomes the vanilla FedAvg.
The overall architecture of our proposed partial personalization scheme for
attention layers is depicted in Fig. 1.

Consistency Regularization between Local Personalized and Global Shared
Weights. The partial personalization scheme enables us to impose an additional
regularization on the local personalized weights. We term the globally shared
subnet in ViT excluding all personalized MSA weights i.e.
$\\{\mathbf{U}_{\mathbf{qkv},l}^{p},\mathbf{U}_{\mathbf{msa},l}^{p}|l=1,\cdots,L\\}$,
as $\operatorname{ViT}^{g}(\cdot)$ [22]. In parallel, we name another client-
specific subnet with only personalized MSA weights as
$\operatorname{ViT}^{p,j}(\cdot)$ where $j$ is the associated client ID. We
impose a consistency regularization to force the local personalized weights
and shared weights ultimately generate predictions that coincide with each
other. To achieve this goal, we propose a mutual consistent regularization,
depicted as follows

$\displaystyle\mathcal{L}_{con}^{j}$
$\displaystyle=\mathbf{KL}(\sigma(\operatorname{ViT}^{g}(\mathbf{x})/T),\sigma(\operatorname{ViT}^{p,j}(\mathbf{x})/T))+$
(5)
$\displaystyle\mathbf{KL}(\sigma(\operatorname{ViT}^{p,j}(\mathbf{x})/T),\sigma(\operatorname{ViT}^{g}(\mathbf{x})/T))\,.$

The softmax $\sigma(\cdot)$ softens the outputs of subnets after being divided
by the temperature $T$. KL-divergence $\mathbf{KL}(\cdot,\cdot)$ is not
interchangeable, hence we symmetry it in Eq. (5). In the local training stage
at $j$th, the overall objective is

$\displaystyle\mathcal{L}^{j}=$
$\displaystyle\mathcal{L}_{ce}+\lambda\cdot\mathcal{L}_{con}^{j},~{}~{}\text{with}$
(6)
$\displaystyle\mathcal{L}_{ce}=\operatorname{CrossEntropy}(\operatorname{ViT}(\mathbf{x}),y),$
(7)

where $y$ is the ground truth associated with input image $\mathbf{x}$.
Coefficient $\lambda$ balances the cross entropy loss and the consistency
regularization term.

##  III Experiments

Datasets. This study retrospectively collects and utilizes CT images of large
patient cohorts from a public dataset NLST (Center 1) [18] and five hospitals
(Center 2-6) i.e. a total number of six medical centers. Post-bronchodilator
pulmonary function testing was available for all patients and applied as the
primary criteria for COPD diagnosis i.e. a forced expiratory volume in 1 sec
(FEV1)/forced vital capacity (FVC) ratio of $<$ 0.7. The inclusion criteria
were (i) participants that underwent single inspiratory breath-hold CT scans
in the supine position; (ii) at least one-time pulmonary function test; and
(iii) aged 18 years and older and with no history of thoracic surgery. The
exclusion criteria were (i) incomplete clinical or radiological data; (ii)
compromised image quality; and (iii) substandard spirometric data. Data from
different centers were collected from similar periods to exclude the selection
bias on the time scale. The NLST public dataset included participants between
55 and 74 years old, with a smoking history of more than 30 pack-years and no
self-reported history of lung cancer, thereby allowing us to include low-dose
CT acquired among smokers in the dataset, which further adds to the data
heterogeneity. A total of 2856 participants were enrolled in the final cohort.
Among them, 1077 patients were diagnosed with COPD and the other 1779 patients
with non-COPD. The data volume of each dataset and data distributions are
depicted in Fig. 2. Briefly, contiguous CT images were reconstructed in the
transverse (axial) plane using standard reconstruction algorithms, creating
three-dimensional volumes with up to 5.0 mm slice thickness.

Figure 2: Data scale (a) and the distributions (b) of COPD CT slides from six
medical centers used in our evaluation.

Implementations. We implemented our methods and the baseline approaches by
Pytorch on one NVIDIA Tesla V100 GPU device, where the distributed environment
of FL is simulated by training the local epoch sequentially. We use a
consistent training scheme. A stochastic gradient descent optimizer is adopted
for each local model update, where the Nesterov momentum and the weight decay
rate are set to 0.9 and $5\times 10^{-4}$ respectively. The total federated
epoch number is set to 50, with the local epoch number to 3. The temperature
$T$ in Eq. (5) is set to 4.0, the balancing coefficient in Eq. (6) as 1, and
the personalization ratio $p=0.6$.

TABLE I: Performance comparison with local training, FL with various network
architectures and PFL with CNNs [2, 13]. We demonstrate the performance of our
approach both with and without the consistent regularization i.e.
$\mathcal{L}_{con}$. ‘Local # j’ denotes the local test AUC of the Center j,
and ‘new’ denotes the new test AUC. The best performance is marked in
boldface, and the second best is underlined.

methods | local#1 | local#2 | local#3 | local#4 | local#5 | local#6 | new
---|---|---|---|---|---|---|---
Local Train+ResNet32 | 0.720 | 0.709 | 0.674 | 0.688 | 0.612 | 0.613 | 0.674
Local Train+ResNet110 | 0.731 | 0.720 | 0.675 | 0.692 | 0.631 | 0.618 | 0.680
Local Train+ViT | 0.712 | 0.714 | 0.678 | 0.701 | 0.621 | 0.634 | 0.687
FedAvg+ResNet32 | 0.821 | 0.834 | 0.832 | 0.827 | 0.790 | 0.829 | 0.783
FedAvg+ResNet110 | 0.828 | 0.836 | 0.835 | 0.815 | 0.799 | 0.835 | 0.794
FedAvg+ViT | 0.835 | 0.839 | 0.841 | 0.842 | 0.811 | 0.831 | 0.805
FedPer+ResNet32 | 0.844 | 0.853 | 0.847 | 0.848 | 0.832 | 0.810 | 0.773
FedPer+ResNet110 | 0.839 | 0.852 | 0.852 | 0.849 | 0.834 | 0.816 | 0.776
LG-FedAvg+ResNet32 | 0.867 | 0.859 | 0.856 | 0.841 | 0.845 | 0.837 | 0.802
LG-FedAvg+ResNet110 | 0.871 | 0.858 | 0.857 | 0.832 | 0.859 | 0.810 | 0.809
Ours (w/o $\mathcal{L}_{con}$) | 0.883 | 0.864 | 0.870 | 0.851 | 0.865 | 0.839 | 0.841
Ours (w $\mathcal{L}_{con}$) | 0.887 | 0.871 | 0.874 | 0.862 | 0.876 | 0.843 | 0.854

Method Comparison. We compare our method with the baseline vanilla FL FedAvg
and local training. We also compare our proposed method with CNN-based PFL
approaches, where ResNet32 and ResNet110 [9] are selected as the CNN backbones
for their compact structures and outstanding performance in the medical
domain. We use ViT-Small [6] as the backbone in our PFL approach. All backbone
networks are trained from scratch, without loading any pre-trained weights for
a fair comparison. Two state-of-the-art personalization are compared, namely
FedPer [2] and LG-FedAvg [13]. For the extra hyper-parameters involved in
these compared methods, we retain their original settings. We use the AUC
(Area Under Curve) as the evaluation metric in two configurations, namely
local test and new test [13]. In the local test, we know precisely the client
ID where the CT images come from, thus we can choose the particular associated
local model. These metrics intend to measure the performance of local
personalized weights. In the new test, the client ID of the test sample is
unknown, thus we employ an ensemble of all local models to derive averaged
predictions, intending to measure the compatibility between local and global
weights.

Results. As shown in Table I, our personalization scheme can improve baseline
FedAvg by 0.012 to 0.065 local test AUC and 0.049 new test AUC. Moreover, our
approach outperforms the state-of-the-art CNN-based PFL methods and
illustrates a higher improvement rate to these methods in terms of local test
AUC. Our method achieves the highest new test AUC indicating that our model is
capable of extracting the common attention with the global shared weights, in
addition to the local personalized attention with the personalized weights.
With the consistent regularization i.e. $\mathcal{L}_{con}$ in Eq. (5), our
method achieves extra performance gain, suggesting the effectiveness of each
component of our method. As shown in Fig. 3, the method achieves the highest
performance in terms of Local Test AUC when $p=0.6$ and New Test AUC with
$p=0.4$, which implies that a competitive proportion of local personalized
knowledge can mitigate the data shift dilemma among different local data
centers.

Figure 3: Ablation on the effect of personalization ratio $p$ in terms of
averaged local test AUC and new test AUC.

##  IV Conclusion

In this paper, a novel partial personalization scheme is proposed,
specifically for vision transformers in the federated learning paradigm. To be
more specific, some heads in the multiheaded self-attention layer are
personalized, while the rest heads remain globally shared. The partial
personalization scheme intends to learn both local attention and global
attention. We evaluate our method on COPD CT images curated from six data
centers. Our method, with a consistent regularization between local and global
self-attention heads, significantly improves the baseline FedAvg and
outperforms other CNN-based PFL methods. One interesting future direction is
to evaluate our approach to different data modalities.

## References

  * [1] George J Annas.  Hipaa regulations—a new era of medical-record privacy?, 2003.
  * [2] Manoj Ghuhan Arivazhagan et al.  Federated learning with personalization layers.  arXiv preprint arXiv:1912.00818, 2019.
  * [3] Jimmy Lei Ba et al.  Layer normalization.  arXiv preprint arXiv:1607.06450, 2016.
  * [4] Tom Brown et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [5] Veronika Cheplygina et al.  Transfer learning for multicenter classification of chronic obstructive pulmonary disease.  IEEE journal of biomedical and health informatics, 22(5):1486–1496, 2017.
  * [6] Alexey Dosovitskiy et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [7] Centers for Disease Control, Prevention (CDC, et al.  Million hearts: strategies to reduce the prevalence of leading cardiovascular disease risk factors–united states, 2011.  MMWR. Morbidity and mortality weekly report, 60(36):1248–1251, 2011\.
  * [8] Kyle A Hasenstab et al.  Automated ct staging of chronic obstructive pulmonary disease severity for predicting disease progression and mortality with a deep learning convolutional neural network.  Radiology: Cardiothoracic Imaging, 3(2):e200477, 2021.
  * [9] K. He et al.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [10] Peter Kairouz et al.  Advances and open problems in federated learning.  arXiv preprint arXiv:1912.04977, 2019.
  * [11] Jing Ke et al.  Style normalization in histology with federated learning.  In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 953–956. IEEE, 2021.
  * [12] Viraj Kulkarni et al.  Survey of personalization techniques for federated learning.  In 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), pages 794–797. IEEE, 2020.
  * [13] Paul Pu Liang et al.  Think locally, act globally: Federated learning with local and global representations.  arXiv preprint arXiv:2001.01523, 2020.
  * [14] Katherine E Lowe et al.  Copdgene® 2019: redefining the diagnosis of chronic obstructive pulmonary disease.  Chronic Obstructive Pulmonary Diseases: Journal of the COPD Foundation, 6(5):384, 2019.
  * [15] H. Brendan McMahan et al.  Communication-efficient learning of deep networks from decentralized data, 2017.
  * [16] Adnan Qayyum et al.  Collaborative federated learning for healthcare: Multi-modal covid-19 diagnosis at the edge.  arXiv preprint arXiv:2101.07511, 2021.
  * [17] Nicola Rieke et al.  The future of digital health with federated learning.  NPJ digital medicine, 3(1):1–7, 2020.
  * [18] National Lung Screening Trial Research Team.  Reduced lung-cancer mortality with low-dose computed tomographic screening.  New England Journal of Medicine, 365(5):395–409, 2011.
  * [19] Jørgen Vestbo et al.  Changes in forced expiratory volume in 1 second over time in copd.  New England Journal of Medicine, 365(13):1184–1192, 2011.
  * [20] Paul Voigt et al.  The eu general data protection regulation (gdpr).  A Practical Guide, 1st Ed., Cham: Springer International Publishing, 10:3152676, 2017.
  * [21] Ruohua Yan et al.  Healthy lifestyle behaviors among individuals with chronic obstructive pulmonary disease in urban and rural communities in china: a large community-based epidemiological study.  International Journal of Chronic Obstructive Pulmonary Disease, 12:3311, 2017.
  * [22] Jiahui Yu et al.  Universally slimmable networks and improved training techniques.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1803–1811, 2019.
  * [23] Weishan Zhang et al.  Dynamic-fusion-based federated learning for covid-19 detection.  IEEE Internet of Things Journal, 8(21):15884–15891, 2021.
  * [24] Yue Zhao et al.  Federated learning with non-iid data.  arXiv preprint arXiv:1806.00582, 2018.