# PromptMind Team at MEDIQA-CORR 2024: Improving Clinical Text Correction with
Error Categorization and LLM Ensembles

Satya K Gundabathula
<EMAIL_ADDRESS>
&Sriram R Kolar
<EMAIL_ADDRESS>

###### Abstract

This paper describes our approach to the MEDIQA-CORR shared task, which
involves error detection and correction in clinical notes curated by medical
professionals. This task involves handling three subtasks: detecting the
presence of errors, identifying the specific sentence containing the error,
and correcting it. Through our work, we aim to assess the capabilities of
Large Language Models (LLMs) trained on a vast corpora of internet data that
contain both factual and unreliable information. We propose to comprehensively
address all subtasks together, and suggest employing a unique prompt-based in-
context learning strategy. We will evaluate its efficacy in this specialized
task demanding a combination of general reasoning and medical knowledge. In
medical systems where prediction errors can have grave consequences, we
propose leveraging self-consistency and ensemble methods to enhance error
correction and error detection performance.

PromptMind Team at MEDIQA-CORR 2024: Improving Clinical Text Correction with
Error Categorization and LLM Ensembles

Satya K Gundabathula<EMAIL_ADDRESS>Sriram R Kolar
<EMAIL_ADDRESS>

##  1 Introduction

With rapid advancements in Natural Language Processing (NLP), we are
witnessing a surge of its applications across various domains, including
healthcare. Incorporating NLP in clinical settings brings about a multitude of
advantages. It enhances clinical decision-making through advanced support,
making health information more accessible, streamlines documentation, and
accelerates research initiatives (Hossain et al., 2023). These developments
contribute to improved patient care, reduced healthcare costs, and alleviated
physician burnout.

NLP for healthcare applications pose inherent challenges due to the need for
medical expertise. However, advancements in LLMs trained on internet data
including medical information have significantly enhanced their knowledge and
reasoning capabilities, enabling them to tackle more complex problems in the
healthcare domain involving text processing and generation. Few recent
applications in healthcare include information extraction, question answering,
summarization, and translation, all while comprehending intricate medical
knowledge (Nazi and Peng, 2023). Despite these advancements, safety and
accuracy remain major concerns as training data may contain unreliable and
misleading information that could have adverse effects if not handled
appropriately. Nevertheless, the effective utilization of these LLMs has the
potential to revolutionize healthcare and bring immense benefits to society
(Clusmann et al., 2023).

In the healthcare industry, there is a need for automated systems capable of
efficiently analyzing and interpreting clinical texts that improves patient’s
safety, quality of care and costs. Processing the clinical texts presents a
unique and significant challenge due to the complexities introduced by medical
jargon, abbreviations, syntactic variations, and context-specific nuances. The
MEDIQA-CORR shared task (Abacha et al., 2024), part of the ClinicalNLP 2024
workshop, seeks to address this issue of identifying and correcting (common
sense) medical errors found in clinical notes.

The MEDIQA-CORR shared task involves three subtasks: detecting errors in
clinical notes, identifying specific error sentences, and correcting those
sentences. Our approach involves tackling all three subtasks simultaneously
using a single prompt for LLMs invoking chain-of-thought. By doing this, we
seek to assess the complex reasoning capabilities of LLMs in the clinical
domain.

First, we analyze the dataset and curate a list of the most common types of
errors in clinical notes. We then utilize this information to create task
specific instructions for the LLM. We leverage contemporary LLMs using in-
context learning (ICL) with a few-shot approach. Since publicly accessible
LLMs are instruction-tuned models, considering the approach of directing them
towards assessing the clinical note based on specific error types maximizes
the objective while only utilizing a few training examples.

By employing single-prompt ICL approaches to solve end-to-end tasks, we pave
the way forward for building more complex healthcare applications using simple
yet intuitive strategies leveraging the capabilities of advanced LLMs.

In addition to the task prompt, an LLM’s performance on a particular task is
mainly influenced by two factors: training techniques and underlying training
data. Consistency, which measures how frequently an LLM produces the same
output given the same instructions, can be viewed as another dimension that
can affect the quality of an LLM. Leveraging self-consistency can
significantly improve the accuracy of an LLM, particularly for complex
reasoning tasks (Wang et al., 2023). In healthcare datasets, where
hallucinations in LLMs can occur more frequently due to training on factually
unverified data, this could lead to serious problems. While self-consistency
is one approach to obtaining more accurate results, LLM ensemble, which has
not yet been fully explored, presents a promising opportunity. We validate the
results of each LLM by using the output of other LLMs that are trained on
different corpora. In our approach, we investigate both self-consistency and
ensemble concepts.

The remainder of the paper includes related work, the MEDIQA-CORR task and
dataset, our approach, results and conclusion.

##  2 Shared Task and Dataset

The shared task focuses on leveraging LLMs for the following three subtasks:

  * •

Binary Classification: Detect if the text from a clinical note includes a
medical error.

  * •

Span Identification: Identify the text span (in the sentence) associated with
the error, if an error exists.

  * •

Natural Language Generation: Generate the corrected text, if an error exists.

###  2.1 Subtasks and Metrics

####  2.1.1 Error Detection

For each clinical text, the prediction is assigned a value of 1 when an error
is detected, and 0 if no error is detected. Given the binary nature of this
classification task, accuracy serves as the primary metric for performance
evaluation.

####  2.1.2 Error Span Identification

Each clinical text comprises sentences associated with unique IDs. The subtask
involves predicting the error ID, which is an integer between 0 and the
highest sentence ID. If no error is detected, the prediction should be -1. The
primary evaluation metric is accuracy, calculated based on all samples,
including those with and without errors.

####  2.1.3 Correct Sentence Generation

If a model identifies an error sentence in the previous subtasks, it should
also generate a corrected sentence as prediction for this subtask. Here, the
full corrected sentence is evaluated against the ground truth sentence for
measuring the performance. Clinical note generation tasks are challenging to
evaluate automatically due to the large number of possible correct answers.
Metric ensembles (Abacha et al., 2023) have been shown to outperform
individual state-of-the-art measures, such as ROGUE for such tasks. The
evaluation metric for this subtask is computed as an unweighted average of the
following three scores:

  * •

ROUGE-1F measures the similarity between system-generated and human-written
texts by measuring the overlap of unigrams (Lin, 2004). It uses the F-1 score
to assess the quality of the generated sentence.

  * •

BERTScore leverages contextual word embeddings obtained from BERT models to
assess the similarity between a candidate sentence and a reference sentence
(Zhang* et al., 2020). In this context, it signifies the F-1 score of the
semantic similarity performed using the DeBERTa XL model (He et al., 2021).

  * •

BLEURT-20 is a learned metric trained on human ratings that aims to better
correlate with human judgments for measuring quality compared to traditional
BLEU (Sellam et al., 2020).

###  2.2 Dataset

The train data consists of clinical texts from MS data while the valid and
test data contains MS and UW collections. Each entry in the datasets includes
a text, its ID and sentences as inputs. Table 1 shows the composition of the
dataset.

Dataset Type | # Samples |  % of Error Samples
---|---|---
MS Training | 2189 | 55.69
MS Validation | 574 | 55.57
UW Validation | 160 | 50.00
MS + UW Test | 925 | 51.35
Table 1: MEDIQA-CORR Dataset

##  3 Approach

We propose to tackle all subtasks concurrently within a single prompt for the
following reasons:

  * •

Comprehensive Evaluation: To enable performance evaluation on a complex task,
rather than assessing them on isolated, simpler tasks. This provides a more
holistic view of the LLMs’ capabilities.

  * •

Efficiency Optimization: To minimize inference costs and developmental efforts
by eliminating the need for multiple models (or) sequential processing. It
streamlines the process, making it more efficient and cost-effective.

  * •

Ease of Adoption: To alleviate the adoption burden in practical applications
and facilitate seamless upgrades to more advanced LLMs amidst the rapid
technological advancements.

Publicly accessible LLMs are models that are fine-tuned to follow
instructions, with the aim of performing user-defined instructions as
accurately as possible. The success of the task then depends on the quality of
the instructions provided and the model’s ability to follow them effectively.
Our approach focuses on refining the instructions for the LLM to facilitate
comprehensive learning of all subtasks using ICL. We initiate this process by
analyzing error types in the dataset followed by curating the prompt and
inferencing with different LLMs.

###  3.1 Error Analysis

In the MEDIQA-CORR task, the definition of an error is loosely defined and can
be interpreted differently by humans or systems without examining the dataset.
To address this, we perform error type classification in clinical texts by
extracting error sentences and corresponding corrected sentences from the
training data. We create an LLM prompting task to broadly categorize the
entities modified from the error sentence to the corrected sentence within the
clinical domain. We utilize GPT-3.5 for categorizing the errors and cluster
these generated free-form categories into a manually defined set. This
categorization results in the identification of various error types as
depicted in figure 1. Finally, we use the well-defined error categories for
the rest of the task while handling "Others" category discreetly.

Figure 1: Error Types Extracted from Training Data

###  3.2 Prompt Curation

When prompted to identify errors in clinical texts without being specific,
LLMs may introduce biases from their training data and flag non-critical
errors adhering to their own standards of composing clinical notes. To
mitigate this, we propose converting an abstract definition of a clinical note
error into a concrete and approximate one by analyzing and categorizing the
errors. Consequently, we expand the original task from identifying the error
to include error classification, which facilitates chain-of-thought for the
LLM. We conduct ablation studies to demonstrate the effectiveness of these
techniques and present in section 4.

To illustrate the task more thoroughly, we incorporate reasoning within the
task prompt for the LLM. Through this, we aim to provide more generalizable
patterns for detecting and identifying errors. Additionally, it adds
explainability to the LLM systems, which is crucial for real-world
applications. Finally, we adopt a few-shot approach utilizing random samples
from the training data to teach the LLM how to detect, identify, classify
errors, provide reasoning and demonstrate the corrected text. We utilize the
same samples for few-shot throughout the task as we need to manually generate
the additional fields such as error category and reason. The designed task
prompt is provided as below. Note that this prompt is a tailored version for
demonstration purposes. Finally, errors that fall into the "Others" category
are processed as "No Error" as they are unimportant for this task.
Prompt Template In this task, you will be given a clinical report presented as
sentences while each sentence is associated with a sentence number. Now, you
need to go through the report line by line and identify if there is any error
in the sentence. The error can fall into one of the following main categories:
1\. Medications
2\. Medical Conditions, Virus or Bacteria
3\. Reports, Diagnosis and Monitoring
4\. Clinical Procedures and Treatments
5\. Clinical Plans and Recommendations
6\. Medical Devices
7\. Others including clarity/improper usage of terminology

The error can be identified by looking at the entities present in each
sentence and check if these entities fall into one of the aforementioned
categories and validate if the entire report is correct with this entity. If
there is an error, you need to output details as shown in the examples. Use
all your medical knowledge and make the right judgements. Here are a few
examples for your understanding:

/* Five random samples from training data with manually curated error category
and reason */
Example Clinical Report:
0 Mr. <Name> is admitted ..
1 He has a surgical ..
2 He is also being managed ..
Output:
{
"Error Sentence ID": 1,
"Error Category": "Medical Devices",
"Reason": "The device should be .."
"Corrected Sentence": "He has a surgical .."
}
…
…
Test Clinical Report:
0 A 45 year old woman ..
1 She is experiencing ..
2 She had prior examamination ..
Output:

###  3.3 Model Selection

Using the designed prompt, we utilize the following LLMs for performing the
task:

  * •

GPT-3.5: A model from the OpenAI’s generative pre-trained transformer (GPT)
family that can understand as well as generate natural language or code (Ye et
al., 2023).

  * •

GPT-4: Latest model from the GPT family with broader general knowledge and
advanced reasoning capabilities (OpenAI et al., 2024).

  * •

Claude-3 Opus: Anthropic’s largest model, released in Feb 2024, which sets new
industry benchmarks across a wide range of cognitive tasks and outperforms its
peers on most of the common evaluation benchmarks for AI systems (Anthropic
and others, 2024).

Due to its affordability, speed, and reliability, GPT-3.5 is an excellent
choice for experimentation. As a result, we employed GPT-3.5 for error
analysis and prompt design, reserving the more advanced GPT-4 and Claude-3
Opus models for the final test runs.

###  3.4 Enhancing Robustness

Due to the potential limitations such as hallucinations and inconsistent
results, which can affect the quality of the LLMs, we investigate two concepts
to improve performance on the subtasks: self-consistency and ensemble. The
Claude-3 Opus model has slower speed, higher cost, and stricter token limits
compared to GPT-4. Therefore, we utilize GPT-4 for self-consistency by
generating four outputs per test sample and aggregate them, while only
generating one output per test sample for Claude-3 Opus.

To enhance the quality of predictions, we combine the results from both models
to predict the outputs for all three subtasks. Figure 2 provides a visual
representation of the overall process. The results aggregator module validates
and combines the outputs i.e. predicted error flag, predicted error sentence
ID and corrected sentence, from GPT-4 and Claude-3 Opus models to generate the
final error flag, error sentence ID and corrected sentence.

Figure 2: LLM Ensemble Approach for MEDIQA-CORR task

##  4 Ablation Study

We begin by presenting the results obtained from incorporating error
categorization into the final prompt, which demonstrates an improvement in
performance on both error detection and text span identification tasks. In
order to make the comparison, we utilize GPT-3.5 with prompts including and
excluding error categorization. Additionally, we assess the performance of
GPT-4 to ascertain the extent to which it surpasses GPT-3.5 for the finalized
prompt. The results obtained using the combined MS and UW validation sets
(during the development phase) are presented in Table 2.

Model | Prompt |  Task-1 Accuracy |  Task-2 Accuracy
---|---|---|---
GPT-3.5 |  No error categories | 48.75% | 22.5%
GPT-3.5 |  Error categories | 58.44% | 38.55%
GPT-4 |  Error categories | 63.07% | 58.17%

Table 2: Performance improvement with error categorization in prompt using
GPT-3.5 and assessing GPT-4 performance

The results indicate that by integrating error categorization which initiates
an intermediate chain-of-thought, results in a significant performance boost
of nearly 10% and 16% for Task-1 and Task-2, respectively. Additionally, GPT-4
outperforms GPT-3.5, confirming its enhanced reasoning capabilities. These
advancements make GPT-4 a preferred choice for final test runs.

##  5 Results

We present the results of individual models first followed by incorporating
self-consistency and ensembles on the test data. The performance of GPT-4 and
Claude-3 Opus using the final prompt on all three subtasks is presented in
Table 3.

Model |  Task-1 Accuracy |  Task-2 Accuracy |  Task-3 Agg- regate Score
---|---|---|---
GPT-4 | 62.05% | 56.43% | 0.6172
Claude-3 Opus | 62.59% | 58.48% | 0.6669

Table 3: Comparison of GPT-4 and Claude-3 Results

Claude-3 Opus surpasses GPT-4 in error detection and significantly in error
sentence identification. However, GPT-4 tends to be more verbose during error
correction, leading to lower scores in metrics such as ROUGE, BERT, and
BLEURT. Although Claude-3 Opus exhibits superior performance, its daily token
limit, slower inference, and shorter test phase duration hinder its usability
for self-consistency. Therefore, we employ GPT-4 to demonstrate the
performance enhancement using self-consistency in large language models
(LLMs). Additionally, we ensemble the self-consistent GPT-4 with Claude-3 Opus
to showcase further improvement. Table 4 presents the results for all subtasks
using the aforementioned methods:

Model |  Task-1 Accuracy |  Task-2 Accuracy |  Task-3 Agg- regate Score
---|---|---|---
GPT-4 with consistency (Majority=3/4) | 62.91% | 59.45% | 0.6390
GPT-4 + Claude-3 Opus (Majority=4/4) | 62.16% | 60.86% | 0.7865
GPT-4 + Claude-3 Opus (Majority=3/4) | 63.78% | 62.48% | 0.7492

Table 4: Self-consistent GPT-4 and its ensemble with Claude-3 Opus Results

The majority ratio x/y for GPT-4 results is a measure of how often the model
produces the same result for a given input. For example, a majority ratio of
3/4 means that at least three out of the four results should be the same to
qualify the predicted error sentence ID as correct. In the ensemble approach,
a prediction is considered correct if the self-consistent GPT-4 result matches
the Claude-3 Opus result. Otherwise, the prediction is considered "no error".
To select the best corrected sentence from the ensemble, the ROGUE score is
used to estimate the distance between each corrected sentence and the error
sentence. The sentence with the highest ROGUE score is used for the evaluation
because LLMs tend to generate verbose corrected sentences which decreases the
aggregate scores. In our experiments, using a majority ratio of 3/4 for GPT-4
in the ensemble resulted in the best Task-1 and Task-2 performances. Using a
majority ratio of 4/4 resulted in the best Task-3 aggregate score. Our best
subtask-3 score is ranked 2nd among all participants in the competition, and
our best subtask-2 performance is among the top 3 according to the reported
scores (Ben Abacha et al., 2024).

##  6 Related Work

In recent years, there has been a surge of research exploring the potential of
prompt engineering techniques with large language models (LLMs) in healthcare.
These techniques have shown promising results in various healthcare tasks,
often achieving state-of-the-art performances (Zhou et al., 2023), (He et al.,
2023). One notable study, MedPrompt, highlighted several research directions
demonstrating the power of prompt exploration for LLMs (Nori et al., 2023).
LLMs exhibited impressive knowledge and reasoning abilities, tackling various
tasks effectively. These advancements showcase the potential of LLMs in
healthcare, offering new opportunities for leveraging language models to
address healthcare challenges.

Evaluating common sense reasoning is essential for computer systems, as it
impacts language comprehension, communication reliability, and general task
performance. SemEval-2020 ComVE aims to address general common sense questions
and seeks logical justification for correct responses, assessing reasoning
abilities. Pretrained language models naturally acquire common sense through
training on vast word tokens obtained from the web (Wang et al., 2020).
MEDIQA-CORR, specifically tailored to identify and correct errors in clinical
notes, offers a valuable resource for evaluating pretrained LLMs in medical
common sense reasoning. Inspired by prompt-based explorations, our research
also focuses on utilizing pretrained LLMs to assess reasoning capabilities in
medical common sense scenarios.

##  7 Conclusion

Our research demonstrates that incorporating error categorization into the
prompt enhances the performance of large language models (LLMs) in detecting,
identifying, and classifying clinical note errors. By initiating an
intermediate chain-of-thought, this approach facilitates better reasoning and
aids the LLM in providing more accurate and explainable results. Furthermore,
our findings suggest that self-consistency and ensembles can further enhance
the robustness and performance of LLMs on these tasks. These advancements pave
the way for the development of more reliable and interpretable AI systems for
clinical documentation analysis, ultimately contributing to improved
healthcare outcomes.

##  8 Limitations and Risks

While promising, our approach has limitations. LLMs trained on general data
may lack specific medical knowledge, potentially leading to misinterpretations
and inaccurate corrections. Despite mitigation efforts, the risk of
hallucinations and inconsistencies in LLM outputs remains a concern.
Additionally, the effectiveness of our approach relies heavily on prompt
engineering, which requires expertise and may not be easily generalizable.

The black box nature of LLMs also presents challenges in terms of
explainability and building trust in medical contexts. To mitigate these
limitations, continuous improvements of training data, more robust evaluation
metrics, and human oversight are crucial. Further research is needed to
explore the full potential and limitations of LLMs in healthcare, ensuring
their safe and responsible application for improved patient care.

##  9 Ethical Considerations

The use of LLMs for medical error detection and correction raises significant
ethical concerns. Potential biases in training data and algorithms must be
carefully mitigated to prevent propagating existing healthcare disparities and
ensure fairness. Transparency in how prompts are designed and how the LLM
reaches its decisions is vital for building trust and ensuring accountability.
Additionally, robust data security and de-identification practices are
paramount for protecting sensitive patient information.

It is essential to remember that LLMs should serve as tools to augment the
expertise of healthcare professionals, not replace them. Clear lines of
responsibility, ongoing human oversight, and continuous research and
collaboration are necessary to address these ethical challenges. This will
ensure the responsible use of LLMs and their positive contribution to improved
healthcare outcomes.

## References

  * Abacha et al. (2024) Asma Ben Abacha, Wen wai Yim, Velvin Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, and Thomas Lin. 2024.  Medec: A benchmark for medical error detection and correction in clinical notes.  _CoRR_.
  * Abacha et al. (2023) Asma Ben Abacha, Wen wai Yim, George Michalopoulos, and Thomas Lin. 2023.  An investigation of evaluation metrics for automated medical note generation.  _ArXiv_ , abs/2305.17364.
  * Anthropic and others (2024) Anthropic and others. 2024.  The claude 3 model family: Opus, sonnet, haiku.  https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf.  Online; accessed March 2024.
  * Ben Abacha et al. (2024) Asma Ben Abacha, Wen wai Yim, Velvin Fu, Zhaoyi Sun, Fei Xia, and Meliha Yetisgen. 2024.  Overview of the mediqa-corr 2024 shared task on medical error detection and correction.  In _Proceedings of the 6th Clinical Natural Language Processing Workshop_ , Mexico City, Mexico. Association for Computational Linguistics.
  * Clusmann et al. (2023) Jan Clusmann, Fiona R Kolbinger, et al. 2023.  The future landscape of large language models in medicine.
  * He et al. (2023) Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. 2023.  A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics.  _Preprint_ , arXiv:2310.05694.
  * He et al. (2021) Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021.  Deberta: Decoding-enhanced bert with disentangled attention.  In _International Conference on Learning Representations_.
  * Hossain et al. (2023) Elias Hossain, Rajib Rana, Niall Higgins, Jeffrey Soar, Prabal Datta Barua, Anthony R. Pisani, and Kathryn Turner. 2023.  Natural language processing in electronic health records in relation to healthcare decision-making: A systematic review.  _Computers in Biology and Medicine_ , 155:106649.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Nazi and Peng (2023) Zabir Al Nazi and Wei Peng. 2023.  Large language models in healthcare and medical domain: A review.  _Preprint_ , arXiv:2401.06775.
  * Nori et al. (2023) Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, Renqian Luo, Scott Mayer McKinney, Robert Osazuwa Ness, Hoifung Poon, Tao Qin, Naoto Usuyama, Chris White, and Eric Horvitz. 2023.  Can generalist foundation models outcompete special-purpose tuning? case study in medicine.  _Preprint_ , arXiv:2311.16452.
  * OpenAI et al. (2024) OpenAI, Josh Achiam, et al. 2024.  Gpt-4 technical report.  _Preprint_ , arXiv:2303.08774.
  * Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020.  BLEURT: Learning robust metrics for text generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7881–7892, Online. Association for Computational Linguistics.
  * Wang et al. (2020) Cunxiang Wang, Shuailong Liang, Yili Jin, Yilong Wang, Xiaodan Zhu, and Yue Zhang. 2020.  SemEval-2020 task 4: Commonsense validation and explanation.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 307–321, Barcelona (online). International Committee for Computational Linguistics.
  * Wang et al. (2023) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023.  Self-consistency improves chain of thought reasoning in language models.  In _The Eleventh International Conference on Learning Representations_.
  * Ye et al. (2023) Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang, and Xuanjing Huang. 2023.  A comprehensive capability analysis of gpt-3 and gpt-3.5 series models.  _Preprint_ , arXiv:2303.10420.
  * Zhang* et al. (2020) Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020.  Bertscore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.
  * Zhou et al. (2023) Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam S. Chen, Peilin Zhou, Yining Hua Junling Liu, Chengfeng Mao, Xian Wu, Yefeng Zheng, Lei Clifton, Zheng Li, Jiebo Luo, and David A. Clifton. 2023.  A survey of large language models in medicine: Progress, application, and challenge.  _arXiv preprint arXiv:2311.05112_.


# Some universal inequalities of eigenvalues and upper bounds for the
$L_{\infty}$ norm of eigenfunctions of the Laplacian

Kei Funano  Division of Mathematics & Research Center for Pure and Applied
Mathematics, Graduate School of Information Sciences, Tohoku University,
6-3-09 Aramaki-Aza-Aoba, Aoba-ku, Sendai 980-8579, Japan<EMAIL_ADDRESS>
###### Abstract.

In this short survey, we derive some weyl-type universal inequalities of
eigenvalues of the Laplacian on a closed Riemannian manifold of nonnegative
Ricci curvature. We also give upper bounds for the $L_{\infty}$ norm of
eigenfunctions of the Laplacian in the same setting. A detailed proof of these
results did not seem to appear in the literature but the results follow from a
simple combination of Milman’s work and Cheng-Li’s work.

###### Key words and phrases:

Eigenvalues of the Laplacian; Universal inequalities; Eigenfunctions of the
Laplacian; Ricci curvature; Convex domain

###### 2010 Mathematics Subject Classification:

35P15, 53C23, 58J50

##  1\. Introduction

For a closed Riemannian manifold $M$ let $\lambda_{k}(M)$ be the $k$-th
nonzero eigenvalue of the Laplacian.

One of the goal of this survey is to give a precise proof of the following
theorem: For $a,b\in\mathbb{R}$, $a\lesssim b$ stands for $a\leq Cb$ for some
universal (and numerical) constant $C>0$.

######  Theorem 1.1.

Let $M$ be an $n$-dimensional closed Riemannian manifold of nonnegative Ricci
curvature. Then for any $k\geq 1$ we have

(1.1) $\displaystyle\lambda_{k}(M)\gtrsim k^{\frac{2}{n}}\lambda_{1}(M).$

Note that the order of $k$ in (1.1) is sharp by the Weyl asymptotic law.

The opposite of (1.1) was obtained by Liu in [10, Theorem 1.1]. Under the same
assumption of Theorem 1.1 Liu showed

(1.2) $\displaystyle\lambda_{k}(M)\lesssim k^{2}\lambda_{1}(M),$

which is sharp with respect to the order of $k$.

Throughout this survey $\mu$ denotes the uniform probability measure on $M$,
i.e., $\mu(\cdot):=\operatorname{vol}(\cdot)/\operatorname{vol}(M)$. For a
measurable function $f:M\to\mathbb{R}$ and $p>0$ we put

$\displaystyle\|f\|_{p}:=\Big{(}\int_{M}|f(x)|^{p}d\mu(x)\Big{)}^{\frac{1}{p}}$

and

$\displaystyle\|f\|_{\infty}:=\operatorname*{ess\,sup}_{x\in M}|f(x)|.$

We also give the proof of the following theorem.

######  Theorem 1.2.

Let $M$ be an $n$-dimensional closed Riemannian manifold of nonnegative Ricci
curvature. Then for any $k\geq 1$ and any eigenfunction $\varphi_{k}$ of the
Laplacian corresponding to $\lambda_{k}(M)$ we have

(1.3)
$\displaystyle\|\varphi_{k}\|_{\infty}\lesssim\Big{(}\frac{c\lambda_{k}(M)}{\lambda_{1}(M)}\Big{)}^{\frac{n}{4}}\|\varphi_{k}\|_{2}.$

for some universal constant $c>0$. In particular we have

(1.4)
$\displaystyle\|\varphi_{k}\|_{\infty}\lesssim(ck)^{\frac{n}{2}}\|\varphi_{k}\|_{2}.$

The inequality (1.4) follows from (1.3) together with (1.2).

A classical result by Hölmander [8] (see also [1], [4], [13], [15], and [16,
Corollary 8.6]) states that

(1.5) $\displaystyle\|\varphi_{k}\|_{\infty}\leq
C\lambda_{k}(M)^{\frac{n-1}{4}}\|\varphi_{k}\|_{2}$

for any closed Riemannian manifold, where the constant $C=C(M,g)$ depends on
$(M,g)$. This inequality is sharp and better than (1.3) when $\lambda_{k}(M)$
is large. In the case where $\lambda_{k}(M)<<\lambda_{1}(M)^{n}$ the
inequality (1.3) is better than the inequaity (1.5).

The proof of Theorems 1.1 and 1.2 follows from a simple combination of
Milman’s result [12] and Cheng-Li’s result [3]. In fact under the
nonnegativity of Ricci curvature, Milman proved the Sobolev inequality in
terms of the first nonzero eigenvalue and Cheng-Li proved lower bounds of
eigenvalues of the Laplacian in terms of the Sobolev constant. As a byproduct
of the proof of Theorem 1.1 we prove Theorem 1.2.

######  Remark 1.1.

Theorems 1.1 and 1.2 hold for a convex domain in a complete Riemannian
manifold of nonnegative Ricci curvature. In that case we impose the Neumann
boundary condition.

##  2\. Proof

Let $M$ be a closed Riemannian manifold and $\mu$ be the uniform probability
measure. For the isoperimetric profile $I=I_{(M,\mu)}:[0,1]\to\mathbb{R}$
(Refer to [12] for the definition) we put
$\tilde{I}(t):=\min\\{I(t),I(1-t)\\}$, $t\in[0,1/2]$.

######  Theorem 2.1 (E. Milman, [12, Remark 2.11]).

Let $M$ be an $n$-dimensional closed Riemannian manifold of nonnegative Ricci
curvature. Then we have the isoperimetric inequality

$\displaystyle\tilde{I}(t)\gtrsim\sqrt{\lambda_{1}(M)}t^{\frac{n-1}{n}}\ \
\forall t\in[0,1/2].$

In [12, Remark 2.11] Milman stated that

$\displaystyle\tilde{I}(t)\gtrsim D_{1,\infty}t^{\frac{n-1}{n}}\ \ \forall
t\in[0,1/2],$

where $D_{1,\infty}$ is the best constant that satisfies the
$(1,\infty)$-Poincaré inequality. Since $D_{1,\infty}$ is equivalent to
$\sqrt{\lambda_{1}(M)}$ by Milman’s result ([12, Theorem 2.4]), we have
Theorem 2.1.

For a measurable function $f:M\to\mathbb{R}$ we define a _median_ of $f$ as a
number $a$ such that

$\displaystyle\mu(f(x)\geq a)\geq\frac{1}{2}\text{ and }\mu(f(x)\leq
a)\geq\frac{1}{2}.$

Note that $a$ might not be unique. Let $M_{\mu}f$ denote one of medians of
$f$. We put

$\displaystyle E_{\mu}f:=\int_{M}f(x)d\mu$

whenever the right-hand side makes sense.

One can transfer the isoperimetric inequality appeared in Theorem 2.1 to a
$(1,\frac{n}{n-1})$-Poincaré inequality via the following theorem. Let
$\mathcal{F}$ be the space of Lipschitz functions on $M$.

######  Theorem 2.2 (Bobkov–Houdré [2], Federer–Fleming [5], Maz’ya [11]).

Let $0<r\leq 1$. Without any assumption on $M$ the $(1/r,1)$-Poincaré
inequality

$\displaystyle D\|f-M_{\mu}f\|_{\frac{1}{r}}\leq\||\nabla f|\|_{1}\ \ \forall
f\in\mathcal{F}$

is equivalent to the isoperimetric inequality

$\displaystyle\tilde{I}(t)\geq Dt^{r}\ \ \forall t\in[0,1/2].$

See [2, Theorem 1.1] for the proof of the above theorem.

Theorem 2.1 together with Theorem 2.2 yields the following.

######  Corollary 2.3.

Let $M$ be an $n$-dimensional closed Riemannian manifold of nonnegative Ricci
curvature. Then we have the $(\frac{n}{n-1},1)$-Poincaré inequality

$\displaystyle\sqrt{\lambda_{1}(M)}\|f-M_{\mu}f\|_{\frac{n}{n-1}}\lesssim\||\nabla
f|\|_{1}\ \ \forall f\in\mathcal{F}.$

From now on we shall study Poincaré inequalities.

######  Lemma 2.4 ([13, Section 1.1.3]).

Suppose that the $(\frac{n}{n-1},1)$-Poincaré inequality

(2.1) $\displaystyle D\|f-M_{\mu}f\|_{\frac{n}{n-1}}\leq\||\nabla f|\|_{1}\ \
\forall f\in\mathcal{F}$

holds for $M$. Then for any $p\geq 1$ we have the
$(\frac{np}{n-p},p)$-Poincaré inequality

(2.2) $\displaystyle
D\|f-M_{\mu}f\|_{\frac{np}{n-p}}\leq\frac{(n-1)p}{n-p}\||\nabla f|\|_{p}\ \
\forall f\in\mathcal{F}.$

In [13, Section 1.1.3] the above lemma is stated for $\mathbb{R}^{n}$. In
there the Poincaré inequalities corresponding to (2.1) and (2.2) are stated
for compactly supported smooth functions and the left-hand sides of (2.1) and
(2.2) in there are the norms of $f$, not the norms of $f-M_{\mu}f$.

###### Proof of Lemma 2.4.

Assume (2.1) and take $f\in\mathcal{F}$. We may assume that $M_{\mu}f=0$.
Setting $\alpha:=\frac{(n-1)p}{n-p}$ we find $\alpha>1$. For any $x\in M$
define

$\displaystyle g(x):=\left\\{\begin{array}[]{ll}|f(x)|^{\alpha}&\text{ if
}f(x)\geq 0\\\ -|f(x)|^{\alpha}&\text{ if }f(x)<0.\end{array}\right.$

Observe that $g\in\mathcal{F}$ and $M_{\mu}g=0$. Thus (2.1) yields that

$\displaystyle D\|f\|^{\alpha}_{\frac{\alpha
n}{n-1}}=D\|g\|_{\frac{n}{n-1}}\leq\alpha\int|f(x)|^{\alpha-1}|\nabla
f(x)|d\mu(x).$

Let $p^{\prime}$ be the conjugate of $p$, i.e., $p^{-1}+p^{\prime-1}=1$. Using
the Hölder inequality we obtain

$\displaystyle
D\|f\|_{\frac{np}{n-p}}^{\frac{(n-1)p}{n-p}}=D\|f\|^{\alpha}_{\frac{\alpha
n}{n-1}}\leq\alpha\||f|^{\alpha-1}\|_{p^{\prime}}\||\nabla
f|\|_{p}=\alpha\|f\|_{\frac{np}{n-p}}^{\frac{n(p-1)}{n-p}}\||\nabla f|\|_{p},$

which gives (2.2). This completes the proof. $\Box$

To connect the $L_{p}$ norm of $f-E_{\mu}f$ with $f-M_{\mu}f$ we use the
following.

######  Lemma 2.5 ([12, Lemma 2.1]).

For any $p\geq 1$ and $f\in\mathcal{F}$ we have

$\displaystyle\frac{1}{2}\|f-E_{\mu}f\|_{p}\leq\|f-M_{\mu}f\|_{p}\leq
3\|f-E_{\mu}f\|_{p}.$

Lemma 2.4 together with Lemma 2.5 gives the following.

######  Lemma 2.6 ([13, Section 1.1.3]).

Suppose that the $(\frac{n}{n-1},1)$-Poincaré inequality

$\displaystyle D\|f-E_{\mu}f\|_{\frac{n}{n-1}}\leq\||\nabla f|\|_{1}\ \
\forall f\in\mathcal{F}$

holds for $M$. Then for any $p\geq 1$ we have the
$(\frac{np}{n-p},p)$-Poincaré inequality

$\displaystyle
D\|f-E_{\mu}f\|_{\frac{np}{n-p}}\lesssim\frac{(n-1)p}{n-p}\||\nabla f|\|_{p}\
\ \forall f\in\mathcal{F}.$

Combining Corollary 2.3 with Lemmas 2.5 and 2.6 we obtain the following.

######  Corollary 2.7.

Let $M$ be an $n$-dimensional closed Riemannian manifold of nonnegative Ricci
curvature. Then for any $p\geq 1$ we have the $(\frac{np}{n-p},p)$-Poincaré
inequality

$\displaystyle\sqrt{\lambda_{1}(M)}\|f-E_{\mu}f\|_{\frac{np}{n-p}}\lesssim\frac{(n-1)p}{n-p}\||\nabla
f|\|_{1}\ \ \forall f\in\mathcal{F}.$

The Hölder inequality implies the following lemma.

######  Lemma 2.8.

Suppose that we have the $(\frac{2n}{n-2},2)$-Poincaré inequality

$\displaystyle D\|f-E_{\mu}f\|_{\frac{2n}{n-2}}\leq\||\nabla f|\|_{2}\ \
\forall f\in\mathcal{F}.$

Then we have

$\displaystyle
D\|f-E_{\mu}f\|_{2}^{\frac{2+n}{n}}\|f-E_{\mu}f\|_{1}^{-\frac{2}{n}}\leq\||\nabla
f|\|_{2}\ \ \forall f\in\mathcal{F}.$

######  Theorem 2.9 (Cheng-Li, [3, Theorem 1]).

Let $M$ be a closed Riemannian manifold. Assume that

(2.3) $\displaystyle
D\|f-E_{\mu}f\|_{2}^{\frac{2+n}{n}}\|f-E_{\mu}f\|_{1}^{-\frac{2}{n}}\leq\||\nabla
f|\|_{2}\ \ \forall f\in\mathcal{F}$

holds. Then we have

$\displaystyle\lambda_{k}(M)\gtrsim k^{\frac{2}{n}}D^{2}$

for any $k$.

###### Proof of Theorem 1.1.

Combining Corollary 2.7 with Lemma 2.8 we have

$\displaystyle\sqrt{\lambda_{1}(M)}\|f-E_{\mu}f\|_{2}^{\frac{2+n}{n}}\|f-E_{\mu}f\|_{1}^{-\frac{2}{n}}\lesssim\||\nabla
f|\|_{2}\ \ \forall f\in\mathcal{F}.$

Theorem 2.9 thereby implies the theorem. This completes the proof. $\Box$

For the completeness of this paper we give the proof of Theorem 2.9 following
[3].

###### Proof of Theorem 2.9.

Let $H(x,y,t)$ denotes the fundamental solution of the heat equation, i.e.,

$\displaystyle\frac{\partial H}{\partial t}(x,y,t)=\Delta_{x}H(x,y,t)$

and

$\displaystyle\int_{M}H(x,y,t)f(y)d\mu(y)\to f(x)\text{ as }t\to 0$

for any $f\in C^{\infty}(M)$ ([6, 7]). We define $G:M\times
M\times[0,\infty)\to\mathbb{R}$ as $G(x,y,t):=H(x,y,t)-1$. Denote by
$\\{\phi_{i}\\}_{i=0}^{\infty}$ be an orthnormal basis of eigenfunctions of
the Laplacian with respect to $\mu$, where $\phi_{i}$ corresponds to the
eigenvalue $\lambda_{k}(M)$. Since $\phi_{0}=1$ we have an expansion

$\displaystyle
G(x,y,t)=\sum_{i=1}^{\infty}e^{-\lambda_{i}(M)t}\phi_{i}(x)\phi_{i}(y).$

Note also that the semi-group property

$\displaystyle G(x,y,t+s)=\int_{M}G(x,z,s)G(z,y,t)d\mu(z)\ \ \forall x,y\in M\
\forall s,t\geq 0$

holds. We thereby have

$\displaystyle G^{\prime}(x,x,t)=\ $
$\displaystyle\int_{M}G^{\prime}(x,y,t/2)G(x,y,t/2)d\mu(y)$ $\displaystyle=\ $
$\displaystyle\int_{M}\Delta_{y}G(x,y,t/2)G(x,y,t/2)d\mu(y).$

Since

$\displaystyle\int_{M}|G(x,y,t)|d\mu(y)\leq\int_{M}|H(x,y,t)|d\mu(y)+1=2,$

the assumption (2.3) and the semi-group property give

$\displaystyle-G^{\prime}(x,x,t)=\ $
$\displaystyle\int_{M}|\nabla_{y}G(x,y,t/2)|^{2}d\mu(y)$ $\displaystyle\geq\ $
$\displaystyle
2^{-\frac{4}{n}}D^{2}\Big{(}\int_{M}G(x,y,t/2)^{2}d\mu(y)\Big{)}^{\frac{2+n}{n}}$
$\displaystyle=\ $ $\displaystyle
2^{-\frac{4}{n}}D^{2}G(x,x,t)^{\frac{2+n}{n}}.$

Since $G(x,x,t)=H(x,x,t)-1\to\infty$ as $t\to 0$, we obtain

$\displaystyle\frac{n}{2}G(x,x,t)^{-\frac{2}{n}}\geq 2^{-\frac{4}{n}}D^{2}t$

It gives

(2.4) $\displaystyle G(x,x,t)\leq
4\Big{(}\frac{2D^{2}}{n}\Big{)}^{-\frac{n}{2}}t^{-\frac{n}{2}}.$

Integrating both-sides yields

$\displaystyle\sum_{i=1}^{\infty}e^{-\lambda_{i}(M)t}\leq
4\Big{(}\frac{2D^{2}}{n}\Big{)}^{-\frac{n}{2}}t^{-\frac{n}{2}}.$

Putting $t:=n/(2\lambda_{k}(M))$ we therefore obtain

$\displaystyle
ke^{-\frac{n}{2}}\leq\sum_{i=1}^{k}e^{-\frac{\lambda_{i}(M)}{2\lambda_{k}(M)}n}\leq\sum_{i=1}^{\infty}e^{-\frac{\lambda_{i}(M)}{2\lambda_{k}(M)}n}\leq
4\Big{(}\frac{2\lambda_{k}(M)}{D^{2}}\Big{)}^{\frac{n}{2}},$

which implies the conclusion of the theorem. This completes the proof. $\Box$

In [3] instead of putting $t=n/(2\lambda_{k}(M))$ in the above proof, Cheng-Li
put $t=1/(2\lambda_{k}(M))$ and the conclusion in there becomes
$\lambda_{k}(M)\gtrsim c_{n}k^{\frac{2}{n}}D^{2}$, where $c_{n}>0$ is a
constant depending only on $n$. In [14, Chapter III §5] it is putted
$t=n/(4\lambda_{k}(M))$ but the assumption there is different. The assumption
there seems the $(\frac{2n}{n-2},2)$-Poincaré inequality

$\displaystyle D\|f\|_{\frac{2n}{n-2}}\leq\||\nabla f|\|_{2}\ \ \forall
f\in\mathcal{F}$

to derive the lower bound of $\lambda_{k}(M)$.

We also give the proof of Theorem 1.2 following [3, Corollary 1].

###### Proof of Theorem 1.2.

Corollary 2.7, Lemma 2.8 and (2.4) yield

$\displaystyle e^{-\lambda_{k}(M)t}\phi_{k}(x)^{2}\leq
4\Big{(}\frac{c\lambda_{1}(M)}{n}\Big{)}^{-\frac{n}{2}}t^{-\frac{n}{2}},$

where $c>0$ is a universal constant. Thus substituting
$t:=n/(2\lambda_{k}(M))$ we obtain (1.3). This completes the proof. $\Box$

## References

  * [1] G. Avakumovic, Über die Eigenfunktionen auf geschlossenen Riemannschen Mannigfaltigkeiten, Math. Z. 65 (1956), p. 327–344.
  * [2] S. G. Bobkov and C. Houdré, Some connections between isoperimetric and Sobolev-type inequalities. Mem. Am. Math. Soc. 129(616) (1997).
  * [3] S. Y. Cheng and P. Li, Heat kernel estimates and lower bound of eigenvalues. Comment.Math. Helv. 56 (1981), no. 3, 327–338.
  * [4] J. J. Duistermaat and V. W. Guillemin, The spectrum of positive elliptic operators and periodic bicharacteristics. Invent Math 29, 39–79 (1975).
  * [5] H. Federer and W. H. Fleming, Normal and integral currents. Ann. Math. (2) 72, 458–520 (1960).
  * [6] A. Grigor’yan, Heat kernel and analysis on manifolds. AMS/IP Studies in Advanced Mathematics, 47. American Mathematical Society, Providence, RI; International Press, Boston, MA, 2009.
  * [7] A. Grigor’yan, Heat kernels on weighted manifolds and applications, The ubiquitous heat kernel, 93–191. Contemp. Math., 398 American Mathematical Society, Providence, RI, 2006.
  * [8] L. Hörmander, The spectral function of an elliptic operator, Acta Math. 121 (1968), 193–218.
  * [9] B. Levitan, On the asymptotic behavior of the spectral function of a self-adjoint differential equation of second order. Isv. Akad. Nauk SSSR Ser. Mat. 16 (1952), p.325–352.
  * [10] S. Liu, An optimal dimension-free upper bound for eigenvalue ratios, preprint arXiv:1405.2213v3.
  * [11] V. G. Maz’ja, Sobolev Spaces. Springer Series in Soviet Mathematics. Springer, Berlin (1985).
  * [12] E. Milman, On the role of convexity in isoperimetry, spectral gap and concentration, Invent. Math. 177 (2009), no. 1, 1–43.
  * [13] L. Saloff-Coste, Aspects of Sobolev-type inequalities. London Math. Soc. Lecture Note Ser., 289 Cambridge University Press, Cambridge, 2002.
  * [14] R. Schoen and S.-T. Yau, Lectures on differential geometry. Lecture notes prepared by Wei Yue Ding, Kung Ching Chang [Gong Qing Zhang], Jia Qing Zhong and Yi Chao Xu. Translated from the Chinese by Ding and S. Y. Cheng. Preface translated from the Chinese by Kaising Tso. Conference Proceedings and Lecture Notes in Geometry and Topology, I. International Press, Cambridge, MA, 1994.
  * [15] C. D. Sogge, Concerning the Lp norm of spectral clusters for second-order elliptic operators on compact manifolds. J. Funct. Anal.77(1988), no.1, 123–138.
  * [16] S. Zelditch, Eigenfunctions of the Laplacian on a Riemannian manifold. CBMS Reg. Conf. Ser. Math., 125 Published for the Conference Board of the Mathematical Sciences, Washington, DC; by the American Mathematical Society, Providence, RI, 2017.