Copyright for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).

Woodstock’20: Symposium on the irreproducible science, June 01–05, 2020,
Woodstock, NY

# NLP is Not enough - Contextualization of User Input in Chatbots

Nathan Dolbir  Triyasha Dastidar  Kaushik Roy  Artificial Intelligence
Institute, University of South Carolina  BITS-Pilani Hyderabad

(2020)

###### Abstract

AI chatbots have made vast strides in technology improvement in recent years
and are already operational in many industries. Advanced Natural Language
Processing techniques, based on deep networks, efficiently process user
requests to carry out their functions. As chatbots gain traction, their
applicability in healthcare is an attractive proposition due to the reduced
economic and people costs of an overburdened system. However, healthcare bots
require safe and medically accurate information capture, which deep networks
aren’t yet capable of due to user text and speech variations. Knowledge in
symbolic structures is more suited for accurate reasoning but cannot handle
natural language processing directly. Thus, in this paper, we study the
effects of combining knowledge and neural representations on chatbot safety,
accuracy, and understanding.

##  1 Introduction

Conversational AI in its brief existence has already been extensively deployed
in nearly every economic sector and is becoming increasingly easier for
businesses to use. Its ability to mirror human conversation has made it a
cost-effective solution to many customer service issues and it can provide
users a stream of knowledge bounded only by the information it can access. The
issue with AI’s ability to mirror human natural language is that it doesn’t
have any true understanding of the language it is processing, but it can
recognize patterns and characteristics of text based on repeated feeding of
test input and by classification of a word and sentence meanings into a user-
designed schema. Pattern recognition has evolved into deep learning of
patterns it recognized in the context of the training a user has performed.
Deep learning models have excelled at Natural Language Processing (NLP) tasks
such as question-answering from the text in recent years [1]. Their widespread
success has led to speedy adoption in real-world industrial applications [2].
However, these models depend on pattern recognition and prediction techniques
which often fail to correctly contextualize bits of input, especially with
open-ended inputs based more on natural speech than an easily processed
formulaic response. When parsing user input, neural networks determine the
significance of each word concerning the other which summarizes into patterns
the network recognized and will continue to observe in following input. The
issue that arises from this is when the network recognizes and utilizes
patterns that are often unnecessary and at times even detrimental to the AI’s
purpose. If, for example, a mental health patient were communicating with a
chatbot and said that they would wish they were not alive, the chatbot may
associate ’not alive’ as a positive sentiment. If the chatbot didn’t have
pretraining done to recognize death as perhaps the most negative sentiment in
the entire mental health domain, the chatbot may recommend that the patient
commit suicide, which unfortunately did happen to a simulated patient using
OpenAI’s GPT-3 neural network [3]. Deep learning’s flaws make it extremely
difficult to use in the mental health interaction setting because the safety
of the patient is of utmost importance, and consequently, there is no room for
error in contextualizing user input and formulating responses. However, mental
health AI absent of all deep learning makes patient interaction monotonous and
devoid of any personalized experience. This form of patient interaction
reflects more of a question and answer with pre-formulated data which the
patient may find useless or likely could simply find online themselves.
Furthermore, deep learning is useful in recognizing patterns in data that
can’t be symbolically processed based only on linguistics and predefined
schema. Therefore, deep learning in combination with symbolic AI processing
would give users the best possible personalized experience with conversational
AI. Thus we study possible cross-overs and their effect on the safety of chat-
bot interactions, the accuracy of medical information parsed, and user
engagement. We conduct a short experiment by training a GPT-2 model on a
therapist conversation dataset from the app counsel chat to analyze its
safety, accuracy, and user engagement before proposing potential fixes to its
shortcomings. Our key contributions are thus:

  * •

We Scrutinize a Language Models ability, once fined-tuned on therapist
conversations, to interact with a mental health patient.

  * •

We identify issues and propose solutions that can use knowledge with deep
learning methods to provide the most optimal user experience.

##  2 Methods

###  2.1 Language Models

Language models generate a probability distribution over a sequence of tokens,
given an input and a learning algorithm [4]. We focus on language models that
are trained using the self-attention mechanism. Each token of the input
sentence learns to attend to each other token in the input sentence while
generating the output response. Thus, they essentially reproduce sentences
based on repetitive patterns of association between the tokens [5]. The
associations’ strength is calculated using the tokens’ embeddings in a vector
space, also known as distributed representations. As an example, "How are you?
-> I am fine, thank you." is a likely input-output pair due to the statistical
frequency of the pair in the training Corpus (data). Therefore, without the
right training data, the model can produce harmful results in an application
such as mental health. Recently, the GPT-3 model asked a patient to commit
suicide [6]; we train a GPT-2 model on therapist conversations from the
counsel-chat app [7] to fine-tune the language model to a therapist
conversation setting. We see some examples and analyze the responses from the
perspective of user safety, the accuracy of parsing medical information, and
user engagement:

###  2.2 Knowledge for dialogue Generation towards specific objectives

Contextualization of knowledge within the domain the chatbot makes symbolic
connections between important ideas within that domain more easily connected
and fluid. A chatbot designed to serve a purpose without knowledge of that
purpose would not be expected to give as accurate of results as if it were
given information on the topic and attributes to its knowledge. In the mental
health domain. For example, a chatbot given information on depression and
quantification of severity for its symptoms would better assign the severity
of the patient’s condition than less contextualized information on the topic.
Knowledge of a domain without conversational input might be able to feed
information relevant to the input a user gave but would probably not be able
to communicate it as effectively because it misses crucial natural language
understanding. Because of this, we analyzed Question/Answer data from
"CounselChat.com", a counseling website that connects counselors and patients,
in addition to the structured knowledge that the medical source
"Mayoclinic.org" provides.

###  2.3 Proposed Solutions

In the broad scope of human-AI interaction, dialogue generated by symbolic
knowledge is far too mechanical for the user to be interested in continuing
talking with the AI, limiting the user data the AI can work with and therefore
hurting its efficiency. Language models’ ability to recognize the attention to
user input and predict optimal word choice given the previous and developing
context make it a far better choice to use for direct conversation. However,
language models cannot categorize words and sentences within a conversation
due to their formulated responses being generated by hidden states not
understandable to people which may lead to the AI forming arbitrary responses
which could be dangerous. We, therefore, propose that a conversational AI may
be most effective when combining language models and symbolically-based
processing techniques for separate tasks which they can be most efficient. In
an information-gathering scenario, for example, a mental health chatbot may
ask a user how they are dealing with a symptom on a certain day. Analysis of
the person’s response at a higher level would be best categorized into a
knowledge graph based on the semantic meaning of the response, but a language
model may serve well as an aid in determining the semantic meaning of a user.
When the user responds to this question with a more natural response (which
may carry more implied meaning than the explicit ’good’ or ’bad’ symbolically-
based processing would attempt to seek out), a language model may better
recognize that the response it received does not align well with the response
prediction it made from previous training data. For example, if a chatbot
asked a person struggling with anxiety "How did the work presentation go?" and
the person responded "It could’ve been worse," a language model may recognize
this phrase as some kind of compromise between ’good’ and ’bad’ because the
user did not have a more explicit response that the model has come to expect
from training data while symbolically-based processing may see ’worse’ and
categorize the presentation as bad because of the negative semantic
association with the word ’worse’. Following this, the conclusion of the
language model found of this response can be categorized and placed in a
symbolic structure such as a knowledge graph for future recollection.

As demonstrated in the above pipeline, the outcome of the project would be
better calculated using a predictive language model than if we were left only
with linguistic, syntactic, and other symbolic processing methods. If in the
future, the user were to describe the presentation again with the language
model finding a clearer, higher-value association with an outcome of the
presentation, the knowledge graph could be updated again and the language
model would have learned its previous decision was incorrect, and this
information would be added to the training data of the language model, further
improving its natural language processing ability for the future.

###  2.4 NLP Datasets for Mental Health Input Contextualization Comparison

The conversational fluency of a chatbot varies greatly with the amount of
training it has had and the inputs it has been trained with. Chatbots trained
to work within a specific domain of knowledge have developed more pattern
recognition toward that domain while generally leaving out miscellaneous
knowledge that may be unhelpful in that conversation setting. The
conversational skill of a chatbot trained in the domain of mental health would
greatly vary based on the input it has been given. A chatbot trained on
Wikipedia sentences may absorb the factual information about mental health
conditions but would be very hard to tune into a conversational agent if it is
given no human conversation to analyze and learn from. In this experiment, we
demonstrate how training a GPT-2-based conversational agent with mental health
chat data from the therapy website "Counselchat.com" makes the agent far more
receptive to language about mental health and strengthens its conversational
ability in that domain [8].

##  3 Results and Analysis

One of the findings of the experiment was that extraneous input makes an
undesirable output more likely, suggesting that narrowing an input to only the
precise question could produce the most effective answer. When giving a
chatbot an input, it must parse each word individually. Words that are not
relevant to the question the user is asking add extraneous information for the
chatbot to compare with its given dataset, and therefore broaden the amount of
information that the chatbot can assign similarity with the input to. If, for
example, a user wanted to inquire about the symptoms of depression, a concise
question such as "What are symptoms of depression" would yield a much more
accurate answer than if the user added extraneous input such as "I was
wondering about this because …".

Extraneous input may be filtered by narrowing the input into question-form. A
question may be present within the input which can be filtered out, commonly
started with an interrogative (what, where, how, etc), and ending with a
question mark ("?"). The placement of an interrogative and question mark (if
not present) would most effectively narrow the input into a question, but the
meaning of the question would still be unknown. Additionally, there may be
input outside of the interrogative and question mark which may affect the
meaning of the personalization of the question, such as if the user asked
about the symptoms of depression after giving their symptoms of depression
(the most effective answer would individually address the symptoms that the
user noted). It is, therefore, necessary to have thorough contextualization of
user input in the domain the chatbot serves. In the mental health domain,
symptoms the user described should already have meaning to the chatbot because
they could be found within a mental health knowledge graph, which is a
collection of subject relation object triples that contains information about
domain-specific information, or other types of knowledge databases [9]. If the
symptom could not be found, it may still be detected as a symptom by the
syntactic structure of the language surrounding the user’s input ("I feel" or
"My symptoms are" followed by the symptoms). Hence, the syntactic structure,
if provided explicitly, can help the neural network’s pattern recognition.
However, recognizing the semantics or the "meaning" required for natural
language understanding needs the use of a knowledge graph. Thus, we conclude
that a purely data-driven approach is insufficient to solve a complex problem
such as mental health counseling without adequate contextualization and
personalization tailored to the specific user. Figure 2 shows that the bot
seems to understand the user’s conversation at first, but Figure 1 illustrates
that wording the question to have the same meaning, but with extraneous input
exposes the lack of natural language understanding in the bot.

Figure 1: In this example, the same question was asked, but the extraneous
input in the second question produced a completely unrelated answer, more
closely aligned with the keywords of the question (’terrible’, ’talk’) than
the question itself. Figure 2: An example of an appropriate response from the
dataset given no clear question within the user input.

##  4 Conclusion and Future Work

Observing how large transformer models are trained on large-scale
conversational data to create a chatbot, conventional wisdom suggested that we
train a language model on counseling conversation to produce a mental health
chatbot. Our experimentation showed, however, that a lack of domain-specific
knowledge led to a lack of understanding of the user’s true inquiry in
conversational responses. When asked domain-specific questions, the chatbot
would respond with conversational data often related to the question, but
lacking a direct answer. Including domain knowledge in the chatbot’s
functionality improved it’s ability to recognize mental health information in
user input and relate it to domain-related knowledge. When user input was more
natural and less of a domain-related inquiry, conversational data often
succeeded in delivering an accurate and direct response to user input. We
conclude that using NLP pattern-recognition with training data in chatbots is
not enough; incorporating domain knowledge and using it to contextualize user
input improves the chatbot’s capability to generate informationally-accurate
and conversationally capable dialogue. In our experiments, articles used as
domain knowledge were fed to the language model as simple text (the only input
the language model takes), making the structure of each article irrelevant
(for example, a Mayo-Clinic article on anxiety listing "Causes" as heading
followed by list of causes of anxiety). Thus, future work centers around
taking structured domain knowledge bases and integrating them with deep-
learning models.

## References

  * Otter et al. [2020] D. W. Otter, J. R. Medina, J. K. Kalita,  A survey of the usages of deep learning for natural language processing,  IEEE Transactions on Neural Networks and Learning Systems (2020).
  * Li et al. [2018] L. Li, K. Ota, M. Dong,  Deep learning for smart industry: Efficient manufacture inspection system with fog computing,  IEEE Transactions on Industrial Informatics 14 (2018) 4665–4673.
  * Brown et al. [2020] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al.,  Language models are few-shot learners,  arXiv preprint arXiv:2005.14165 (2020).
  * Qiu et al. [2020] X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, X. Huang,  Pre-trained models for natural language processing: A survey,  Science China Technological Sciences (2020) 1–26.
  * Vaswani et al. [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin,  Attention is all you need,  arXiv preprint arXiv:1706.03762 (2017).
  * Roy et al. [2021] K. Roy, U. Lokala, V. Khandelwal, A. Sheth,  " is depression related to cannabis?": A knowledge-infused model for entity and relation extraction with limited supervision,  arXiv preprint arXiv:2102.01222 (2021).
  * Bertagnolli [2020] N. Bertagnolli, Counsel chat: Bootstrapping high-quality therapy data, 2020.
  * Budzianowski and Vulić [2019] P. Budzianowski, I. Vulić,  Hello, it’s gpt-2–how can i help you? towards the use of pretrained language models for task-oriented dialogue systems,  arXiv preprint arXiv:1907.05774 (2019).
  * Fensel et al. [2020] D. Fensel, U. Şimşek, K. Angele, E. Huaman, E. Kärle, O. Panasiuk, I. Toma, J. Umbrich, A. Wahler,  Introduction: What is a knowledge graph?,  in: Knowledge Graphs, Springer, 2020, pp. 1–10.


V\rVert,{\cdot})$ with respect to $\sigma_{V}$ (which is a well-defined Borel
function in $\operatorname{Dens}(V)$ and can be extended, for instance, to be
$0$ outside $\operatorname{Dens}(V)$; the chosen extension does not influence
the approximate continuity, because by (5.28) $x$ has density $1$ in
$\operatorname{Dens}(V)$, where $\Theta^{k}(\lVert V\rVert,{\cdot})$ is well-
defined).

By Lebesgue differentiation theorem and by Remark 5.1, we have

(5.29) $\sigma_{V}(\operatorname{Dens}(V)\setminus A)=0.$

In addition, by Theorem 1.4, for every $x\in A$ the map
$\rho\mapsto\varphi_{x}(\rho)$ is monotone increasing and converge pointwise
to $0$ as $\rho\downarrow 0$, that is

$\lim_{\rho\to 0}\varphi_{x}(\rho)=0\qquad\forall x\in A.$

Thus, by Egoroff’s Theorem, for every $h\in\mathbb{N}$ there exists a set
$F_{h}\subset A$ such that

(5.30) $\sigma_{V}(A\setminus F_{h})\leq
1/h,\qquad\varphi_{x}(\rho)\operatorname*{\overset{unif}{\longrightarrow}}_{\rho\to
0}0\mbox{ on }F_{h}.$

Up to removing sets of $\sigma_{V}$-measure $0$, we can assume that every
$x\in F_{h}$ is a point of density $1$ with respect to $\sigma_{V}$, that is

(5.31) $\lim_{\rho\to 0}\frac{\sigma_{V}\big{(}F_{h}\cap
B_{\rho}(x)\big{)}}{\sigma_{V}\big{(}B_{\rho}(x)\big{)}}=1\qquad\forall x\in
F_{h}.$

We now define

$F=A\cap\Big{(}\bigcup_{h\in\mathbb{N}}F_{h}\Big{)}.$

By (5.29) and (5.30) it follows

$\sigma_{V}^{*}(\operatorname{Dens}(V)\setminus F)=0.$

Let us fix $x_{0}\in F$ and let us consider $h\in\mathbb{N}$ such that
$x_{0}\in F_{h}$. Without loss of generality we can assume $x_{0}=0$.

Now let us fix $r_{j}\downarrow 0$. Using the notations for the scalings,
since $F\subset\operatorname{Dens}(V)$, by Lemma 5.4 there exists a
subsequence, not relabeled, such that
$V_{j}\operatorname*{\overset{\ast}{\rightharpoonup}}C$ with
$C\in\operatorname{Tan}(V,0)$ and $C$ is a rectifiable cone with
$\Theta^{k}(\lVert C\rVert,y)\geq 1$ for $\lVert C\rVert$-a.e.
$y\in\mathbb{R}^{n}$.

We now need a technical remark that is useful in the rest of the proof:
recalling that $H_{j}(y)=r_{j}H(r_{j}y)$, by a simple change of variables one
obtains

$r_{j}x\in\operatorname{Dens}(V)\quad\Leftrightarrow\quad
x\in\operatorname{Dens}(V_{j}).$

More precisely, if we call $\varphi_{x}^{j}$ the function for $V_{j}$ defined
in Theorem 1.4, we get

(5.32) $\varphi_{x}^{j}(\rho)=\varphi_{r_{j}x}(r_{j}\rho).$

  2. Step 2:

We can now begin with the proof. If $\sigma_{C}=0$ there is nothing to prove.
Thus we can assume $\sigma_{C}\neq 0$ and
$\emptyset\neq\operatorname{supp}\sigma_{C}\subset\partial
T_{0}^{+}\mathcal{M}$. Since $C$ is a cone, By (5.26) we have

$\Theta^{k}(\lVert C\rVert,y)\leq\Theta^{k}(\lVert C\rVert,0)\qquad\forall
y\in\partial T_{0}^{+}\mathcal{M}.$

By contradiction, let us assume that there exists
$y\in\operatorname{supp}\sigma_{C}$ and $\varepsilon>0$ such that

(5.33) $\Theta^{k}(\lVert C\rVert,y)<\Theta^{k}(\lVert
C\rVert,0)-\varepsilon.$

Since $\sigma_{C}$ is scaling invariant by Lemma 5.4, without loss of
generality we can assume that $y\in B_{1/2}$. By the uniform convergence
(5.30) and by definition of density, there exists $\rho\in(0,1/2)$ such that

(5.34) $\frac{\lVert
C\rVert\big{(}B_{\rho}(y)\big{)}}{\rho^{k}}+\varphi_{z}(\rho)\leq\Theta^{k}(\lVert
C\rVert,y)+\frac{\varepsilon}{8}\qquad\forall z\in F_{h}.$

Since $V_{j}\operatorname*{\overset{\ast}{\rightharpoonup}}C$, without loss of
generality we can choose $\rho$ so that there exists $J\in\mathbb{N}$ and a
small $0<r<\rho$ for which

(5.35) $\left|\frac{\lVert
C\rVert\big{(}B_{\rho}(y)\big{)}}{\rho^{k}}-\frac{\lVert
V_{j}\rVert\big{(}B_{\rho}(y)\big{)}}{(\rho-r)^{k}}\right|<\frac{\varepsilon}{8}\qquad\forall
j>J.$

Let us choose $j>J$; for every $z\in B_{r}(y)$ such that $r_{j}z\in F_{h}$ we
have

$\begin{split}\Theta^{k}(\lVert V_{j}\rVert,z)&\leq\frac{\lVert
V_{j}\rVert\big{(}B_{\rho-r}(z)\big{)}}{(\rho-r)^{k}}+\varphi_{z}^{j}(\rho-r)\\\
&\leq\frac{\lVert
V_{j}\rVert\big{(}B_{\rho}(y)\big{)}}{(\rho-r)^{k}}+\varphi_{r_{j}z}(r_{j}\rho)\\\
&\stackrel{{\scriptstyle\eqref{eq:closeness_density_cones_scalings}}}{{\leq}}\frac{\lVert
C\rVert\big{(}B_{\rho}(y)\big{)}}{\rho^{k}}+\varphi_{r_{j}z}(r_{j}\rho)+\frac{\varepsilon}{8}\\\
&\stackrel{{\scriptstyle\eqref{eq:density_cone_close_limit}}}{{\leq}}\Theta^{k}(\lVert
C\rVert,y)+\frac{\varepsilon}{4}\\\
&\stackrel{{\scriptstyle\eqref{eq:density_cone_contradiction}}}{{\leq}}\Theta^{k}(\lVert
C\rVert,0)-\frac{3\varepsilon}{4}\\\ &=\Theta^{k}(\lVert
V_{j}\rVert,0)-\frac{3\varepsilon}{4}.\end{split}$

where we used the fact that every $\varphi_{x}$ is increasing and (5.32). This
shows that, for $j>J$,

$B_{r}(y)\cap\frac{1}{r_{j}}F_{h}\subseteq\left\\{z\in
B_{1}\mid\left|\Theta^{k}(\lVert V_{j}\rVert,z)-\Theta^{k}(\lVert
V_{j}\rVert,0)\right|>\frac{\varepsilon}{2}\right\\}.$

  3. Step 3:

We now want to estimate from below the measure of this set to get a
contradiction with the approximate continuity of $\Theta^{k}(\lVert
V\rVert,{\cdot})$ in $0$.

By approximate continuity of the $\Theta^{k}(\lVert V\rVert,{\cdot})$ in $0$
with respect to $\sigma_{V}$ we have

(5.36) $\begin{split}0&=\limsup_{j\to\infty}\frac{\sigma_{V}\left(\left\\{z\in
B_{r_{j}}\mid\left|\Theta^{k}(\lVert V\rVert,z)-\Theta^{k}(\lVert
V\rVert,0)\right|>\frac{\varepsilon}{2}\right\\}\right)}{\sigma_{V}(B_{r_{j}})}\\\
&\geq\limsup_{j\to\infty}\frac{\sigma_{V}\big{(}B_{rr_{j}}(r_{j}y)\cap
F_{h}\big{)}}{\sigma_{V}(B_{r_{j}})}\\\
&=\limsup_{j\to\infty}\frac{\sigma_{V}\big{(}B_{rr_{j}}(r_{j}y)\big{)}}{\sigma_{V}(B_{r_{j}})}\\\
&=\limsup_{j\to\infty}\frac{\sigma_{j}\big{(}B_{r}(y)\big{)}}{\sigma_{j}(B_{1})},\end{split}$

where the second identity is consequence of (5.31) and the last one follows by
(5.6). We want to estimate from below the last term to get a contradiction. To
do so, let us notice that, by
$\sigma_{j}\stackrel{{\scriptstyle\ast}}{{\rightharpoonup}}\sigma_{C}$ and
$y\in\operatorname{supp}\sigma_{C}$, there exist two constants $c,\beta>0$
such that, for $j$ sufficiently large,

$\sigma_{j}\big{(}B_{1}\big{)}\leq
c,\qquad\sigma_{j}\big{(}B_{r}(y)\big{)}\geq\beta,$

which contradicts (5.36).

This also prove the inclusion $\operatorname{supp}\sigma_{C}\subset D_{C}$. To
prove the other inclusion, let us notice that the scaling invariance of
$\sigma_{C}$ and $\operatorname{supp}\sigma_{C}\neq\emptyset$ imply
$0\in\operatorname{supp}\sigma_{C}$. Since $\sigma_{C}$ is invariant by
translations along $D_{C}$ by Lemma 5.6, we have the opposite inclusion and
$\operatorname{supp}\sigma_{C}=D_{C}$.

∎

We next prove that, for every $x\in F$ (where $F$ is the set defined in the
previous Lemma) such that $\Theta_{*}^{k-1}(\sigma_{V},x)>0$ and for every
$C\in\operatorname{Tan}(V,x)$, $\sigma_{C}$ is the surface measure of a
$(k-1)$-plane, which coincides with $D_{C}$.

######  Lemma 5.8.

Let $V$ satisfy Assumption 1 and let $F$ be the set defined in Lemma 5.7. For
every $x_{0}\in F$ such that $\Theta_{*}^{k-1}(\sigma_{V},x)>0$ and for every
$C\in\operatorname{Tan}(V,x_{0})$, the invariant subspace $D_{C}$ of $C$ is
$(k-1)$-dimensional; moreover there exists $\alpha_{0}>0$ such that

$\sigma_{C}=\alpha_{0}\mathcal{H}^{k-1}\llcorner D_{C}.$

###### Proof.

Let $x_{0}\in F$ be a fixed point and let us assume
$C\in\operatorname{Tan}(V,x_{0})$. Without loss of generality we can assume
that $x_{0}=0$ and that $T_{0}\partial\mathcal{M}$ is the subspace
$\\{x_{n}=0\\}$. Since $C\in\operatorname{Tan}(V,0)$ there exists
$r_{j}\downarrow 0$ such that, using the notations for the scalings,
$V_{j}\operatorname*{\overset{\ast}{\rightharpoonup}}C$. Lemma 5.4 asserts $C$
is a rectifiable cone and that that
$\sigma_{j}\stackrel{{\scriptstyle\ast}}{{\rightharpoonup}}\sigma_{C}$, where
$\sigma_{C}$ is the measure relative to $C$ given by Corollary 4.6.

We first recall that the condition

(5.37) $\Theta_{*}^{k-1}(\sigma_{V},0)=\liminf_{r\to
0}\frac{\sigma_{V}\big{(}B_{r}(x_{0})\big{)}}{r^{k-1}}>0,$

together with
$\sigma_{j}\stackrel{{\scriptstyle\ast}}{{\rightharpoonup}}\sigma_{C}$,
implies that $\sigma_{C}\neq 0$. Thus, Lemma 5.7 provides
$\operatorname{supp}\sigma_{C}=D_{C}$, where $D_{C}$ is the invariant subspace
$D_{C}$ of $C$, given by Lemma 5.6. $D_{C}$ is a linear subspace of
$\mathbb{R}^{n}$ and throughout this proof we call $m=\dim D_{C}$ its
dimension. By definition of $D_{C}$, we clearly have
$D_{C}\subset\\{x_{n}=0\\}$. Thus $m\leq n-1$. After a suitable change of
coordinates, we can assume that $D_{C}=\\{x_{m+1}=\dots=x_{n}=0\\}$.

For any $y\in D_{C}$ and any $r>0$, we denote by $Q_{D_{C}}(y,r)$ the closed
cube included in $D_{C}$ with center $y$, side of length $r$ and faces
parallel to the coordinate vectors $e_{1},\dots,e_{m}$. If we set

(5.38)
$\alpha_{0}=\liminf_{j\to\infty}\frac{\sigma_{V}\big{(}B_{r_{j}}\big{)}}{\omega_{k-1}r_{j}^{k-1}}.$

we have $\alpha_{0}>0$, by (5.37). Since $\sigma_{C}$ is invariant by scalings
(by Lemma 5.4) and by translations in $D_{C}$ (by Lemma 5.6), there exists a
fixed $\beta_{0}>0$ such that

(5.39) $\sigma_{C}\big{(}Q_{D_{C}}(y,r)\big{)}=\beta_{0}r^{k-1}\qquad\forall
y\in D_{C}\quad\forall r>0.$

We are going to show that (5.39) implies $m=k-1$. We argue by contradiction
and by cases:

  * •

Let us assume, by contradiction, that $m<k-1$. For each $l\in\mathbb{N}$,
there exists a covering $\\{Q_{i}^{l}\\}_{i=1}^{2^{lm}}$ of $Q_{D_{C}}(0,1)$
such that each $Q_{i}^{l}$ is a cube included in $D_{C}$ and of side length
$2^{-l}$. Therefore

$\sigma_{C}\big{(}Q_{D_{C}}(0,1)\big{)}\leq\sum_{i=1}^{2^{lm}}\sigma_{C}\big{(}Q_{i}^{l}\big{)}=2^{lm}\beta_{0}2^{-l(k-1)}\leq\beta_{0}2^{-l}\xrightarrow[l\to\infty]{}0.$

Thus $\sigma_{C}\big{(}Q_{D_{C}}(0,1)\big{)}=0$. By translation invariance of
$\sigma_{C}$, it follows that $\sigma_{C}=0$, which is a contradiction.

  * •

Let us assume now that $m\geq k$. We first observe that by approximation,
(5.39) holds also for cubes that are open in $D_{C}$. Hence, taking for every
$l\in\mathbb{N}$ a covering $\\{Q_{i}^{l}\\}_{i=1}^{2^{lm}}$ of
$Q_{D_{C}}(0,1)$ of cubes included in $D_{C}$ with disjoint interiors and of
side length $2^{-l}$, we have

$\sigma_{C}\big{(}Q_{D_{C}}(0,1)\big{)}\geq\sum_{i=1}^{2^{lm}}\sigma_{C}\Big{(}{\kern
0.0pt(Q_{i}^{l})}^{\mathrm{o}}\Big{)}=2^{lm}\beta_{0}2^{-l(k-1)}\geq\beta_{0}2^{l}\xrightarrow[l\to\infty]{}+\infty,$

(where ${\kern 0.0pt(Q_{i}^{l})}^{\mathrm{o}}$ is intended in the topology of
$D_{C}$) which is a contradiction.

This shows that $\dim D_{C}=k-1$. Since $\sigma_{C}$ is invariant by scalings
and translations in $D_{C}$, we have

$\frac{\sigma_{C}\big{(}B(r(y)\big{)}}{\mathcal{H}^{k-1}\big{(}D_{C}\cap
B_{r}(y)\big{)}}=\alpha_{0}\qquad\forall y\in D_{C},\,\forall r>0.$

By Radon-Nikodym Theorem [12, Theorem 4.7], we obtain that
$\sigma_{C}=\alpha_{0}\mathcal{H}^{k-1}\llcorner D_{C}$. ∎

###  5.3. Proof of Theorem 1.2

We can now prove Theorem 1.2.

###### Proof of Theorem 1.2.

By definition of $\sigma_{V}^{*}$ we have

(5.40)
$0<\Theta_{*}^{k-1}(\sigma_{V}^{*},x)\leq\Theta^{*(k-1)}(\sigma_{V}^{*},x)<+\infty\qquad\mbox{for
}\sigma_{V}^{*}\mbox{-a.e. }x\in\partial\mathcal{M},$

thus $\sigma_{V}^{*}$ satisfies the first condition of the Marstrand-Mattila
Rectifiability Criterion (Theorem 5.1).

To check the second condition of the criterion, let us notice that Corollary
5.5 yield

(5.41) $\operatorname{Tan}^{k-1}(\sigma_{V}^{*},x)=\\{\sigma_{C}\mid
C\in\operatorname{Tan}(V,x)\\}\qquad\mbox{for }\sigma_{V}^{*}\mbox{-a.e.
}x\in\partial\mathcal{M}.$

Since $\Theta_{*}^{k-1}(\sigma_{V},x)>0$ for every $x\in E$ where $E$ is
defined in (1.8), Lemma 5.7 and Lemma 5.8 yields

(5.42) $\\{\sigma_{C}\mid
C\in\operatorname{Tan}(V,x)\\}\subset\\{\alpha\mathcal{H}^{k-1}\llcorner S\mid
S\,(k-1)\mbox{-dimensional plane},\alpha>0\\}\qquad\forall x\in F\cap E,$

where $F$ is the set defined in Lemma 5.7. Moreover

(5.43) $\begin{split}\sigma_{V}^{*}\big{(}\partial\mathcal{M}\setminus(F\cap
E)\big{)}\leq&\,\sigma_{V}^{*}\big{(}\partial\mathcal{M}\setminus\operatorname{Dens}(V)\big{)}+\sigma_{V}^{*}\big{(}\operatorname{Dens}(V)\setminus
F\big{)}+\sigma_{V}^{*}\big{(}\partial\mathcal{M}\setminus E\big{)}\\\
=&\,0.\end{split}$

Every set in the right-hand side is $\sigma_{V}^{*}$-negligible because:
$\sigma_{V}^{*}\big{(}\partial\mathcal{M}\setminus\operatorname{Dens}(V)\big{)}=0$
by Lemma 5.2 and $p>1$; $\sigma_{V}^{*}\big{(}\operatorname{Dens}(V)\setminus
F\big{)}=0$ by Lemma 5.7 and $\sigma_{V}^{*}\ll\sigma_{V}$;
$\sigma_{V}^{*}\big{(}\partial\mathcal{M}\setminus E\big{)}=0$ by definition
of $\sigma_{V}^{*}$.

Summarizing (5.41), (5.42) and (5.43), we obtain

$\operatorname{Tan}^{k-1}(\sigma_{V}^{*},x)\subset\\{\alpha\mathcal{H}^{k-1}\llcorner
S\mid\alpha>0,\,S\text{ is a }(k-1)\mbox{-dimensional plane}\\}\qquad\mbox{for
}\sigma_{V}^{*}\mbox{-a.e. }x\in\partial\mathcal{M}.$

Since this is the second condition for the Marstrand-Mattila Rectifiability
Criterion, we have that $\sigma_{V}^{*}$ is $(k-1)$-rectifiable. ∎

## References

  * [1] W.. Allard  “On the First Variation of a Varifold”  In _Annals of Mathematics_ Annals of Mathematics, 1972, pp. 417–491
  * [2] W.. Allard  “On the First Variation of a Varifold: Boundary Behavior”  In _Annals of Mathematics_ Annals of Mathematics, 1975, pp. 418–446
  * [3] T. Bourni  “Allard-type boundary regularity for $C^{1,\alpha}$ boundaries”  In _Advances in Calculus of Variations_ , 2015
  * [4] C. De Lellis  “Rectifiable Sets, Densities and Tangent Measures”, Zurich lectures in advanced mathematics  European Mathematical Society, 2008
  * [5] G. De Philippis, A. De Rosa and F. Ghiraldin  “Rectifiability of varifolds with locally bounded first variation with respect to anisotropic surface energies”  In _Comm. Pure App. Math._ , 2016
  * [6] N. Edelen  “The free-boundary Brakke flow”  In _Journal für die reine und angewandte Mathematik (Crelles Journal)_ 2020, 2018
  * [7] L.C. Evans and R.F. Gariepy  “Measure Theory and Fine Properties of Functions, Revised Edition”, Textbooks in Mathematics  CRC Press, 2015
  * [8] H. Federer and W.. Ziemer  “The Lebesgue set of a function whose distribution derivatives are p-th power summable”  In _Indiana University Mathematics Journal_ 22.2, 1972, pp. 139–158
  * [9] M. Grüter and J. Jost  “Allard type regularity results for varifolds with free boundaries”  In _Annali della Scuola Normale Superiore di Pisa - Classe di Scienze_ 13.1 Scuola normale superiore, 1986, pp. 129–169
  * [10] P. Mattila  “Geometry of Sets and Measures in Euclidean Spaces: Fractals and Rectifiability”  Cambridge University Press, 1995
  * [11] U. Menne  “Some applications of the isoperimetric inequality for integral varifolds”  In _Advances in Calculus of Variations_ 2.3, 01 Jul. 2009, pp. 247–269
  * [12] L. Simon  “Lectures on geometric measure theory”, Proceedings of the Centre for Mathematical Analysis  Australian National University, 1984
  * [13] L. Simon  “Theorems on Regularity and Singularity of Energy Minimizing Maps”, Lectures in Mathematics. ETH Zürich  Birkhäuser Basel, 1996
  * [14] B. White  “Stratification of minimal surfaces, mean curvature flows, and harmonic maps.”  In _Journal für die reine und angewandte Mathematik_ 488, 1997, pp. 1–36