# Fourier Coefficients of Hilbert Modular Forms at Cusps

Tim Davis

###### Abstract.

The aim of this article is to study the fields generated by the Fourier
coefficients of Hilbert newforms at arbitrary cusps. Precisely, given a
cuspidal Hilbert newform $f$ and a matrix $\sigma$ in (a suitable conjugate
of) the Hilbert modular group, we give a cyclotomic extension of the field
generated by the Fourier coefficients at infinity which contains all the
Fourier coefficients of $f||_{k}\sigma$.

###### Contents

  1. 1 Introduction
  2. 2 Background Setting and Notation
  3. 3 The Whittaker Model
  4. 4 Modular Forms Case
  5. 5 Preliminary Results
  6. 6 Main Result

##  1\. Introduction

Let $f$ be a normalised Hecke eigenform for $\Gamma_{0}(N)$ of weight $k$. It
is known that the field generated by the Fourier coefficients of $f$ is a
number field, see [19, Proposition 2.8]. This number field we denote by
${\mathbb{Q}}(f)$. Furthermore, for any matrix
$g\in\operatorname{SL}_{2}({\mathbb{Z}})$, an application of the $q$-expansion
principle shows that the Fourier coefficients of $f|_{k}g$ lie in the
cyclotomic extension ${\mathbb{Q}}(f)(\zeta_{N})$, see [6, Remark 12.3.5] for
more details. In recent years, algorithms have been developed to compute these
Fourier coefficients. These were developed by several different authors, see
[4, 8] for more details about these algorithms. Their methods use the
knowledge that these numbers are in fact algebraic numbers. This means if we
know more about the number field where these Fourier coefficients lie, this
could speed up the computations.

Therefore the question is given a Hecke eigenform (which we may assume to be a
newform) $f$ of level $N$ and weight $k$ and
$\sigma\in\operatorname{SL}_{2}({\mathbb{Z}})$; what is the number field that
the Fourier coefficients of $f|_{k}\sigma$ generate? Or a slightly weaker
question is: can one write down an explicit subfield of
${\mathbb{Q}}(f)(\zeta_{N})$, depending on the entries of $\sigma$, which
contains all the Fourier coefficients of $f|_{k}\sigma$? This was answered in
a paper of Brunault and Neururer, who proved the following result [2, Theorem
4.1].

Let $f$ be a normalised newform on $\Gamma_{0}(N)$ with weight $k$. Let
${\mathbb{Q}}(f)$ be the field generated by all the Fourier coefficients of
$f$. Let $\sigma=(\begin{smallmatrix}a&b\\\
c&d\end{smallmatrix})\in\operatorname{SL}_{2}(\mathbb{Z})$. Then the Fourier
coefficients of $f|_{k}\sigma$ lie in the cyclotomic extension
${\mathbb{Q}}(f)(\zeta_{N^{\prime}})$ where $N^{\prime}=N/(cd,N)$.

The method of Brunault and Neururer was classical. They use a result of
Shimura [18, Theorem 8], which studied the connections between two actions on
spaces of modular forms: the action of
$\operatorname{GL}_{2}^{+}({\mathbb{Q}})$ via the slash-operator and the
action of $\operatorname{Aut}({\mathbb{C}})$ on the Fourier coefficient of a
modular form. The proof of Brunault and Neururer also applies to modular forms
of $\Gamma_{0}(N)$ that are not necessarily newforms but in that case it is
not known if the field is optimal. In this paper we give a new proof of the
result of Brunault and Neuruer as well as a substantial generalisation (to the
case of Hilbert modular forms) using adelic and local representation-theoretic
methods. Specifically we use local Whittaker functions and their invariance
properties. The starting point is an explicit formula for the Fourier
coefficients of $f|_{k}\sigma$ given in [5, Section 3], in terms of local
Whittaker functions. From there we find sufficient conditions for
$\tau\in\operatorname{Aut}({\mathbb{C}})$ to fix this product. The advantage
of this method is that it gives an insight into how one could prove analogous
results for other families of modular forms, specifically it can be
generalised to Hilbert modular forms. One would suspect that a similar method
can also be used to prove a result of the same style for Whittaker
coefficients (or other factorisable periods) of automorphic forms lying in
cohomological automorphic representations of higher rank groups.

The main result of this paper is an extension of Brunault and Neururer stated
above to the case of cuspidal Hilbert newforms. Let $F$ be a totally real
number field of degree $n$ with narrow class group of size $h$. Let
${\mathfrak{n}}$ be a fixed integral ideal of ${\mathcal{O}}_{F}$. For
$\mu=1,...,h$, we define the congruence subgroup
$\Gamma_{\mu}({\mathfrak{n}})$ of $\operatorname{GL}_{2}(F)$ as

${\Gamma_{\mu}({\mathfrak{n}})}=\bigg{\\{}\begin{pmatrix}a&b\\\
c&d\end{pmatrix}:a,d\in{\mathcal{O}}_{F},b\in(t_{\mu})^{-1}\mathfrak{D}_{F}^{-1},c\in{\mathfrak{n}}t_{\mu}\mathfrak{D}_{F},ad-
bc\in{\mathcal{O}}_{F}^{\times}\bigg{\\}},$

where ${\mathfrak{D}}_{F}$ is the absolute different of $F$ and $t_{\mu}$ is a
representative of the narrow class group.

We identify $\alpha\in\operatorname{GL}_{2}(F)$ with an element of
$\operatorname{GL}_{2}({\mathbb{R}})^{n}$ via the various embeddings of $F$ in
${\mathbb{R}}$ and for $y=(y_{1},...,y_{n})\in{\mathbb{C}}^{n}$ and
$k=(k_{1},...,k_{n})\in{\mathbb{Z}}^{n}$ we use the notation
$y^{k}:=y_{1}^{k_{1}}...y_{n}^{k_{n}}$.

Then $f:{\mathbb{H}}^{n}\rightarrow{\mathbb{C}}$ is a Hilbert modular form of
weight $k=(k_{1},...,k_{n})$ and level ${\Gamma_{\mu}({\mathfrak{n}})}$ if $f$
is holomorphic on ${\mathbb{H}}^{n}$ and cusps and we have
$f||_{k}\alpha(z):=\det\alpha^{k/2}(cz+d)^{-k}f(\alpha z)=f(z)$ for every
$\alpha=(\begin{smallmatrix}a&b\\\
c&d\end{smallmatrix})\in\Gamma_{\mu}({\mathfrak{n}})$. Any $f$ of the above
form has a Fourier expansion of the form

$f(z)=\sum_{\xi}a(\xi;f)e^{2\pi i\operatorname{Tr}(\xi z)}.$

Following Shimura [19, 2.24], we define for $f$ as above, $1\leq\mu\leq h$,

$c_{\mu}(\xi;f)=N(t_{\mu}{\mathcal{O}}_{F})^{-k_{0}/2}a(\xi;f)\xi^{(k_{0}\mathbf{1}-k)/2},$

where $\mathbf{1}=(1,...,1)$ and $k_{0}=\max\\{k_{1},...,k_{n}\\}$.

We say $f$ is a Hilbert cuspform if the the constant term in the Fourier
expansion of $f||_{k}\gamma$ is zero for every
$\gamma\in\operatorname{GL}_{2}^{+}(F)$. A cuspidal Hilbert newform of weight
$k$ and level ${\mathfrak{n}}$ is a tuple ${\mathbf{f}}=(f_{1},...,f_{h})$
where $f_{\mu}$ is a Hilbert cuspform for $\Gamma_{\mu}({\mathfrak{n}})$ and
such that ${\mathbf{f}}$ satisfies additional properties (it does not come
from a form of lower level and is a Hecke eigenform), see [19, Section 2] for
more details. A normalised cuspidal Hilbert newform ${\mathbf{f}}$ (also
called primitive form) is one such that the first Fourier coefficient of
${\mathbf{f}}$ (which is non-zero) is normalised to equal 1. A result of
Shimura [19, Proposition 2.8], tells us that the set of all
$\\{c_{\mu}(\xi;f_{\mu}):1\leq\mu\leq h\ \text{and}\ \xi\in F_{+}\\}$
generates a totally real or CM number field, denoted
${\mathbb{Q}}({\mathbf{f}})$, under the assumption that $k_{1}\equiv...\equiv
k_{n}\pmod{2}$. We prove the following result.

######  Theorem 1 (Theorem 6.1).

Let ${\mathbf{f}}=(f_{1},...,f_{h})$ be a normalised Hilbert newform of weight
$k=(k_{1},...,k_{n})$ with $k_{1}\equiv...\equiv k_{n}\pmod{2}$ and level
${\mathfrak{n}}$. Let $1\leq\mu\leq h$ and let
$\sigma=(\begin{smallmatrix}a&b\\\ c&d\end{smallmatrix})\in\Gamma_{\mu}(1)$.
Let $f_{\mu}||_{k}\sigma$ have the Fourier expansion

$f_{\mu}||_{k}\sigma(z)=\sum_{\xi}a_{\mu}(\xi;\sigma)e^{2\pi
i\operatorname{Tr}(\xi z)},$

and define

$c_{\mu}(\xi;f_{\mu}||_{k}\sigma)=N(t_{\mu}{\mathcal{O}}_{F})^{-k_{0}/2}a_{\mu}(\xi;\sigma)\xi^{(k_{0}\mathbf{1}-k)/2}.$

Then $c_{\mu}(\xi;f_{\mu}||_{k}\sigma)$ lie in the number field
${\mathbb{Q}}({\mathbf{f}})(\zeta_{N_{0}})$ where $N_{0}$ is the integer such
that
$N_{0}{\mathbb{Z}}={\mathfrak{n}}/(cdt_{\mu}^{-1}\mathfrak{D}_{F}^{-1},{\mathfrak{n}})\cap{\mathbb{Z}}$.

To prove this theorem we study the classical action of
$\operatorname{Aut}({\mathbb{C}})$ on the Fourier coefficients via the action
of $\operatorname{Aut}({\mathbb{C}})$ on local Whittaker newforms. We give an
explicit formula for the Fourier coefficients of a classical Hilbert
automorphic form in terms of a global Whittaker function. We can then apply
this general result to the specific newform in the theorem. This generalises
two results of [5], namely Lemma 3.1 and Propostion 3.2, where the authors
proved analogous results for modular forms. The global Whittaker newform can
be broken up into a product of local Whittaker newforms. Therefore we have
that we can write the Fourier coefficients of $f_{\mu}||_{k}\sigma$ in terms
of local Whittaker newforms. Thus studying the action of
$\operatorname{Aut}({\mathbb{C}})$ on these Fourier coefficients is equivalent
to studying the action of $\operatorname{Aut}({\mathbb{C}})$ on local
Whittaker newforms.

### Acknowledgments

The author would like to thank his supervisor, Abhishek Saha, for suggesting
this problem and for many helpful discussions. This research is supported by
the Leverhulme Trust, grant number RPG-2018-401.

##  2\. Background Setting and Notation

Here we discuss the background setting and collect the notation which will be
used throughout this paper.

###  2.1. Background notation

For an integer $N\geq 1$, we define $\zeta_{N}=e^{2\pi i/N}$. For ease of
notation we will write $e^{2\pi ix}$ as $e(x)$. We will use the shorthand for
the following matrices, $a(x):=\big{(}\begin{smallmatrix}x&\\\
&1\end{smallmatrix}\big{)}$ and $n(x):=\big{(}\begin{smallmatrix}1&x\\\
&1\end{smallmatrix}\big{)}$. We denote the upper half plane by ${\mathbb{H}}$.
We let $\operatorname{GL}_{2}^{+}({\mathbb{R}})$ denote the real two by two
matrices with positive determinant. This group acts on ${\mathbb{H}}$ via the
action $(\begin{smallmatrix}a&b\\\ c&d\end{smallmatrix})z=\frac{az+b}{cz+d}$.
For a representation $\Pi$ we define the representation ${}^{\tau}\Pi$ as
follows. Let $V$ be the space of $\Pi$ and let $V^{\prime}$ be any vector
space such that $t:V\rightarrow V^{\prime}$ is a $\tau$-linear isomorphism. We
define the representation $({}^{\tau}\Pi,V^{\prime})$ via
${}^{\tau}\Pi(g)=t\circ\Pi(g)\circ t^{-1}$.

###  2.2. Number fields

Let $F$ be a totally real number field of degree $n$, with narrow class group
of size $h$. We denote the ring of integers of $F$ by ${\mathcal{O}}_{F}$. The
embeddings of $F$ in ${\mathbb{C}}$ we write as $\eta_{1},...,\eta_{n}$. With
respect to these embeddings we have that $F$ naturally embeds into
${\mathbb{R}}^{n}$. For an element $\alpha$ of $F$ write
$(\alpha_{1},...,\alpha_{n})$ for $(\eta_{1}(\alpha),...,\eta_{n}(\alpha))$ as
an element of ${\mathbb{R}}^{n}$. We write
$\alpha^{k}=\prod_{j=1}^{n}\alpha_{j}^{k_{j}}$ for $k=(k_{1},...,k_{n})$. For
a subset $S$ of $F$, $S_{+}$ denotes the totally positive elements of $S$.
Throughout ${\mathfrak{n}}$ will be a fixed integral ideal of $F$. We denote
the trace map of $F$ to ${\mathbb{Q}}$ by $\operatorname{Tr}$. Let
${\mathfrak{D}}_{F}$ denote the absolute different of $F$, that is,
${\mathfrak{D}}_{F}^{-1}=\\{x\in
F:\operatorname{Tr}(x{\mathcal{O}}_{F})\subset{\mathbb{Z}}\\}$. Let $F_{v}$ be
the completion of $F$ at a non-archimedean place $v$. The ring of integers of
$F_{v}$ are denoted by ${\mathcal{O}}_{v}$. We denote the maximal ideal of
${\mathcal{O}}_{v}$ by ${\mathfrak{p}}_{v}$ and a generator of
${\mathfrak{p}}_{v}$ by $\varpi_{v}$. We write
${\mathfrak{n}}_{v}={\mathfrak{n}}\otimes_{{\mathcal{O}}_{F}}{\mathcal{O}}_{v}$
and
$\mathfrak{D}_{v}=\mathfrak{D}_{F}\otimes_{{\mathcal{O}}_{F}}{\mathcal{O}}_{v}$
to be the $v$-part of ${\mathfrak{n}}$ and ${\mathfrak{D}}_{F}$ respectively.
For an ideal ${\mathfrak{n}}_{v}$ of ${\mathcal{O}}_{v}$ we define the
subgroup $K_{v}({\mathfrak{n}}_{v})$ of $\operatorname{GL}_{2}(F_{v})$ as

$K_{v}({\mathfrak{n}}_{v}):=\bigg{\\{}\begin{pmatrix}a&b\\\
c&d\end{pmatrix}\in\operatorname{GL}_{2}(F_{v}):\begin{aligned}
&a{\mathcal{O}}_{v}+{\mathfrak{n}}_{v}={\mathcal{O}}_{v},\
b\in\mathfrak{D}_{v}^{-1}\\\ &c\in{\mathfrak{n}}_{v}\mathfrak{D}_{v},\
d\in{\mathcal{O}}_{v}\end{aligned},ad-
bc\in{\mathcal{O}}_{F}^{\times}\bigg{\\}}.$ (1)

By abuse of notation, given an ideal ${\mathfrak{n}}$ of ${\mathcal{O}}_{F}$
use $K_{v}({\mathfrak{n}})$ to denote $K_{v}({\mathfrak{n}}_{v})$ where
${\mathfrak{n}}_{v}={\mathfrak{n}}\otimes_{{\mathcal{O}}_{F}}{\mathcal{O}}_{v}$
is the $v$-part of ${\mathfrak{n}}$. We define $n_{v}$ and $d_{v}$ via
${\mathfrak{n}}_{v}=\varpi_{v}^{n_{v}}{\mathcal{O}}_{v}$ and
${\mathfrak{D}}_{v}=\varpi_{v}^{d_{v}}{\mathcal{O}}_{v}$ respectively.

###  2.3. Adeles

The ring of adeles of $F$ is denoted by ${\mathbb{A}}_{F}$. In the specific
case that $F={\mathbb{Q}}$ we denote ${\mathbb{A}}_{\mathbb{Q}}={\mathbb{A}}$.
The narrow class group can be viewed as

$F^{\times}\backslash{\mathbb{A}}_{F}^{\times}/F_{\infty+}^{\times}\prod{\mathcal{O}}_{v}^{\times}.$

We fix elements, for $1\leq\mu\leq h$, $t_{\mu}\in{\mathbb{A}}_{F}^{\times}$
such that

$(t_{\mu})_{v}\begin{cases}=1,\ \text{if}\ v\in\infty\ \text{or}\
v|{\mathfrak{n}}\\\ \in{\mathcal{O}}_{v},\ \text{if}\ v\nmid\infty\
\text{and}\ v\nmid{\mathfrak{n}}\end{cases}$

and $\\{t_{\mu}\\}$ are representatives of the narrow class group. We let
$(t_{\mu}{\mathcal{O}}_{F})$ denote the ideal of ${\mathcal{O}}_{F}$
corresponding to $t_{\mu}$. We have the following disjoint union decomposition

${\mathbb{A}}_{F}^{\times}=\bigcup_{1\leq\mu\leq
h}t_{\mu}F^{\times}F^{\times}_{\infty,+}\prod_{v<\infty}{\mathcal{O}}_{v}^{\times}.$

We denote $x_{\mu}=(\begin{smallmatrix}1&\\\ &t_{\mu}\end{smallmatrix})$ and
for a finite place $v$ we denote $x_{\mu,v}=(\begin{smallmatrix}1&\\\
&(t_{\mu})_{v}\end{smallmatrix})$. For
$g\in\operatorname{GL}_{2}({\mathbb{A}}_{F})$ we define $\iota_{\text{f}}(g)$
to equal $g$ at all finite places and the identity at infinite places.

###  2.4. Measures

We normalise the measures so that Vol$({\mathcal{O}}_{v})=1$ and in a
multiplicative setting Vol$({\mathcal{O}}_{v}^{\times})=1$. We also normalise
so that Vol$(F\backslash{\mathbb{A}_{F}})=1$.

###  2.5. Additive characters

Let
$\psi_{\mathbb{Q}}:{\mathbb{Q}}\backslash{\mathbb{A}}\rightarrow{\mathbb{C}}^{\times}$
be an additive character defined by
$\psi_{\mathbb{Q}}=\prod_{p}\psi_{{\mathbb{Q}},p}$ where
$\psi_{{\mathbb{Q}},\infty}(x)=e(x)$ for $x\in{\mathbb{R}}$ and
$\psi_{{\mathbb{Q}},p}(x)=1$ for $x\in{\mathbb{Z}}_{p}$. We then define an
additive character $\psi$ on $F\backslash{\mathbb{A}}_{F}$ by composing
$\psi_{\mathbb{Q}}$ with the trace map from $F$ to ${\mathbb{Q}}$, that is,
$\psi=\psi_{\mathbb{Q}}\circ\operatorname{Tr}:F\backslash{\mathbb{A}}_{F}\rightarrow{\mathbb{C}}^{\times}$.
If $\psi=\otimes\psi_{v}$ then the local characters are defined analogously
and we have $\psi_{\infty}(x)=e(\operatorname{Tr}(x))$. Using [17, Lemma
2.3.1] we have a formula for computing the value of the exponent of the
character $\psi_{v}$ where $v$ is a non-archimedean place of $F$. For a place
$v$ of $F$, letting $c(\psi_{v})$ denote the smallest integer $c_{v}$ such
that $\psi_{v}$ is trivial on ${\mathfrak{p}}_{v}^{c_{v}}$, we have by [17,
Lemma 2.3.1]

$c(\psi_{v})=d_{v},$ (2)

where $d_{v}$ is defined as above, that is, via
${\mathfrak{D}}_{v}=\varpi_{v}^{d_{v}}{\mathcal{O}}_{v}$.

##  3\. The Whittaker Model

Recall that each infinite dimensional admissible representation of
$\operatorname{GL}_{2}$ over a local field admits a unique Whittaker model see
[3, Chapter 3] and [11, Chapter 4]. Furthermore each cuspidal automorphic
representation of $\operatorname{GL}_{2}$ admits a unique global Whittaker
model. We give below the definition and basic properties of the Whittaker
model for representation of trivial central character.

For any place $v$ of $F$ let $\Pi_{v}$ be an irreducible admissible infinite
dimensional representation of $\operatorname{GL}_{2}(F_{v})$. Let
$\mathcal{W}(\psi_{v})$ be the space of smooth functions
$W_{v}:\operatorname{GL}_{2}(F_{v})\rightarrow{\mathbb{C}}$ satisfying

$W_{v}\Bigg{(}\begin{pmatrix}1&x\\\
&1\end{pmatrix}g\Bigg{)}=\psi_{v}(x)W_{v}(g),\ \text{for}\ x\in F_{v}\
\text{and}\ g\in\operatorname{GL}_{2}(F_{v}).$

The local Whittaker model, denoted $\mathcal{W}(\Pi_{v},\psi_{v})$, is the
unique subspace of $\mathcal{W}(\psi_{v})$ such that the representation of
$\operatorname{GL}_{2}(F_{v})$ on the space $\mathcal{W}(\Pi_{v},\psi_{v})$
(given by right translation) is equivalent to the representation $\Pi_{v}$.

Let $(\Pi,V_{\Pi})$ be a cuspidal automorphic representation of
$\operatorname{GL}_{2}({\mathbb{A}}_{F})$. Then the global Whittaker model,
$\mathcal{W}(\Pi,\psi)$, for $\Pi$ with respect to a fixed non-trivial
additive character $\psi$, consists of the space generated by functions
$W_{\phi}$ on $\operatorname{GL}_{2}({\mathbb{A}_{F}})$ given by

$W_{\phi}(g):=\int_{{\mathbb{A}}_{F}/F}\phi\Bigg{(}\begin{pmatrix}1&x\\\
&1\end{pmatrix}g\Bigg{)}\overline{\psi(x)}\ dx,$ (3)

as $\phi$ varies in $V_{\Pi}$. The representation $\mathcal{W}(\Pi,\psi)$
decomposes as a restricted tensor product of local Whittaker models,
$\mathcal{W}(\Pi_{v},\psi_{v})$.

For $\Pi_{v}$ an irreducible admissible infinite dimensional representation of
$\operatorname{GL}_{2}(F_{v})$, with unramified central character, let
$a(\Pi_{v})$ be the smallest integer $n$ such that $\Pi_{v}$ has a
$K_{v}({\mathfrak{p}}_{v}^{n})$-fixed vector. Then the normalised Whittaker
newform (with respect to $\psi_{v}$) is the unique
$K_{v}({\mathfrak{p}}_{v}^{a(\Pi_{v})})$-invariant vector
$W_{v}\in\mathcal{W}(\Pi_{v},\psi_{v})$ satisfying $W_{v}(1)=1$.

We now state a key result which will be used throughout this paper.

######  Lemma 3.1.

Let $\Pi_{v}$ be an irreducible admissible infinite dimensional representation
of $\operatorname{GL}_{2}(F_{v})$ with unramified central character. Let
$K_{v}({\mathfrak{p}}_{v}^{a(\Pi_{v})})$ be defined as in (1). Let $W_{v}$ be
the local normalised Whittaker newform. Let
$\tau\in\operatorname{Aut}({\mathbb{C}})$ and
$g\in\operatorname{GL}_{2}(F_{v})$. Then we have
$\tau(W_{v}(g))=W_{{}^{\tau}\Pi_{v}}(a(\alpha_{\tau})g)$ where $\alpha_{\tau}$
is defined via

$\displaystyle\operatorname{Aut}($ $\displaystyle{\mathbb{C}}/{\mathbb{Q}})$
$\displaystyle\rightarrow$
$\displaystyle\operatorname{Gal}(\overline{{\mathbb{Q}}}/{\mathbb{Q}})$
$\displaystyle\rightarrow$
$\displaystyle\operatorname{Gal}({\mathbb{Q}}(\mu_{\infty})/{\mathbb{Q}})$
$\displaystyle\rightarrow$
$\displaystyle\hat{{\mathbb{Z}}}^{\times}\cong\Pi_{v}{\mathbb{Z}}_{v}^{\times}$
$\displaystyle\subset$
$\displaystyle\Pi_{v}\Pi_{{\mathfrak{p}}|v}{\mathcal{O}}_{\mathfrak{p}}^{\times}$
$\displaystyle\tau$ $\displaystyle\mapsto$
$\displaystyle\qquad\tau|_{\overline{{\mathbb{Q}}}}$ $\displaystyle\mapsto$
$\displaystyle\qquad\tau|_{{\mathbb{Q}}(\mu_{\infty})}$ $\displaystyle\mapsto$
$\displaystyle\qquad\alpha_{\tau}$ $\displaystyle\mapsto$
$\displaystyle\alpha_{\tau}=(\alpha_{\tau,{\mathfrak{p}}})_{\mathfrak{p}}.$

###### Proof.

See [5, Section 2] and [15, Section 3.2.3]. ∎

Let $\Pi_{v}$ be an irreducible admissible infinite dimensional representation
of $\operatorname{GL}_{2}(F_{v})$ with unramified central character and
normalised Whittaker newform $W_{v}$. Then consider the representation
$\Pi_{v}\otimes|\ |_{v}^{k_{0}/2}$, for some integer $k_{0}$. We now apply the
above result to this representation. Therefore using Lemma 3.1 we have

$\tau\big{(}W_{\Pi_{v}\otimes|\
|^{k_{0}/2}}(g)\big{)}=W_{{}^{\tau}(\Pi_{v}\otimes|\
|^{k_{0}/2})}(a(\alpha_{\tau})g).$

This is equivalent to

$\tau\big{(}W_{v}(g)\big{)}\tau\big{(}|\det(g)|^{k_{0}/2}_{v}\big{)}=W_{{}^{\tau}(\Pi_{v}\otimes|\
|^{k_{0}/2})}(a(\alpha_{\tau})g).$

So if $\Pi_{v}\otimes|\ |^{k_{0}/2}\cong{}^{\tau}(\Pi_{v}\otimes|\
|^{k_{0}/2})$, we have that

$\tau\big{(}W_{v}(g)\big{)}\tau\big{(}|\det(g)|^{k_{0}/2}_{v}\big{)}=W_{v}(a(\alpha_{\tau})g)|\det(g)|^{k_{0}/2}_{v}.$
(4)

##  4\. Modular Forms Case

In this section we study the case of modular forms. The reason we do this is
because the proof of the theorem stated below follows a similar method to the
one used to prove Theorem 1 but is less technical. As a consequence of this we
are able to prove Thereon 4.1 below, which is an alternative version of [2,
Theorem 4.1] whereby we make some simplifying assumptions on the matrix
$\sigma\in\operatorname{SL}_{2}({\mathbb{Z}})$. The general form of [2,
Theorem 4.1] will follow from applying Theorem 1 in the setting of classical
newforms.

###  4.1. Fourier expansion

For a function $f:{\mathbb{H}}\rightarrow{\mathbb{C}}$ we define a function
$f|_{k}g$ on ${\mathbb{H}}$ for an integer $k$ and some
$g=(\begin{smallmatrix}a&b\\\
c&d\end{smallmatrix})\in\operatorname{GL}_{2}^{+}({\mathbb{R}})$ as
$f|_{k}g(z)=(\det g)^{k/2}(cz+d)^{-k}f(gz)$. Let $N\geq 1$. Let $\Gamma$ be a
congruence subgroup of $\operatorname{SL}_{2}({\mathbb{Z}})$. Let $f$ be a
modular form on $\Gamma$ of weight $k$. Then we have that $f$ has a Fourier
expansion at infinity of the form

$f(z)=\sum_{n\geq 0}a_{f}(n)e^{2\pi inz/w},$

where $a_{f}(n)$ are the Fourier coefficients of $f$ and $w$ is the smallest
integer such that $(\begin{smallmatrix}1&w\\\ 0&1\end{smallmatrix})\in\Gamma$.
Note that in the case that $\Gamma=\Gamma_{0}(N)$ we have that $w=1$. If the
first Fourier coefficient of $f|_{k}g$ is zero for every
$g\in\operatorname{SL}_{2}({\mathbb{Z}})$ then we say that $f$ is a cuspform.
If we also have that $a_{f}(1)=1$ we say that $f$ is normalised. Recall that
if $f$ is a modular form for $\Gamma_{0}(N)$ which is non-zero we have that
$k$ is even.

Let $\sigma\in\operatorname{SL}_{2}({\mathbb{Z}})$ be such that
${\mathfrak{a}}=\sigma\infty$, where ${\mathfrak{a}}$ is a cusp of
$\Gamma_{0}(N)\backslash{\mathbb{H}}$. If $f$ is a modular form for
$\Gamma_{0}(N)$ then $f|_{k}\sigma$ is a modular form for the congruence
subgroup $\sigma^{-1}\Gamma_{0}(N)\sigma$. We have that $f|_{k}\sigma$ has a
Fourier expansion of the form

$f|_{k}\sigma(z)=\sum_{n>0}a_{f}(n;\sigma)e^{2\pi inz/w({\mathfrak{a}})},$

where $a_{f}(n;\sigma)$ are the Fourier coefficients of $f|_{k}\sigma$ and
$w({\mathfrak{a}})$ is the width of the cusp. The width of the cusp is defined
to be the integer $N/(L^{2},N)$ where $L$ is the denominator of the cusp
${\mathfrak{a}}$.

###  4.2. Result for modular forms

######  Theorem 4.1.

Let $f$ be a normalised newform on $\Gamma_{0}(N)$ with weight $k$. Let
${\mathbb{Q}}(f)$ be the field generated by all the Fourier coefficients of
$f$. Let ${\mathfrak{a}}=\sigma\infty$ be a cusp of
$\Gamma_{0}(N)\backslash{\mathbb{H}}$ with $\sigma=(\begin{smallmatrix}a&b\\\
L&d\end{smallmatrix})\in\operatorname{SL}_{2}(\mathbb{Z}),\ L|N,\
(a,N)=(d,N)=1$, so that ${\mathfrak{a}}$ corresponds to the cusp $a/L$. Then
the Fourier coefficients of $f|_{k}\sigma$ lie in the cyclotomic extension
${\mathbb{Q}}(f)(\zeta_{N/L})$.

######  Remark 4.2.

Recall that to show a complex number $\rho\in{\mathbb{C}}$ is an element of a
number field it is equivalent to showing that $\rho$ is fixed by the Galois
group of that number field.

###### Proof.

Let $\tau\in\operatorname{Aut}({\mathbb{C}})$ fix
${\mathbb{Q}}(f)(\zeta_{N/L})$. So in particular $\tau$ fixes
${\mathbb{Q}}(f)$ and all the $\zeta_{N/L}$ roots of unity.

Let $n=n_{0}\prod_{p|N}p^{n_{p}}$ with $(n_{0},N)=1$. In the proof of
Proposition 3.3 in [5] the authors give an explicit formula for
$a_{f}(n,\sigma)$, given by

$a_{f}(n,\sigma)=\frac{a_{f}(n_{0})}{n^{k/2}}\Big{(}\frac{n}{\delta({\mathfrak{a}})}\Big{)}^{k/2}\prod_{p|N}W_{p}(a(n/\delta({\mathfrak{a}}))\sigma^{-1}),$

where $\delta({\mathfrak{a}})$ is a rational number related to the cusp
${\mathfrak{a}}$ and $W_{p}$ are the local Whittaker newforms associated to
$f$. Note that $\frac{n}{\delta({\mathfrak{a}})}\in{\mathbb{Q}}$ and since $k$
is even we have that the second term is automatically fixed by $\tau$. Since
$\tau$ fixes ${\mathbb{Q}}(f)$ we have that $\tau$ fixes $a_{f}(n_{0})$.

Since $\tau$ fixes $\zeta_{N/L}$ we have that $\alpha_{\tau}\equiv
1\pmod{p^{n_{p}-l_{p}}}$. This implies

$\sigma\begin{pmatrix}\alpha_{\tau}&0\\\
0&1\end{pmatrix}\sigma^{-1}=\begin{pmatrix}ad\alpha_{\tau}-bc&ab(1-\alpha_{\tau})\\\
dL(\alpha_{\tau}-1)&ad-bL\alpha_{\tau}\end{pmatrix}\in K_{p}(p^{n_{p}}).$

Since $\tau$ fixes ${\mathbb{Q}}(f)$ we have that $\pi_{p}={}^{\tau}\pi_{p}$.
Therefore using Lemma 3.1 we have that

$\tau(W_{p}(a(n/\delta({\mathfrak{a}}))\sigma^{-1}))=W_{p}(a(\alpha_{\tau})a(n/\delta({\mathfrak{a}}))\sigma^{-1}).$

Recall that $W_{p}$ is right invariant by $K_{p}(p^{n_{p}})$. Thus from the
condition that $\sigma a(\alpha_{\tau})\sigma^{-1}\in K_{p}(p^{n_{p}})$ we
have that

$W_{p}(a(n/\delta({\mathfrak{a}}))\sigma^{-1})=W_{p}(a(\alpha_{\tau})a(n/\delta({\mathfrak{a}}))\sigma^{-1}).$

Thus $\tau$ fixes $a_{f}(n;\sigma)$ and so
$a_{f}(n;\sigma)\in{\mathbb{Q}}(f)(\zeta_{N/L})$. ∎

##  5\. Preliminary Results

In this section we define the classical Hilbert newform and how we can
reinterpret this definition in the adelic setting. We then prove some general
results needed to prove Theorem 1. Specifically we show that the Fourier
coefficients of Hilbert newform equals the global Whittaker newform.

###  5.1. Hilbert Modular Forms

We define

${\mathbb{H}}^{n}=\\{z=(z_{1},...,z_{n}):\operatorname{Im}(z_{j})>0,\forall
j=1,...,n\\},$

where we write $z=x+iy$ to mean for $z=(z_{1},...,z_{n})$ and
$z_{j}=x_{j}+iy_{j}$ for each $j$. We have that $\operatorname{GL}_{2}(F)$
acts on ${\mathbb{H}}^{n}$ in the following way

$g\cdot
z=\Bigg{(}\frac{\eta_{1}(a_{1})z_{1}+\eta_{1}(b_{1})}{\eta_{1}(c_{1})z_{1}+\eta_{1}(d_{1})},...,\frac{\eta_{n}(a_{n})z_{n}+\eta_{n}(b_{n})}{\eta_{n}(c_{n})z_{n}+\eta_{n}(d_{n})}\Bigg{)}.$

We define the congruence subgroup ${\Gamma_{\mu}({\mathfrak{n}})}$ of
$\operatorname{GL}_{2}(F)$ for each $\mu$ as

${\Gamma_{\mu}({\mathfrak{n}})}=x_{\mu}(\operatorname{GL}_{2}^{+}(F_{\infty})K_{0}({\mathfrak{n}}))x_{\mu}^{-1}\cap\operatorname{GL}_{2}(F),$
(5)

where

$K_{0}({\mathfrak{n}})=\prod_{v<\infty}K_{v}({\mathfrak{n}}).$ (6)

Then $f_{\mu}:{\mathbb{H}}^{n}\rightarrow{\mathbb{C}}$ is a Hilbert modular
form of level ${\Gamma_{\mu}({\mathfrak{n}})}$ and weight
$k=(k_{1},...,k_{n})$ if $f_{\mu}$ is holomorphic on ${\mathbb{H}}^{n}$ and at
cusps and

$f_{\mu}||_{k}\alpha(z):=\det\alpha^{k/2}(cz+d)^{-k}f_{\mu}(\alpha\cdot
z)=f_{\mu}(z),$

for every $\alpha\in{\Gamma_{\mu}({\mathfrak{n}})}$.

######  Remark 5.1.

One can also consider this definition with a character $\chi$ on
${\mathbb{A}}_{F}^{\times}/F^{\times}$, which we will assume is trivial. For a
treatment of $\chi$ being non-trivial see [15, Section 4] and the references
therein.

If the constant term in the Fourier expansion of $f_{\mu}||_{k}\gamma$ is zero
for every $\gamma\in\operatorname{GL}_{2}^{+}(F)$ then we say $f_{\mu}$ is a
cuspform. For the space of cuspforms to be nonempty we require that $k_{j}\geq
1$, see [9, Section 1.7] for a proof of this fact. In this case $f_{\mu}$ has
a Fourier expansion at infinity of the form

$f_{\mu}(z)=\sum_{\xi\in(t_{\mu}{\mathcal{O}}_{F})_{+}}a_{\mu}(\xi)e^{2\pi
i\text{Tr}(\xi z)}.$

The space of cuspforms of level ${\Gamma_{\mu}({\mathfrak{n}})}$ and weight
$k$ is denoted $S_{k}({\Gamma_{\mu}({\mathfrak{n}})})$. For each $\mu$ choose
a cuspform in $S_{k}({\Gamma_{\mu}({\mathfrak{n}})})$ and put
${\mathbf{f}}=(f_{1},...,f_{h})$. Then we say that ${\mathbf{f}}$ is a
cuspform of level ${\mathfrak{n}}$ and weight $k=(k_{1},...,k_{n})$. The space
of all such ${\mathbf{f}}$ is denoted by $S_{k}({\mathfrak{n}})$ and we have

$S_{k}({\mathfrak{n}})=\bigoplus_{\mu=1}^{h}S_{k}({\Gamma_{\mu}({\mathfrak{n}})}).$

An element ${\mathbf{f}}\in S_{k}({\mathfrak{n}})$ is said to be a cuspidal
classical Hilbert newform if it lies in the orthogonal complement of the
oldspace and is a common eigenfunction for the Hecke operators at almost all
primes, see [19, Section 2].

###  5.2. Classical and adelic correspondence

Here we give a description on how one can go between classical Hilbert
newforms and adelic newforms (as in Section 5.1 we only consider the case of
trivial character $\chi$). We will give an outline of how this description is
constructed for a full discussion see [9, Chapter 3] and [15, Section 4]. Let
$A_{k}({\mathfrak{n}})$ be the subspace of
$A_{\text{cusp}}(\operatorname{GL}_{2}(F)\backslash\operatorname{GL}_{2}({\mathbb{A}}_{F}))$
that consists of functions such that

  1. (i)

$\phi(gr(\theta))=e^{-ik\theta}\phi(g)$, where
$r(\theta)=\Big{\\{}\begin{pmatrix}\cos\theta_{j}&-\sin\theta_{j}\\\
\sin\theta_{j}&\cos\theta_{j}\end{pmatrix}\Big{\\}}_{j}\in\text{SO}(2)^{n}$.

  2. (ii)

$\phi(gk_{0})=\phi(g)$, where $k_{0}\in K_{0}({\mathfrak{n}})$.

  3. (iii)

$\phi$ is an eigenfunction of the Casimir element
$\Delta:=(\Delta_{1},...,\Delta_{n})$ as a function of
$\operatorname{GL}_{2}({\mathbb{R}})^{n}$ with its eigenvalue
$\lambda=\prod_{j=1}^{n}\frac{k_{j}}{2}(1-\frac{k_{j}}{2})$.

Then $A_{k}({\mathfrak{n}})$ is isomorphic to $S_{k}({\mathfrak{n}})$. To see
this we define, for each ${\mathbf{f}}=(f_{1},...,f_{h})\in
S_{k}({\mathfrak{n}})$

$\phi(\gamma x_{\mu}g_{\infty}k_{0})=f_{\mu}||_{k}g_{\infty}(\mathbf{i})$ (7)

where
$\gamma\in\operatorname{GL}_{2}(F),g_{\infty}\in\operatorname{GL}_{2}^{+}(F_{\infty}),k_{0}\in
K_{0}({\mathfrak{n}})$ and $\mathbf{i}=(i,...,i)$. This is well defined
because of (5) and (6). One can also check that $\phi$ as above gives an
element of $A_{k}({\mathfrak{n}})$. Conversely, given $\phi\in
A_{k}({\mathfrak{n}})$ define

$f_{\mu}(z)=y^{-k/2}\phi(x_{\mu}g_{z}),$

where $g_{z}\mathbf{i}=z$. Then we have that
${\mathbf{f}}:=(f_{1},...,f_{h})\in S_{k}({\mathfrak{n}})$.

We say that $\phi\in A_{k}({\mathfrak{n}})$ is an adelic newform if it
generates an irreducible cuspidal automorphic representation $\Pi=\Pi_{\phi}$
of $\operatorname{GL}_{2}({\mathbb{A}_{F}})$ of conductor ${\mathfrak{n}}$.
The set of adelic newforms in $A_{k}({\mathfrak{n}})$ is in bijective
correspondence with the set of classical cuspidal Hilbert newforms $f\in
S_{k}({\mathfrak{n}})$. Precisely given $\phi$ as above it corresponds to a
classical cuspidal Hilbert newform ${\mathbf{f}}=(f_{1},...,f_{h})$ of level
${\mathfrak{n}}$ and weight $k$ defined via
$f_{\mu}(z)=y^{-k/2}\phi(x_{\mu}g_{z})$.

Given an adelic newform $\phi_{\mathbf{f}}$ attached to ${\mathbf{f}}\in
S_{k}({\mathfrak{n}})$, the automorphic representation of
$\operatorname{GL}_{2}({\mathbb{A}_{F}})$ generated by it is known to be
irreducible, see for example [15, Theorem 4.7]. The following lemma gives an
explicit relation between the action of $\operatorname{Aut}({\mathbb{C}})$ on
newforms and representations.

######  Lemma 5.2.

Let ${\mathbf{f}}=(f_{1},...,f_{h})$ be a cuspidal classical Hilbert newform.
Given $\tau\in\operatorname{Aut}({\mathbb{C}})$ let ${\mathbf{f}}^{\tau}$ be
defined as in [19, Proposition 2.6]. Let $\Pi$ the automorphic representation
generated by $\phi_{\mathbf{f}}$ and $\Pi^{\prime}$ be the automorphic
representation generated by $\phi_{{\mathbf{f}}^{\tau}}$. Then for each finite
place $v$ we have

$\Pi^{\prime}_{v}\otimes|\ |_{v}^{k_{0}/2}\cong{}^{\tau}(\Pi_{v}\otimes|\
|_{v}^{k_{0}/2}).$

###### Proof.

See [15, Theorem 4.19]. ∎

###  5.3. Whittaker expansion

We now give the explicit relation between the Fourier coefficients of a
classical Hilbert newform at a cusp and the Whittaker newform. We start with a
general result which holds for a larger class of Hilbert automorphic forms.

######  Lemma 5.3.

Let $M$ be an ideal of ${\mathcal{O}}_{F}$ with localisation
$M_{v}=\varpi_{v}^{m_{v}}{\mathcal{O}}_{v}$ for every finite place $v$ such
that for almost all $v$ we have that $m_{v}=0$. Let $\phi$ be a Hilbert
automorphic form in the space of $\Pi$ such that $\phi$ is right invariant by
$\big{(}\begin{smallmatrix}1&\varpi_{v}^{m_{v}}\mathcal{O}_{v}\\\
0&1\end{smallmatrix}\big{)}$ for every finite place and satisfies (i) and
(iii) in Section 5.2. Let $h$ be a holomorphic function on $\mathbb{H}^{n}$
defined by the equation

$h(z)=j(g_{z},\mathbf{i})^{k}\phi(g_{z}).$

Then $h$ has a Fourier expansion of the form

$h(z)=\sum_{\xi\in M^{-1}\mathfrak{D}_{F}^{-1}}a_{h}(\xi)e^{2\pi
i\operatorname{Tr}(\xi z)}.$ (8)

Moreover

$W_{\phi}(a(\xi)g_{z})=\begin{cases}y^{k/2}a_{h}(\xi)e^{2\pi i\text{Tr}(\xi
z)},\ \text{if}\ \xi\in M^{-1}\mathfrak{D}_{F}^{-1}\\\ 0,\
\text{otherwise}.\end{cases}$

###### Proof.

From the fact that $\phi$ satisfies (i) and (iii) in Section 5.2, it follows
that $h$ is holomorphic on ${\mathbb{H}}^{n}$ and $h(z+m)=h(z)$, for every
$m\in M$. This shows that

$f(z)=\sum_{\xi\in F}a_{h}(\xi)e^{2\pi i\operatorname{Tr}(\xi z)},$

for some complex numbers $a_{h}(\xi)$.

Suppose $a_{h}(\xi)\neq 0$. From the property that $h$ is invariant by $M$ we
have that $\operatorname{Tr}(\xi m)\in{\mathbb{Z}}$ for every $m\in M$. This
property is equivalent to that of $\xi\in M^{-1}\mathfrak{D}_{F}^{-1}$. To see
this note that for every place $v<\infty$ we have that
$\operatorname{Tr}(\xi_{v}m_{v})\in{\mathbb{Z}}_{v}$, for every $m_{v}\in
M_{v}$. Thus
$\operatorname{Tr}((\varpi_{v}^{m_{v}}\xi_{v})n_{v})\in{\mathbb{Z}}_{v}$ for
every $n_{v}\in{\mathcal{O}}_{v}$. This is equivalent to the following

$\varpi_{v}^{m_{v}}\xi_{v}\in\mathfrak{D}_{v}^{-1}\Longleftrightarrow\xi_{v}\in\varpi_{v}^{-m_{v}}\mathfrak{D}_{v}^{-1}\Longleftrightarrow\xi\in
M^{-1}\mathfrak{D}_{F}^{-1}.$

This shows that $h$ has a Fourier expansion of the form (8). Next let $\xi\in
F^{\times}$ and let $z=x+iy\in{\mathbb{H}}^{n}$. Then using the definition of
$W_{\phi}$, (3), we have that

$\displaystyle W_{\phi}(a(\xi)g_{z})$
$\displaystyle=\int_{{\mathbb{A}_{F}}/F}\phi\big{(}n(x^{\prime})a(\xi)g_{z}\big{)}\overline{\psi(x^{\prime})}\
dx^{\prime}$
$\displaystyle=\int_{{\mathbb{A}_{F}}/F}\phi\big{(}a(\xi)n(\xi^{-1}x^{\prime})g_{z})\psi(-x^{\prime})\
dx^{\prime}$
$\displaystyle=\int_{{\mathbb{A}_{F}}/F}\phi\big{(}n(\xi^{-1}x^{\prime})g_{z})\psi(-x^{\prime})\
dx^{\prime},$
$\displaystyle=\int_{{\mathbb{A}_{F}}/F}\phi\big{(}n(x^{\prime})g_{z})\psi(-\xi
x^{\prime})\ dx^{\prime},$
$\displaystyle=\int_{{\mathbb{A}_{F}}/F}\phi(n(x^{\prime}+x)a(y))\psi(-\xi
x^{\prime})\ dx^{\prime},$
$\displaystyle=\int_{{\mathbb{A}_{F}}/F}\phi(n(x^{\prime})a(y))\psi(-\xi
x)\psi(\xi x)\ dx^{\prime},$ $\displaystyle=e(\operatorname{Tr}(\xi
x))\int_{{\mathbb{A}_{F}}/F}\phi\big{(}n(x^{\prime})a(y))\psi(-\xi
x^{\prime})\ dx^{\prime}.$

We denote the integral in the last line by $I_{\phi}(y,\xi)$. We now want to
split this integral into the product of two integrals, using the fundamental
domain

${\mathbb{A}_{F}}/F=F_{\infty}/M\times\prod_{v<\infty}\varpi_{v}^{m_{v}}{\mathcal{O}}_{v},$

using strong approximation. Therefore using $y\in F_{\infty}$ and the right
invariance of $\phi$ by $n(\varpi_{v}^{m_{v}}{\mathcal{O}}_{v})$ we have that

$\displaystyle\text{Vol}(F_{\infty}/M)I_{\phi}(y,\xi)$
$\displaystyle=\int_{F_{\infty}/M}\phi(n(x_{\infty})a(y))\psi_{\infty}(-\xi
x_{\infty})\
dx_{\infty}\prod_{v<\infty}\int_{{\mathcal{O}}_{v}}\psi_{v}(-\xi\varpi_{v}^{m_{v}}x_{v})\
dx_{v},$
$\displaystyle=\int_{F_{\infty}/M}y^{k/2}h(x_{\infty}+iy)e(-\operatorname{Tr}(\xi
x_{\infty}))\
dx_{\infty}\prod_{v<\infty}\int_{{\mathcal{O}}_{v}}\psi_{v}(-\xi\varpi_{v}^{m_{v}}x_{v})\
dx_{v}.$

We have that $I_{\phi}(y,\xi)$ will only be non-zero when

$\int_{{\mathcal{O}}_{v}}\psi_{v}(-\xi\varpi_{v}^{m_{v}}x_{v})\ dx_{v}\neq 0$

for each $v$. Recall that
$\psi_{v}=\psi_{p}\circ\operatorname{Tr}_{F_{v}/{\mathbb{Q}_{p}}}$ and
$\psi_{p}(x)=1$ for every $x\in{\mathbb{Z}}_{p}$. If the finite place integral
is non-zero for each $v$ we write $\xi=\varpi_{v}^{n}u$ where
$n\in{\mathbb{Z}}$ and $u\in{\mathcal{O}}_{v}^{\times}$. Therefore using (2)
we have that

$\int_{{\mathcal{O}}_{v}}\psi_{v}(\xi\varpi_{v}^{m_{v}}x_{v})\
dx_{v}=\int_{{\mathcal{O}}_{v}}\psi_{v}(\varpi_{v}^{m_{v}+n}uk)\
dk=\begin{cases}0,\ \text{if}\ m_{v}+n<d_{v}\\\ 1,\ \text{if}\ m_{v}+n\geq
d_{v},\end{cases}$

Therefore we have that

$I_{\phi}(y,\xi)=\begin{cases}\frac{1}{\text{Vol}(F_{\infty}/M)}\int_{F_{\infty}/M}y^{k/2}h(x_{\infty}+iy)e(-\operatorname{Tr}(\xi
x_{\infty}))\ dx_{\infty},\ \text{if}\ \xi\in M^{-1}\mathfrak{D}_{F}^{-1}\\\
0,\ \text{otherwise}.\end{cases}$

Using the Fourier expansion of $h$,

$h(x_{\infty}+iy)=\sum_{\xi^{\prime}\in
M^{-1}\mathfrak{D}_{F}^{-1}}a_{h}(\xi^{\prime})e^{2\pi
i\operatorname{Tr}(\xi^{\prime}x_{\infty})}e^{-2\pi\operatorname{Tr}(\xi^{\prime}y)},$

we have that

$I_{\phi}(y,\xi)=\begin{cases}y^{k/2}a_{h}(\xi)e^{-2\pi i\operatorname{Tr}(\xi
y)},\ \text{if}\ \xi\in M^{-1}\mathfrak{D}_{F}^{-1}\\\ 0,\
\text{otherwise}.\end{cases}$

Therefore for $\xi\in M^{-1}\mathfrak{D}_{F}^{-1}$ we have

$W_{\phi}(a(\xi)g_{z})=e(\operatorname{Tr}(\xi
x))y^{k/2}a_{h}(\xi)e(i\operatorname{Tr}(\xi
y))=y^{k/2}a_{h}(\xi)e(\operatorname{Tr}(\xi z)),$

and for $\xi\notin M^{-1}\mathfrak{D}_{F}^{-1}$ we have
$W_{\phi}(a(\xi)g_{z})=0$. ∎

From now on we let $\phi$ be an adelic newform in the space of $\Pi$ and
${\mathbf{f}}$ be the corresponding classical Hilbert newform. We shall now
also assume that $k_{j}\geq 1$ for every $j=1,...,n$. Let
$\sigma=(\begin{smallmatrix}a&b\\\ c&d\end{smallmatrix})\in\Gamma_{\mu}(1)$
then recall that if $f_{\mu}$ is a Hilbert modular form for
${\Gamma_{\mu}({\mathfrak{n}})}$ we have $f_{\mu}||_{k}\sigma(z)$ is a Hilbert
modular form for $\sigma^{-1}\Gamma_{\mu}({\mathfrak{n}})\sigma$. We have for
$\sigma\in\Gamma_{\mu}(1)$, using (7)

$\displaystyle f_{\mu}||_{k}\sigma(z)$ $\displaystyle=y^{-k/2}(f_{\mu}||\sigma
g_{z})(\mathbf{i})$
$\displaystyle=y^{-k/2}\phi(x_{\mu}\iota_{\infty}(\sigma)g_{z})$
$\displaystyle=y^{-k/2}\phi(\iota_{\infty}(\sigma)x_{\mu}g_{z})$
$\displaystyle=y^{-k/2}\phi({\iota_{\text{f}}(\sigma^{-1})}x_{\mu}g_{z})$
$\displaystyle=y^{-k/2}\phi(g_{z}{\iota_{\text{f}}(\sigma^{-1})}x_{\mu}),$

where ${\iota_{\text{f}}(\sigma^{-1})}x_{\mu}$ is an element of the finite
adeles.

Our aim is to apply Lemma 5.3 to the specific case of $f_{\mu}||_{k}\sigma$
where $\sigma\in\Gamma_{\mu}(1)$. By the above calculation, this corresponds
to $({\iota_{\text{f}}(\sigma^{-1})}x_{\mu})\phi$ in the setup of Lemma 5.3.
The first step to do this is by finding the ideal $M^{\prime}$ of
${\mathcal{O}}_{F}$ corresponding to
$M_{v}^{\prime}=\varpi_{v}^{m^{\prime}_{v}}{\mathcal{O}}_{v}$ such that
$\big{(}\begin{smallmatrix}1&M^{\prime}_{v}\\\
&1\end{smallmatrix}\big{)}\in\sigma^{-1}\Gamma_{\mu}({\mathfrak{n}})\sigma$.
That is we require $\sigma\big{(}\begin{smallmatrix}1&M^{\prime}_{v}\\\
&1\end{smallmatrix}\big{)}\sigma^{-1}\in{\Gamma_{\mu}({\mathfrak{n}})}$.
Equivalently for each place $v$, we require

$\begin{pmatrix}1&\varpi_{v}^{m^{\prime}_{v}}{\mathcal{O}}_{v}\\\
&1\end{pmatrix}\in\sigma^{-1}x_{\mu,v}K_{v}(\varpi_{v}^{n_{v}})x_{\mu,v}^{-1}\sigma.$

We can view

$\sigma=x_{\mu,v}(\begin{smallmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{smallmatrix})(\begin{smallmatrix}a^{\prime}&b^{\prime}\\\
c^{\prime}&d^{\prime}\end{smallmatrix})(\begin{smallmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{smallmatrix})x_{\mu,v}^{-1},$ (9)

where $A:=(\begin{smallmatrix}a^{\prime}&b^{\prime}\\\
c^{\prime}&d^{\prime}\end{smallmatrix})\in\operatorname{GL}_{2}({\mathcal{O}}_{F})$
and $x_{\mu,v}=(\begin{smallmatrix}1&\\\
&\varpi_{v}^{t_{v}}\end{smallmatrix})$. Therefore

$\displaystyle\begin{pmatrix}1&\varpi_{v}^{m^{\prime}_{v}}{\mathcal{O}}_{v}\\\
&1\end{pmatrix}\in x_{\mu,v}\begin{pmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{pmatrix}A^{-1}\begin{pmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{pmatrix}K_{v}(\varpi_{v}^{n_{v}})\begin{pmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{pmatrix}A\begin{pmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{pmatrix}x_{\mu,v}^{-1}$
$\displaystyle\Longleftrightarrow\begin{pmatrix}1&\varpi_{v}^{m^{\prime}_{v}+t_{v}+d_{v}}{\mathcal{O}}_{v}\\\
&1\end{pmatrix}\in A^{-1}\begin{pmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{pmatrix}K_{v}(\varpi_{v}^{n_{v}})\begin{pmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{pmatrix}A$ $\displaystyle\Longleftrightarrow
A\begin{pmatrix}1&\varpi_{v}^{m^{\prime}_{v}+t_{v}}{\mathcal{O}}_{v}\\\
&1\end{pmatrix}A^{-1}\in\begin{pmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{pmatrix}K_{v}(\varpi_{v}^{n_{v}})\begin{pmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{pmatrix}.$ (10)

On computing the left hand side of (10) we require

$\begin{pmatrix}\varepsilon^{-1}(a^{\prime}d^{\prime}-b^{\prime}c^{\prime}-a^{\prime}c^{\prime}\varpi_{v}^{m_{v}^{\prime}+t_{v}+d_{v}}\alpha)&\varepsilon^{-1}(a^{\prime})^{2}\varpi_{v}^{m_{v}^{\prime}+t_{v}+d_{v}}\alpha\\\
-\varepsilon^{-1}(c^{\prime})^{2}\varpi_{v}^{m_{v}^{\prime}+t_{v}+d_{v}}\alpha&\varepsilon^{-1}(a^{\prime}d^{\prime}-b^{\prime}c^{\prime}+a^{\prime}c^{\prime}\varpi_{v}^{m_{v}^{\prime}+t_{v}+d_{v}}\alpha)\end{pmatrix}\in
y_{v}^{-1}K_{v}(\varpi_{v}^{n_{v}})y_{v},$

where $\alpha\in{\mathcal{O}}_{v},\ y_{v}=(\begin{smallmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{smallmatrix})$ and $\varepsilon=\det
A\in{\mathcal{O}}_{F}^{\times}$. If we now study the
${\mathfrak{p}}$-valuation we obtain the following conditions

  1. (1)

$2v_{\mathfrak{p}}(a^{\prime})+m_{v}^{\prime}+t_{v}\geq 0$,

  2. (2)

$2v_{\mathfrak{p}}(c)+m_{v}^{\prime}+t_{v}\geq n_{v}$.

On rearranging these conditions we obtain

$m_{v}^{\prime}\geq-d_{v}-t_{v}-2v_{\mathfrak{p}}(a^{\prime}),\
m_{v}^{\prime}\geq-t_{v}-d_{v}+n_{v}-2v_{\mathfrak{p}}(c^{\prime}).$

Hence

$m_{v}^{\prime}\geq-
t_{v}-d_{v}+\max\\{-2v_{\mathfrak{p}}(a^{\prime}),n_{v}-2v_{\mathfrak{p}}(c^{\prime})\\}.$

Let

$w_{v}(\sigma,{\mathfrak{n}})=\max\\{-2v_{\mathfrak{p}}(a^{\prime}),n_{v}-2v_{\mathfrak{p}}(c^{\prime})\\}=n_{v}-\min\\{2v_{\mathfrak{p}}(a^{\prime})+n_{v},2v_{\mathfrak{p}}(c^{\prime})\\}.$

Then let
$\mathfrak{w}_{v}(\sigma,{\mathfrak{n}})=\varpi_{v}^{w_{v}(\sigma,{\mathfrak{n}})}{\mathcal{O}}_{v}$
and $\mathfrak{w}(\sigma,{\mathfrak{n}})$ the corresponding ideal in
${\mathcal{O}}_{F}$, so

$\mathfrak{w}(\sigma,{\mathfrak{n}})={\mathfrak{n}}(((a^{\prime})^{2}{\mathfrak{n}},(c^{\prime})^{2}))^{-1}.$

This leads to $f_{\mu}||_{k}\sigma$ having a Fourier expansion of the form

$f_{\mu}||_{k}\sigma(z)=\sum_{\xi\in((t_{\mu}{\mathcal{O}}_{F})\mathfrak{w}(\sigma,{\mathfrak{n}})^{-1})_{+}}a_{\mu}(\xi,\sigma)e(\operatorname{Tr}(\xi
z)).$

We define

$c_{\mu}(\xi;f_{\mu}||_{k}\sigma)=N(t_{\mu}{\mathcal{O}}_{F})^{-k_{0}/2}a_{\mu}(\xi;\sigma)\xi^{(k_{0}\mathbf{1}-k)/2},$
(11)

recall $k_{0}=\max\\{k_{1},...,k_{n}\\}$ and
$k_{0}\mathbf{1}=(k_{0},...,k_{0})$.

######  Remark 5.4.

Note from (9) we can view a matrix $\sigma=(\begin{smallmatrix}a&b\\\
c&d\end{smallmatrix})\in\Gamma_{\mu}(1)$ as

$x_{\mu,v}(\begin{smallmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{smallmatrix})(\begin{smallmatrix}a^{\prime}&b^{\prime}\\\
c^{\prime}&d^{\prime}\end{smallmatrix})(\begin{smallmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{smallmatrix})x_{\mu,v}^{-1},$

with $(\begin{smallmatrix}a^{\prime}&b^{\prime}\\\
c^{\prime}&d^{\prime}\end{smallmatrix})\in\operatorname{GL}_{2}({\mathcal{O}}_{F})$.
Therefore we have that

$\sigma_{v}=\begin{pmatrix}a_{v}&b_{v}\\\
c_{v}&d_{v}\end{pmatrix}=\begin{pmatrix}a^{\prime}_{v}&b^{\prime}_{v}\\\
\varpi_{v}^{t_{v}+d_{v}}c^{\prime}_{v}&d_{v}\end{pmatrix}.$

######  Proposition 5.5.

Let $\mathfrak{w}(\sigma,{\mathfrak{n}})$ be the ideal of ${\mathcal{O}}_{F}$
defined by
$\mathfrak{w}(\sigma,{\mathfrak{n}})={\mathfrak{n}}(\mathrm{gcd}((a^{\prime})^{2}{\mathfrak{n}},(c^{\prime})^{2}))^{-1}$.
Let $\xi\in F^{\times}$. Then

$W_{\phi}(a(\xi)g_{z}\iota_{\emph{f}}(\sigma^{-1})x_{\mu})=\begin{cases}y^{k/2}a_{\mu}(\xi;\sigma)e(\operatorname{Tr}(\xi
z)),\ \text{if}\
\xi\in\big{(}(t_{\mu}{\mathcal{O}}_{F})\mathfrak{w}(\sigma,{\mathfrak{n}})^{-1}\big{)}_{+}\\\
0,\ \text{otherwise}.\end{cases}$

###### Proof.

This follows from applying Lemma 5.3 to the automorphic form
$\phi^{\prime}=\Pi({\iota_{\text{f}}(\sigma^{-1})}x_{\mu})\phi$ and using the
fact
$W_{\phi}(a(\xi)g_{z}{\iota_{\text{f}}(\sigma^{-1})}x_{\mu})=W_{\phi^{\prime}}(a(\xi)g_{z})$.
∎

We are able to evaluate $W_{\infty}$ via,

$W_{\infty}(a(\xi)g_{z})=(\xi y)^{k/2}e(\operatorname{Tr}(\xi z)),$ (12)

see for example [16, Section 3].

##  6\. Main Result

In this section we prove our main result. We do this by applying a similar
method to that used in the proof of Theorem 4.1.

In Section 5 we showed that $a_{\mu}(\xi;\sigma)$ can be written in terms of
the global Whittaker function. Recall that the global Whittaker function can
be broken up into a product of local Whittaker functions. This means that
finding sufficient conditions for $\tau$ to fix
$c_{\mu}(\xi;f_{\mu}||_{k}\sigma)$, boils down to finding sufficient
conditions such that $\tau$ fixes the local Whittaker newforms. With this in
mind we have the following decomposition

$W_{\phi}(g)=C\prod_{v|\infty}W_{\infty}(g_{\infty})\prod_{v<\infty}W_{v}(g_{v}),$
(13)

where $C$ is a constant. This decomposition is due to the uniqueness of
Whittaker functionals. We wish to compute the constant $C$. To do this let
$g_{\infty}=g_{z}$ and $g_{v}=x_{1,v}$ for every $v<\infty$. Then using
Proposition 5.5 and the fact that ${\mathbf{f}}$ is normalised, that is,
$a_{\mu}(1;1)=1$ we have

$W_{\phi}(g_{z}x_{1})=y^{k/2}a_{1}(1;1)e(\operatorname{Tr}(z))=y^{k/2}e(\operatorname{Tr}(z)).$

On the other hand using (12), (13) and the fact that $W_{v}(1)=1$ we obtain

$W_{\phi}(g_{z}x_{1})=Cy^{k/2}e(\operatorname{Tr}(z)).$

Thus $C=1$.

Let ${\mathbf{f}}=(f_{1},...,f_{h})$ a normalised cuspidal Hilbert newform
with weight $k=(k_{1},...,k_{n})$ such that $k_{1}\equiv...\equiv
k_{n}\pmod{2}$. We let ${\mathbb{Q}}({\mathbf{f}})$ denote the number field
generated by $c_{\mu}(\xi;f_{\mu})$ as $\xi$ varies over $F$ and $\mu$ varies
over $1\leq\mu\leq h$. Then for ${\mathbf{f}},1\leq\mu\leq h,\ \text{and}\
\sigma\in\Gamma_{\mu}(1)$ we let ${\mathbb{Q}}({\mathbf{f}},\mu,\sigma)$
denote the field generated by $c_{\mu}(\xi;f_{\mu}||_{k}\sigma)$ as $\xi$
varies over $F$. Note that ${\mathbb{Q}}({\mathbf{f}})$ is the compositum of
the fields ${\mathbb{Q}}(f,\mu,1)$ as $\mu$ varies over $1\leq\mu\leq h$.

We are now able to state and prove the main result of this article.

######  Theorem 6.1.

Let ${\mathbf{f}}=(f_{1},...,f_{h})$ be a normalised cuspidal Hilbert newform
of level ${\mathfrak{n}}$ and weight $k=(k_{1},...,k_{n})$ with
$k_{1}\equiv...\equiv k_{n}\pmod{2}$. Let $1\leq\mu\leq h$ and
$\sigma=(\begin{smallmatrix}a&b\\\ c&d\end{smallmatrix})\in\Gamma_{\mu}(1)$.
Then ${\mathbb{Q}}({\mathbf{f}},\mu,\sigma)$ lies in the number field
${\mathbb{Q}}({\mathbf{f}})(\zeta_{N_{0}})$ where $N_{0}$ is the integer such
that
$N_{0}{\mathbb{Z}}={\mathfrak{n}}/(cdt_{\mu}^{-1}\mathfrak{D}_{F}^{-1},{\mathfrak{n}})\cap{\mathbb{Z}}$.

###### Proof.

Let $\tau\in\operatorname{Aut}({\mathbb{C}})$ fix
${\mathbb{Q}}({\mathbf{f}})(\zeta_{N_{0}})$ where $N_{0}$ is the unique
positive integer such that
$N_{0}{\mathbb{Z}}={\mathfrak{n}}/(cdt_{\mu}^{-1}\mathfrak{D}_{F}^{-1},{\mathfrak{n}})\cap{\mathbb{Z}}$.
Thus specifically $\tau$ fixes ${\mathbb{Q}}({\mathbf{f}})$ and all the
$\zeta_{N_{0}}$ roots of unity.

From Proposition 5.5 and evaluating $W_{\infty}$ we have that

$\displaystyle a_{\mu}(\xi;\sigma)$
$\displaystyle=y^{-k/2}e(-\operatorname{Tr}(\xi
z))W_{\phi}(a(\xi)g_{z}{\iota_{\text{f}}(\sigma^{-1})}x_{\mu})$
$\displaystyle=y^{-k/2}e(-\operatorname{Tr}(\xi z))(\xi
y)^{k/2}e(\operatorname{Tr}(\xi
z))\prod_{v<\infty}W_{v}(a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu})$
$\displaystyle=\xi^{k/2}\prod_{v<\infty}W_{v}(a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu}).$

Recall from (11) we have

$c_{\mu}(\xi;f_{\mu}||_{k}\sigma)=N(t_{\mu}{\mathcal{O}}_{F})^{-k_{0}/2}\xi^{(k_{0}\mathbf{1}-k)/2}a_{\mu}(\xi;\sigma).$

Therefore

$c_{\mu}(\xi;f_{\mu}||_{k}\sigma)=N(t_{\mu}{\mathcal{O}}_{F})^{-k_{0}/2}\xi^{k_{0}\mathbf{1}/2}\prod_{v<\infty}W_{v}(a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu}).$
(14)

From the fact that $\tau$ fixes ${\mathbb{Q}}({\mathbf{f}})$ and using [19,
Proposition 2.6], multiplicity one and Lemma 5.2 we have that
${\mathbf{f}}={\mathbf{f}}^{\tau}$. Therefore, using Lemma 5.2 we have

$\Pi_{v}\otimes|\ |_{v}^{k_{0}/2}\cong{}^{\tau}(\Pi_{v}\otimes|\
|_{v}^{k_{0}/2}).$

So, by (4), we have that for each non-archimedean place $v$

$\tau(W_{v}(a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu}))\tau\big{(}(|\xi|_{v}|t_{\mu}|_{v})^{k_{0}/2}\big{)}=W_{v}(a(\alpha_{\tau})a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu})(|\xi|_{v}|t_{\mu}|_{v})^{k_{0}/2}.$
(15)

Since all the $N_{0}$ roots of unity are fixed by $\tau$ we have that

$\alpha_{\tau}\equiv
1\pmod{{\mathfrak{n}}_{v}/(c_{v}d_{v}\varpi_{v}^{-t_{v}-d_{v}},{\mathfrak{n}}_{v})},$

where $c_{v}$ and $d_{v}$ are the $v$ parts of $c$ and $d$ respectively.
Consider the product

$x_{\mu}^{-1}\begin{pmatrix}a&b\\\
c&d\end{pmatrix}\begin{pmatrix}\alpha_{\tau}&\\\
&1\end{pmatrix}\begin{pmatrix}a&b\\\ c&d\end{pmatrix}^{-1}x_{\mu}.$

On using (9) we have this product is equal to

$\begin{pmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{pmatrix}\begin{pmatrix}a^{\prime}&b^{\prime}\\\
c^{\prime}&d^{\prime}\end{pmatrix}\begin{pmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{pmatrix}\begin{pmatrix}\alpha_{\tau}&\\\
&1\end{pmatrix}\begin{pmatrix}1&\\\
&\varpi_{v}^{d_{v}}\end{pmatrix}\begin{pmatrix}a^{\prime}&b^{\prime}\\\
c^{\prime}&d^{\prime}\end{pmatrix}\begin{pmatrix}1&\\\
&\varpi_{v}^{-d_{v}}\end{pmatrix}.$ (16)

On evaluating (16) we have

$\begin{pmatrix}\varepsilon^{-1}(a^{\prime}d^{\prime}\alpha_{\tau}-b^{\prime}c^{\prime})&\varepsilon^{-1}a^{\prime}b^{\prime}(1-\alpha_{\tau})\varpi_{v}^{-d_{v}}\\\
\varepsilon^{-1}c^{\prime}d^{\prime}(\alpha_{\tau}-1)\varpi_{v}^{d_{v}}&\varepsilon^{-1}(a^{\prime}d^{\prime}-b^{\prime}c^{\prime}\alpha_{\tau})\end{pmatrix},\
\text{where}\ \varepsilon=\det(\begin{smallmatrix}a^{\prime}&b^{\prime}\\\
c^{\prime}&d^{\prime}\end{smallmatrix}).$

We want to show that this matrix is an element of $K_{v}({\mathfrak{n}})$.
Therefore we need the following conditions to be satisfied

  1. (1)

$\varepsilon^{-1}a^{\prime}b^{\prime}(1-\alpha_{\tau})\varpi_{v}^{-dv}\in\mathfrak{D}_{v}^{-1}$

  2. (2)

$\varepsilon^{-1}c^{\prime}d^{\prime}(\alpha_{\tau}-1)\varpi_{v}^{d_{v}}\in{\mathfrak{n}}_{v}\mathfrak{D}_{v}$

  3. (3)

$\varepsilon^{-1}(a^{\prime}d^{\prime}-b^{\prime}c^{\prime}\alpha_{\tau})\in{\mathcal{O}}_{v}$.

Recall that $a^{\prime},b^{\prime},c^{\prime}$ and $d^{\prime}$ are elements
of ${\mathcal{O}}_{F}$ and $\varepsilon^{-1}\in{\mathcal{O}}_{F}^{\times}$ so
conditions (1) and (3) are satisfied with no additional assumptions on
$\alpha_{\tau}$. Now consider condition (2). From the fact we have
$\alpha_{\tau}\equiv
1\pmod{{\mathfrak{n}}_{v}/(c_{v}d_{v}\varpi_{v}^{-t_{v}-d_{v}},{\mathfrak{n}}_{v})}$
which is equivalent to $\alpha_{\tau}\equiv
1\pmod{n_{v}/(c^{\prime}_{v}d^{\prime}_{v},{\mathfrak{n}}_{v})}$, using Remark
5.4. Thus we have that condition (2) is also satisfied. Therefore

$\begin{pmatrix}\varepsilon^{-1}(a^{\prime}d^{\prime}\alpha_{\tau}-b^{\prime}c^{\prime})&\varepsilon^{-1}a^{\prime}b^{\prime}(1-\alpha_{\tau})\varpi_{v}^{-dv}\\\
\varepsilon^{-1}c^{\prime}d^{\prime}(\alpha_{\tau}-1)\varpi_{v}^{d_{v}}&\varepsilon^{-1}(a^{\prime}d^{\prime}-b^{\prime}c^{\prime}\alpha_{\tau})\end{pmatrix}\in
K_{v}({\mathfrak{n}}).$

Recall that in general for $W_{v}(g_{1})=W_{v}(g_{2})$ it suffices that
$g_{1}^{-1}g_{2}\in K_{v}({\mathfrak{n}})$. Therefore we have that

$W_{v}(a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu})=W_{v}(a(\alpha_{\tau})a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu}),$
(17)

for every finite place. Combining (15) and (17), we have

$\frac{\tau\big{(}W_{v}(a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu})\big{)}}{W_{v}(a(\xi){\iota_{\text{f}}(\sigma^{-1})}x_{\mu})}=\frac{|\xi
t_{\mu}|_{v}^{k_{0}/2}}{\tau(|\xi t_{\mu}|_{v})^{k_{0}/2}}.$ (18)

We now consider the quotient

$\frac{\tau\big{(}c_{\mu}(\xi;f_{\mu}||_{k}\sigma)\big{)}}{c_{\mu}(\xi;f_{\mu}||_{k}\sigma)}.$

To prove the result it now suffices to show this quotient is equal to one.
Therefore by (14) and (18) we have

$\displaystyle\frac{\tau\big{(}c_{\mu}(\xi;f_{\mu}||_{k}\sigma)\big{)}}{c_{\mu}(\xi;f_{\mu}||_{k}\sigma)}$
$\displaystyle=\frac{\tau\big{(}N(t_{\mu}{\mathcal{O}}_{F})^{-k_{0}/2}\big{)}\tau(\xi^{k_{0}/2})}{N(t_{\mu}{\mathcal{O}}_{F})^{-k_{0}/2}\xi^{k_{0}/2}}\prod_{v<\infty}\frac{|\xi
t_{\mu}|_{v}^{k_{0}/2}}{\tau(|\xi t_{\mu}|_{v})^{k_{0}/2}}$
$\displaystyle=\frac{\tau\big{(}\prod_{v<\infty}|t_{\mu}|_{v}^{k_{0}/2}\big{)}}{\prod_{v<\infty}|t_{\mu}|_{v}^{k_{0}/2}}\frac{\tau\big{(}\prod_{v|\infty}|\xi|_{v}^{k_{0}/2}\big{)}}{\prod_{v|\infty}|\xi|_{v}^{k_{0}/2}}\frac{\big{(}\prod_{v<\infty}|t_{\mu}|_{v}|\xi|_{v}\big{)}^{k_{0}/2}}{\tau\big{(}\prod_{v<\infty}|t_{\mu}|_{v}|\xi|_{v}\big{)}^{k_{0}/2}}$
$\displaystyle=\frac{\tau\big{(}\prod_{v|\infty}|\xi|_{v}^{k_{0}}\big{)}}{\prod_{v|\infty}|\xi|_{v}^{k_{0}}},\
\text{using global norm formula}.$

We have that $\prod_{v|\infty}|\xi|_{v}=|N(\xi)|\in{\mathbb{Q}}$. Hence

$\frac{\tau\big{(}\prod_{v|\infty}|\xi|_{v}^{k_{0}}\big{)}}{\prod_{v|\infty}|\xi|_{v}^{k_{0}}}=\frac{\prod_{v|\infty}|\xi|_{v}^{k_{0}}}{\prod_{v|\infty}|\xi|_{v}^{k_{0}}}=1.$

Therefore we have that $\tau$ fixes $c_{\mu}(\xi;f_{\mu}||_{k}\sigma)$ and so

$c_{\mu}(\xi;f_{\mu}||_{k}\sigma)\in{\mathbb{Q}}({\mathbf{f}})(\zeta_{N_{0}}).$

∎

######  Remark 6.2.

We see that in the special case of $F={\mathbb{Q}}$ we obtain the result of
Brunault and Neururer for newforms.

######  Question 6.3.

Our method to prove Theorem 4.1 and Theorem 6.1 was to find sufficient
conditions not necessary ones. Therefore one can ask if the number field
${\mathbb{Q}}({\mathbf{f}})(\zeta_{N_{0}})$ is optimal, that is, do we have
${\mathbb{Q}}({\mathbf{f}},\mu,\sigma)={\mathbb{Q}}({\mathbf{f}})(\zeta_{N_{0}})$?

Note that if $F={\mathbb{Q}}$ this is known to be true [2, Theorem 7.6], so
one can restrict their attention to the case of $n>1$.

######  Question 6.4.

It would be interesting to generalise the results in this paper to the setting
of Hilbert newforms with non-trivial central character and specifically what
will the number field be.

## References

  * [1] J. H. Bruinier, G. van der Geer, G. Harder & D. Zagier The 1-2-3 of modular forms, Universitext, Springer-Verlag, Berlin, 2008.
  * [2] F. Brunault & M. Neururer Fourier expansions at cusps, J. Ramanujan Math., 53, 423–437 (2020).
  * [3] D. Bump Automorphic forms and representations, Cambridge Studies in Advanced Mathematics, 55. Cambridge University Press, Cambridge, 1997.
  * [4] Henri Cohen Expansions at Cusps and Petersson Products in Pari/GP, Johannes Blümlein; Carsten Schneider; Peter Paule. Elliptic Integrals, Elliptic Functions and Modular Forms in Quantum Field Theory, Springer, 2019, Texts & Monographs in Symbolic Computation. hal-01883070
  * [5] A. Corbett & A. Saha On the order of vanishing of newforms at cusps, Mathematical Research Letters, vol. 25 (2018), no. 6, 1771-1804.
  * [6] F. Diamond & J. Im Modular forms and modular curves, in: Seminar on Fermat’s Last Theorem, CMS Conf. Proc. 17, Amer. Math. Soc., Providence, RI, 1995, 39-133.
  * [7] F. Diamond & J. Shurman A first course in modular forms, Graduate Texts in Mathematics, vol. 228, Springer-Verlag, New York, 1999.
  * [8] M. Dickson & M. Neururer Products of Eisenstein series and Fourier expansions of modular forms at cusps, Journal of Number Theory, 188:137–164, 2018.
  * [9] P. Garrett Holomorphic Hilbert modular forms, The Wadsworth & Brooks/Cole Mathematics Series. Wadsworth & Brooks/Cole Advanced Books & Software, Pacific Grove, CA, 1990.
  * [10] S. Gelbart Automorphic forms on adele groups, Annals of Math. Studies, Number 83, Princeton University Press, 1975.
  * [11] D. Goldfeld & J. Hundley Automorphic representations and $L$-functions for the general linear group Vol I, Cambridge University Press, 2011.
  * [12] LMFDB, The L-functions and Modular forms Database, https://www.lmfdb.org
  * [13] J. Neukirch Algebraic number theory, A Series of Comprehensive Studies in Mathematics, vol. 322, Springer-Verlag, Berlin, 1999.
  * [14] D. Ramakrishnan & R. Valenza Fourier analysis on number fields, Graduate Texts in Mathematics, vol. 186, Springer-Verlag, New York, 1999.
  * [15] A. Raghuram & N. Tanabe Notes on the arithmetic of Hilbert modular forms, J. Ramanujan Math. Soc. 26 (2011), no. 3, 261–319.
  * [16] A. Saha Large values of newforms on GL(2) with highly ramified central character, Int. Math. Res. Not. (IMRN) (2016), 2016(13): 4103-4131.
  * [17] R. Schmidt Some remarks on local newforms for $\operatorname{GL}(2)$, J. Ramanujan Math. Soc. 17 (2002), no. 2, 115-147.
  * [18] G. Shimura On some arithmetic properties of modular forms of one and several variables, Ann. of Math. (2), 102(3):491–515, 1975.
  * [19] G. Shimura The special values of the zeta functions associated with Hilbert modular forms, Duke Math. J. 45 (1978), no. 3, 637-679.

SCHOOL OF MATHEMATICAL SCIENCES, QUEEN MARY UNIVERSITY OF LONDON, LONDON E1
4NS, UK

E-mail address<EMAIL_ADDRESS>


# Anatomical and Diagnostic Bayesian Segmentation in Prostate MRI —Should
Different Clinical Objectives Mandate Different Loss Functions?

Anindo Saha1, Joeran Bosma1, Jasper Linmans1,2, Matin Hosseinzadeh1, Henkjan
Huisman1
1Diagnostic Image Analysis Group, Radboud University Medical Center, The
Netherlands
2Computational Pathology Group, Radboud University Medical Center, The
Netherlands
{anindya.shaha,joeran.bosma,jasper.linmans,
<EMAIL_ADDRESS>

###### Abstract

We hypothesize that probabilistic voxel-level classification of anatomy and
malignancy in prostate MRI, although typically posed as near-identical
segmentation tasks via U-Nets, require different loss functions for optimal
performance due to inherent differences in their clinical objectives. We
investigate distribution, region and boundary-based loss functions for both
tasks across 200 patient exams from the publicly-available ProstateX dataset.
For evaluation, we conduct a thorough comparative analysis of model
predictions and calibration, measured with respect to multi-class volume
segmentation of the prostate anatomy (whole-gland, transitional zone,
peripheral zone), as well as, patient-level diagnosis and lesion-level
detection of clinically significant prostate cancer. Notably, we find that
distribution-based loss functions (in particular, focal loss) are well-suited
for diagnostic or panoptic segmentation tasks such as lesion detection,
primarily due to their implicit property of inducing better calibration.
Meanwhile, (with the exception of focal loss) both distribution and
region/boundary-based loss functions perform equally well for anatomical or
semantic segmentation tasks, such as quantification of organ shape, size and
boundaries.

##  1 Introduction

### Anatomical versus Diagnostic

Anatomical (organ-level) and diagnostic (pathology-level) segmentation are
fundamentally different, with respect to their clinical objectives. For
instance, within the clinical workflow of prostate MRI, segmentation of the
prostate anatomy {whole-gland (WG), transitional zone (TZ), peripheral zone
(PZ)} is used to estimate its volume, boundaries and geometry –enabling the
calculation of prostate-specific antigen (PSA) density, guiding treatment
planning and future interventions [1, 2, 3]. Meanwhile, segmentation of
clinically significant prostate cancer (csPCa) is primarily used to detect the
number of malignant lesions present in the prostate gland (if any), and
characterize each instance for diagnosis, risk stratification and/or targeted
biopsies (similar to PI-RADS guidelines [4]). In other words, while the former
anatomical task leans towards semantic single-object quantification, the
latter diagnostic task can be framed as panoptic multi-object detection [5,
6]. Despite these differences, in medical image computing, these objectives
are often treated and trained as near-identical segmentation tasks using
U-Nets, and evaluated in a similar manner using the Dice Similarity
Coefficient (DSC) [7, 8, 9, 10]. Recent studies confirm that while DSC can be
appropriate for single-object quantification, its inability to measure multi-
object detection makes it unsuitable for lesion localization [11, 12, 13, 5].
We hypothesize that such considerations should not only be made during
evaluation, but should also be incorporated at train-time by using task-
specific loss function(s).

### Segmentation Loss Functions

We can distinguish between loss functions that minimize mismatch in underlying
distributions and those that minimize mismatch in segmentation
regions/boundaries [14]. Distribution-based loss functions, such as the widely
adopted cross-entropy (CE) loss, minimize dissimilarity between the target
distribution (ground-truth) and its approximation (model predictions), over
the course of training. Balanced cross-entropy (BCE) loss is an extension of
CE, introducing class weights $\alpha$ to address imbalanced class frequencies
in the dataset. Focal loss (FL) represents a further extension of BCE,
introducing an additional hyperparameter $\gamma$ to differentiate between
easy and difficult examples [15]. When $\gamma=0$, FL reduces down to CE or
BCE depending on the value of $\alpha$. On the other hand, region or boundary-
based loss functions are derived from integrals over segmentation regions
(e.g. soft Dice loss [16]) or distance metrics in the space of contours (e.g.
boundary loss [17]). Unlike distribution-based losses, which can benefit from
class re-weighting, the sum of Dice and boundary loss (DB) is implicitly
impervious to class imbalance, and in turn, recommended for highly imbalanced
datasets [17].

##  2 Experiments and Analysis

### Materials

We used 200 prostate bpMRI (T2W, high b-value DWI, ADC) exams from the
publicly-available ProstateX dataset [18], paired with voxel-level
delineations of WG, TZ, PZ and csPCa [19]. All images were resampled to
$0.5\times 0.5\times 3.0$ mm3 resolution, center-cropped to $160\times
160\times 20$ voxels and intensity-normalized (T2W, DWI: z-score; ADC: linear)
[20], prior to usage.

### Bayesian Segmentation Model

We used a probabilistic adaptation [21] of the deep attentive 3D U-Net,
developed and validated specifically for prostate bpMRI in our previous work
[20]. Monte-Carlo dropout nodes were added to capture both epistemic and
aleatoric uncertainty during inference (as recommended by Hu et al. [22]).
Cosine annealing learning rate [23] (decaying from $10^{\textnormal{-}4}$ to
$10^{\textnormal{-}7}$) and AMSGrad optimizer [24] were used to train the
model. Data augmentations comprised of additive Gaussian noise (standard
deviation $0$-$0.5$), horizontal flip, rotation ($\pm$$7.5$°), translation
($0$-$15$% horizontal and/or vertical shifts) and scaling ($0$-$15$%) centered
along the axial plane. Fig. 1 illustrates the train-time schematic of the
model, and its source code is made publicly-available at
https://github.com/DIAGNijmegen/prostateMR_3D-CAD-csPCa.

### Experiments

We investigated four different loss policies (CE, BCE, FL, DB) as the
segmentation loss function ($\textbf{{L}}_{\textbf{S}}$) in our model for
anatomical ({WG, TZ, PZ} in T2W MRI) and diagnostic (csPCa in bpMRI)
segmentation. For the former task, $\alpha$ is set as [$0.05$, $0.30$, $0.65$]
as per the empirical class distribution of the dataset, and for the latter
task, $\alpha$ is set as [$0.75$, $0.25$] as per the findings of previous
studies [25, 20]. In both cases, $\gamma$ is set to its default value of
$2.00$ as used in [15, 25, 20]. All metrics were computed in 3D over 3
independent runs $\times$ 5-fold cross-validation $\times$ mean of 100
executions of probabilistic inference per image. Identical data splits were
maintained for all configurations across both tasks.

Figure 1: Train-time schematic for the Bayesian segmentation model.
$\textbf{{L}}_{\textbf{KL}}$ denotes the Kullback–Leibler divergence loss
between prior distribution P and posterior distribution Q.
$\textbf{{L}}_{\textbf{S}}$ denotes the segmentation loss between prediction p
and ground-truth Y. For each execution of the model, one sample z
$\in\textbf{{Q}}$ (train-time) or z $\in\textbf{{P}}$ (test-time) is drawn to
predict one segmentation mask p [21].

##  3 Results and Discussion

|  |
---|---|---
(a) |  (c) |  (d)

|  |
---|---|---
(b) |  (e) |  (f)

Figure 2: Anatomical (top-row) and diagnostic (bottom-row) segmentation of
{WG, TZ, PZ} in T2W MRI and csPCa in bpMRI, respectively. For each
$\textbf{{L}}_{\textbf{S}}$ configuration, we evaluated (a, b) model
calibration with negative log-likelihood (NLL), (c, d) segmentation quality
with mean soft/hard Dice Similarity Coefficient (DSC) across all 600
observations (3 runs $\times$ 5 folds $\times$ 40 validation samples), (e)
patient-level diagnosis with Area Under Receiver Operating Characteristic
(AUROC), (f) lesion-level localization with Free-Response Receiver Operating
Characteristic (FROC). Transparent areas in (f) FROC indicate $95$% confidence
intervals. Arrows ($\uparrow$ or $\downarrow$) across (a-f) indicate whether
the corresponding metric should be maximized/minimized for ideal performance.

### Calibration

We find that distribution-based loss functions (CE, BCE, FL) produce
implicitly well calibrated predictions, in comparison to DB [26] (as shown in
Fig. 2 (a, b) and Appendix A). Among them, FL exhibits notably better
calibration. Our findings concur with that of Mukhoti et al. [27], where the
authors demonstrate that FL is formulated as such, that it minimizes
dissimilarity between ground-truth and predicted distributions, while
increasing entropy of the latter –thereby limiting NLL overfitting and
inducing both calibration and regularization. Such a property can also lead to
more useful uncertainty estimates [27, 28].

### Anatomical Segmentation

From Fig. 2 (c), it is clear that while CE, BCE and DB achieve similar
segmentation quality with respect to raw softmax predictions ({WG: $81$-$89$;
TZ: $82$-$85$; PZ: $71$-$74$} mean DSC), FL performs worse ({WG: $70$; TZ:
$64$; PZ: $50$} mean DSC). When considering binarized argmax(softmax)
predictions, as seen in Fig. 2 (d), overall FL performance is only marginally
lower than that of CE, BCE and DB ({WG: $88$-$90$; TZ: $84$-$86$; PZ:
$72$-$76$} mean DSC). We attribute this decline in FL performance to its high
predictive uncertainty near class boundaries (as seen in Appendix A), which
limits its ability to produce precise contour definitions.

### Diagnostic Segmentation

From Fig. 2 (e, f), we observe that while all loss functions are similar in
patient-level diagnosis ($76$-$78$ mean AUROC), results indicate that
distribution-based loss functions, especially FL, perform substantially better
than region/boundary-based losses (DB) at lesion detection (with an average
increase of $15.7\%.$ in maximum detection sensitivity and $0.25$ in partial
area under FROC between $0.1$-$2.5$ false positives per patient). We attribute
this improvement to the implicit train-time calibration induced by FL, which
can facilitate better risk stratification and higher detection sensitivities
across difficult diagnostic tasks (e.g. via retaining low-confidence lesion
predictions, as opposed to muting them out –refer to Appendix A). We believe
that the growing success of csPCa detection models trained using FL
derivatives (such as the FocalNet [25], among others [20, 29, 30]) can, in
part, be attributed to this phenomenon rather than architectural enhancements
or class weighting alone (note BCE performs near-identical to CE, if not
worse, across both tasks). On the other hand, while DB loss is a natural fit
for maximizing DSC, it is ill-posed to optimize panoptic segmentation
objectives [5, 11]. Subsequently, its miscalibration (refer to Fig. 2 (a, b))
or polarized predictions (refer to Appendix A), translate to a relatively flat
FROC curve with lower maximum detection sensitivity (refer to Fig. 2 (f))
–presumably detection of low-confidence, difficult or small lesions are
skipped in favour of maximizing confidence and overlap of clear lesions [11].

In conclusion, we recommend distribution-based loss functions (in particular,
FL) for diagnostic or panoptic segmentation tasks. For anatomical or semantic
segmentation tasks, we observe that although distribution-based losses improve
calibration, (with the exception of FL) they are equivalent to
region/boundary-based losses in terms of standard/hard DSC. Hence, CE, BCE, DB
or presumably their composites (as applied by the nnU-Net [31] and
investigated by J. Ma et al. [14]), can be recommended. Further analyses are
required, using larger datasets and multiple medical imaging modalities, to
draw out definitive conclusions.

## Broader Impact

Prostate cancer is one of the most prevalent cancers in men worldwide [32]. In
the absence of experienced radiologists, its multifocality, morphological
heterogeneity and strong resemblance to numerous non-malignant conditions in
MR imaging, can lead to low inter-reader agreement ($<50\%$) and sub-optimal
interpretation [33, 34, 35]. The development of automated, reliable detection
algorithms has therefore become an important research focus in medical image
computing —offering the potential to support radiologists with consistent
quantitative analysis, improve diagnostic accuracy, and in turn, minimize
unnecessary biopsies in patients [36, 37].

To the best of our knowledge, this study has no foreseeable negative societal
effects.

## Acknowledgments and Disclosure of Funding

This research is supported, in parts, by the European Union H2020: ProCAncer-I
project (EU grant 952159), Health~Holland (LSHM20103), ContextVision AB
(Linköping, Sweden) and Siemens Healthineers (CID: C00225450).

## References

  * Eklund et al. [2021] M. Eklund, F. Jäderling, A. Discacciati, and M. Bergman et al.  MRI-Targeted or Standard Biopsy in Prostate Cancer Screening.  _New England Journal of Medicine_ , 2021.  doi: 10.1056/NEJMoa2100852.
  * Falagario et al. [2020] U. G. Falagario, I. Jambor, A. Lantz, and O. Ettala et al.  Combined Use of Prostate-specific Antigen Density and Magnetic Resonance Imaging for Prostate Biopsy Decision Planning: A Retrospective Multi-institutional Study Using the Prostate Magnetic Resonance Imaging Outcome Database (PROMOD).  _European Urology Oncology_ , 2020.  ISSN 2588-9311.  doi: 10.1016/j.euo.2020.08.014.
  * M. Valerio and Y. Cerantola and S. E. Eggener and H. Lepor et al. [2017] M. Valerio and Y. Cerantola and S. E. Eggener and H. Lepor et al.  New and Established Technology in Focal Ablation of the Prostate: A Systematic Review.  _European Urology_ , 71(1):17–34, 2017.  ISSN 0302-2838.  doi: 10.1016/j.eururo.2016.08.044.
  * Weinreb et al. [2016] J.C. Weinreb, J.O. Barentsz, P.L. Choyke, and F. Cornud.  PI-RADS Prostate Imaging – Reporting and Data System: 2015, Version 2.  _European Urology_ , 69(1):16 – 40, 2016.
  * Kirillov et al. [2019] A. Kirillov, K. He, R. Girshick, and C. Rother et al.  Panoptic Segmentation.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * Yu et al. [2020] X. Yu, B. Lou, D. Zhang, D. Winkel, and L. Joskowicz.  Deep Attentive Panoptic Model for Prostate Cancer Detection Using Biparametric MRI Scans.  In _Medical Image Computing and Computer Assisted Intervention (MICCAI 2020)_ , pages 594–604, 2020.  ISBN 978-3-030-59719-1.
  * Artan and Yetik [2012] Y. Artan and I. S. Yetik.  Prostate Cancer Localization Using Multiparametric MRI based on Semisupervised Techniques With Automated Seed Initialization.  _IEEE Transactions on Information Technology in Biomedicine_ , 16(6):1313–1323, 2012.  doi: 10.1109/TITB.2012.2201731.
  * Chung et al. [2015] A. G. Chung, F. Khalvati, M. J. Shafiee, M. A. Haider, and A. Wong.  Prostate Cancer Detection via a Quantitative Radiomics-Driven Conditional Random Field Framework.  _IEEE Access_ , 3:2531–2541, 2015.  doi: 10.1109/ACCESS.2015.2502220.
  * Kohl et al. [2017] S. Kohl, D. Bonekamp, H. P. Schlemmer, and K. Maier-Hein et al.  Adversarial Networks for the Detection of Aggressive Prostate Cancer, 2017.
  * Jäger [2020] P. F. Jäger.  Challenges and Opportunities of End-to-End Learning in Medical Image Classification, 2020.
  * A. Reinke and M. D. Tizabi and M. Eisenmann and L. Maier-Hein [2021] A. Reinke and M. D. Tizabi and M. Eisenmann and L. Maier-Hein.  Common Pitfalls and Recommendations for Grand Challenges in Medical Artificial Intelligence.  _European Urology Focus_ , 7(4):710–712, 2021\.  ISSN 2405-4569.  doi: 10.1016/j.euf.2021.05.008.
  * Carass et al. [2020] A. Carass, S. Roy, A. Gherman, and I. Oguz et al.  Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis.  _Scientific Reports_ , 10(1):8242, 2020.  ISSN 2045-2322.
  * Yeghiazaryan and Voiculescu [2018] V. Yeghiazaryan and I. D. Voiculescu.  Family of Boundary Overlap Metrics for the Evaluation of Medical Image Segmentation.  _Journal of Medical Imaging_ , 5(1):1 – 19, 2018\.  doi: 10.1117/1.JMI.5.1.015006.
  * J. Ma and J. Chen and M. Ng and A. L. Martel et al. [2021] J. Ma and J. Chen and M. Ng and A. L. Martel et al.  Loss Odyssey in Medical Image Segmentation.  _Medical Image Analysis_ , 71:102035, 2021.  ISSN 1361-8415.  doi: 10.1016/j.media.2021.102035.
  * Lin et al. [2017] T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár.  Focal Loss for Dense Object Detection.  In _2017 IEEE International Conference on Computer Vision (ICCV)_ , pages 2999–3007, 2017.
  * F. Milletari, N. Navab, S. Ahmadi [2016] F. Milletari, N. Navab, S. Ahmadi.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation.  In _4th International Conference on 3D Vision (3DV)_ , 2016.
  * H. Kervadec and J. Bouchtiba and C. Desrosiers and E. Granger et al. [2021] H. Kervadec and J. Bouchtiba and C. Desrosiers and E. Granger et al.  Boundary Loss for Highly Unbalanced Segmentation.  _Medical Image Analysis_ , 67:101851, 2021.  ISSN 1361-8415.
  * S. G. Armato and H. Huisman and K. Drukker and L. Hadjiiski et al. [2018] S. G. Armato and H. Huisman and K. Drukker and L. Hadjiiski et al.  PROSTATEx Challenges for Computerized Classification of Prostate Lesions from Multiparametric Magnetic Resonance Images.  _Journal of Medical Imaging_ , 5(4):1 – 9, 2018\.  doi: 10.1117/1.JMI.5.4.044501.
  * Cuocolo et al. [2021] R. Cuocolo, A. Stanzione, A. Castaldo, and D. R. De Lucia et al.  Quality Control and Whole-Gland, Zonal and Lesion Annotations for the PROSTATEx Challenge Public Dataset.  _European Journal of Radiology_ , 138:109647, 2021.  doi: https://doi.org/10.1016/j.ejrad.2021.109647.
  * Saha et al. [2021] A. Saha, M. Hosseinzadeh, and H. Huisman.  End-to-end Prostate Cancer Detection in bpMRI via 3D CNNs: Effects of Attention Mechanisms, Clinical Priori and Decoupled False Positive Reduction.  _Medical Image Analysis_ , 73:102155, 2021.  ISSN 1361-8415.  doi: 10.1016/j.media.2021.102155.
  * Kohl et al. [2018] S. A. A. Kohl, B. Romera-Paredes, C. Meyer, and O. Ronneberger et al.  A Probabilistic U-Net for Segmentation of Ambiguous Images.  In _Medical Imaging Meets NeurIPS Workshop – 32nd Conference on Neural Information Processing Systems (NeurIPS)_ , 2018.
  * Hu et al. [2019] S. Hu, D. Worrall, S. Knegt, and M. Welling et al.  Supervised uncertainty quantification for segmentation with multiple annotations.  In _Medical Image Computing and Computer Assisted Intervention – MICCAI 2019_ , pages 137–145, 2019.  ISBN 978-3-030-32245-8.
  * Loshchilov and Hutter [2016] I. Loshchilov and F. Hutter.  SGDR: Stochastic Gradient Descent with Warm Restarts.  In _International Conference on Learning Representations (ICLR)_ , 2016.
  * Reddi et al. [2018] S. J. Reddi, S. Kale, and S. Kumar.  On the Convergence of Adam and Beyond.  In _International Conference on Learning Representations (ICLR)_ , 2018.
  * Cao et al. [2019] R. Cao, A. Mohammadian Bajgiran, S. Afshari Mirak, S. Shakeri, X. Zhong, D. Enzmann, S. Raman, and K. Sung.  Joint Prostate Cancer Detection and Gleason Score Prediction in mp-MRI via FocalNet.  _IEEE Transactions on Medical Imaging_ , 38(11):2496–2506, 2019.
  * Mehrtash et al. [2020] A. Mehrtash, W. M. Wells, C. M. Tempany, and T. Kapur et al.  Confidence calibration and predictive uncertainty estimation for deep medical image segmentation.  _IEEE Transactions on Medical Imaging_ , 39(12):3868–3878, 2020.  doi: 10.1109/TMI.2020.3006437.
  * Mukhoti et al. [2020] J. Mukhoti, V. Kulharia, A. Sanyal, and P. K. Dokania et al.  Calibrating deep neural networks using focal loss.  In _34th Conference on Neural Information Processing Systems (NeurIPS 2020)_ , 2020.
  * Kuleshov et al. [2018] V. Kuleshov, N. Fenner, and S. Ermon.  Accurate Uncertainties for Deep Learning Using Calibrated Regression.  In _35th International Conference on Machine Learning (ICML 2018)_ , 2018.
  * Cao et al. [2019] R. Cao, X. Zhong, S. Shakeri, and K. Sung et al.  Prostate Cancer Detection and Segmentation in Multi-parametric MRI via CNN and Conditional Random Field.  In _2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)_ , pages 1900–1904. IEEE, 2019.
  * Qian et al. [2021] Y. Qian, Z. Zhang, and B. Wang.  ProCDet: A New Method for Prostate Cancer Detection based on MR Images.  _IEEE Access_ , 2021.  doi: 10.1109/ACCESS.2021.3114733.
  * Isensee et al. [2021] F. Isensee, S. A. A. Kohl, J. Petersen, P. F. Jaeger, and K. H. Maier-Hein.  nnU-Net: A Self-Configuring Method for Deep Learning-based Biomedical Image Segmentation.  _Nature Methods_ , 18(2):203–211, Feb 2021.  ISSN 1548-7105.  doi: 10.1038/s41592-020-01008-z.
  * Miller et al. [2019] K.D. Miller, L. Nogueira, A.B. Mariotto, and R.L. et al. Siegel.  Cancer Treatment and Survivorship Statistics, 2019.  _CA: A Cancer Journal for Clinicians_ , 69(5):363–385, 2019.
  * Israël et al. [2020] B. Israël, M. van der Leest, M. Sedelaar, A.R. Padhani, P. Zámecnik, and J.O. Barentsz.  Multiparametric Magnetic Resonance Imaging for the Detection of Clinically Significant Prostate Cancer: What Urologists Need to Know. Part 2: Interpretation.  _European Urology_ , 77(4):469–480, 2020.
  * Smith et al. [2019] C.P. Smith, S.A. Harmon, T. Barrett, and L.K. Bittencourt.  Intra- and Interreader Reproducibility of PI-RADS v2: A Multireader Study.  _Journal of Magnetic Resonance Imaging_ , 49(6):1694–1703, 2019.
  * Rosenkrantz et al. [2016] A.B. Rosenkrantz, L.A. Ginocchio, D. Cornfeld, and A.T. Froemming.  Interobserver Reproducibility of the PI-RADS Version 2 Lexicon.  _Radiology_ , 280(3):793–804, 2016.
  * Elwenspoek et al. [2019] M.M.C. Elwenspoek, A.L. Sheppard, M.D.F. McInnes, and P. Whiting.  Comparison of Multiparametric Magnetic Resonance Imaging and Targeted Biopsy With Systematic Biopsy Alone for the Diagnosis of Prostate Cancer: A Systematic Review and Meta-Analysis.  _JAMA Network Open_ , 2(8):e198427, 2019.
  * Schelb et al. [2019] P. Schelb, J.P. Kohl, S.and Radtke, and D. Bonekamp.  Classification of Cancer at Prostate MRI: Deep Learning versus Clinical PI-RADS Assessment.  _Radiology_ , 293(3):607–617, 2019.

##  Appendix A Appendix: Model Predictions

Figure 3: Anatomical (WG, TZ, PZ) and diagnostic (csPCa) segmentations for a
single validation scan, across all four loss functions (CE, BCE, FL, DB), are
shown above. Predictions are overlaid on the T2W scan as reference, where
white lines indicate the corresponding ground-truth annotation for the given
task. For each case, a histogram of class probabilities over the predicted
segment(s) has been illustrated below, indicating the degree of implicit
calibration induced by the loss function.


# Self-supervision of wearable sensors time-series data for influenza
detection

Arinbjörn Kolbeinsson
Evidation Health
<EMAIL_ADDRESS>
Piyusha Gade
Evidation Health
<EMAIL_ADDRESS>
Raghu Kainkaryam
Evidation Health
<EMAIL_ADDRESS>
Filip Jankovic
Evidation Health
<EMAIL_ADDRESS>
Luca Foschini
Evidation Health
<EMAIL_ADDRESS>

###### Abstract

Self-supervision may boost model performance in downstream tasks. However,
there is no principled way of selecting the self-supervised objectives that
yield the most adaptable models. Here, we study this problem on daily time-
series data generated from wearable sensors used to detect onset of influenza-
like illness (ILI). We first show that using self-supervised learning to
predict next-day time-series values allows us to learn rich representations
which can be adapted to perform accurate ILI prediction. Second, we perform an
empirical analysis of three different self-supervised objectives to assess
their adaptability to ILI prediction. Our results show that predicting the
next day’s resting heart rate or time-in-bed during sleep provides better
representations for ILI prediction. These findings add to previous work
demonstrating the practical application of self-supervised learning from
activity data to improve health predictions.

##  1 Introduction

Ground-truth labels for health data are both difficult and expensive to
acquire. Lack of access to labelers (e.g. physicians) and patients not
reporting symptoms are common sources of missing labels. As a result, many
healthcare data are partially or completely unlabelled, with larger
prospective studies often having proportionally fewer labels. This trade-off
in study size and accuracy is particularly severe in healthcare, although it
manifests in most real-world studies across domains. We are interested in
learning from person-generated activity time-series data that has heavily
imbalanced or missing labels. As ground-truth labels are a requirement for
supervised learning, we must look to other approaches in order to make use of
the vast information contained in the unlabelled data.

One potential method to fit this need is transfer learning, where a model is
first trained on a different but near-identical population to the final task.
However, on closer inspection, this becomes difficult to implement in
practice. Firstly, study design critically affects the signals that connect
the observed data to the output labels. It is impractical to expect access to
a different dataset created through the same study design. Secondly, in the
case where we want to study a rare disease, the low prevalence will make it a
challenge to find a large, separate initial training set. Therefore, transfer
learning alone does not satisfy all our criteria.

Self-supervised learning allows for learning of representations without
defined labels. Although the theoretical principles of these methods have yet
to be described, the empirical performance has been widely reported. Most
notably in natural language processing [1] and other syntactic tasks such as
programming [2], but also in image classification [3]. The taxonomy of these
models is debated, with some scholars [4] describing pre-trained, self-
supervised, meta-learning models as _Foundation Models_. A core concept of
self-supervised learning is that the objective is defined as some function of
the data itself. However, selecting an optimal self-supervised objective task
is not trivial. Here, we study and compare three objective tasks and gauge
their performance on influenza-like illness (ILI) prediction with a lab-
validated ground-truth.

We take inspiration from natural language processing, where next-word
prediction is a task that enables rich representations to be learned [1]. Our
end-to-end transformer models initially learn through self-supervision to
predict next-day activity levels using activity data from the previous ten
days in a large population. These models are then fine-tuned on the ILI
prediction task and evaluated on a small dataset with laboratory-confirmed ILI
labels. Final evaluation on an ILI prediction task in a held-out testing set
reveals that performance is dependent on the self-supervision task.

##  2 Data and study set-up

We use the HomeKit dataset collected by Evidation Health between February and
May 2020. $5\,229$ individuals took part in a prospective cohort study where
wearable (Fitbit) device data (including resting heart-rate, step count, and
sleep) were recorded. Along with these raw data, Fitbit provides API to
multiple processed features that they have pre-calculated, including but not
limited to nap count, total minutes asleep, total minutes in bed, total
minutes spent sedentary, calories spent on activity and total calories spent.

Every day, each participant was prompted to complete a survey on ILI-related
symptoms. If a participant reported two specific symptoms (cough and one of
body ache, feeling feverish, chills, sweats) on the same day, they were sent a
kit to self-administer a nasal swab which was then tested in a laboratory
using an approved polymerase chain reaction (PCR) method. The PCR results
revealed that infections identified in the study were caused by a host of
respiratory viruses including, but not limited to, influenza virus,
respiratory syncytial virus, rhinovirus and SARS-CoV-2.

Although label coverage is almost complete in the original study, we devise a
set of experiments to simulate a scenario where we have access to two
populations: A) a large set of unlabeled activity data and B) a small set of
labeled data. In both sets, the recorded activity features (inputs) are
identical. A schematic of this is shown in Figure 1A.

For self-supervised learning, we train three models to predict next-day
resting heart rate, time-in-bed minutes and activity calories expended,
respectively. For every model, the input data consisted of resting heart rate,
minutes spent in bed, estimated activity calorie output, and missingness flags
for each feature. For evaluation, we apply the model to flu prediction. The
input data has the same form as that of SSL training, i.e. ten days of resting
heart rate, minutes spent in bed, estimated activity calorie output, and
missingness flags for each feature. The label is the laboratory-confirmed
cases of respiratory viral infection, not limited to influenza.

Figure 1: Overview schematic of the study design and model architecture. A)
The study design showing how the input data sequences are used to train the
model, first on a selected self-supervised objective. The model is then
adapted to ILI prediction by retraining only the final layer. B) A simplified
architectural overview of the transformer-based model.

##  3 Model and learning

We use a decoder-only transformer-based architecture, inspired by state-of-
the-art natural learning processing approaches [1]. It is composed of four
transformer blocks. Each block contains a dense self-attention layer with
dimensionality $2\,048$, dropout ($p=0.1$) and ReLU activation. Self attention
is calculated as $attention(Q,K,V)=softmax(QK^{T}/\sqrt{d_{k}})V$ [5]. Where
$Q$, $K$ and $V$ are the query, key and value matrices, respectively and
$d_{k}$ is the dimensionality of the keys. Layer normalisation was applied
after each transformer block. $LayerNorm(v)=\gamma\frac{v-\mu}{\sigma}+\beta$
[6]. Where $\mu$ and $\sigma$ are the mean and standard deviation of the
vector, respectively and $\gamma$ and $\beta$ are learnable parameters.
Position embeddings for the sequence are learned from random initialization
and added to the input sequence. A simplified overview of the model
architecture is shown in Figure 1B.

The final layer maps the output of the last transformer block to the task-
dependent prediction. In regression tasks, this is a fully-connected layer
with a single output unit. In the case of binary classification, the output is
a vector of length two that, when combined with softmax activation, represent
the predicted class pseudo-probabilities.

For self-supervised learning, we train three models for 50 epochs with the
objective of minimizing the mean squared error (MSE) of next-day prediction of
three features: resting heart rate, time spent in bed, and activity calories
expended. The three models have identical architectures as described above and
are randomly initialized. In this setting, we only train on sequences where
the next-day sequence is known. Fine-tuning on ILI prediction classification
was done by minimizing the binary cross entropy of the true and predicted
outputs.

All models are minimized with Adam [7], initial learning rate of $1$ with
cosine annealing and gradient clipping at $1.0$ to accelerate training [1, 8].
We train with a batch size of 64 for 50 epochs. For fine-tuning on the
adaptation task, we freeze the transformer weights and retrain a new fully-
connected output layer from random initialization for 30 epochs.

##  4 Results

We compare adaptation of the three models on five adaptation sets of different
sizes, ranging from 25 to 400 individuals. Final evaluation is performed on a
held-out test set of 64 individuals. The performance of the three models is
visualised in Figure 2 and listed in Table 1. For all three models, the
performance, as measured by AUC on the ILI prediction test set, increases
logarithmically as the number of samples in the adaptation training step
increases.

Above $\sim 200$ adaptation samples, the resting-heart rate and time-in-bed
models appear to saturate in performance. However, the results suggest that
the model trained on calories might benefit from additional adaptation
samples.

Figure 2: Comparison of models trained with three different self-supervised
objective tasks. The AUC (y-axis) quantifies the performance on the adaptation
task, ILI prediction.

Out of the three models, the self-supervised objective on resting-heart rate
and time-in-bed gave the strongest result on the downstream task. Training on
activity calories expended resulted in a weaker adaptation model. With only 25
adaptation samples, the resting-heart rate model achieved an AUC of $0.55$,
noticeably better than the other two models. With more adaptation samples, the
model adapted from time-in-bed eventually matches the one trained on resting-
heart rate.

Table 1: Comparison of ILI prediction performance (area under ROC curve) of models. | Self-supervised objective
---|---
N adaptation | Resting heart-rate | Time-in-bed | Calories
$25$ | $0.55$ | $0.49$ | $0.49$
$50$ | $0.67$ | $0.60$ | $0.55$
$100$ | $0.74$ | $0.74$ | $0.55$
$200$ | $0.77$ | $0.79$ | $0.62$
$400$ | $0.78$ | $0.79$ | $0.65$

##  5 Discussion

In this work, we showed that self-supervised learning of time-series data
provides rich representations that can augment ILI identification. We also
explored three different self-supervision objective tasks and found that a
model self-trained on predicting next-day resting heart-rate or time-in-bed
outperforms a model trained using active calories expended.

The most related work in the literature is a study on transformer-based
transfer learning from fatigue prediction to ILI classification [9]. The
significant difference between the two works is that ours uses self-
supervision to learn the initial representations instead of transferring from
a set of collected labels. Our work also makes use of day-level aggregates
versus high-frequency minute-level data. While minute-level features offer
significantly more information, they are orders of magnitude more
computationally intensive. Other work on wearable sensors and attention-based
models is the learning of individualized activity responses for cardiovascular
risk-factor prediction [10]. Self-attention models have also been applied to
forecasting influenza prevalence in a population [11].

The application of self-supervised approaches in the field creates its own set
of challenges, such as the one we explored here: self-supervision task
selection. Fundamentally, this is highly related to the problem of task-
relatedness and has been explored in many subfields of machine learning,
including transfer learning [12] and meta-learning [13], and has connections
to functional symmetry [14].

From a practical perspective, it might be possible to omit task selection and
predict all features jointly. Given that heart-rate, sleep and movement are
not independent, a model learning all three might capture more diverse
relationships and give better results on related downstream tasks, such as ILI
prediction. This is an obvious next step for research in this area.

Further improvements can also be made to the study design implemented for the
ILI prediction task. Here, we are predicting positive test results, not
symptoms. There is a time lag between symptom onset and lab results, on the
order of 1 to 3 days. The results must be interpreted with this caveat in
mind, as the forecasting capabilities of these models have not been
quantified. Additional artifacts for which the effect on the results is
unknown include reduced wear-time around illness and the effect of
unidentified ILI cases in the self-supervised training set.

##  6 Conclusion

The application of self-supervised learning has great potential in medicine
and health, where labels are expensive and often missing. Our results
highlight this potential and demonstrate a practical use case where a large
set of unlabelled health data can be used and adapted to a different but
related task. The remaining open problems offer a large scope for future
research which, along with robust baselines and theoretical analysis, would
set a strong foundation for applying self-supervised learning for person-
generated health data.

## Acknowledgments and Disclosure of Funding

This project has been funded in whole or in part with Federal funds from the
Department of Health and Human Services; Office of the Assistant Secretary for
Preparedness and Response; Biomedical Advanced Research and Development
Authority, under Contract No. 75A50120C00091.

## References

  * [1] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry and Amanda Askell  “Language models are few-shot learners”  In _arXiv preprint arXiv:2005.14165_ , 2020
  * [2] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harri Edwards, Yura Burda, Nicholas Joseph and Greg Brockman  “Evaluating large language models trained on code”  In _arXiv preprint arXiv:2107.03374_ , 2021
  * [3] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold and Sylvain Gelly  “An image is worth 16x16 words: Transformers for image recognition at scale”  In _arXiv preprint arXiv:2010.11929_ , 2020
  * [4] Rishi Bommasani et al.  “On the Opportunities and Risks of Foundation Models”  In _arXiv preprint_ , 2021  arXiv: https://arxiv.org/abs/2108.07258
  * [5] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser and Illia Polosukhin  “Attention is all you need”  In _Advances in neural information processing systems_ , 2017, pp. 5998–6008
  * [6] Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang and Tieyan Liu  “On layer normalization in the transformer architecture”  In _International Conference on Machine Learning_ , 2020, pp. 10524–10533  PMLR
  * [7] Diederik P. Kingma and Jimmy Ba  “Adam: A Method for Stochastic Optimization”  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , 2015  URL: http://arxiv.org/abs/1412.6980
  * [8] Jingzhao Zhang, Tianxing He, Suvrit Sra and Ali Jadbabaie  “Why gradient clipping accelerates training: A theoretical justification for adaptivity”  In _arXiv preprint arXiv:1905.11881_ , 2019
  * [9] Mike A Merrill and Tim Althoff  “Transformer-Based Behavioral Representation Learning Enables Transfer Learning for Mobile Sensing in Small Datasets”  In _arXiv preprint arXiv:2107.06097_ , 2021
  * [10] Haraldur T Hallgrímsson, Filip Jankovic, Tim Althoff and Luca Foschini  “Learning individualized cardiovascular responses from large-scale wearable sensors data”  In _arXiv preprint arXiv:1812.01696_ , 2018
  * [11] Neo Wu, Bradley Green, Xue Ben and Shawn O’Banion  “Deep transformer models for time series forecasting: The influenza prevalence case”  In _arXiv preprint arXiv:2001.08317_ , 2020
  * [12] Sinno Jialin Pan and Qiang Yang  “A survey on transfer learning”  In _IEEE Transactions on knowledge and data engineering_ 22.10 IEEE, 2009, pp. 1345–1359
  * [13] Timothy Hospedales, Antreas Antoniou, Paul Micaelli and Amos Storkey  “Meta-learning in neural networks: A survey”  In _arXiv preprint arXiv:2004.05439_ , 2020
  * [14] Daniel Lengyel, Janith Petangoda, Isak Falk, Kate Highnam, Michalis Lazarou, Arinbjörn Kolbeinsson, Marc Peter Deisenroth and Nicholas R Jennings  “GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability”  In _arXiv preprint arXiv:2011.07407_ , 2020