# Real moments of the logarithmic derivative of characteristic polynomials in
random matrix ensembles

Fan Ge  Department of Mathematics, William & Mary, Williamsburg, VA, United
States<EMAIL_ADDRESS>
###### Abstract.

We prove asymptotics for real moments of the logarithmic derivative of
characteristic polynomials evaluated at $1-\frac{a}{N}$ in unitary, even
orthogonal, and symplectic ensembles, where $a>0$ and $a=o(1)$ as the size $N$
of the matrix goes to infinity. Previously, such asymptotics were known only
for integer moments (in the unitary ensemble by the work of Bailey, Bettin,
Blower, Conrey, Prokhorov, Rubinstein and Snaith [BBB+19], and in orthogonal
and symplectic ensembles by the work of Alvarez and Snaith [AS20]), except
that in the odd orthogonal ensemble real moments asymptotics were obtained by
Alvarez, Bousseyroux and Snaith [ABS23]. Our proof is new and does not make
use of the known integer moments results in [BBB+19] and [AS20], and is
different from the method in [ABS23] for the odd orthogonal ensemble.

###### Key words and phrases:

Riemann zeta-function, logarithmic derivative, random matrices, real moments

##  1\. Introduction

Zeros of the Riemann zeta-function have been of central importance in number
theory. One way to investigate zeta zeros is to study the logarithmic
derivative $\zeta^{\prime}/\zeta$, which encodes information of zeta zeros via
the Hadamard factorization formula. This brings considerable interest to the
study of $\zeta^{\prime}/\zeta$. Besides its relation to zeta zeros, the
logarithmic derivative function $\zeta^{\prime}/\zeta$ also offers connections
between the zeros and the critical points of zeta; see for instance [LM74,
Sou98, Zha01, GY07, Ki08, FK12, Rad14, Ge17b, Ge17a] in which the logarithmic
derivative plays a key role in the study of critical points. In addition,
$\zeta^{\prime}/\zeta$ has intimate connections to the prime numbers. Indeed,
Riemann’s original plan for proving the Prime Number Theorem makes use of
$\zeta^{\prime}/\zeta$ in a crucial way. Futhermore, the mean values of
$\zeta^{\prime}/\zeta$ are closely related to primes in short intervals; see
[Sel43, GGM01].

As is often the case, one seeks for simple quantities that can more or less
represent important properties of the function itself, and in the theory of
the Riemann zeta-function, mean values or moments often serve such purposes.
To this end, let us define

$\displaystyle\mathcal{I}_{K}(\sigma,T)=\int_{T}^{2T}\left|\frac{\zeta^{\prime}}{\zeta}\left(\sigma+it\right)\right|^{2K}dt.$

It turns out that the range of $\sigma$ has a crucial impact on the bahavior
of $\mathcal{I}_{K}(\sigma,T)$. We separate the range of $\sigma$ into the
following four cases. (Note that the average gap between zeta zeros near
height $T$ is of order $1/\log T$.)

  * •

The macroscopic range: $\sigma_{0}\leq\sigma\leq 1$ for some constant
$\sigma_{0}>1/2$.

  * •

The mesoscopic range: $\sigma=\dfrac{1}{2}+\dfrac{a}{\log T}$, where $a=o(\log
T)$ and $a\to\infty$ as $T\to\infty$.

  * •

The microscopic range: $\sigma=\dfrac{1}{2}+\dfrac{a}{\log T}$, where $a$ is
of constant size.

  * •

The nanoscopic range: $\sigma=\dfrac{1}{2}+\dfrac{a}{\log T}$, where $a\to 0$
as $T\to\infty$.

An important and well-known formula of Selberg [Sel46] expresses
$\zeta^{\prime}/\zeta$ in terms of a prime sum plus a typically controllable
error, provided that $\sigma$ is not too close to the critical line. In the
macroscopic and mesoscopic range, it follows from Selberg’s formula that
$\zeta^{\prime}/\zeta$ can be approximated by a short prime sum for most $t$.
If we assume the Riemann Hypothesis (RH) then such approximation holds for all
$t$, and as a result, the $2K$-th moment $\mathcal{I}_{K}$ of
$\zeta^{\prime}/\zeta$ can be approximated by the $2K$-th moment of the prime
sum, which can be computed for positive integers $K$ using the mean value
theorem of Montgomery and Vaughan [MV74]. Moreover, in the mesoscopic range
the distribution of $\zeta^{\prime}/\zeta$ obeys a central limit theorem; see
Lester [Les14] and Guo [Guo96].

It seems that in the macroscopic and mesoscopic range, the behavior of
$\zeta^{\prime}/\zeta$ is not much affected by the vertical distribution of
zeta zeros, and thus it reflects less such information of them. The situation
changes dramatically when we move closer to the critical line, that is, to the
microscopic range. In this range, the behavior of $\zeta^{\prime}/\zeta$ is
largely affected by the distribution of zeta zeros. In fact, Goldston, Gonek,
and Montgomery [GGM01] showed that an asymptotic estimate of $\mathcal{I}_{1}$
in this range is equivalent to Montgomery’s Pair Correlation Conjecture. See
also [Sel43, GM87, Gol88] for connections to primes in short intervals.
Therefore, obtaining asymptotic behavior for moments of $\zeta^{\prime}/\zeta$
in the microscopic range is highly important, but we also expect it to be
extremely difficult. Indeed, it is also known that $\mathcal{I}_{K}$ has close
connections to other important properties of zeta and primes (see Farmer,
Gonek, Lee and Lester [FGLL13] for a connection to higher correlation
functions of zeta zeros, Baluyot [Bal16] for a discussion of $\mathcal{I}_{1}$
under the Alternative Hypothesis, and Ki [Ki08] for an estimate of
$\mathcal{I}_{1}$ under an assumption on zeros of $\zeta^{\prime}(s)$, as well
as [Sou98, Zha01, Ge17a] for implication of Ki’s assumption to small gaps
between zeta zeros), yet obtaining an unconditional asymptotic estimate for
$\mathcal{I}_{K}$ in the microscopic range seems to be far out of reach. In
fact, even assuming knowledge on correlation functions of zeta zeros, it is
still not clear how to obtain such asymptotic estimates for a general integer
$K$; see [FGLL13] for some relevant discussion.

The extent to which the behavior of $\mathcal{I}_{K}$ reflects the
distribution of typical sizes of zero gaps fades down when we move even closer
to the critical line, namely, to the nanoscopic range. The first result
illustrating such type of phenomena is a result of Goldston et al [GGM01] who
showed that an asymptotic for $\mathcal{I}_{1}$ is equivalent to the Essential
Simplicity hypothesis, which roughly speaking asserts that there are not too
many small gaps between zeta zeros. Therefore, the second moment of
$\zeta^{\prime}/\zeta$ is only capable of recognizing certain information of
very small zero gaps, rather than their typical sizes. This result was
generalized by the author [Ge23] to all even integer moments, in which we
established asymptotic estimates for $\mathcal{I}_{K}$ for all integers $K$
conditional on RH and a so-called $2K$-tuple Essential Simplicity hypothesis.
This result also provides a conditional proof of a conjecture of Bailey,
Bettin, Blower, Conrey, Prokhorov, Rubinstein and Snaith [BBB+19]. Thus, in
the nanoscopic range, on one hand the quantities $\mathcal{I}_{K}$ capture
only partial information on vertical gaps between zeros, but on the other hand
the asymptotics for $\mathcal{I}_{K}$ become available under reasonable
assumptions.

The above mentioned conjecture of Bailey et al was motivated by their
investigation of analogous problems in the setting of random unitary matrices
(also called the Circular Unitary Ensemble, or CUE for short). It is widely
believed that the characteristic polynomials $P$ in the unitary ensemble
$U(N)$ model the Riemann zeta-function regarding zero statistics, value
distribution, and more (for example, see [Mon73] and [KS00]). In particular,
it is expected that the logarithmic derivative $P^{\prime}/P$ of the
characteristic polynomials models $\zeta^{\prime}/\zeta$ and is therefore also
important and interesting. What is good about CUE is that, unlike zeta, the
distribution of zeros of $P$ is known, and thus, it is often hopeful to obtain
unconditional results whose analogues are formidable in the zeta case.
Corresponding to the moments of $\zeta^{\prime}/\zeta$, let us define for
$a\in\mathbb{R}$

$\displaystyle\mathcal{J}_{K}=\mathcal{J}_{K}(a,N)=\int_{U(N)}\left|\frac{P^{\prime}}{P}\Big{(}1-\frac{a}{N}\Big{)}\right|^{2K},$

where the integral is taken with respect to the Haar measure on $U(N)$. Note
that in the unitary ensemble the distribution of eigenvalues is invariant
under rotation, and therefore, the integral remains unchanged if we introduce
an extra averaging over the circle centered at the origin with radius $1-a/N$,
which would look more directly like an analogue of $\mathcal{I}_{K}$ in the
zeta case. The four ranges in this context are as follows.

  * •

The macroscopic range: $0<\frac{a}{N}<1$ is of constant size.

  * •

The mesoscopic range: $a=o(N)$ and $a\to\infty$ as $N\to\infty$.

  * •

The microscopic range: $a$ is of constant size.

  * •

The nanoscopic range: $a\to 0$ as $T\to\infty$.

An important set of closed form formulas, known as the ratios theorems (see
[CS08, CFZ08]), is useful for evaluating integrals like $\mathcal{J}_{K}$.
Indeed, in the macroscopic and mesoscopic ranges the leading term of
$\mathcal{J}_{K}$ for positive integers $K$ may be obtained by a direct
application of the ratios formula. Moreover, like zeta, in the mesoscopic
range we have a central limit theorem for $P^{\prime}/P$ as proved in [Ge24].
The microsopic range is again the most difficult to deal with: while we have
the powerful ratios theorems, deriving asymptotic estimates in the microscopic
range for $\mathcal{J}_{K}$ from the ratios theorems is not easy. In fact, the
situation becomes significantly more complicated as $K$ grows, and it is still
an unsolved problem how to obtain asymptotics for $\mathcal{J}_{K}$ for a
general $K\in\mathbb{N}$. See [BBB+19, FGLL13] for some relevant discussion.
Now let us move closer to the unit circle and consider the nanoscopic range.
In this range, although we may lose finer information of typical zero gaps (as
suggested by the zeta case), it becomes possible to obtain unconditional
asymptotics for $\mathcal{J}_{K}$. Indeed, in [BBB+19] Bailey et al proved
that if $K$ is a positive integer and $a>0$ with $a\to 0$ as $N\to\infty$,
then

$\displaystyle\mathcal{J}_{K}\sim\dfrac{N^{2K}}{(2a)^{2K-1}}\cdot\binom{2K-2}{K-1}$
(1)

as $N\to\infty$. Based on this, they also conjectured a similar estimate for
$\mathcal{I}_{K}$ in the zeta case, and this is the aforementioned conjecture
of theirs which was studied in [Ge23].

The conjectural connection between zeta and the unitary ensemble was developed
further by Katz and Sarnak [KS99] into a more general philosophy that families
of $L$-functions can be modelled by random matrices of certain symmetry types.
This brings considerable interest to the study of orthogonal and symplectic
ensembles as well. In [AS20] Alvarez and Snaith proved analogous results of
(1) for orthogonal and symplectic random matrices. More precisely, for even
orthogonal ensemble $SO(2N)$ their result is

$\displaystyle\int_{SO(2N)}\left(\frac{P^{\prime}}{P}\Big{(}1-\frac{a}{N}\Big{)}\right)^{K}\sim(-1)^{K}\dfrac{2N^{K}}{a^{2K-1}}\cdot\dfrac{(2K-3)!!}{(K-1)!}$
(2)

for integers $K\geq 2$, while for odd orthogonal ensemble $SO(2N+1)$ they
showed that

$\displaystyle\int_{SO(2N+1)}\left(\frac{P^{\prime}}{P}\Big{(}1-\frac{a}{N}\Big{)}\right)^{K}=(-1)^{K}\left[\left(\frac{N}{a}\right)^{K}-\frac{N^{K}}{a^{K-1}}K\right]+O\left(\frac{N^{K-1}}{a^{K-1}}+\frac{N^{K}}{a^{K-2}}\right)\quad$
(3)

for integers $K\geq 1$. For symplectic ensemble $USp(2N)$ their result is

$\displaystyle\int_{USp(2N)}\left(\frac{P^{\prime}}{P}\Big{(}1-\frac{a}{N}\Big{)}\right)^{K}\sim(-1)^{K}\frac{2}{3}\dfrac{N^{K}}{a^{K-3}}\cdot\dfrac{(2K-5)!!}{(K-1)!}$
(4)

for integers $K\geq 4$. We remark that in their work Alvarez and Snaith also
obtained asymptotic results for $K=1$ in $SO(2N)$ and for $K=1,2,3$ in
$USp(2N)$, but these results have different shapes from the above.

It is desirable to extend the above asymptotics to real moments (see the
introduction of [ABS23] for some discussion on this matter). The method used
in proving (1), (2), (3) and (4) in [BBB+19, AS20] makes use of ratios
theorems to transform the integral, and therefore is capable of treating
integer moments but does not generalize to real moments directly. Very
recently, Alvarez, Bousseyroux and Snaith [ABS23] extended the corresponding
result for the odd orthogonal ensemble to real moments. Precisely, they proved
that (3) is true for all real $K>0$. The reason that they can treat real
moments for the odd orthogonal ensemble is that in $SO(2N+1)$ all matrices
have a fixed eigenvalue at $1$. It turns out that this fixed eigenvalue at $1$
gives the main contribution in moments, and they are able to control the
smaller order terms by making use of the integer moments result (3). Note that
in other ensembles we do not have any fixed eigenvalues, and as a result, the
method in [ABS23] does not work directly for unitary, even orthogonal, or
symplectic ensembles.

The purpose of this paper is to prove real moments asymptotics for unitary,
even orthogonal, and symplectic ensembles, as follows.

######  Theorem 1.1.

Let $K>1$ be a real number, and $a>0$ with $a\to 0$ as $N\to\infty$. Then

$\displaystyle\int_{U(N)}\left|\frac{P^{\prime}}{P}\Big{(}1-\frac{a}{N}\Big{)}\right|^{K}$
$\displaystyle\sim\dfrac{N}{2\pi}\cdot\int_{-\infty}^{\infty}\left(\dfrac{1}{\left(\frac{a}{N}\right)^{2}+\theta^{2}}\right)^{\frac{K}{2}}d\theta$
$\displaystyle=\dfrac{N}{2\pi}\cdot\left(\dfrac{N}{a}\right)^{K-1}\cdot\sqrt{\pi}\cdot\frac{\Gamma\left(\frac{K-1}{2}\right)}{\Gamma\left(\frac{K}{2}\right)}\
.$ (5)

######  Theorem 1.2.

Let $K>1$ be a real number, and $a>0$ with $a\to 0$ as $N\to\infty$. Then

$\displaystyle\int_{SO(2N)}\left|\frac{P^{\prime}}{P}\Big{(}1-\frac{a}{N}\Big{)}\right|^{K}$
$\displaystyle\sim\dfrac{2N}{\pi}\cdot\int_{0}^{\infty}\left(\dfrac{2\cdot{\left(\frac{a}{N}\right)}}{\left(\frac{a}{N}\right)^{2}+\theta^{2}}\right)^{K}d\theta$
$\displaystyle=\dfrac{2N}{\pi}\cdot
2^{K}\cdot\left(\dfrac{N}{a}\right)^{K-1}\cdot\frac{\sqrt{\pi}}{2}\cdot\frac{\Gamma\left(K-\frac{1}{2}\right)}{\Gamma\left(K\right)}\
.$ (6)

######  Theorem 1.3.

Let $K>3$ be a real number, and $a>0$ with $a\to 0$ as $N\to\infty$. Then

$\displaystyle\int_{USp(2N)}\left|\frac{P^{\prime}}{P}\Big{(}1-\frac{a}{N}\Big{)}\right|^{K}$
$\displaystyle\sim\dfrac{2N^{3}}{3\pi}\cdot\int_{0}^{\infty}\left(\dfrac{2\cdot{\left(\frac{a}{N}\right)}}{\left(\frac{a}{N}\right)^{2}+\theta^{2}}\right)^{K}\cdot\theta^{2}d\theta$
$\displaystyle=\dfrac{2N^{3}}{3\pi}\cdot
2^{K}\cdot\left(\dfrac{N}{a}\right)^{K-3}\cdot\frac{\sqrt{\pi}}{4}\cdot\frac{\Gamma\left(K-\frac{3}{2}\right)}{\Gamma\left(K\right)}\
.$ (7)

######  Remark 1.

In Theorems 1.2 and 1.3 we consider $|P^{\prime}/P|^{K}$ rather than
$(P^{\prime}/P)^{K}$ as in (2) and (4). For $K>1$ in the case of $SO(2N)$ and
$K>3$ in the case of $USp(2N)$, the difference between $|P^{\prime}/P|^{K}$
and $(P^{\prime}/P)^{K}$ is minor. This is because from the proof of the
theorems we can see that, for such $K$, the main contributing quantities in
the integrals have the same sign. We choose to work with $|P^{\prime}/P|^{K}$
because it avoids unnecessary complication caused by complex logarithm.

######  Remark 2.

Our proofs are new and do not make use of integer moments results (1), (2) or
(4). By evaluating the gamma functions at half integers explicitly, one easily
checks that our theorems agree with the known integer moments.

######  Remark 3.

Our method is robust for obtaining asymptotics for real moments when the
parameter $K$ is not too small. The reason we require $K$ to be not too small
is that, roughly speaking, the smaller $K$ is, the less influential large
values of $P^{\prime}/P$ are on the $K$-th moment. As our method eventually
extracts the large values and uses them to compute moments, there is a
threshold for $K$ above which we are able to do so. While this approach does
not seem to work for $|P^{\prime}/P|^{K}$ when $K$ is small, the results for
$(P^{\prime}/P)^{K}$ for small integers $K$ are known (see the remark
following equation (4)).

######  Remark 4.

Some ideas in our proofs could be adapted to treat nanoscopic real moments in
the zeta case, extending the integer moments results in [Ge23]. The $2K$-tuple
essential simplicity hypothesis may be replaced by a similar hypothesis for
$2\lceil K\rceil$-tuples. Such hypotheses are used to control error terms in
the context of zeta. We also note that the method in [Ge23] for zeta may be
adapted to treat integer moments in the case of the unitary ensemble, by
introducing an additional average over the unit circle (and this is because
the unitary ensemble is rotation-invariant). However, this approach does not
work for orthogonal or symplectic ensembles.

We shall give a complete proof for the unitary case, and indicate necessary
changes in other ensembles.

##  2\. Unitary ensemble

Recall that $a>0$ and $a=o(1)$ as $N\to\infty$. Let

$\displaystyle z_{0}=1-\frac{1}{N}\quad\text{ and }\quad z=1-\frac{a}{N}.$

We will need to choose a suitable parameter $c$ such that

$\displaystyle c$ $\displaystyle=o(1),$ (8) $\displaystyle a$
$\displaystyle=o(c),$ (9) $\displaystyle\text{and }\quad c^{-K}$
$\displaystyle=o(a^{1-K}).$ (10)

For example, we may take

$\displaystyle c=a^{\frac{K-1}{2K}}.$

Note that these restrictions of $c$ require $K>1$.

Let $z_{j}=e^{i\theta_{j}}$ be the eigenvalues and write

$\displaystyle\frac{P^{\prime}}{P}(z)=\sum_{|\theta_{j}|<\frac{c}{N}}\frac{1}{z-z_{j}}+X_{1}+X_{2}-X_{3},$

where

$\displaystyle X_{1}$
$\displaystyle=\frac{P^{\prime}}{P}(z_{0})=\sum_{j=1}^{N}\frac{1}{z_{0}-z_{j}},$
$\displaystyle X_{2}$
$\displaystyle=\sum_{|\theta_{j}|\geq\frac{c}{N}}\left(\frac{1}{z-z_{j}}-\frac{1}{z_{0}-z_{j}}\right),$
$\displaystyle X_{3}$
$\displaystyle=\sum_{|\theta_{j}|<\frac{c}{N}}\frac{1}{z_{0}-z_{j}}.$

We require the following three lemmas.

######  Lemma 2.1.

Let $\ell\in\mathbb{Z}^{+}$. Then

$\displaystyle\mathbb{E}|X_{1}|^{2\ell}\ll_{\ell}N^{2\ell}.$ (11)

This is Propostion 2.1 in [Ge24].

######  Lemma 2.2.

We have

$\displaystyle X_{2}\ll\frac{1}{c}\cdot(N+|X_{1}|)$

Proof. This is obtained implicitly in the proof of Proposition 2.2 in [Ge24].
We sketch the argument here. First we write

$\displaystyle X_{2}$
$\displaystyle=\sum_{|\theta_{j}|\geq\frac{c}{N}}\left(\frac{1}{z-z_{j}}-\frac{1}{z_{0}-z_{j}}\right)$
$\displaystyle=\sum_{|\theta_{j}|\geq\frac{c}{N}}\frac{z_{0}-z}{(z-z_{j})(z_{0}-z_{j})}$
$\displaystyle\ll\frac{1}{N}\sum_{|\theta_{j}|\geq\frac{c}{N}}\frac{1}{|z-z_{j}|\cdot|z_{0}-z_{j}|}.$

Now by definitions of $z$ and $z_{0}$ and the assumption $|\theta_{j}|\geq
c/N$ we have

$\displaystyle c|z_{0}-z_{j}|\ll|z-z_{j}|.$

It follows that

$\displaystyle X_{2}$
$\displaystyle\ll\frac{1}{cN}\sum_{|\theta_{j}|\geq\frac{c}{N}}\frac{1}{|z_{0}-z_{j}|^{2}}$
$\displaystyle\ll\frac{1}{cN}\sum_{j=1}^{N}\frac{1}{|z_{0}-z_{j}|^{2}}.$

By a change of variable, transforming the unit circle into the imaginary axis,
one can show that (see equation (8) in [Ge24])

$\displaystyle\sum_{j=1}^{N}\frac{1}{|z_{0}-z_{j}|^{2}}\ll N(N+|X_{1}|).$ (12)

This gives

$\displaystyle X_{2}\ll\frac{1}{c}\cdot(N+|X_{1}|).$

∎

######  Lemma 2.3.

We have

$\displaystyle X_{3}\ll N+|X_{1}|.$

This is obtained in the proof of Proposition 2.3 in [Ge24].

From the above three lemmas we can write

$\displaystyle\frac{P^{\prime}}{P}(z)=M+E,$ (13)

where the main term

$\displaystyle M=\sum_{|\theta_{j}|<\frac{c}{N}}\frac{1}{z-z_{j}}$ (14)

and the error term

$\displaystyle E=X_{1}+X_{2}-X_{3}\ll\frac{1}{c}\cdot(N+|X_{1}|).$ (15)

From (15) and (11) it is clear that for every $\ell\in\mathbb{Z}^{+}$ we have

$\displaystyle\mathbb{E}|E|^{2\ell}\ll_{\ell}\left(\frac{N}{c}\right)^{2\ell}.$
(16)

A simple application of Jensen’s (or Hölder’s) inequality extends this to all
real moments, as follows.

######  Proposition 2.4.

For all $K>0$ we have

$\displaystyle\mathbb{E}|E|^{K}\ll_{K}\left(\frac{N}{c}\right)^{K}.$

We will also prove a moment estimate for the main term $M$, as follows.

######  Proposition 2.5.

$\displaystyle\mathbb{E}|M|^{K}$
$\displaystyle\sim\dfrac{N}{2\pi}\cdot\int_{-\infty}^{\infty}\left(\dfrac{1}{\left(\frac{a}{N}\right)^{2}+\theta^{2}}\right)^{\frac{K}{2}}d\theta$
$\displaystyle=\dfrac{N}{2\pi}\cdot\left(\dfrac{N}{a}\right)^{K-1}\cdot\sqrt{\pi}\cdot\frac{\Gamma\left(\frac{K-1}{2}\right)}{\Gamma\left(\frac{K}{2}\right)}.$

Proof of Proposition 2.5. We let

$\displaystyle\mathcal{T}_{0}=\\{U\in U(N):U\text{ has no eigenvalues in
}|\theta|<c/N\\},$ $\displaystyle\mathcal{T}_{1}=\\{U\in U(N):U\text{ has
exactly $1$ eigenvalue in }|\theta|<c/N\\},$
$\displaystyle\mathcal{T}_{2}=\\{U\in U(N):U\text{ has at least $2$
eigenvalues in }|\theta|<c/N\\},$

and so,

$\displaystyle\mathbb{E}|M|^{=}\left(\int_{\mathcal{T}_{0}}+\int_{\mathcal{T}_{1}}+\int_{\mathcal{T}_{2}}\right)\left|\sum_{|\theta_{j}|<\frac{c}{N}}\frac{1}{z-z_{j}}\right|^{K}dU.$

The $\mathcal{T}_{0}$ integral is trivial $0$ since the integrand is $0$.

Consider the $\mathcal{T}_{1}$ integral. By definition

$\displaystyle\int_{\mathcal{T}_{1}}=\int_{\mathcal{T}_{1}}\left|\frac{1}{z-e^{i\theta}}\right|^{K}dU,$

where $e^{i\theta}$ is the unique eigenvalue of $U$ in the region
$|\theta|<c/N$. We may rewrite this as

$\displaystyle\int_{\mathcal{T}_{1}}=\int_{-c/N}^{c/N}\frac{1}{|z-e^{i\theta}|^{K}}\cdot
f(\theta)\cdot P(\theta)d\theta,$

where

$\displaystyle f(\theta)=f_{U(N)}(\theta):=\text{ the likelihood that
}\theta\text{ is an eigenangle of some }U\in U(N)$

and

$\displaystyle P(\theta):=\Pr\left(\text{conditional on }\theta\text{ is an
eigenangle, there is exactly one eigenangle in
}\left[\dfrac{-c}{N},\dfrac{c}{N}\right]\right).$

Here the notation $\Pr(\cdot)$ means probability. From the standard $1$-level
density estimates we know that

$\displaystyle f(\theta)=\frac{N}{2\pi}.$ (17)

Next, we show

$\displaystyle P(\theta)=1+o(1)$ (18)

for $\theta\in[-c/N,c/N]$. This is intuitively true since the condition (8)
$c=o(1)$ implies that it is not likely to see two or more eigenvalues in
$[-c/N,c/N]$. To argue rigorously, we shall show that for $|\theta|<c/N$ and
for every positive integer $m$,

$\displaystyle P_{m}(\theta)$ $\displaystyle:=\Pr(\text{exactly }m\text{
eigenangles in }[-c/N,c/N],\text{ and one in }[\theta,\theta+d\theta])$
$\displaystyle\leq\frac{N}{2\pi}d\theta\cdot
m\left(\frac{c}{\pi}\right)^{m-1}.$ (19)

To prove this, recall the $m$-level density formula for $U(N)$ is (see
[Con05])

$\displaystyle\int_{U(N)}\sum_{\begin{subarray}{c}J\subset\\{1,...,N\\}\\\
J=\\{j_{1},...,j_{n}\\}\end{subarray}}F(\theta_{j_{1}},...,\theta_{j_{n}})dU$
$\displaystyle\quad=\frac{1}{(2\pi)^{n}n!}\int_{[0,2\pi]^{n}}F(\theta_{1},...,\theta_{n})\det_{n\times
n}S_{N}(\theta_{k}-\theta_{j})d\theta_{1}\cdots d\theta_{n},$ (20)

where

$\displaystyle S_{N}(x)=\dfrac{\sin\frac{Nx}{2}}{\sin\frac{x}{2}}.$

Apply this with

$F=\mathbbm{1}_{[\theta,\theta+d\theta]}\times\mathbbm{1}_{[-c/N,c/N]}^{m-1}+\mathbbm{1}_{[-c/N,c/N]}\times\mathbbm{1}_{[\theta,\theta+d\theta]}\times\mathbbm{1}_{[-c/N,c/N]}^{m-2}+\cdots+\mathbbm{1}_{[-c/N,c/N]}^{m-1}\times\mathbbm{1}_{[\theta,\theta+d\theta]}$

for $|\theta|<c/N$, and (2) follows since the left-hand side of (2) is at
least $P_{m}(\theta)$ while the right-hand side of (2) is

$\displaystyle\leq\frac{N}{2\pi}d\theta\cdot
m\left(\frac{c}{\pi}\right)^{m-1},$

where the last assertion can be seen by expanding the determinant as a
permutation sum. Therefore, we have

$\displaystyle P_{2}(\theta)+P_{3}(\theta)+\cdots+P_{N}(\theta)$
$\displaystyle\ll Nd\theta\cdot\sum_{m\geq 2}mc^{m-1}$
$\displaystyle\ll(Nd\theta)c,$

while

$\displaystyle
P_{1}(\theta)+P_{2}(\theta)+P_{3}(\theta)+\cdots+P_{N}(\theta)=f(\theta)d\theta=\frac{N}{2\pi}d\theta.$

Since $c=o(1)$, it follows that

$\displaystyle P_{1}(\theta)\sim f(\theta)d\theta,$

and thus, the conditional probability

$\displaystyle P(\theta)=\frac{P_{1}(\theta)}{f(\theta)d\theta}\sim 1.$

Therefore, with $f(\theta)$ and $P(\theta)$ estimated, we arrive at

$\displaystyle\int_{\mathcal{T}_{1}}\sim\int_{-c/N}^{c/N}\frac{1}{|z-e^{i\theta}|^{K}}\cdot\frac{N}{2\pi}d\theta.$

Recall that $z=1-\frac{a}{N}$. Thus,

$\displaystyle|z-e^{i\theta}|^{2}$
$\displaystyle=\left(1-\frac{a}{N}-\cos\theta\right)^{2}+\sin^{2}\theta$
$\displaystyle\sim\left(-\frac{a}{N}+\frac{\theta^{2}}{2}+O(\theta^{4})\right)^{2}+\theta^{2}$
$\displaystyle\sim\left(\frac{a}{N}\right)^{2}+\theta^{2}.$

It follows that

$\displaystyle\int_{\mathcal{T}_{1}}\sim\int_{-c/N}^{c/N}\left(\frac{1}{(a/N)^{2}+\theta^{2}}\right)^{K/2}\cdot\frac{N}{2\pi}d\theta.$

By a change of variable and the condition (9) that $a=o(c)$, it is easy to see
that

$\displaystyle\int_{\mathcal{T}_{1}}\sim\int_{-\infty}^{\infty}\left(\frac{1}{(a/N)^{2}+\theta^{2}}\right)^{K/2}\cdot\frac{N}{2\pi}d\theta.$

It is now straightforward to compute the above integral and we conclude that

$\displaystyle\int_{\mathcal{T}_{1}}\sim\dfrac{N}{2\pi}\cdot\left(\dfrac{N}{a}\right)^{K-1}\cdot\sqrt{\pi}\cdot\frac{\Gamma\left(\frac{K-1}{2}\right)}{\Gamma\left(\frac{K}{2}\right)}$

which gives the leading term in Proposition 2.5.

It remains to show that

$\int_{\mathcal{T}_{2}}=o\left(\int_{\mathcal{T}_{1}}\right).$

A familiar inequality says that for $m\in\mathbb{Z}^{+}$,
$s\in\mathbb{R}_{\geq 1}$ and $a_{1},...,a_{m}\in\mathbb{R}^{+}$ we have

$\displaystyle
m\cdot(a_{1}+\cdots+a_{m})^{s}\leq(ma_{1})^{s}+\cdots+(ma_{m})^{s}.$

This inequality holds essentially because when $s\geq 1$ the function $x^{s}$
is convex, and one can prove it easily by first proving for $m=2$ and then
applying two-term averaging repeatedly. This inequality implies

$\displaystyle(a_{1}+\cdots+a_{m})^{s}\leq(a_{1}^{s}+\cdots+a_{m}^{s})\cdot
m^{s-1}.$

Now in the $\mathcal{T}_{2}$ integral we first apply the triangle inequality
and then the above inequality to the integrand , and we obtain

$\displaystyle\int_{\mathcal{T}_{2}}=\int_{\mathcal{T}_{2}}\left|\sum_{|\theta_{j}|<\frac{c}{N}}\frac{1}{z-z_{j}}\right|^{K}dU$
$\displaystyle\leq\int_{\mathcal{T}_{2}}\left(\sum_{|\theta_{j}|<\frac{c}{N}}\frac{1}{|z-z_{j}|}\right)^{K}dU$
$\displaystyle\leq\int_{\mathcal{T}_{2}}\left(\sum_{|\theta_{j}|<\frac{c}{N}}\frac{1}{|z-z_{j}|^{K}}\right)m^{K-1}dU,$

where $m=m(U)$ is the number of $\theta$’s in $[-c/N,c/N]$ for $U$. As we did
in the $\mathcal{T}_{1}$ integral, we interchange summation in the last
integral and arrive that

$\displaystyle\int_{\mathcal{T}_{2}}\leq\int_{-c/N}^{c/N}\left(\frac{1}{|z-e^{i\theta}|^{K}}\cdot\int_{\mathcal{T}_{2},\text{
and }\theta\text{ is an eigenangle}}m^{K-1}d_{\theta}U\right)\cdot
f(\theta)d\theta,$ (21)

where $d_{\theta}U$ is the conditioning Haar measure on the subset of $U(N)$
when $\theta$ is an eigenangle. Now recall (2) that for $\theta\in[-c/N,c/N]$,

$\displaystyle\Pr(\text{exactly }m\text{ eigenangles in }[-c/N,c/N],\text{ and
one in }[\theta,\theta+d\theta])\leq\frac{N}{2\pi}d\theta\cdot
m\left(\frac{c}{\pi}\right)^{m-1}.$

Thus, for $\theta\in[-c/N,c/N]$ we have

$\displaystyle\Pr(\text{conditional on an eigenangle at }\theta,\text{ there
are exactly }m\text{ eigenangles in }[-c/N,c/N])$ $\displaystyle\quad\ll
mc^{m-1}.$

It follows that

$\displaystyle\int_{\mathcal{T}_{2},\text{ and }\theta\text{ is an
eigenangle}}m^{K-1}d_{\theta}U$
$\displaystyle\quad=\sum_{m=2}^{N}m^{K-1}\cdot\Pr\big{(}\text{conditional on
an eigenangle at }\theta,$ $\displaystyle\qquad\qquad\qquad\qquad\qquad\text{
there are exactly }m\text{ eigenangles in }[-c/N,c/N]\big{)}$
$\displaystyle\quad\ll\sum_{m=2}^{N}m^{K}\cdot c^{m-1}$
$\displaystyle\quad\ll_{K}c=o_{K}(1),$

where in the last line we recall again that $c=o(1)$ by (8). Plugging this
into (21) we obtain

$\displaystyle\int_{\mathcal{T}_{2}}$
$\displaystyle=o_{K}(1)\cdot\int_{-c/N}^{c/N}\frac{1}{|z-e^{i\theta}|^{K}}f(\theta)d\theta$
$\displaystyle=o_{K}(1)\cdot\int_{\mathcal{T}_{1}}.$

This finishes the proof of Proposition 2.5. ∎

Now recall the condition (10) that $c^{-K}=o(a^{1-K})$. Thus, by Propositions
2.4 and 2.5 we see that

$\displaystyle\mathbb{E}|E|^{K}=o\left(\mathbb{E}|M|^{K}\right).$ (22)

From this we shall deduce the following proposition, which together with
Proposition 2.5 gives Theorem 1.1.

######  Proposition 2.6.

$\displaystyle\mathbb{E}|M+E|^{K}\sim\mathbb{E}|M|^{K}.$

Proof of Proposition 2.6. We write

$\displaystyle\mathbb{E}|M+E|^{K}=\int_{|M|>3|E|}|M+E|^{K}dU+\int_{|M|\leq
3|E|}|M+E|^{K}dU.$

For the second integral we have

$\displaystyle\int_{|M|\leq 3|E|}|M+E|^{K}$ $\displaystyle\leq\int_{|M|\leq
3|E|}(|M|+|E|)^{K}dU$ $\displaystyle\leq\int_{|M|\leq 3|E|}(4|E|)^{K}dU$
$\displaystyle\ll_{K}\int_{U(N)}|E|^{K}dU$ $\displaystyle=\mathbb{E}|E|^{K}$
$\displaystyle=o\left(\mathbb{E}|M|^{K}\right)$

by (22).

For the first integral we write

$\displaystyle A=|M|^{2},\quad B=|E|^{2}+M\overline{E}+\overline{M}E,$

and so

$\displaystyle\int_{|M|>3|E|}|M+E|^{K}dU$
$\displaystyle=\int_{|M|>3|E|}(|M|^{2}+|E|^{2}+M\overline{E}+\overline{M}E)^{K/2}dU$
$\displaystyle=\int_{|M|>3|E|}(A+B)^{K/2}dU$
$\displaystyle=\int_{|M|>3|E|}|M|^{K}\left(1+\frac{B}{A}\right)^{K/2}dU$
$\displaystyle=\int_{|M|>3|E|}|M|^{K}\left(1+\frac{K}{2}\frac{B}{A}+O_{K}\left(\frac{B^{2}}{A^{2}}\right)\right)dU,$

where in the last line we have used the Taylor expansion with remainder that
for real $\alpha$ and $|x|<\frac{7}{9}$, say,

$\displaystyle(1+x)^{\alpha}=1+\alpha x+O_{\alpha}(x^{2}),$

and note that the condition $|M|>3|E|$ guarantees $|B/A|<7/9$. Thus,

$\displaystyle\int_{|M|>3|E|}|M+E|^{K}dU$
$\displaystyle=\int_{|M|>3|E|}|M|^{K}dU+O_{K}\left(\int|M|^{K-1}|E|+|M|^{K-2}|E|^{2}dU\right).$

As before we have

$\displaystyle\int_{|M|\leq 3|E|}|M|^{K}=o\left(\mathbb{E}|M|^{K}\right),$

so that

$\displaystyle\int_{|M|>3|E|}|M|^{K}dU\sim\mathbb{E}|M|^{K}.$

For the $O_{K}$-terms we use Hölder’s inequality together with Propositions
2.4 and 2.5 to obtain again an upper bound $o\left(\mathbb{E}|M|^{K}\right)$.
This finishes the proof of Proposition 2.6. ∎

##  3\. Orthogonal and Symplectic ensembles

In this section we give a proof sketch for Theorems 1.2 and 1.3. We shall only
indicate the differences from the unitary case.

The three lemmas cited from [Ge24] can be used or easily adapted. For example,
Lemma 2.1 (that is, even integer moments bounds for $X_{1}$) is proved by
applying a ratios theorem in [CS08] together with a discretization argument.
The analogous results for orthogonal and symplectic ensembles can be deduced
by applying the ratios theorems in [MS18] for these ensembles together with a
similar discretization argument as in [Ge24].

In $SO(2N)$, eigenvalues appear in conjugate pairs, and we shall only label
eigenvalues in the upper half plane, as $z_{1},...,z_{N}$. This explains why
the integral in (1.2) is from $0$ rather than $-\infty$. Also, in the
corresponding summation (14) for $M$ we group conjugate pairs and so each
summand looks like

$\displaystyle\frac{1}{z-z_{j}}+\frac{1}{z-\overline{z_{j}}}=2\Re\frac{1}{z-z_{j}},$

and if $\theta_{j}<c/N$ the above is

$\displaystyle\sim\dfrac{2\cdot{\left(-\frac{a}{N}+O(\theta^{2})\right)}}{\left(\frac{a}{N}\right)^{2}+\theta_{j}^{2}}.$

The $O$-term makes a small contribution toward the integral, and this explains
the shape of the integrand in the integral in (1.2). The likelihood function
$f_{SO(2N)}$ can be calculated using $1$-level density estimate for $SO(2N)$
and the result is

$\displaystyle f_{SO(2N)}\sim\frac{2N}{\pi},$

and this should replace (17). The $P(\theta)$ and $P_{m}(\theta)$ terms in
(18) and (2) can be estimated in the same way and the results are similar as
the unitary case. The main term in (1.2) is of order $N^{K}a^{1-K}$, and so we
require $K>1$ in order for the parameter $c$ to exist.

In $USp(2N)$, eigenvalues again appear in conjugate pairs, and so the integral
in (1.3) is from $0$. As in $SO(2N)$, summands in the corresponding summation
(14) for $M$ are

$\displaystyle\sim\dfrac{2\cdot{\left(-\frac{a}{N}+O(\theta^{2})\right)}}{\left(\frac{a}{N}\right)^{2}+\theta_{j}^{2}}.$

if $\theta_{j}<c/N$, and again the $O$-term is negligible since its
contribution toward the integral is small. The likelihood function
$f_{USp(2N)}$ can be calculated using $1$-level density estimate for $USp(2N)$
and the result is

$\displaystyle f_{USp(2N)}\sim\frac{2N^{3}\theta^{2}}{3\pi}.$

The $P_{m}(\theta)$ terms can be treated in a similar way (with a bit more
care dealing with Taylor expansions) and the bounds look like

$\displaystyle N^{3}\theta^{2}d\theta\cdot m(c^{3})^{m-1}.$

From this we again have $P(\theta)\sim 1$. This time the main term in (1.3) is
of order $N^{K}a^{3-K}$, so we require $K>3$ to guarantee the existence of
$c$.

##  4\. Acknowledgment

The author would like to thank Nina Snaith for very helpful communication.

## References

  * [ABS23] Emilia Alvarez, Pierre Bousseyroux, and Nina C. Snaith.  Asymptotics of non-integer moments of the logarithmic derivative of characteristic polynomials over $SO(2N+1)$.  arXiv preprint arXiv:2303.07813, 2023.
  * [AS20] E. Alvarez and N. C. Snaith.  Moments of the logarithmic derivative of characteristic polynomials from $SO(N)$ and $USp(2N)$.  J. Math. Phys., 61(10):103506, 32, 2020.
  * [Bal16] Siegfred Alan C. Baluyot.  On the pair correlation conjecture and the alternative hypothesis.  J. Number Theory, 169:183–226, 2016.
  * [BBB+19] E. C. Bailey, S. Bettin, G. Blower, J. B. Conrey, A. Prokhorov, M. O. Rubinstein, and N. C. Snaith.  Mixed moments of characteristic polynomials of random unitary matrices.  J. Math. Phys., 60(8):083509, 26, 2019.
  * [CFZ08] Brian Conrey, David W. Farmer, and Martin R. Zirnbauer.  Autocorrelation of ratios of $L$-functions.  Commun. Number Theory Phys., 2(3):593–636, 2008.
  * [Con05] Brian Conrey.  Notes on eigenvalue distributions for the classical compact groups.  In Recent perspectives in random matrix theory and number theory, volume 322 of London Math. Soc. Lecture Note Ser., pages 111–145. Cambridge Univ. Press, Cambridge, 2005.
  * [CS08] John Brian Conrey and Nina Claire Snaith.  Correlations of eigenvalues and Riemann zeros.  Commun. Number Theory Phys., 2(3):477–536, 2008.
  * [FGLL13] D. W. Farmer, S. M. Gonek, Y. Lee, and S. J. Lester.  Mean values of $\zeta^{\prime}/\zeta(s)$, correlations of zeros and the distribution of almost primes.  Q. J. Math., 64(4):1057–1089, 2013.
  * [FK12] David W. Farmer and Haseo Ki.  Landau-Siegel zeros and zeros of the derivative of the Riemann zeta function.  Adv. Math., 230(4-6):2048–2064, 2012.
  * [Ge17a] Fan Ge.  The distribution of zeros of $\zeta^{\prime}(s)$ and gaps between zeros of $\zeta(s)$.  Adv. Math., 320:574–594, 2017.
  * [Ge17b] Fan Ge.  The number of zeros of $\zeta^{\prime}(s)$.  Int. Math. Res. Not. IMRN, (5):1578–1588, 2017.
  * [Ge23] Fan Ge.  Mean values of the logarithmic derivative of the riemann zeta-function near the critical line.  Mathematika, 69(2):562–572, 2023.
  * [Ge24] Fan Ge.  On the logarithmic derivative of characteristic polynomials for random unitary matrices.  Bull. Lond. Math. Soc., 56(3):1114–1128, 2024.
  * [GGM01] D. A. Goldston, S. M. Gonek, and H. L. Montgomery.  Mean values of the logarithmic derivative of the Riemann zeta-function with applications to primes in short intervals.  J. Reine Angew. Math., 537:105–126, 2001.
  * [GM87] Daniel A. Goldston and Hugh L. Montgomery.  Pair correlation of zeros and primes in short intervals.  In Analytic number theory and Diophantine problems (Stillwater, OK, 1984), volume 70 of Progr. Math., pages 183–203. Birkhäuser Boston, Boston, MA, 1987.
  * [Gol88] D. A. Goldston.  On the pair correlation conjecture for zeros of the Riemann zeta-function.  J. Reine Angew. Math., 385:24–40, 1988.
  * [Guo96] Charng Rang Guo.  On the zeros of the derivative of the Riemann zeta function.  Proc. London Math. Soc. (3), 72(1):28–62, 1996.
  * [GY07] M.Z. Garaev and C.Y. Yıldırım.  On small distances between ordinates of zeros of $\zeta(s)$ and $\zeta^{\prime}(s)$.  Int. Math. Res. Not. IMRN, (21):Art. ID rnm091, 14, 2007.
  * [Ki08] Haseo Ki.  The zeros of the derivative of the Riemann zeta function near the critical line.  Int. Math. Res. Not. IMRN, (16):Art. ID rnn064, 23, 2008.
  * [KS99] Nicholas M. Katz and Peter Sarnak.  Random matrices, Frobenius eigenvalues, and monodromy, volume 45 of American Mathematical Society Colloquium Publications.  American Mathematical Society, Providence, RI, 1999.
  * [KS00] J. P. Keating and N. C. Snaith.  Random matrix theory and $\zeta(1/2+it)$.  Comm. Math. Phys., 214(1):57–89, 2000.
  * [Les14] S. J. Lester.  The distribution of the logarithmic derivative of the Riemann zeta-function.  Q. J. Math., 65(4):1319–1344, 2014.
  * [LM74] Norman Levinson and Hugh L. Montgomery.  Zeros of the derivatives of the Riemann zeta-function.  Acta Math., 133:49–65, 1974.
  * [Mon73] H. L. Montgomery.  The pair correlation of zeros of the zeta function.  In Analytic number theory (Proc. Sympos. Pure Math., Vol. XXIV, St. Louis Univ., St. Louis, Mo., 1972), pages 181–193, 1973.
  * [MS18] A. M. Mason and N. C. Snaith.  Orthogonal and symplectic $n$-level densities.  Mem. Amer. Math. Soc., 251(1194):v+93, 2018.
  * [MV74] H. L. Montgomery and R. C. Vaughan.  Hilbert’s inequality.  J. London Math. Soc. (2), 8:73–82, 1974.
  * [Rad14] Maksym Radziwiłł.  Gaps between zeros of $\zeta(s)$ and the distribution of zeros of $\zeta^{\prime}(s)$.  Adv. Math., 257:6–24, 2014.
  * [Sel43] Atle Selberg.  On the normal density of primes in small intervals, and the difference between consecutive primes.  Arch. Math. Naturvid., 47(6):87–105, 1943.
  * [Sel46] Atle Selberg.  Contributions to the theory of the Riemann zeta-function.  Arch. Math. Naturvid., 48(5):89–155, 1946.
  * [Sou98] K. Soundararajan.  The horizontal distribution of zeros of $\zeta^{\prime}(s)$.  Duke Math. J., 91(1):33–59, 1998.
  * [Zha01] Yitang Zhang.  On the zeros of $\zeta^{\prime}(s)$ near the critical line.  Duke Math. J., 110(3):555–572, 2001.


# Explainable Deep Learning Analysis for Raga Identification in Indian Art
Music

Parampreet Singh, Vipul Arora This paper was produced by the IEEE Publication
Technology Group. They are in Piscataway, NJ.Manuscript received …. .., ….;
revised ….. .., ….

###### Abstract

The task of Raga Identification is a very popular research problem in Music
Information Retrieval. Few studies that have explored this task employed
various approaches, such as signal processing, Machine Learning (ML) methods,
and more recently Deep Learning (DL) based methods. However, a key question
remains unanswered in all of these works: do these ML/DL methods learn and
interpret Ragas in a manner similar to human experts? Besides, a significant
roadblock in this research is the unavailability of ample supply of rich,
labeled datasets, which drives these ML/DL based methods. In this paper, we
introduce ”Prasarbharti Indian Music” version-1 (PIM-v1), a novel dataset
comprising of 191 hours of meticulously labeled Hindustani Classical Music
(HCM) recordings, which is the largest labeled dataset for HCM recordings to
the best of our knowledge. Our approach involves conducting ablation studies
to find the benchmark classification model for Automatic Raga Identification
(ARI) using PIM-v1 dataset. We achieve a chunk-wise f1-score of 0.89 for a
subset of 12 Raga classes. Subsequently, we employ model explainability
techniques to evaluate the classifier’s predictions, aiming to ascertain
whether they align with human understanding of Ragas or are driven by
arbitrary patterns. We validate the correctness of model’s predictions by
comparing the explanations given by two ExAI models with human expert
annotations. Following this, we analyze explanations for individual test
examples to understand the role of regions highlighted by explanations in
correct or incorrect predictions made by the model.

###### Index Terms:

Hindustani Classical Music, Raga Identification, Deep Learning, Explainable
AI, Music Information Retrieval

##  I Introduction

Indian Art music (IAM) is a culturally rich and complex musical tradition that
has evolved over centuries. It is categorized into two main categories:
Hindustani Classical Music (HCM) and Carnatic music (CM). HCM is predominant
in the northern regions of India, while Carnatic music is more prevalent in
the south. Both forms of music have their own distinct nature and way of
singing. Central to IAM is the concept of a Raga, which can be termed as
melodic structures that provide a framework for improvisation and composition,
making it fundamental descriptor for IAM. It represents the mood of any
musical composition. Ragas also form the basis of a lot of indian film or folk
music[1]. Detailed description about IAM Ragas can be found at [2, 3].

To compare with western music, in HCM, a raga is not merely defined by its set
of notes, but also by the way these notes are performed, with emphasis on
smooth melodic glides and the space between notes. This contrasts with
Eurocentric music genres, where a scale is primarily defined by its set of
notes[4]. Also, in western music, there is more emphasis on harmony, whereas
HCM focuses on melody[5]. Raga is also different from a mode in Eurocentric
music as it encompasses a broader range of musical features including scale,
notes’ sequence, hierarchy, relative duration, micro-tonal intonation,
ornamentation, melodic movement, specific phrases, and mood[4]. Due to these
distinctions, the task of Automatic ARI in IAM differs significantly from
genre classification in Western music. In IAM, categories such as Dhrupad,
Khayal, Thumri, Dadra, Tappa, and Bhajan etc can be considered distinct
genres[5], but not Ragas.

IAM’s emphasis on improvisation within the framework of Raga underscores the
intricate nature of this musical tradition, setting it apart from Western
music genres.

For data-driven methods used in computational musicology and music information
retrieval (MIR), digital music artifacts with metadata and expert annotations
are a prerequisite[6]. Today, searching for music based on identifiers like
ragas is not as simple as searching for text on search engines like Google.
Efforts are underway to automate this process, starting with the automatic
identification of ragas. Music enthusiasts and researchers who view music as
more than just entertainment require a music/audio retrieval system based on
melody or raga[7]. ARI has several other applications, including music
recommendation, music teaching, and automatic annotation.

The progress of research in IAM is often hindered by the scarcity of a variety
of open labeled datasets. Introduction of new datasets in IAM can be useful
for making tasks like melody extraction, ARI, tala identification, source
separation, automatic mixing, and performance analysis more robust and
generalisable.

For the task of ARI, various Machine Learning(ML)/deep learning(DL) based
approaches are being used, but we have no way to know if the models are making
predictions as if an IAM expert would do or not. This motivates us to look for
these answers from the field of Explainable AI (ExAI), which has barely been
explored in the field of IAM analysis. ExAI is a popular research area within
ML that aims to provide explanations about complex deep learning models,
making them interpretable. ExAI focuses on explaining the behavior of models
and their predictions, in various known and unknown situations, in a post-hoc
manner, hence increasing transparency and understanding of these models,
ultimately improving their reliability and trustworthiness. In this study, we
take up the task of ARI for IAM and then use ExAI methods to show the
reliability of the classifier.

The Key contributions of this work are as follows:

  * •

We Introduce a novel HCM dataset along with manual annotations, and benchmark
the dataset for the task of Raga classification by undergoing Ablation
Studies. We release this dataset in the form of 30-second audio chunks along
with labels and additional metadata. The sample audios can be accesed at:
https://github.com/ParampreetSingh97/PIM_v1_ExAI.git.

  * •

We make one of the first attempts on using ExAI techniques for raga
identification, and subsequently validating the significance and correctness
of the Raga classifier musically.

  * •

We show that the ARI model in fact does make musically correct predictions,
making it usable for various HCM tasks like music teaching, automatic
labeling, music suggestion and search, and other tasks involving IAM Ragas.

##  II Related Work

###  II-A HCM Datasets

The Indian Art Music Raga Recognition Dataset (RRD)[8], offers 116 hours of
HCM with 300 recordings featuring 30 unique ragas, with 10 recordings per
raga. Then there is Saraga dataset [9], which comprises of 108 recordings of
HCM in 61 unique Ragas and 9 unique talas, totaling 43.6 hours. Both of these
datasets feature live performances by various experts, along with providing
labels like Raga names, manually marked melodic phrases, and features like
tonic and pitch extracted using various algorithms. While Saraga allows
download access with annotations, RRD requires access requests for audio file
viewing. Notably, we observe a significant overlap of audio files between
Saraga and RRD. Some other works [10, 11] utilize self-made HCM datasets for
ARI, but these datasets have not been made publicly available.

Our dataset stands out as the largest labeled HCM dataset to the best of our
knowledge, surpassing both Saraga and RRD, in terms of total duration, number
of recordings, unique Ragas, talas and manually labeled Raga and tonic
information.

###  II-B Raga Identification

For HCM, the paper [10] uses their own generated instrumental dataset of 20
second snippets for 12 Raga classification. They make use of a combination of
features like Pitch, MFCC and other statistics derived from mfcc together for
the classification task. Another paper [4] does 2-Raga classification using a
subset from Saraga Dataset and report that using Swargram features is helpful
for Raga classification, which is nothing but Chromagram features shifted in
pitch. The paper [12] uses pitch class distribution and N-gram approach on the
extracted pitch for raga identification and reports that longer audio clip
length is beneficial for the task. Another work [13] uses a dataset downloaded
from youtube for the task of 6-class classification. In paper [14], KNN and
SVM machine learning algorithms were used for classifying different ragas like
Yaman and Bhairavi, and the KNN was found to be performing slightly better.
The paper [15] proposed a hierarchical deep learning system called PhonoNet,
which used a CNN model in the first stage to predict the raga of a small audio
chunk, and a two-stage recurrent LSTM network in the second stage to look at
various parts of the audio input and prioritize observed patterns.

There are some other works related to Carnatic Music[1, 16, 17, 18, 19, 20,
21, 22], and most of them use basic features like extracted pitch, MFCCs,
Spectral centroid, chromagram, spectrogram, scalogram etc and apply different
ML classifiers like Naive Bayes, SVM, Random forest, GMMs, and some also use
DL models like RNN, LSTM, CNN etc.

We observe that we have more research centered around Carnatic music,
primarily because of higher availability of labeled datasets as compared to
HCM. Also, to the best of our knowledge, none of the works in the literature
have explored ExAI methods for the task of Raga Identification in any form.

###  II-C ExAI methods

Until now, we have a lot of research already available focused on ExAI methods
for DL models used for various tasks involving Images, and some works for
audio and music also. We will now discuss some notable works in both. One very
prominent kind of approach for post-hoc explainability involves the use of
model’s Gradients from various layers of a CNN or ANN model and using them in
a intelligent way to visualize explanations of the classifier in the form of
saliency map, superimposed over the original image itself, highlighting the
most important region for classification by the classifier [23, 24, 25, 26,
27, 28, 29, 30]. We will be using GradCAM++[28], which is a very popular
method for CNN explanations. It weighs each of the pixels in each feature map
of the last CNN layer separately, with weights determined using the gradients
of these layers in a clever way, and then finally stacking all the weighted
feature maps together to get the final attribution map, which highlights
important regions in an image. In the audio domain, we have some works [31,
32] which treat audio feature representations as images and use the ExAI
methods for images to generate explanations.

Other type is of black box models, whose gradients are not accessible and we
have to design ExAI methods by querying the model multiple times. There are
some notable works for black box model explainability, with LIME[33]
generating locally faithful explanations by training an interpretable model on
perturbed instances of the input data sample. Another method SHAP[34], uses
game theory to assign a unique contribution score to each feature in the
input, indicating its impact on the model’s output. SHAP is computationally
intensive, especially for models with a large number of features. Another
approach Anchors[35] express explanations as simple IF-THEN rules, making them
more easily understandable, but it struggles with large and complex datasets.

For audio , we have a variant of LIME in SoundLIME[36], which works on
spectrograms or other audio based features, treats them as image data only and
then replicate the LIME framework.

For our work, we employ a very popular white-box ExAI model in Grad-CAM++[28],
and we use SoundLIME out of the black box models. We employ both white-box and
black-box ExAI models to ensure comprehensive coverage of interpretability
techniques, enabling a thorough comparison and analysis of the performance of
our ARI classifier model.

##  III Dataset

The dataset comprises a total of 501 HCM songs with each song being a
polyphonic audio containing vocals of the main artist, percussion instruments
like tabla, Pakhawaj, along with Manjeera in some audios, along with non-
percussion instruments like Harmonium, Sarangi, Violin, and also tanpura
(drone) which provides the tonic for the whole music performance. These music
recordings are sung in variety of Ragas and Talas with all of the songs being
of varying lengths. The dataset includes 144 unique Ragas performed by 169
vocal artists. The Distribution of Ragas is shown in table: I. Harmonium is
the most common instrument, present in 372 songs, while some performances
feature multiple string instruments. All of the songs include drone (tanpura)
playing in the background of the performance. The dataset includes
performances in 20 different talas accompnying the vocals, with teentaal (121
instances) and ektaal (146 instances) being the most frequently used talas as
shown in fig:1. Each song is available in the form of 30-second chunks,
totaling 23,005 audio files, approximately 191.70 hours of music.

Figure 1: Distribution of number of audio files per tala

###  III-A Musical Styles

The dataset covers a variety of musical styles belonging to HCM, such as
Khayal, Dhrupad, Dadra, Thumari, Tarana, Gat, Bhajan, Tappa, Dhammar, Kajri,
and Jhoola (folk), showcasing a wide range of songs from different regions
throughout India.

###  III-B Recording Source and Quality

All recordings are sourced from the archives of Prasar Bharti (Akashwani,
Delhi), which are not available online elsewhere. They feature performances by
HCM ustaads (experts) and are mainly studio recordings or recordings from live
shows, ensuring high audio quality. Most of the featured vocal, percussion, or
string instrument artists are well-known in the HCM community. All audio files
are mono tracks. Some of the recordings feature speech at the start which
includes introduction of the artists and some details about their performance
and ranges from a few seconds to about a few minutes in some recordings.

###  III-C Metadata

The metadata contains details such as the names of vocal and instrumental
artists, the instruments played, the event where the performance took place,
the name of the song, and the number of 30-second files for each song. We also
add to the metadata the Raga name, the tonic and the start and end times of
music and speech for each of those songs, labeled manually by our annotators.

###  III-D Manual Annotation Process

Two music experts with over 10 years of experience in vocal music teaching and
singing for HCM carried out the task of annotation. They identified the Raga
and tonic of each of the 501 music performances, listed in the metadata. This
annotation task involves labelling the whole audio with a single Raga and
Tonic label, as they do not change throughout a single HCM performance. As the
task does not involve frame-wise labelling or subjective labelling, there is
negligible chance of discrepancies among annotators, so inter-annotator
agreement is not necessary.

TABLE I: Distribution of number of audio files per Raga No. of files |  Ragas
---|---
1 |  Abhogi_kanada, Alhaiya_bilawal, Asavari, Asavari (Komal_rishabh), Basant_bhairav, Bhimpalasi, Bhoop_nat, Dagori, Deepak, Des_and_tilak_kamod_combination, Deshi, Desi_todi, Devgandhar, Devgiri_bilawal, Gaud_sarang, Gunkali, Jhoola, Kafi_malhar, Kans_gandhar, Kaushiki_kanada, Lalit, Madhmad_sarang, Malgunji, Mian_ki_todi, Mishra_gara, Mishra_jhinjhoti, Mishra_jinjhoti, Mishra_kirwani, Nat_bhairav, Nat_bihag, Nat_kedar, Pancham, Prabhat_bhairav, Purvi, Ramkali, Sahna_kanada, Salagwarali, Shahana_kanada, Shiv_kalyan, Shivranjani, Shuddha_bairav, Shuddha_basant, Shuddha_sarang, Sohni (pancham), Suha_malhar, Tilak_shyam, Todi, Todi (ahiri), Vrindavani_sarang
2 |  Anand_bhairav, Bageshri_bahar, Bahar, Bhairav, Bhatiyar, Bihari, Chandrakauns (Bageshri_ang), Chandrakauns (Bhairavi_ang), Charukeshi, Darbari, Deshkaar, Hemant, Hindol, Jait_kalyan, Jayant_malhar, Kirwani, Marwa, Paraj, Paraj_kalingda, Pilu, Puriya_dhanashri, Raysa_kanada, Sampoorna_malkauns, Shahana, Shivmat_bhairav, Swanandi, Tilak_kamod
3 |  Abhogi, Bageshri_kanada, Basant_bahar, Basant_kedar, Bhinna_shadaja, Durga, Gavati, Gujri_todi, Hameer, Hansdhwani, Janasammohini, Jhinjhoti, Kafi_kanada, Kaunsi_kanada, Mishra_bhairavi, Mishra_desh, Mishra_pahadi, Mishra_tilak_kamod, Mishra_tilang, Sawani_bihag, Shyam_kalyan, Tilang
4 |  Aheer_bhairav, Basant, Bihagda, Gaud_malhar, Jaunpuri, Kamod, Kaushik_dhwani, Majh_khamaj, Megh, Nand, Rageshree, Saraswati, Surdasi_malhar
5 |  Adana, Gorakh_kalyan, Madhukauns, Mishra_pilu, Puriya
6 |  Chayanat, Darbari_kanada, Jaijaiwanti, Kalavati, Mian_malhar, Shankara
8 |  Chandrakauns, Des, Kafi, Pahadi
9 |  Kedar, Mishra_khamaj, Puriya_kalyan
10 |  Jog_kauns, Yaman
11 |  Malkauns, Maru_bihaag
12 |  Bageshri, Bhopali, Bihag, Nayaki_kanada, Shuddha_kalyan
13 |  Khamaj
14 |  Jog
15 |  Bhairavi, Sohni

##  IV Methodology

In this study, we use our PIM dataset to extract chromagram features from the
audios and train a CNN-LSTM model to perform multi-class classification for 12
Raga classes. Next, we employ Grad-CAM++ (GC) and SoundLIME (SL), two popular
ExAI models, for saliency visualisation and comparing them with annotations
from music experts. The sub-sections below explain the entire procedure.

###  IV-A Feature Extraction

We use chromagram features, extracted from the audios using librosa library on
python for our task. In our dataset, we have performances by various artists,
each singing at different tonics. Tonic is just the reference note for
construction of any Raga. Music performances in the same Raga can exhibit
structural similarities when they use the same tonic. We performed tonic
normalization on the extracted chromagram features, which aligns all songs to
a common reference note, facilitating the classification model’s ability to
discern Raga patterns irrespective of the original tonic used by the artists.

###  IV-B Tonic Normalisation

For a given audio signal x, let the short time fourier transform is given by
$X(n,k)$. The center frequency of a pitch in an equal-tempered midi scale is
given by:

$F_{pitch}(p)=440\times 2^{(p-69)/12}$ (1)

From these, we get a pitch-based log-frequency spectrogram, which converts the
whole spectrogram having frequency axis equal to NFFT to have 128 values
corresponding to the 128 midi frequencies. We basically assign each of the
$X(n,k)$ to the closest $f_{coef(k)}$ frequency. So, for each of the 128 pitch
values $p\in[0:127]$, we have:

$P(p)={k:F_{pitch}(p-0.5)<=F_{coef}(k)<F_{pitch}(p+0.5)}$ (2)

So, we have the log frequency spectrogram $Y_{LF}$ given by:

$Y_{LF}(n,p)=\sum_{k\in P(p)}|X(n,k)|^{2}$ (3)

To obtain chromagram features, we simply sum up the pitch values which differ
by some integer multiple of an octave as they are perceived to be the same
note musically:

$C(n,c)=\sum_{p\in[0:127]:pmod12=c}Y_{LF}(n,p)$ (4)

Now, to shift the chromagram representation of any song by a factor of $"a"$
semitones, we are essentially looking to get a new $X^{\prime}(n,k)$ such
that:

$X^{\prime}(n,k)=X(n,k\times 2^{a/12})$ (5)

Shifting by $"a"$ semitones essentially means multiplying the frequency values
in a spectrogram by $2^{a/12}$, where $"a"$ is a discrete value such that
$a\in[1:11]$. This range for $"a"$ is chosen because a value greater than 11
would result in an octave shift, which is not a consideration for us since it
would return to the same musical note. Now, we will get log-frequency
spectrogram as:

$Y_{LF}^{\prime}(n,p)=\sum_{k\in P(p)}|X(n,k^{\prime})|^{2}$ (6)

where, $k^{\prime}=k\times 2^{a/12}$
Now, using Equation: 1,

$F_{pitch}(p)=k^{\prime}=440\times 2^{(p-69)/12}$ (7)

Substituting the value for $k^{\prime}$, and re-arranging terms, we get:

$\displaystyle k$ $\displaystyle=440\times 2^{((p-a)-69)/12}$ (8)

Comparing Equation: 8 with 7, we can see that for a given $k^{\prime}$, which
gets shifted ${}^{\prime}a^{\prime}$ semitones, the set of frequency values
given by $P(p^{\prime})$ is just all the frequency values present
${}^{\prime}a^{\prime}$ chromagram bins higher. i.e., the pitch values,
represented by $F_{\text{pitch}}(p)$, undergo a shift when transitioning from
$k^{\prime}$ to $k$ or vice versa, where the difference between the two
pitches is given by the factor ${}^{\prime}a^{\prime}$. This shift signifies a
displacement of the frequency values associated with the musical notes,
particularly within the context of a chromagram bin. It implies that the set
of pitch values $P(p)$, forming a bin in the chromagram, undergoes a uniform
displacement by a factor of ${}^{\prime}a^{\prime}$. Consequently, while the
bins themselves remain unchanged, the pitch values contained within each bin
experience a consistent shift, maintaining the relative positioning of musical
notes while adjusting their absolute frequencies. So, in practice, for
applying tonic normalisation to our audio data, we just shift the extracted
chromagram bins through the shorter dimension in a cyclic manner.

Figure 2: Model Architecture for CN2+LSTM+T

###  IV-C Classification Model Description

To determine the best performing model, we conduct an ablation study,
experimenting with various CNN and LSTM-based architectures. Our first model,
CN1+T, utilizes a simple CNN architecture with convolutional, max-pooling, and
batch normalization layers, followed by a flatten layer, in a very similar way
to a common Image classification model. Then the CN1+LSTM+T model extends this
by replacing the flatten layer with an LSTM layer, ideal for processing time-
series data like audio signals. Also, we provide tonic normalization as
discussed in Section:IV-B and test the model performance. The CN2+LSTM model
features a more complex CNN-LSTM architecture with 1D max-pooling to preserve
temporal information, as shown in Fig: 2. Then the CN2+LSTM+T model just
includes tonic normalization in the input features, following the exact
configuration of CN2+LSTM model. The selection of layer configurations and
activation functions is carefully optimized for classification performance.
These models will be further detailed in sectionV, where we evaluate their
effectiveness in ARI.

###  IV-D Classification Model Training

We train our DL models for the 12-class classification task, while using
softmax activation in the last layer. Let $z_{i}$ represents the output logit
for the $i^{th}$ neuron in the last dense layer having 12 neurons. Softmax
function applied to these logits converts them into probability values as:

$p_{i}=softmax(z_{i})=\frac{e^{z_{i}}}{\sum_{j=1}^{N}e^{z_{j}}}$ (9)

Here, $p_{i}$ is the probability score for $i^{th}$ neuron in the layer,
$N=12$ as we have a 12-class classification problem. Also, $i\in[1:N]$, so we
get 12 probability values, such that $\sum_{i}{p_{i}}=1$. Then for training,
we use categorical cross-entropy loss function given by:

$L_{CCE}=\sum_{i=1}^{N}t_{i}log(p_{i})$ (10)

Here, $t_{i}$ is the $i^{th}$ true label. In this case, $t_{i}$ will be 1 for
only 1 neuron and 0 for all other neurons corresponding to a particular input.
The training is carried out for multiple epochs until the model performances
converges measured by precision, recall and f-1 score. In our experiments, we
use cross-validation to assess the robustness and generalization ability of
our models for the classification task.

###  IV-E Grad-CAM++

GradCAM++ builds upon GradCAM (Gradient-weighted Class Activation Maps), with
a key advancement to it. Initially, we reload the CN2+LSTM+T model trained in
above steps where the last CNN layer comprises 256 learned feature maps in our
case as visible in the Fig: 2. To create a class-specific GradCAM++ activation
map, each of these feature maps is weighted and then a weighted sum is taken
to generate the final activation map or saliency map for the target class. We
compute the weights of feature maps using the weights of those last CNN
blocks, given by the equation:

$\alpha_{c}^{k}=\frac{1}{Z}\sum_{i}\sum_{j}\frac{\partial y^{c}}{\partial
A^{k}_{ij}}$ (11)

Here, $A^{k}_{ij}$ represents each feature map (so we have $k$ feature maps
and $k\in[1:256]$). Also, we have $c$ classes with $c\in[1:12]$.
$\alpha_{c}^{k}$ represents the weights of $k^{th}$feature map for $c^{th}$
class. $Z$ is the dimension of each feature map, and $y^{c}$ represents the
output logits whose gradients we compute using the chain rule. Using these
computed weights as the weights of each of the $k$ feature maps, we get the
final Grad-CAM attribution map, given by:

$L^{c}_{Grad-CAM}=ReLU\Bigl{(}\sum_{k}\alpha_{k}^{c}A^{k}\Bigl{)}$ (12)

$L^{c}_{Grad-CAM}$ represents the final attribution map for GradCAM, which is
obtained by summing all the feature maps up-sampled to the original input
size, given by $A^{k}$, with each $A^{k}$ weighted with a corresponding weight
$\alpha_{k}^{c}$ according to a given output prediction class $c$. After that,
ReLU is applied to keep only the positively impacting salience regions and
remove the regions with negative gradient in the output saliency map or
attribution map. But, to obtain the activation map for GradCAM++, we weight
each pixel separately rather than weighting the whole feature map with a
single weight. The new weights are given by:

$\alpha_{c}^{k}=\sum_{i}\sum_{j}w^{kc}_{ij}ReLU\Bigl{(}\frac{\partial
y^{c}}{\partial A^{k}_{ij}}\Bigl{)}$ (13)

Here, $A^{k}_{ij}$ represents each pixel in the $k^{th}$ feature map. This
$w^{kc}_{ij}$ aims to weigh each pixel separately, rather than the whole
activation map. So, this $w^{kc}_{ij}$ is given by:

$w^{kc}_{ij}=\frac{\frac{\partial y^{c}}{(\partial
A^{k}_{ij})^{2}}}{2\frac{\partial y^{c}}{(\partial
A^{k}_{ij})^{2}}+\sum_{a}\sum_{b}A^{k}_{ab}\Bigl{(}\frac{\partial^{3}Y^{c}}{(\partial
A^{k}_{ij})^{3}}\Bigl{)}}$ (14)

Using these, we finally obtain the attribution map for Grad-CAM++ given by:

$L^{c}_{Grad-CAM++}=ReLU\Bigl{(}\sum_{k}\alpha_{k}^{c}A^{k}\Bigl{)}$ (15)

This activation map given by $L^{c}_{Grad-CAM++}$ represents the desired
visualisations for the regions of interest in a particular test example for a
given class for the classifier model.

###  IV-F SoundLIME

SoundLIME[36] is specifically designed for audio, analogous to LIME’s[33]
application in images. Here also, we need the pre-trained classifier model,
but we treat it as a black-box model with no access to its gradients. Here, we
temporally divide the chromagram into 30 equal parts or super-pixels, with one
super-pixel representing the chromagram representation for each second of the
audio. Next, we create 150 random perturbations for the given test input by
applying random multi-hot masks of shape 30x1, meaning that each element of
the mask either suppresses or retains each super-pixel. An exponential kernel
is defined on L2 distance function $D(x,z)$ between the original image $x$ and
the perturbed image $z$, denoted by $\pi_{x}(z)$:

$\pi_{x}(z)=exp(-D(x,z)^{2}/\sigma^{2})$ (16)

We train a weighted linear regression model, using the mask as input and
querying the original classifier model for prediction of the masked input
image of the chromagram, using that as the target.The significance of this
approach can be understood from the fact that if most of the salient regions
are already present in the masked input example and only non-salient regions
are masked, the classifier model predicts the correct class. That becomes the
label for our linear regression model that whether the given perturbed example
results in the desired prediction or not. So, our weighted Linear regression
model trained on these 150 perturbed data points learns the coefficients for
each dimension of the mask and ultimately outputs which regions (super-pixels)
of the mask are more salient for the prediction of the current test example.

Let $g\in G$ represents a family of interpretable models, in our case, $g$
represents the linear regression model. $f$ is the classifier model.
$\omega(g)$ represents the complexity of the model, in our case it could be
represented by the number of non-zero weights. The LIME explanation can be
shown mathematically as:

$\zeta(x)=argmin_{g\in G}L(f,g,\pi_{x})+\omega(g)$ (17)

###  IV-G Saliency Evaluation Method

To evaluate the salient regions identified by the two ExAI models, we use the
output of ExAI models, and compare them with the expert-annotated salient
regions. We compute the precision for top 1 to 10 seconds for both the ExAI
models. In our task, the concern lies solely with determining whether the top
predicted regions are correct. Hence, precision is the best metric to employ
in this situation, whereas recall and accuracy are not helpful. We also
explain one correct and one incorrect prediction of the classifier and analyse
their salient region. Also, the evaluation of correctness of ExAI model
predictions is not required in our case, as we are solely focused on the
salient regions’ correctness within the top few seconds in context of HCM,
rather than assessing the ExAI model’s performance.

##  V Experiments and Results

###  V-A Data Pre-processing

To prepare the audio files for analysis, all non-musical speech has been
removed. So, we firstly distribute the whole songs to train, test and
validation sets, making sure that there is at least 1 song of each class in
the test and validation sets. Then we split the songs into 30 second snippets
for all the 3 splits with percentages of approximately 79%, 15%, and 6%, for
training, validation, and test sets respectively as shown in Fig: 3 Notice
that we split the dataset into train,test and validation sets first on file
level, and then extract split them into 30-second clips. We do not directly
create 30-seconds clips and then shuffle them as we have a possibility that we
have snippets of the same song in train, test and validation set, which will
add an additional bias in our dataset and will hinder the generalizability of
our model.

Figure 3: Dataset pre-processing for the network

Also, the last part of each music recording is removed as it is mostly found
to be having only 3 to 10 seconds of music, which may not even contain any
salient region for getting classified to a particular raga and just serves the
purpose of confusing the model. Also, generally in HCM performances, the
endings of clips lasting 20 seconds or more may simply consist of a single
note prolonged for a few seconds, which may not accurately represent the
current Raga class. Thus, removing these endings should not adversely affect
the analysis.

For our work, we extract chromagram features for these audio snippets, making
each input sample of shape (938,12), having 938 time frames having 12
chromagram bins each.

All the songs, marked with their respective tonic by the music experts go
through tonic normalisation as explained in IV-B. We make Note ”A” as the
reference note for all and normalise accordingly. To achieve this, we just
cyclically shift the obtained chromagram features through their shorter
dimension throughout the 938 frames for each 30-sec file to obtain the desired
normalisation.

###  V-B Classification results

We chose audio files belonging to 12 classes having the most number of audio
files out of the dataset and train a 12-class classifier using the models
discussed in IV-C. Classification results for our Ablation studies are shown
in table: II (all values rounded off to 2 decimal places)

TABLE II: Combined Classification Report for Different Models Classification
Report for Different Models
---
Raga | CN1+T | CN1+LSTM+T | CN2+LSTM | CN2+LSTM+T
| Precision | Recall | F1 | Precision | Recall | F1 | Precision | Recall | F1 | Precision | Recall | F1
Bhairavi | 0.89 | 0.53 | 0.67 | 0.83 | 0.74 | 0.78 | 0.0 | 0.0 | 0.0 | 0.93 | 0.91 | 0.92
Bihag | 0.42 | 0.31 | 0.36 | 0.87 | 0.77 | 0.81 | 0.40 | 0.74 | 0.52 | 0.90 | 0.91 | 0.90
Des | 0.09 | 0.08 | 0.08 | 0.63 | 0.74 | 0.68 | 0.06 | 0.08 | 0.07 | 0.78 | 0.66 | 0.70
Jog | 0.85 | 0.95 | 0.90 | 0.97 | 0.86 | 0.91 | 0.88 | 0.88 | 0.88 | 0.91 | 0.89 | 0.90
Kedar | 0.72 | 0.81 | 0.76 | 0.92 | 0.75 | 0.82 | 0.0 | 0.0 | 0.0 | 0.93 | 0.89 | 0.91
Khamaj | 0.24 | 0.38 | 0.29 | 0.70 | 0.85 | 0.77 | 0.0 | 0.0 | 0.0 | 0.55 | 0.68 | 0.59
Malkauns | 0.94 | 0.89 | 0.92 | 0.90 | 0.96 | 0.93 | 0.64 | 0.61 | 0.62 | 0.99 | 0.96 | 0.97
Maru-Bihag | 0.57 | 0.56 | 0.56 | 0.80 | 0.90 | 0.84 | 0.52 | 0.68 | 0.60 | 0.88 | 0.85 | 0.86
Nayaki-Kanada | 0.64 | 1.0 | 0.78 | 0.91 | 0.96 | 0.93 | 0.17 | 0.20 | 0.19 | 0.93 | 0.92 | 0.92
Shudha-Kalyan | 0.95 | 0.37 | 0.53 | 0.79 | 0.75 | 0.77 | 0.67 | 0.67 | 0.67 | 0.89 | 0.94 | 0.92
Sohni | 0.35 | 0.61 | 0.45 | 0.88 | 0.90 | 0.88 | 0.0 | 0.0 | 0.0 | 0.89 | 0.83 | 0.85
Yaman | 0.66 | 0.86 | 0.75 | 0.87 | 0.81 | 0.84 | 0.68 | 0.52 | 0.60 | 0.90 | 0.91 | 0.90
Weighted Avg | 0.68 | 0.64 | 0.63 | 0.85 | 0.83 | 0.83 | 0.50 | 0.50 | 0.50 | 0.90 | 0.89 | 0.89

####  V-B1 CN1+T

This model achieved an overall weighted f1-score of 0.78 on the test data for
all 12 classes after cross-validation. The maximum and minimum f1-scores
during cross-validation were 0.80 and 0.73, respectively. Clearly, this
architecture is not be suitable for audio data, as flattening does not
appropriately handle the dependent framewise information required for Raga
detection in case of such time-series data.

####  V-B2 CN1+LSTM+T

This model showed improved classification performance with an f1-score of 0.83
after cross-validation. The maximum and minimum f1-score observed during
cross-validation were 0.87 and 0.79, respectively. This model outperformed the
CN1+T model, highlighting the importance of LSTM in stead of flattening in
such scenarios.

####  V-B3 CN2+LSTM

Despite multiple training attempts and hyper-parameter tunings, this model did
not perform well on the training data itself, even with a similar architecture
to our main model (CNT+LSTM+T) or a even deeper CNN-LSTM model. This
underscores the importance of tonic normalization for the ARI task. We also
experimented with removing classes with very low f1-scores, simplifying the
problem. While this led to some improvement in training, it suggests a lack of
generalization to any number of Raga classes using this approach.

####  V-B4 CN2+LSTM+T

This model consistently achieved the highest performance among all models,
with an f1-score of 0.89 after 6-fold cross-validation. For four classes, the
CN1+LSTM+T model showed better results, however, the proposed model closely
follows, demonstrating strong training across all 12 classes and better
generalizability. The maximum and minimum f1-scores observed during cross-
validation are 0.94 and 0.84, respectively.

In HCM performances involving Khayal or Dhrupad singing, the starting 5 to 10
minutes of a performance generally involves the ”Vilambhit” (very slow tempo)
section, where the singer will be singing just straight notes stretched to a
few seconds, until they start singing in ”Madhya Laya” (medium tempo of the
range 100 to 150). Considering that this dataset is composed of actual HCM
performances, it is expected that some of the 30 second chunks from the
starting portions of such singing performances will have just a few straight
notes stretched for longer periods of times. These kind of audio files
(30-second) will most probably confuse the model since these would come out to
be very similar for any ragas which have similar ”swaras” in their ”Aaroh” and
”Avroh”. Considering that, the achieved performance is remarkable. For our
current analysis, we are focused on identifying salient regions based on
predictions, so we report ARI results at the chunk-level only and do not
predict Raga on song-level for our dataset.

Now, we carry out further analysis in terms of ExAI analysis to understand the
credibility of this classifier.

Figure 4: Raga class distribution for test files

###  V-C Saliency Evaluation

####  V-C1 Saliency annotation

The music expert is given a randomly selected 64 audio files for which the
model predicts the correct class. The task is to look for such regions in the
audio file which would allow any hindustani music expert to judge if that
audio snippet belongs to a particular Raga class. The Expert is asked to keep
only those audio files which have at least 5 to 7 seconds of such region and
at max 15 to 17 seconds. As it would make no sense to mark just one or two
seconds, or the whole audio as the salient region. The salient region can be a
single region, or may be 2 or 3 disjoint regions throught the 30 second audio
chunk. Also, it was made sure that we have at least 3 audio examples per
class. So after the annotation, we get 47 labelled test examples to evaluate
our saliency methods. The distribution of test classes is shown in figure 4.

####  V-C2 Saliency evaluation

To evaluate the salient regions identified by the two ExAI models, we compare
the predictions with the human-annotated saliency regions. For GC, we sum the
activation map values across the frequency to get a 938x1 vector, showing
saliency values for all 938 frames. We also convert the annotations of music
experts into a multi-hot vector of length 938, where we mark 1 for salient
region and 0 for non-salient. Now, to compare, we try out different windows
starting from top 1 seconds upto top 10 seconds of GC prediction values and
compute precision in each case.
For SL model, we get coefficients of linear regression model trained on the
test data, in terms of 30 masks. We upscale this to get a multi-hot vector of
938x1 to match the dimensions of expert annotations. Now we compare the
salient region given by this model in the same way as GC and compute precision
values.
In both these cases, we compute the model’s precision for upto top 10 seconds
out of the salient region, starting from 1 second. This is done so that we are
able to judge how correct the top salient region of our model is. If the model
is giving out decent results, it can be used as snippet for being
representative of the particular Raga to use that for music teaching other
applications.

TABLE III: Saliency report for GC and SL Time frame | GC Precision | SL Precision
---|---|---
1 second | 0.61 | 0.68
2 seconds | 0.60 | 0.66
3 seconds | 0.59 | 0.64
4 seconds | 0.58 | 0.60
5 seconds | 0.57 | 0.58
6 seconds | 0.57 | 0.58
7 seconds | 0.56 | 0.58
8 seconds | 0.56 | 0.56
9 seconds | 0.55 | 0.55
10 seconds | 0.55 | 0.54

####  V-C3 Saliency results

Table: III shows the achieved average precision of explanations over the 47
test samples, compared with expert annotations for the most-salient N-seconds
for both the ExAI models, where $N\in[1:10]$. Observe that for a smaller N,
precision is high and it decreases as N increases, indicating that the top N
seconds of highlighted salient regions by these ExAI models are very close to
actual salient regions for that particular audio chunk, implying that the
model is indeed relying on the time frames (the music in those time-frames) to
make its predictions for current Raga class. Also, we can observe that SL
shows higher performance which is also expected since GC considers activations
only upto the CNN layers, leaving out the LSTM layers. Although there is no
direct comparison metric among the two methods, we argue here that SoundLIME
explanations will be more reliable as it takes predictions from the whole
model (including the LSTM layers) while generating the explanations, hence
these should prove to be a better representative of the predictions of the
classification model.

TABLE IV: Analysis of individual samples True | Predicted | softmax | GC | SL
---|---|---|---|---
Sample | Sample | probability | Precision | Precision
Des | Des | 0.95 | 0.77 | 0.8
Bihag | Maru-Bihag | 0.71 | 0.12 | 0.0
Figure 5: Salient Regions for Raga Des sample

Now, we study two individual test audios for salient regions. We select a
ramdom correctly predicted test sample belonging to Raga class “Des”. The
classifier shows a softmax probability score of 0.95 as shown in Table: IV
with signifying that the model is quite confident about this prediction. The
true salient region marked by the music expert, the salient region as reported
by GC and SL are shown in Figures 5. The marked true region by the music
expert came out to be around 15 seconds. We are concerned about finding how
much of the region predicted by the model falls in the correct category. The
precision score for GC and SL come out to be 0.77 and 0.8 respectively for 10
seconds, signifying a significant overlap and hence a significant amount of
the prediction being based on the true concepts related to music only.

Figure 6: Salient Regions for Raga Bihag sample

We also conduct an individual study on two closely related Ragas in HCM,
namely Bihag and Maru-Bihag. Maru-Bihag, being a musical blend of two Ragas,
”Maru” and ”Bihag”, may exhibit some characteristics of Raga Bihag in its
composition, and vice versa. We can observe in the confusion matrix shown in
Fig: 7 also that there are a few number of samples of class Bihag which have
been mis-classified as Maru-Bihag.

Figure 7: Confusion Matrix for 12-class classifier

We carefully select such a false negative example and get it annotated for
salient region showing the current Raga (Bihag in this case). Fig: 6 shows the
true salient region, predicted salient region from GC and SL. We obtained top
k seconds of salient region to match the length of salient region obtained
from expert annotations from GC and SL models, shown in Fig: 6. The softmax
probabilities as shown in table:IV of this model are 0.26 (true class: Bihag)
and 0.71 (predicted class: Maru-Bihag), signifying that the model does
recognise the true class (Bihag) to some extent in the given audio, but
prediction of Maru-Bihag suggests that there is some ambiguous region in that
particular audio snippet which confuses the model to differentiate between the
two Raga classes. The same is also evident from the fact that the expert was
only able to find a very small region from the whole audio that prominently
shows the characteristics of Raga Bihag, implying that the audio chunk itself
could be a bit ambiguous. Also, The precision score for GC and SL come out to
be 0.12 and 0.0 respectively. We can clearly see in the figure6 that the
salient regions of both GC and SL do not overlap the true region and points to
totally different region, implying that there were frames which are similar to
Raga Maru-Bihag in that audio sample and the frames implying Raga Bihag might
not be that prominent, which can be confirmed by listening to the audio also.

##  VI Conclusion and Future Scope

In this paper, we introduce a novel labeled dataset for Hindustani Classical
Music (HCM) analysis and achieve an f1-score of 0.89 for 12-class Raga
classification task by carrying out Ablation studies. We observe that,
providing tonic normalization to the extracted Chromagram features
significantly enhances the performance of the Raga classifier. Following that,
we employ two Explainable AI (ExAI) models to identify the salient regions for
the classifier’s predictions and analyze their overlap with human annotations.
This comparison reveals a significant overlap, affirming that the patterns or
characteristics theoretically associated with the identification of a Raga for
any audio are present and are being used by the Raga Identification model for
the task. This serves as a correctness check for the music theory that
specific motifs, phrases or characteristics are indeed responsible for the
identification of any Raga by the Deep Learning model built on a dataset that
represents actual HCM performances.

This work provides a valuable contribution to the field of explainability in
Raga identification in Indian Art Music, setting the foundation for future
research. The meticulously labeled dataset, including Ragas and tonic
information, expands the landscape of available high-quality datasets in HCM
analysis, and opens avenues for further research in music classification and
analysis. Moving forward, Incorporating musicians’ perspectives and musical
concepts like ”Khatka” and ”murki” to refine Raga classifiers can ensure
correctness of Raga identification according to musical conventions and
nuances.

Future research can focus on developing methods to generate listenable or
musical explanations, which could serve as standalone identifiers of the Raga.
This would include extracting phrases from audio clips that correspond to the
Raga, offering potential applications in music pedagogy and beyond. It is
important to note that experts may not always agree on the same set of phrases
or regions, indicating the subjective nature of some aspects of music
analysis. Another direction for future research can be to extend this work to
other musical traditions and explore the generalizability of the approach.

## References

  * [1] S. Gulati, J. Serrà, V. Ishwar, S. Sentürk, and X. Serra, “Phrase-based rĀga recognition using vector space modeling,” in _2016 IEEE ICASSP_ , 2016, pp. 66–70.
  * [2] J. Bor, S. Rao, W. V. D. Meer, and J. Harvey, “The raga guide: a survey of 74 hindustani ragas,” _Nimbus Records with Rotterdam Conservatory of Music_ , 1999.
  * [3] R. Widdess, “The rāgas of early indian music: modes, melodies and musical notations from the gupta period to c. 1250,” _Clarendon Press, Oxford_ , 1995.
  * [4] S. Paschalidou and I. Miliaresi, “Multimodal Deep Learning Architecture for Hindustani Raga Classification,” _Sensors & Transducers_, vol. 261, no. 2, pp. 77–86, Feb. 2024.
  * [5] P. Agarwal, H. Karnick, and B. Raj, “A comparative study of indian and western music forms,” in _ISMIR Conference_ , 2013.
  * [6] X. Serra, “The computational study of a musical culture through its digital traces,” _Acta Musicologica_ , vol. 89, no. 1, p. 24–44, Jun. 2017.
  * [7] P. Kirthika and R. Chattamvelli, “A review of raga based music classification and music information retrieval (mir),” in _IEEE AICERA_ , 2012, pp. 1–5.
  * [8] S. Gulati, J. Serrà, K. K. Ganguli, S. Sentürk, and X. Serra, “Indian Art Music Raga Recognition Dataset (features),” Nov. 2022.
  * [9] A. Srinivasamurthy, S. Gulati, R. Caro Repetto, and X. Serra, “Saraga: Open datasets for research on indian art music,” _Empirical Musicology Review_ , vol. 16, no. 1, p. 85–98, Dec. 2021.
  * [10] A. A. Bidkar, R. S. Deshpande, and Y. H. Dandawate, “A north indian raga recognition using ensemble classifier,” _IJEET_ , vol. 12, no. 6, pp. 251–258, 2021.
  * [11] H. Sharma and R. S.Bali, “Comparison of ML classifiers for Raga recognition,” _Int J Sci Res Publ_ , vol. 5, no. 10, 10 2015.
  * [12] M. Velankar, A. Deshpande, and P. Kulkarni, “Melodic pattern recognition in Indian classical music for raga identification,” _International journal of information technology_ , vol. 251-258, no. 1, sep 4 2018.
  * [13] P. Mishra.et.al, “Transformer-based technique to classify raags in hindustani classical music,” _IEEE IATMSI_ , 2022.
  * [14] D. Joshi, J. Pareek, and P. Ambatkar, “Indian classical raga identification using machine learning.” in _ISIC_ , 2021, pp. 259–263.
  * [15] S. Chowdhuri, “Phononet: multi-stage deep neural networks for raga identification in hindustani classical music,” in _ICMR_ , 2019.
  * [16] S. Padi, S. Breiner, E. Subrahmanian, and R. D. Sriram, “Modeling and analysis of indian carnatic music using category theory,” _IEEE Transactions on Systems, Man, and Cybernetics: Systems_ , vol. 48, no. 6, pp. 967–981, 2018.
  * [17] A. Anand, “Raga identification using convolutional neural network,” in _ICACCP_ , 2019, pp. 1–6.
  * [18] S. John, M. S. Sinith, S. R. S, and L. P. P, “Classification of indian classical carnatic music based on raga using deep learning,” in _IEEE RAICS_ , 2020, pp. 110–113.
  * [19] P. Dighe, P. Agrawal, H. Karnick, S. Thota, and B. Raj, “Scale independent raga identification using chromagram patterns and swara based features,” in _IEEE ICMEW_ , 2013, pp. 1–4.
  * [20] A. K. Sharma, G. Aggarwal, S. Bhardwaj, P. Chakrabarti, T. Chakrabarti, J. H. Abawajy, S. Bhattacharyya, R. Mishra, A. Das, and H. Mahdin, “Classification of indian classical music with time-series matching deep learning approach,” _IEEE Access_ , vol. 9, pp. 102 041–102 052, 2021.
  * [21] K. Waghmare and B. Sonkamble, “Raga identification techniques for classifying indian classical music: A survey,” _International Journal of Signal Processing Systems_ , vol. 5, pp. 130–135, 12 2017.
  * [22] K. C. Waghmare and B. A. Sonkamble, “Analyzing acoustics of indian music audio signal using timbre and pitch features for raga identification,” in _ICISPC_ , 2019, pp. 42–46.
  * [23] K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” _CoRR_ , vol. abs/1312.6034, 2014.
  * [24] J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. A. Riedmiller, “Striving for simplicity: The all convolutional net,” _CoRR_ , 2014.
  * [25] D. Smilkov, N. Thorat, B. Kim, F. B. Viégas, and M. Wattenberg, “Smoothgrad: removing noise by adding noise,” _ArXiv_ , 2017.
  * [26] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in _IEEE CVPR_ , 2016, pp. 2921–2929.
  * [27] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in _ICCV_ , 2017, pp. 618–626.
  * [28] A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, “Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks,” in _WACV_ , 2018, pp. 839–847.
  * [29] A. Binder, G. Montavon, S. Bach, K.-R. Müller, and W. Samek, “Layer-wise relevance propagation for neural networks with local renormalization layers,” 2016.
  * [30] M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in _International Conference on Machine Learning_ , 2017.
  * [31] S.-Y. Lim, D.-K. Chae, and S.-C. Lee, “Detecting deepfake voice using explainable deep learning techniques,” _Applied Sciences_ , vol. 12, no. 8, p. 3926, 2022.
  * [32] A. Govindu, P. Kale, A. Hullur, A. Gurav, and P. Godse, “Deepfake audio detection and justification with explainable artificial intelligence (xai),” 10 2023.
  * [33] V. Haunschmid, E. Manilow, and G. Widmer, “audiolime: listenable explanations using source separation,” _arXiv preprint arXiv:2008.00582_ , 2020.
  * [34] S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in _Neural Information Processing Systems_ , 2017.
  * [35] M. T. Ribeiro, S. Singh, and C. Guestrin, “Anchors: High-Precision Model-Agnostic Explanations,” _AAAI’18_ , vol. 32, no. 1, 2018.
  * [36] S. Mishra, B. L. Sturm, and S. Dixon, “Local Interpretable Model-Agnostic Explanations for Music Content Analysis.” in _ISMIR_ , Sep. 2018, pp. 537–543.