# Explicit Formula for the $n$-th Derivative of a Quotient

Roudy El Haddad
Université La Sagesse, Faculté de génie, Polytech
<EMAIL_ADDRESS>
###### Abstract

Leibniz’s rule for the $n$-th derivative of a product is a very well known and
extremely useful formula. In this article, we introduce an analogous explicit
formula for the $n$-th derivative of a quotient of two functions. Later, we
use this formula to derive new partition identities and to develop expressions
for some special $n$-th derivatives.

######  Keywords.

$n$-th derivative of a quotient, generalized quotient rule, partitions.
MSC 2020: primary 26A24 secondary 28A15, 11P99

##  1 Introduction

If we chose two functions $u$ and $v$ and went around asking mathematicians to
compute the $n$-th derivative of their product, the first idea that would come
to their mind is to use Leibniz’s formula. However, what if instead we asked
them to compute the $n$-th derivative of the quotient, what formula would come
to their mind? For many, the answer is none. A large portion of the
mathematical community is unaware of the existence of such a formula. This is
because, while Leibniz’s formula is a subject that is studied in practically
all calculus courses, when it comes to talk about an analogous formula for the
quotient of two functions, the topic is rarely discussed. Although many wonder
if such a formula exists, not much work has been done on the subject. In 1967
[13], the first step was taken as a simpler question was answered, that is, a
recursive formula for the $n$-th derivative of $1/f(x)$ was presented. Later,
in 1980, F. Gerrish [8] noticed an interesting pattern linking the $n$-th
derivative of a quotient to a special determinant. In 2008, this special
connection was used to establish a recursive formula for such a derivative
[14]. So if such a formula exists, why do most of us not know about it? There
are two major reasons: The first is that the existing formulas are not very
practical as they are recursive rather than explicit. The second is that such
a formula was thought to be useless. F. Gerrish [8] even went as far as
calling it the “Useless Formula”. However, since then, this formula has found
a variety of applications and has been used to deal with a multitude of topics
[7, 2, 9, 10, 12, 4, 3]. Hence, in this article, we propose to revisit the
subject and develop an explicit formula for the $n$-th derivative of a
quotient analogous to the generalized product rule, in hopes this formula will
become a standard like Leibniz’s rule. More precisely, it will be analogous to
the generalized product rule for the product of several functions. Note that
we mean by analogous that the formula will be explicit and that it will have
the same form (that is, it will be in terms of a sum over partitions). Let us
begin by noting that, in the same way Leibniz’s formula is often referred to
as the product rule, in this article, for simplicity, we will refer to the
formula for the $n$-th derivative of a quotient as the quotient rule. We will
begin by deriving a new formula for the $n$-th derivative of $1/f(x)$ (Section
2). Although, such a formula already exists, the formula presented in [13] is
rather complicated. We propose a simpler formula involving partitions. The
formula we will present also has the advantage of being explicit rather than
recursive. We will refer to this particular case as the reciprocal rule.
Similarly, although a recursive formula already exists for the quotient rule,
no explicit formula exists. Therefore, in Section 3, by combining the
reciprocal formula with Leibniz’s formula, we develop an explicit formula for
the $n$-th derivative of the quotient of two functions. Finally, in Section 4,
we apply the reciprocal and quotient rules developed to derive new partition
identities as well as expressions for some special $n$-th derivatives.

##  2 n-th derivative of $1/v(x)$ (Reciprocal rule)

We begin by introducing the concept of partitions as partitions are an
essential part of the quotient rule we will develop. As defined by the author
in [6, 5], a partition can be defined as follows:

######  Definition.

A partition of a non-negative integer $m$ is a set of positive integers whose
sum equals $m$. We can represent a partition of $m$ as an ordered set
$(y_{k,1},\ldots,y_{k,m})$ that verifies

$y_{k,1}+2y_{k,2}+\cdots+my_{k,m}=\sum_{i=1}^{m}{i\,y_{k,i}}=m.$ (1)

The coefficient $y_{k,i}$ is the multiplicity of the integer $i$ in the $k$-th
partition of $m$. Note that $0\leq y_{k,i}\leq m$ while $1\leq i\leq m$. Also
note that the number of partitions of an integer $m$ is given by the partition
function denoted $p(m)$ and hence, $1\leq k\leq p(m)$. In the remainder of
this text, the subscript $k$ will be added to indicate that a given parameter
is associated with a given partition. Similarly, for simplicity, we will omit
the bounds of $i$ and write $\sum{iy_{k,i}}=m$ and $\sum{y_{k,i}}$.
Furthermore, we define the following partition notation:

$\pi_{k}=\sum{iy_{k,i}},$ (2) $r_{k}=\sum{y_{k,i}}.$ (3)

As partitions are not the main focus of this article, we will not go into more
details. For readers interested in a more in-depth explanation about
partitions, see [1].

Before we begin proving the main results of this section, let us introduce the
following notation: In the remainder of this article, the letters $u$ and $v$
will be used to indicate a function of $x$. In other words, $u$ represents
$u(x)$ and $v$ represents $v(x)$.

######  Definition.

Let us define the following shorthand notation:

$\left(v\right)^{(n)}=v^{(n)}=\frac{d^{n}}{dx^{n}}\left(v(x)\right).$ (4)

In order to prove the reciprocal rule, we need to first prove the following
lemma.

######  Lemma 2.1.

We have that

$\begin{split}\sum_{j=0}^{n-1}{\binom{\sum{Y_{k,i}}-1}{Y_{k,1},\ldots,Y_{k,{n-j}}-1,\ldots,Y_{k,{n}}}}=\sum_{j=1}^{n}{\binom{\sum{Y_{k,i}}-1}{Y_{k,1},\ldots,Y_{k,{j}}-1,\ldots,Y_{k,{n}}}}=\binom{\sum{Y_{k,i}}}{Y_{k,1},\ldots,Y_{k,{n}}}.\end{split}$

###### Proof.

$\begin{split}\sum_{j=0}^{n-1}{\binom{\sum{Y_{k,i}}-1}{Y_{k,1},\ldots,Y_{k,{n-j}}-1,\ldots,Y_{k,{n}}}}&=\sum_{j=0}^{n-1}{\frac{(\sum{Y_{k,i}}-1)!}{Y_{k,1}!\cdots
Y_{k,{n-j}}!\cdots Y_{k,{n}}!}(Y_{k,{n-j}})}\\\
&=\frac{(\sum{Y_{k,i}}-1)!}{Y_{k,1}!\cdots
Y_{k,{n}}!}\sum_{j=1}^{n}{(Y_{k,{j}})}\\\
&=\frac{(\sum{Y_{k,i}})!}{Y_{k,1}!\cdots
Y_{k,{n}}!}=\binom{\sum{Y_{k,i}}}{Y_{k,1},\ldots,Y_{k,{n}}}.\end{split}$

∎

Using the recursive formula for the quotient rule [14], we derive the
reciprocal rule.

######  Theorem 2.1 (Reciprocal rule).

Let $v$ be an $n$ times differentiable function of $x$, for any
$n\in\mathbb{N}$ and at every point where $v\neq 0$, we have that

$\left(\frac{1}{v}\right)^{(n)}=\frac{d^{n}}{dx^{n}}\left(\frac{1}{v}\right)=n!\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}.$

######  Remark.

A very interesting and compact way of rewriting this theorem is as follows:

$\left(\frac{1}{v}\right)^{(n)}=n!\sum_{\sum{iy_{k,i}}=n}{C_{k}\prod_{i=1}^{n}{\frac{1}{y_{k,i}!}\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}$

where

$C_{k}=\frac{d^{\sum{y_{k,i}}}}{dv^{\sum{y_{k,i}}}}\left(\frac{1}{v}\right)=\frac{(\sum{y_{k,i}})!(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}=\frac{(-1)^{r_{k}}r_{k}!}{v^{r_{k}+1}}.$
(5)

As we can see, the general reciprocal rule using the $C_{k}$ notation is
simple and easy to memorize.

######  Remark.

Let us define the notation $\\{a\\}_{b}$ corresponds to writing $b$ times the
value $a$. Let $I_{k}=(\\{1\\}_{y_{k,1}},\ldots,\\{n\\}_{y_{k,n}})$.
Similarly, let $P_{k}=(y_{k,1},\ldots,y_{k,n})$. Other interesting ways of
writing the theorem are:

$\left(\frac{1}{v}\right)^{(n)}=\sum_{\sum{iy_{k,i}}=n}{\binom{n}{I_{k}}C_{k}\prod_{i=1}^{n}{\frac{\left[v^{(i)}\right]^{y_{k,i}}}{y_{k,i}!}}}=\frac{1}{v}\sum_{\sum{iy_{k,i}}=n}{\binom{n}{I_{k}}\binom{\sum{y_{k,i}}}{P_{k}}\prod_{i=1}^{n}{\left[-\frac{v^{(i)}}{v}\right]^{y_{k,i}}}}.$

###### Proof.

1\. Base case: verify true for $n=1$.

$1!\sum_{\sum{iy_{k,i}}=1}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}=\binom{1}{1}\frac{(-1)^{1}}{v^{2}}\left[\frac{v^{{}^{\prime}}}{1!}\right]^{1}=-\frac{v^{\prime}}{v^{2}}=\frac{d}{dx}\left(\frac{1}{v}\right).$

######  Remark.

We can also verify true for $n=0$. It is important to note that the partition
assumed to correspond to zero is $(0,0,\ldots)$. Hence,

$0!\sum_{\sum{iy_{k,i}}=0}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}=\binom{0}{0,0,\ldots}\frac{(-1)^{0}}{v^{1}}\left(1\right)=\frac{1}{v}=\left(\frac{1}{v}\right)^{(0)}.$

2\. Induction hypothesis: assume the statement is true until
$(n-1)\in\mathbb{N}$.

$\left(\frac{1}{v}\right)^{(n-1)}=(n-1)!\sum_{\sum{iy_{k,i}}=n-1}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,{n-1}}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n-1}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}.$

3\. Induction step: we will show that this statement is true for $n$.
We have to show the following statement to be true:

$\left(\frac{1}{v}\right)^{(n)}=n!\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,{n}}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}.$

Using the recursive formula developped in [14] with $u=1$, we have

$\left(\frac{1}{v}\right)^{(n)}=\frac{(-1)n!}{v}\sum_{j=1}^{n}{\frac{v^{(n+1-j)}}{(n+1-j)!}\frac{\left(\frac{1}{v}\right)^{(j-1)}}{(j-1)!}}=\frac{(-1)n!}{v}\sum_{j=0}^{n-1}{\frac{v^{(n-j)}}{(n-j)!}\frac{\left(\frac{1}{v}\right)^{(j)}}{j!}}.$

Applying the induction hypothesis, we get

$\begin{split}\left(\frac{1}{v}\right)^{(n)}&=\frac{(-1)n!}{v}\sum_{j=0}^{n-1}{\frac{v^{(n-j)}}{(n-j)!}\sum_{\sum{iy_{k,i}}=j}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,{j}}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{j}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}\\\
&=n!\sum_{j=0}^{n-1}{\frac{v^{(n-j)}}{(n-j)!}\sum_{\sum{iy_{k,i}}=j}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,{j}}}\frac{(-1)^{\sum{y_{k,i}}+1}}{v^{\sum{y_{k,i}}+2}}\prod_{i=1}^{j}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}.\end{split}$

Let us define an extension $(y_{k,1},\ldots,y_{k,n})$ of
$(y_{k,1},\ldots,y_{k,j})$ where $y_{k,j+1}=\cdots=y_{k,n}=0$. Hence, we can
write that

$\begin{split}\left(\frac{1}{v}\right)^{(n)}&=n!\sum_{j=0}^{n-1}{\frac{v^{(n-j)}}{(n-j)!}\sum_{\sum{iy_{k,i}}=j}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,{n}}}\frac{(-1)^{\sum{y_{k,i}}+1}}{v^{\sum{y_{k,i}}+2}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}\\\
&=n!\sum_{j=0}^{n-1}{\sum_{\sum{iy_{k,i}}+(n-j)\cdot
1=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,{n}}}\frac{(-1)^{\sum{y_{k,i}}+1}}{v^{\sum{y_{k,i}}+2}}{\left[\frac{v^{(n-j)}}{(n-j)!}\right]}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}.\end{split}$

Notice that $1\leq n-j\leq n$ as $0\leq j\leq n-1$. Now, for all
$(n-j)\in[1,n]$, let us associate with each partition
$(y_{k,1},\ldots,y_{k,n})$, the partition $(Y_{k,1},\ldots,Y_{k,n})$ such that

$\begin{cases}Y_{k,i}=y_{k,i}+1,&\text{for }i=n-j,\\\
Y_{k,i}=y_{k,i},&\text{otherwise}.\end{cases}$

Notice that $\sum{Y_{k,i}}=\sum{y_{k,i}}+1$ and that $\sum{iY_{k,i}}=n$.
Hence, we can write

$\begin{split}\left(\frac{1}{v}\right)^{(n)}&=n!\sum_{j=0}^{n-1}{\sum_{\sum{iY_{k,i}}=n}{\binom{\sum{Y_{k,i}}-1}{Y_{k,1},\ldots,Y_{k,{n-j}}-1,\ldots,Y_{k,{n}}}\frac{(-1)^{\sum{Y_{k,i}}}}{v^{\sum{Y_{k,i}}+1}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{Y_{k,i}}}}}\\\
&=n!\sum_{\sum{iY_{k,i}}=n}{\frac{(-1)^{\sum{Y_{k,i}}}}{v^{\sum{Y_{k,i}}+1}}\left(\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{Y_{k,i}}}\right)}\sum_{j=0}^{n-1}{\binom{\sum{Y_{k,i}}-1}{Y_{k,1},\ldots,Y_{k,{n-j}}-1,\ldots,Y_{k,{n}}}}.\end{split}$

Applying Lemma 2.1 to the inner sum, we obtain

$\left(\frac{1}{v}\right)^{(n)}=n!\sum_{\sum{iY_{k,i}}=n}{\binom{\sum{Y_{k,i}}}{Y_{k,1},\ldots,Y_{k,{n}}}\frac{(-1)^{\sum{Y_{k,i}}}}{v^{\sum{Y_{k,i}}+1}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{Y_{k,i}}}}.$

This concludes our proof by induction. ∎

######  Remark.

As we can see, the reciprocal rule derived (Theorem 2.1) is very similar to
the product rule for the product of several functions:

$\left(u_{1}\cdots
u_{m}\right)^{(n)}=\sum_{\ell_{1}+\cdots+\ell_{m}=n}{\binom{n}{\ell_{1},\ldots,\ell_{m}}\prod_{i=1}^{m}{u_{i}^{(\ell_{i})}}}$
(6)

There exists other alternatives for proving Theorem 2.1. In what follows, we
will present a few propositions that will be useful for doing so.

First, let us prove the following useful proposition for the derivative of a
product.

######  Proposition 2.1.

Let $u_{1}$, $\ldots$, $u_{n}$ be differentiable functions of $x$, we have
that

$\frac{d}{dx}\left(\prod_{i=1}^{n}{u_{i}}\right)=\left(\prod_{i=1}^{n}{u_{i}}\right)\sum_{j=1}^{n}{\frac{u_{i}^{{}^{\prime}}}{u_{i}}}.$

###### Proof.

Let $f(x)=u_{1}\cdots u_{n}$. Taking the logarithm of both sides, we get

$\ln f(x)=\ln\left(\prod_{i=1}^{n}{u_{i}}\right)=\sum_{i=1}^{n}{\ln u_{i}}.$

Differentiating both sides, we get

$-\frac{f^{\prime}(x)}{f(x)}=-\sum_{i=1}^{n}{\frac{u_{i}^{{}^{\prime}}}{u_{i}}}.$

Canceling the minus sign, we obtain the desired formula. ∎

Now we prove the following partition identity involving a special sum of
multinomial coefficients. This expression is equivalent to Lemma 2.1 that we
used to prove Theorem 2.1.

######  Proposition 2.2.

We have that

$\sum_{\begin{subarray}{c}\sum{\varphi_{i}}=\sum{Y_{i}}-1\\\ \varphi_{i}\leq
Y_{i}\end{subarray}}{\binom{\sum{Y_{i}}-1}{\varphi_{1},\ldots,\varphi_{n}}}=\binom{\sum{Y_{i}}}{Y_{1},\ldots,Y_{n}}.$

###### Proof.

Let $Z_{i}=Y_{i}-\varphi_{i}$ for $1\leq i\leq n$.

$\begin{split}\sum_{\begin{subarray}{c}\sum{\varphi_{i}}=\sum{Y_{i}}-1\\\
\varphi_{i}\leq
Y_{i}\end{subarray}}{\binom{\sum{Y_{i}}-1}{\varphi_{1},\ldots,\varphi_{n}}}&=\frac{(\sum{Y_{i}}-1)!}{Y_{1}!\cdots
Y_{n}!}\sum_{\begin{subarray}{c}\sum{\varphi_{i}}=\sum{Y_{i}}-1\\\
\varphi_{i}\leq Y_{i}\end{subarray}}{\frac{Y_{1}!\cdots
Y_{n}!}{\varphi_{1}!\cdots\varphi_{n}!}}\\\
&=\frac{(\sum{Y_{i}}-1)!}{Y_{1}!\cdots
Y_{n}!}\sum_{\begin{subarray}{c}\sum{Z_{i}}=1\\\ Z_{i}\geq
0\end{subarray}}{\frac{Y_{1}!\cdots
Y_{n}!}{(Y_{1}-Z_{1})!\cdots(Y_{n}-Z_{n})!}}.\end{split}$

Knowing that the $Z_{i}$’s are non-negative integers, the only way for their
sum to be equal to 1 is if one of them is equal to 1 and the others are equal
to 0. Hence, we have that

$\sum_{\begin{subarray}{c}\sum{Z_{i}}=1\\\ Z_{i}\geq
0\end{subarray}}{\frac{Y_{1}!\cdots
Y_{n}!}{(Y_{1}-Z_{1})!\cdots(Y_{n}-Z_{n})!}}=\sum_{i=1}^{n}{\frac{Y_{i}!}{(Y_{i}-1)!}}=\sum_{i=1}^{n}{Y_{i}}.$

Substituting back, we obtain the proposition. ∎

##  3 n-th derivative of $u(x)/v(x)$ (Quotient rule)

In this section, we combine Theorem 2.1 with Leibniz’s rule to obtain the
general quotient rule.

######  Theorem 3.1 (Quotient rule).

Let $u$ and $v$ be $n$ times differentiable functions of $x$, for any
$n\in\mathbb{N}$ and at every point where $v\neq 0$, we have that

$\begin{split}\left(\frac{u}{v}\right)^{(n)}=\frac{d^{n}}{dx^{n}}\left(\frac{u}{v}\right)&=n!\sum_{\ell=0}^{n}{\frac{u^{(n-\ell)}}{(n-\ell)!}\sum_{\sum{iy_{k,i}}=\ell}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,\ell}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{\ell}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}\\\
&=n!\sum_{\pi_{k}=0}^{n}{\frac{u^{(n-\pi_{k})}}{(n-\pi_{k})!}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,\pi_{k}}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{\pi_{k}}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}.\end{split}$

###### Proof.

Applying Leibniz’s rule to Theorem 2.1, we obtain this theorem. ∎

######  Remark.

A much more compact way of expressing this theorem is as follows:

$\left(\frac{u}{v}\right)^{(n)}=n!\sum_{\ell=0}^{n}{\sum_{\sum{iy_{k,i}}=n-\ell}{C_{k}\left[\frac{u^{(\ell)}}{\ell!}\right]\prod_{i=1}^{n-\ell}{\frac{1}{y_{k,i}!}\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}=n!\sum_{\pi_{k}=0}^{n}{{C_{k}\frac{u^{(n-\pi_{k})}}{(n-\pi_{k})!}\prod_{i=1}^{\pi_{k}}{\frac{1}{y_{k,i}!}\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}$

where $C_{k}$ is as defined in Eq. (5) and $\pi_{k}$ is as defined in Eq. (2).
Using the $C_{k}$ and $\pi_{k}$ notation, we obtain a simple and easy to
memorize expression for the general quotient rule that could potentially be
taught to university students at the same time as the general product rule.

######  Remark.

Let $I_{k}=(\\{1\\}_{y_{k,1}},\ldots,\\{\pi_{k}\\}_{y_{k,\pi_{k}}})$ and
$P_{k}=(y_{k,1},\ldots,y_{k,\pi_{k}})$. We can write the following interesting
but not very practical expressions:

$\begin{split}\left(\frac{u}{v}\right)^{(n)}&=\sum_{\pi_{k}=0}^{n}{{C_{k}\binom{n}{I_{k},n-\pi_{k}}u^{(n-\pi_{k})}\prod_{i=1}^{\pi_{k}}{\frac{\left[v^{(i)}\right]^{y_{k,i}}}{y_{k,i}!}}}}\\\
&=\sum_{\pi_{k}=0}^{n}{{\binom{n}{I_{k},n-\pi_{k}}\binom{\sum{y_{k,i}}}{P_{k}}\frac{u^{(n-\pi_{k})}}{v}\prod_{i=1}^{\pi_{k}}{\left[-\frac{v^{(i)}}{v}\right]^{y_{k,i}}}}}.\end{split}$

##  4 Applications

Let us first define some notation to simplify the expressions we will derive.
For a given partition $(y_{k,1},\ldots,y_{k,n})$ of $n$, we define the
following notation:

$\displaystyle c_{k}$
$\displaystyle=\prod_{i=1}^{n}{\frac{1}{i^{y_{k,i}}y_{k,i}!}},$
$\displaystyle\overline{c}_{k}$
$\displaystyle=\prod_{i=1}^{n}{\frac{(-1)^{y_{k,i}}}{i^{y_{k,i}}y_{k,i}!}}.$
(7) $\displaystyle p_{k}$
$\displaystyle=\prod_{i=1}^{n}{\frac{1}{i!^{y_{k,i}}y_{k,i}!}},$
$\displaystyle\overline{p}_{k}$
$\displaystyle=\prod_{i=1}^{n}{\frac{(-1)^{y_{k,i}}}{i!^{y_{k,i}}y_{k,i}!}}.$
(8) $\displaystyle q_{k}$
$\displaystyle=\prod_{i=1}^{n}{\frac{1}{i!^{y_{k,i}}}},$
$\displaystyle\overline{q}_{k}$
$\displaystyle=\prod_{i=1}^{n}{\frac{(-1)^{y_{k,i}}}{i!^{y_{k,i}}}}.$ (9)

###  4.1 Partition identities

In this section, we will show how the quotient rule developed can be used to
derive partition identities. In particular, we will derive a few special
partition identities.

######  Proposition 4.1.

For any $n\in\mathbb{N}$, we have that

$\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}\prod_{i=1}^{n}{\frac{1}{i!^{y_{k,i}}}}}=\frac{(-1)^{n}}{n!}.$

Using the notation, this proposition can be expressed as

$\sum_{\sum{iy_{k,i}}=n}{\binom{r_{k}}{y_{k,1},\ldots,y_{k,n}}(-1)^{r_{k}}q_{k}}=\sum_{\sum{iy_{k,i}}=n}{\binom{r_{k}}{y_{k,1},\ldots,y_{k,n}}\overline{q}_{k}}=\frac{(-1)^{n}}{n!}.$

######  Remark.

We can also rewrite it as follows:

$\sum_{\sum{iy_{k,i}}=n}{r_{k}!(-1)^{r_{k}}\prod_{i=1}^{n}{\frac{1}{i!^{y_{k,i}}y_{k,i}!}}}=\frac{(-1)^{n}}{n!}.$

Using the notation, this proposition can be expressed as

$\sum_{\sum{iy_{k,i}}=n}{r_{k}!(-1)^{r_{k}}p_{k}}=\sum_{\sum{iy_{k,i}}=n}{r_{k}!\overline{p}_{k}}=\frac{(-1)^{n}}{n!}.$

###### Proof.

From Theorem 2.1 with $v(x)=e^{x}$ and knowing that $v^{(i)}(x)=e^{x}$ for all
$i$, we get

$\begin{split}\frac{d^{n}}{dx^{n}}{\left(\frac{1}{e^{x}}\right)}&=n!\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{(e^{x})^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n}{\left[\frac{e^{x}}{i!}\right]^{y_{k,i}}}}\\\
&=n!e^{-x}\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}\prod_{i=1}^{n}{\frac{1}{i!^{y_{k,i}}}}}.\end{split}$

Noticing that

$\frac{d^{n}}{dx^{n}}{\left(\frac{1}{e^{x}}\right)}=\frac{d^{n}}{dx^{n}}{\left({e^{-x}}\right)}=(-1)^{n}e^{-x},$

we obtain the proposition. ∎

######  Proposition 4.2.

For any $n\in\mathbb{N}$ and any $m\in\mathbb{N}^{*}$, we have that

$\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}\prod_{i=1}^{n}{\left[\binom{m}{i}\right]^{y_{k,i}}}}=(-1)^{n}\binom{n+m-1}{m-1}.$

###### Proof.

Let $v(x)=x^{m}$, then $v^{(i)}=i!\binom{m}{i}x^{m-i}$. Hence, from Theorem
2.1, we have

$\begin{split}\frac{d^{n}}{dx^{n}}{\left(\frac{1}{x^{m}}\right)}&=n!\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{x^{m\sum{y_{k,i}}+m}}\prod_{i=1}^{n}{\left[x^{m-i}\binom{m}{i}\right]^{y_{k,i}}}}\\\
&=n!\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{x^{m\sum{y_{k,i}}+m}}(x^{m\sum{y_{k,i}}-n})\prod_{i=1}^{n}{\left[\binom{m}{i}\right]^{y_{k,i}}}}\\\
&=n!x^{-m-n}\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}\prod_{i=1}^{n}{\left[\binom{m}{i}\right]^{y_{k,i}}}}.\end{split}$

Noticing that

$\frac{d^{n}}{dx^{n}}{\left(\frac{1}{x^{m}}\right)}=\frac{d^{n}}{dx^{n}}{\left({x^{-m}}\right)}=(-1)^{n}n!\binom{n+m-1}{m-1}x^{-m-n},$

we obtain the proposition. ∎

######  Corollary 4.1.

Setting $m=n$, we get

$\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}\prod_{i=1}^{n}{\left[\binom{n}{i}\right]^{y_{k,i}}}}=(-1)^{n}\binom{2n-1}{n-1}.$

######  Proposition 4.3.

For any $n\in\mathbb{N}$ and any $m\in\mathbb{N}^{*}$, we have that

$\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}\prod_{i=1}^{n}{\left[\binom{i+m-1}{m-1}\right]^{y_{k,i}}}}=(-1)^{n}\binom{m}{n}.$

###### Proof.

Let $v(x)=x^{-m}$, then $v^{(i)}=(-1)^{i}i!\binom{i+m-1}{m-1}x^{-(m+i)}$.
Hence, from Theorem 2.1, we have

$\begin{split}\frac{d^{n}}{dx^{n}}{\left(\frac{1}{x^{-m}}\right)}&=n!\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{x^{-m\sum{y_{k,i}}-m}}\prod_{i=1}^{n}{\left[(-1)^{i}\binom{i+m-1}{m-1}x^{-(m+i)}\right]^{y_{k,i}}}}\\\
&=n!(-1)^{n}\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}(x^{-m\sum{y_{k,i}}-n})}{x^{-m\sum{y_{k,i}}-m}}\prod_{i=1}^{n}{\left[\binom{i+m-1}{m-1}\right]^{y_{k,i}}}}\\\
&=(-1)^{n}n!x^{m-n}\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}\prod_{i=1}^{n}{\left[\binom{i+m-1}{m-1}\right]^{y_{k,i}}}}.\end{split}$

Noticing that

$\frac{d^{n}}{dx^{n}}{\left(\frac{1}{x^{m}}\right)}=\frac{d^{n}}{dx^{n}}{\left({x^{-m}}\right)}=n!\binom{m}{n}x^{m-n},$

we obtain the proposition. ∎

An extremely interesting result that can be derived from Proposition 4.3 is
that for the alternating sum over partitions of multinomial coefficients.

######  Corollary 4.2.

Setting $m=1$, we get

$\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}(-1)^{\sum{y_{k,i}}}}=(-1)^{n}\binom{1}{n}=\begin{cases}(-1)^{n},&n=0,1,\\\
0,&n\geq 2.\end{cases}$

###  4.2 Special n-th derivatives

Because of the absence of a general quotient rule, there were many $n$-th
derivatives for which we could not obtain an explicit expression. In this
section, we will use the quotient rule derived to develop an expression for
some of these derivatives.

The first special $n$-th derivative is that of $\log_{x}{a}$ as well as that
of the reciprocal of $\ln x$. In 2014, Feng Qi [11] introduced the following
expression for the reciprocal of $\ln x$:

$\left(\frac{1}{\ln
x}\right)^{(n)}=\frac{(-1)^{n}}{x^{n}}\sum_{i=2}^{n+1}{\frac{a_{n,i}}{(\ln
x)^{i}}},$ (10)

where

$a_{n,2}=(n-1)!$ (11)

and, for $n+1\geq i\geq 3$,

$a_{n,i}=(i-1)!(n-1)!\sum_{\ell_{1}=1}^{n-1}{\frac{1}{\ell_{1}}\sum_{\ell_{2}=1}^{\ell_{1}-1}{\frac{1}{\ell_{2}}\cdots\sum_{\ell_{i-3}=1}^{\ell_{i-4}-1}{\frac{1}{\ell_{i-3}}\sum_{\ell_{i-2}=1}^{\ell_{i-3}-1}{\frac{1}{\ell_{i-2}}}}}}.$
(12)

The expression seems simple, however, the $a_{n,i}$ terms correspond to a kind
of multiple harmonic sum. Such sums are very tedious to compute, thus, making
Eq. (10) a bit tedious to use. In what follows, using the general reciprocal
rule, we will derive a simpler expression.

######  Proposition 4.4.

For any $a\in\mathbb{N^{*}}$, the $n$-th derivative of $\log_{x}{a}$ is given
by

$\left(\log_{x}{a}\right)^{(n)}=\left(\frac{\ln a}{\ln
x}\right)^{(n)}=(\log_{x}{a})\frac{(-1)^{n}n!}{x^{n}}\sum_{\sum{iy_{k,i}}=n}{\frac{({\sum{y_{k,i}}})!}{(\ln
x)^{\sum{y_{k,i}}}}\prod_{i=1}^{n}{\frac{1}{i^{y_{k,i}}y_{k,i}!}}}.$

Using the notation, we can rewrite it as follows:

$\left(\log_{x}{a}\right)^{(n)}=\left(\frac{\ln a}{\ln
x}\right)^{(n)}=(\log_{x}{a})\frac{(-1)^{n}n!}{x^{n}}\sum_{\sum{iy_{k,i}}=n}{c_{k}\frac{r_{k}!}{(\ln
x)^{r_{k}}}}.$

###### Proof.

From Theorem 2.1 with $v=\ln x$, we have

$\begin{split}\left(\log_{x}{a}\right)^{(n)}=\left(\frac{\ln a}{\ln
x}\right)^{(n)}&=n!\frac{\ln a}{\ln
x}\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{(\ln
x)^{\sum{y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{(\ln
x)^{(i)}}{i!}\right]^{y_{k,i}}}}.\end{split}$

Knowing that, for $i\geq 1$,

$(\ln x)^{(i)}=\frac{(-1)^{i-1}(i-1)!}{x^{i}},$

hence, by substituting back and simplifying, we get

$\begin{split}\left(\log_{x}{a}\right)^{(n)}&=n!(\log_{x}{a})\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}}}{(\ln
x)^{\sum{y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{(-1)^{i-1}}{ix^{i}}\right]^{y_{k,i}}}}\\\
&=\frac{(-1)^{n}n!}{x^{n}}(\log_{x}{a})\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{1}{(\ln
x)^{\sum{y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{1}{i}\right]^{y_{k,i}}}}.\end{split}$

Replacing the multinomial coefficient by its factorial definition, we obtain
this proposition. ∎

Another special $n$-th derivative is that of $\ln{v(x)}$.

######  Proposition 4.5.

The $n$-th derivative of $\ln v(x)$ is given by

$\begin{split}(\ln
v)^{(n)}&=n!\sum_{\sum{iy_{k,i}}=n}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n}}\frac{(-1)^{\sum{y_{k,i}}-1}}{(\sum{y_{k,i}})!v^{\sum{y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}\\\
&=n!\sum_{\sum{iy_{k,i}}=n}{\frac{(\sum{y_{k,i}}-1)!(-1)^{\sum{y_{k,i}}-1}}{v^{\sum{y_{k,i}}}}\prod_{i=1}^{n}{\frac{1}{y_{k,i}!}\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}.\end{split}$

###### Proof.

From Theorem 3.1, we have

$\begin{split}(\ln v)^{(n)}&=\left(\frac{v^{\prime}}{v}\right)^{(n-1)}\\\
&=(n-1)!\sum_{\ell=0}^{n-1}{\frac{(v^{\prime})^{(\ell)}}{\ell!}\sum_{\sum{iy_{k,i}}=n-\ell-1}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n-\ell-1}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n-\ell-1}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}\\\
&=(n-1)!\sum_{\ell=0}^{n-1}{\frac{v^{(\ell+1)}(\ell+1)}{(\ell+1)!}\sum_{\sum{iy_{k,i}}=n-\ell-1}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n-\ell-1}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n-\ell-1}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}\\\
&=(n-1)!\sum_{\ell=1}^{n}{\frac{v^{(\ell)}}{\ell!}\ell\sum_{\sum{iy_{k,i}}=n-\ell}{\binom{\sum{y_{k,i}}}{y_{k,1},\ldots,y_{k,n-\ell}}\frac{(-1)^{\sum{y_{k,i}}}}{v^{\sum{y_{k,i}}+1}}\prod_{i=1}^{n-\ell}{\left[\frac{v^{(i)}}{i!}\right]^{y_{k,i}}}}}.\end{split}$

Similar to what was done in the proof of Theorem 2.1, we defined an extension
$(y_{k,1},\cdots,y_{k,n})$ of each partition $(y_{k,1},\cdots,y_{k,n-\ell})$
such that $y_{k,n-\ell+1}=\cdots=y_{k,n}=0$. Now, for every $\ell\in[1,n]$,
let us associate with each partition $(y_{k,1},\ldots,y_{k,n})$, the partition
$(Y_{k,1},\ldots,Y_{k,n})$ such that

$\begin{cases}Y_{k,i}=y_{k,i}+1,&\text{for }i=\ell,\\\
Y_{k,i}=y_{k,i},&\text{otherwise}.\end{cases}$

Notice that $\sum{Y_{k,i}}=\sum{y_{k,i}}+1$ and that $\sum{iY_{k,i}}=n$.
Hence, we can write

$\begin{split}(\ln
v)^{(n)}&=(n-1)!\sum_{\ell=1}^{n}{\ell\sum_{\sum{iY_{k,i}}=n}{\binom{\sum{Y_{k,i}-1}}{Y_{k,1},\ldots,Y_{k,\ell}-1,\ldots,Y_{k,n}}\frac{(-1)^{\sum{Y_{k,i}}-1}}{v^{\sum{Y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{Y_{k,i}}}}}\\\
&=(n-1)!\sum_{\sum{iY_{k,i}}=n}{\frac{(-1)^{\sum{Y_{k,i}}-1}}{v^{\sum{Y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{Y_{k,i}}}}\sum_{\ell=1}^{n}{\ell\binom{\sum{Y_{k,i}-1}}{Y_{k,1},\ldots,Y_{k,\ell}-1,\ldots,Y_{k,n}}}\\\
&=(n-1)!\sum_{\sum{iY_{k,i}}=n}{\binom{\sum{Y_{k,i}}}{Y_{k,1},\ldots,Y_{k,n}}\frac{(-1)^{\sum{Y_{k,i}}-1}}{(\sum{Y_{k,i}})!v^{\sum{Y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{Y_{k,i}}}}\sum_{\ell=1}^{n}{\ell
Y_{k,\ell}}\\\
&=n!\sum_{\sum{iY_{k,i}}=n}{\binom{\sum{Y_{k,i}}}{Y_{k,1},\ldots,Y_{k,n}}\frac{(-1)^{\sum{Y_{k,i}}-1}}{(\sum{Y_{k,i}})!v^{\sum{Y_{k,i}}}}\prod_{i=1}^{n}{\left[\frac{v^{(i)}}{i!}\right]^{Y_{k,i}}}}.\end{split}$

∎

## References

  * Andrews [1998] Andrews, G. E. (1998).  The theory of partitions.  Number 2. Cambridge university press.
  * Barabesi [2020] Barabesi, L. (2020).  The computation of the probability density and distribution functions for some families of random variables by means of the wynn-$\rho$ accelerated post-widder formula.  Communications in Statistics-Simulation and Computation, 49(5):1333–1351.
  * [3] BASU, R.  A new formula for investigating delay integro-differential equations using differential transform method involving quotient of two functions.
  * Cao [2017] Cao, R. (2017).  Hierarchical stochastic modelling in multistable perception.
  * El Haddad [2021a] El Haddad, R. (2021a).  Multiple sums and partition identities.  arXiv preprint arXiv:2102.00821.
  * El Haddad [2021b] El Haddad, R. (2021b).  Recurrent sums and partition identities.  arXiv preprint arXiv:2101.09089.
  * Furrer [2008] Furrer, E. M. (2008).  Asymptotic behavior of a continuous approximation to the kriging weighting function.  Technical Note NCAR/TN476+ STR.
  * Gerrish [1980] Gerrish, F. (1980).  64.2 a useless formula?  The Mathematical Gazette, 64(427):52–52.
  * Liu [2014] Liu, Y. (2014).  Asymptotic moments of symmetric self-normalized sums.  Scientiae Mathematicae Japonicae, 77(1):59–67.
  * Mahmudov and Matar [2017] Mahmudov, N. and Matar, M. M. (2017).  Existence of mild solution for hybrid differential equations with arbitrary fractional order.  TWMS Journal of Pure and Applied Mathematics, 8(2):160–169.
  * Qi [2014] Qi, F. (2014).  Explicit formulas for computing bernoulli numbers of the second kind and stirling numbers of the first kind.  Filomat, 28(2):319–327.
  * Rafeiro and Samko [2010] Rafeiro, H. and Samko, S. (2010).  Characterization of the variable exponent bessel potential spaces via the poisson semigroup.  Journal of mathematical analysis and applications, 365(2):483–497.
  * Shieh and Verghese [1967] Shieh, P. and Verghese, K. (1967).  A general formula for the nth derivative of 1/f(x).  American Mathematical Monthly, 74:1239.
  * Xenophontos [2007] Xenophontos, C. (2007).  A formula for the nth derivative of the quotient of two functions.  Communicated by Masanobu Taniguchi, 70.


# Generalizability and Application of the Skin Reflectance Estimate Based on
Dichromatic Separation (SREDS)

Joseph Drahos  Richard Plesh  Keivan Bahmani  Mahesh Banavar  Stephanie
Schuckers 111Department of Electrical and Computer Engineering, 8 Clarkson
Ave, Potsdam, NY,
{drahosj, pleshro, bahmank, mbanavar, sschucke} @clarkson.edu
This material is based upon work supported by the Center for Identification
Technology Research and the National Science Foundation (NSF) under Grant
No.$1650503$.

###### Abstract

Face recognition (FR) systems have become widely used and readily available in
recent history. However, differential performance between certain demographics
has been identified within popular FR models. Skin tone differences between
demographics can be one of the factors contributing to the differential
performance observed in face recognition models. Skin tone metrics provide an
alternative to self-reported race labels when such labels are lacking or
completely not available e.g. large-scale face recognition datasets. In this
work, we provide a further analysis of the generalizability of the Skin
Reflectance Estimate based on Dichromatic Separation (SREDS) against other
skin tone metrics and provide a use case for substituting race labels for
SREDS scores in a privacy-preserving learning solution. Our findings suggest
that SREDS consistently creates a skin tone metric with lower variability
within each subject and SREDS values can be utilized as an alternative to the
self-reported race labels at minimal drop in performance. Finally, we provide
a publicly available and open-source implementation of SREDS to help the
research community. Available at https://github.com/JosephDrahos/SREDS

###### keywords:

Face Recognition, Privacy-Preserving, Feature Unlearning, Skin Reflectance.

##  1 Introduction

Face recognition systems are increasingly used as a form of biometric
authentication for many new and existing systems. Research on the differential
performance between demographics is an important topic of study to mitigate
bias and ensure fairness [dFPM22, Ho22]. Modern facial recognition systems use
deep learning pipelines to take an image of a person’s face and create a
unique template for that person. In such systems, the demographic information
of a dataset is needed to assess or mitigate the differential performance of a
particular face recognition algorithm. However, many of the large-scale
datasets which have been aggregated from public images on the internet and
used to train and benchmark face recognition networks lack self-reported race
labels. Additionally, the large scale of such datasets makes it impractical
and expensive to efficiently label demographics by human annotators. As a
result, methods to automatically label a dataset can provide a valuable asset.

Our research focuses on the intersection of privacy preservation, bias
mitigation, and skin tone metrics. We present our analysis of the Skin
Reflectance Estimate based on Dichromatic Separation (SREDS) skin tone metric
from [Ba21]. SREDS is a continuous skin tone metric that can be used to
automatically label skin tones on face datasets. Our goal is to evaluate
SREDS’ ability to label datasets compared to other skin tone metrics, assess
the generalizability of SREDS on unseen data, and demonstrate an application
of SREDS using a sensitive information removal approach when race labels are
not available.

##  2 Background

###  2.1 Skin Tone Metrics

Previous methods formulated for generating a metric for subject skin tones to
more accurately describe skin color are listed as follows: Fitzpatrick Skin
Type (FST), Monk Skin Tone (MST) Scale, Individual Typology Angle (ITA), and
Relative Skin Reflectance (RSR) [Fi88, Mo19, CCH91, Co19]. FST and its
successor MST require a manual calculation from a survey of the subject, while
ITA and RSR can be computed automatically via an algorithm. RSR was created to
analyze skin tone for a specific dataset by fitting a Principal Component
Analysis (PCA) model to the RGB space of the dataset. RSR is not resistant to
changes in lighting and is specific to a particular dataset. The need for a
skin tone metric that can be computed automatically and is more resistant to
changes in lighting prompted the research that led to the Skin Reflectance
Estimate based on a Dichromatic Separation (SREDS) [Ba21]. SREDS aims to
decompose patches of skin into specular and diffuse components using the
dichromatic reflectance model. A Kernal Principal Component Analysis (KPCA) is
fit onto the diffuse components extracted from the dataset, resulting in a
data-driven skin tone metric.

###  2.2 Bias Mitigation

The inclusion of demographic information in a dataset is to observe and
attempt to eliminate the differential performance between demographic groups
in FR models. Differing methods of bias mitigation have been attempted and
documented at the feature, comparison, and post-comparison levels. A method of
bias mitigation at the feature level is the triplet mining approach of [Se22]
which used a triplet loss for discrimination-aware learning. Closely related
triplets are mined based on race information to try and train a new
representation that mitigates biased learning within the face embedding space
of a pre-trained model. At the comparison level, a learning classifier method
reduces ethnic bias by introducing group and individual fairness to the
decision process at the cost of matching performance [Te20c]. At the post-
comparison level, an unsupervised method of score normalization has been
presented to reduce bias between ethnic groups while increasing the
performance of the system [Te20b].

###  2.3 Soft Biometric Privacy Preservation

Soft biometric information such as gender, race, age, etc. is stored within
the templates created from FR systems and can be extracted without the user’s
consent [Te20a]. Methods of privacy preservation have been studied and
introduced to protect users’ sensitive information. The efforts in[OR15]
produced a technique that morphed the input face with another face to mask the
soft biometrics while maintaining matching performance. Another technique that
added a perturbing element to the initial face image that would mask sensitive
information while maintaining performance is [MR17]. Information removal
networks attempt to remove sensitive information from the feature embedding
space of the FR deep network. These methods require complex loss functions to
maintain the performance accuracy of the network while also suppressing the
racial information from the learned space, as performed in [Xu18]. A method
that combines the methods from [Se22, Xu18] and was used within this research
is [Mo20], which attempts to maintain the inter-identity distance using
triplet loss and simultaneously unlearn222The term unlearn will be used
throughout the paper in the same context as introduced in the literature
[Mo20]. the facial features used to differentiate between demographic classes.

(a) Intra-subject variation between diffuse and specular components extracted
using SREDS.

(b) SensitiveNets model implementation

Figure 1:

##  3 Methodology

###  3.1 Skin Tone Metrics Assessment

The skin tone metrics outlined in section 2.1 will be used as a baseline to
compare the previously developed methods to the performance of the SREDS
measure. Individual typology angle (ITA) is a type of colorimetric analysis
designed to measure acquired tanning [CCH91]. An RGB image is converted into
CIE-Lab space [CF97], as follows: (1) the ‘$L$’ component which quantifies
luminance, (2) the ‘$a$’ component - absence or presence of redness, and (3)
the ‘$b$’ component - yellowness. Using the ‘$L$’ and ‘$b$’ components, Pixel-
wise ITA value, in degrees, can be estimated throughout an image as:

$ITA=\frac{{\rm arctan}(L-50)}{b}*\frac{180}{\pi}.$ (1)

To find suitable skin pixels in the image, a landmark extractor based on Dlib
is used to detect the forehead, left cheek, and right cheek facial regions
[Ki06]. For each facial region, ITA is computed over each pixel and smoothed
using an averaging filter. The mode from each region’s resulting values is
averaged to result in a single skin tone estimate for a face.

Relative Skin Reflectance (RSR) is a process designed to relate the physical
properties of the skin to the performance of facial recognition [Co19]. The
pipeline works by removing the confounding effects of imaging artifacts on
skin pixels and fitting a line in the direction of the greatest variance in
the RGB color space using PCA. The resulting metric is related to the skin
tone of each subject relative to the rest of the photos in the dataset.
Assumptions include consistent lighting, the same acquisition camera, and
constant background. As a further limitation, the metric only indicates where
a subject lies regarding net skin reflectance relative to the other subjects
in the dataset, rather than an absolute measure.

The process to compute SREDS begins by extracting patches of skin from the
forehead, right, and left cheeks using Dlib landmarks of each face image.
Using the dichromatic reflection model as a guide, Non-Negative Matrix
Factorization (NNMF) is used to estimate the diffuse and specular components
of the selected skin patches. KPCA is utilized on the extracted diffuse
components to learn a skin tone gradient across the dataset. The averaged
value of the first principal components of the extracted diffuse bases for a
particular face defines that person’s SREDS score. The KPCA model used for
SREDS is data-driven, so the generalizability of the KPCA model onto unseen
datasets is a point of interest within this study. A full description of the
extraction of SREDS is found in [Ba21].

###  3.2 Datasets

For our experiments, we selected datasets that included demographic
information of subjects across race, age, gender, orientation, and lighting.
We utilized CMU Multi-PIE, MEDS-II, and Morph-II datasets [SBB03, Fo11, RT06].
Multi-PIE contains 750,000 sample images from 337 subjects images under 15
viewpoints with 19 illumination conditions. We selected three viewpoints (14
0, 05 1, 05 0) where full views of the face were captured for our testing,
which reduced our sample images to 150,668 from 314 subjects. MEDS-II contains
only 836 sample images from 425 subjects imaged in a controlled mugshot
setting. Morph-II is a dataset from a longitudinal study that contains 55,063
sample images from 13,000 subjects within a controlled setting over 5 years.
While MEDS-II and Morph-II datasets include uncontrolled illumination,
Multipie includes controlled illumination samples. ITA, RSR, and SREDS scores
were generated for all samples of each dataset.

Figure 2: Distribution of SREDS Scores for Morph-II dataset separated by Race.
Subjects with specific SREDS scores are shown for the range across the
distribution.

###  3.3 Cross-Dataset Analysis

In prior work [Ba21], the intra-subject variance was used as a metric to
describe the variance of a specific subject’s skin tone score across multiple
samples. The low intra-subject variance shows the metric can produce a
consistent value of the same subject independent of external conditions. An
example of intra-subject variation can be seen in figure 1a. We evaluate and
compare the intra-subject variance across all of our evaluation datasets and
compare it to other methods. In addition, we test the generalizability of the
SREDS metric by calibrating the skin tone gradient on one dataset and applying
it to another, thereby testing its robustness to unseen datasets. The same
experiment is run using the RSR PCA models for comparison. ITA does not have a
training component and is only reported per dataset.

###  3.4 SREDS Agnostic Facial Recognition Model

To show the potential of SREDS for use in the replacement of race labels, we
compared the outcome performance of SREDS versus ground-truth race labels when
incorporated into SensitiveNets, a sensitive information removal network
[Mo20]. SensitiveNets provides a novel privacy-preserving neural network
feature representation to suppress the sensitive information of a learned
space while maintaining the utility of the data. We reimplemented the
sensitive removal network as our model for analysis of the suppression of race
and skin tone. A diagram of our model is seen in figure 1b. SensitiveNets
contains sensitive information removal dense layers added on top of a pre-
trained face recognition backbone. Within our testing, we used a Resnet50
model pre-trained on VGGFace2 as the backbone, consistent with the cited
literature [Mo20, Ca18, Xi]. The model’s loss function requires a race
classifier that acts as the sensitive information detector. The softmax
probability from this detector describes the amount of racial information
present within a subject’s template and the goal of the loss function is to
remove the sensitive race information and trend the classifier towards 50%
accuracy. In our experiments, this classifier is either trained on race labels
or SREDS scores binned into predetermined groups. The sensitive information
removal $\varphi$ layers are then added and trained sequentially using an
adversarial approach of triplet loss and an adversarial sensitivity
regularizer loss which reduces the amount of sensitive race information from
the embedding space. An in-depth look at the model and loss function can be
found in the SensitiveNets literature [Mo20].

##  4 Experiment Results

Our experiments were performed to analyze how the consistency of SREDS
performed relative to other skin tone metrics and the outcome of replacing
race labels with SREDS-generated labels in a privacy preservation method.

###  4.1 Cross-Dataset Analysis Results

We performed the cross-dataset analysis of the two skin tone metrics described
in Section 2 and SREDS across the three datasets listed in Section 3. We
generated ITA, RSR, and SREDS for all subjects from the mentioned datasets. As
part of background normalization, RSR assumed consistent lighting, the same
acquisition camera, and a constant background. Only the Multi-PIE dataset
meets all conditions. However, due to the lack of constant background in MEDS-
II and MORPH-II, the background normalization step was bypassed for these
datasets. ITA is a non-trainable method so we collected the ITA values from
each subject of each dataset. To test SREDS consistency on unseen data we used
the Kernal Principal Component Analysis (KPCA) fit to one dataset’s diffuse
components and used it to transform another dataset’s diffuse components. The
same process was recreated using the RSR PCA models on the same datasets’
selected skin pixel values in order to compare these two methods.

| Testing Dataset
---|---
| Morph-II | MEDS-II | Multi-Pie (Mugshot)
Training Dataset | SREDS | RSR | ITA | SREDS | RSR | ITA | SREDS | RSR | ITA
Morph-II | 0.419 | 0.539 | 0.645 | 0.681 | 0.493 | N/A | 0.157 | 0.468 | N/A
MEDS-II | 0.457 | 0.540 | N/A | 0.463 | 0.493 | 0.448 | 0.186 | 0.470 | N/A
Multi-Pie (Mugshot) | 0.399 | 0.538 | N/A | 0.674 | 0.493 | N/A | 0.138 | 0.304 | 0.401
Table 1: Cross dataset intra-subject variability analysis between SREDS, RSR,
and ITA skin tone metrics. Bolded values are the lowest recorded intra-subject
variability in that testing dataset. SREDS scores result in the least variable
metric from Morph-II and MultiPie datasets and the second least variable
metric in MEDS-II, behind ITA.

We computed the intra-subject variability of each dataset’s skin tone metrics
by calculating the standard deviation of each subject’s individual skin tone
measures and averaging across the dataset. The results of this analysis are
seen in Table 1 and suggest that the learning-based algorithms (RSR and SREDS)
perform better than ITA when evaluated on the dataset they are calibrated on.
Viewing our cross-dataset results, we observe that in larger datasets (Morph,
Multi-pie), SREDS outperforms both ITA and RSR even when calibrated on a
different dataset, suggesting the generalizability of this approach.

Figure 3: SensitiveNets Sensitive Information Classifier ROCs for both race
labels and SREDS scores before and after training on Morph-II dataset. The
goal of SensitiveNets training is for 50% classification accuracy. The
unlearned classifier accuracy for both the RDM and SDM is nearly 50%, which
shows SREDS scores and race labels perform similarly in this experiment.

###  4.2 Distribution of SREDS

To utilize SREDS by replacing race labels we needed a process to convert
continuous SREDS scores into discrete labels. To understand the distribution
of scores, the SREDS scores across the Morph-II dataset were plotted within
Figure 2. We split the dataset in half by the median SREDS score of -0.01 and
binned the subjects into low and high SREDS scores to create a discrete
labeling of the Morph-II dataset.

###  4.3 Comparison of Race Labels and SREDS in Sensitive Feature Unlearning

To see the effects of SREDS scores being used in place of self-reported race
labels, we implemented two SensitiveNets models. One model is trained using
the black and white subject race labels from the Morph-II dataset while the
second model is trained using the binned SREDS value for the same subjects.

Backbone | Classifier | Trained On | Tested On | ICA | FCA
---|---|---|---|---|---
Resnet50 | Race | Race Triplets | Morph | 0.985 | 0.47
Resnet50 | SREDS | SREDS Triplets | Morph | 0.937 | 0.48
Table 2: Sensitive Information Removal Network Experiment Results
ICA: Initial Classification Accuracy, FCA: Final Classification Accuracy (Goal
of sensitive information removal is for FCA to be 0.50)

The first model trained on race labels and the second model trained on SREDS
scores will be referred to as the Race Unlearned Model (RUM) and the SREDS
Unlearned Model (SUM) respectively. An outline of this testing plan is seen in
Table 2 with the initial and final classification accuracy of the
SensitiveNets classifiers. For both models, the sensitive information
classifier ROCs were calculated and shown in Figure 3.

(a) ROC comparison for baseline (Resnet50), RUM, and SUM models

(b) ROC comparison between baseline and RUM model for White and Black
subjects.

(c) ROC comparison between baseline and SUM model for White and Black
subjects.

Figure 4: Comparison of biometric performance (matching) ROCs of baseline,
RDM, and SDM, categorized by race labels on Morph-II dataset. Shows RDM and
SDM suffer from a similar drop in matching performance when race or skin tone
information is removed, respectively.

The two trained SensitiveNets models matching performances are compared to the
baseline Resnet50 matching performance to evaluate the results of the training
on matching performance in Figure 4. The feature unlearning experiments to
preserve privacy show a similar drop in performance between training with race
labels and training with SREDS scores. The results suggest only a small
(0.027) difference between the True Positive Rate (TPR) of RUM and SUM at
$10^{-3}$ False Positive Rate (FPR).

##  5 Conclusions

The existing feature unlearning methods in FR rely on large-scale and
expensive-to-collect demographically-labeled datasets. Within this study, we
demonstrate the ability of SREDS to mitigate this reliance by automatically
extracting consistent skin tone information from face images. We have shown
that SREDS outperforms other available skin tone metrics in producing
continuous and less-variable skin tone estimates while generalizing well to
unseen data. We have presented an application of extracted SREDS scores in the
absence of race labels in a feature unlearning method and shown that SREDS
could be used as a replacement.

###  5.1 Limitations and Future Work

Limitations of this work include our analysis of only black and white subjects
due to the under-representation of other races in our datasets. This led to us
only using two SREDS bins when categorizing our datasets to match the binary
race labels. We tested using only one face matcher within our privacy-
preserving method and have not seen how different networks affect our results.
A limitation of using skin tone as a way to label datasets is that skin tone
does not encapsulate the entirety of a self-reported race label. Skin tone is
one physical characteristic that makes up race and cannot be used as an exact
replacement.

Future work planned includes further analysis of the mapping of SREDS to
multi-race demographic information and its use in different downstream
biometric tasks, recreating our experiments with addition face matches [De22],
and attempting a bias mitigation solution using SREDS scores and evaluating
using fairness metrics [dFPM22, Ho22] on an even larger scale dataset (BUPT-
Globalface) [WZD21].

## References

  * [Ba21] Bahmani, Keivan; Plesh, Richard; Sahu, Chinmay; Banavar, Mahesh; Schuckers, Stephanie: SREDS: A dichromatic separation based measure of skin color.  In: 2021 IEEE International Workshop on Biometrics and Forensics (IWBF).  IEEE, pp. 1–6, 2021.
  * [Ca18] Cao, Qiong; Shen, Li; Xie, Weidi; Parkhi, Omkar M.; Zisserman, Andrew: VGGFace2: A dataset for recognising faces across pose and age, 2018.
  * [CCH91] Chardon, A; Cretois, I; Hourseau, C: Skin colour typology and suntanning pathways.  Int J Cosmet Sci, 13(4):191–208, August 1991.
  * [CF97] Connolly, C.; Fleiss, T.: A study of efficiency and accuracy in the transformation from RGB to CIELAB color space.  IEEE Transactions on Image Processing, 6(7):1046–1048, 1997.
  * [Co19] Cook, Cynthia M.; Howard, John J.; Sirotin, Yevgeniy B.; Tipton, Jerry L.; Vemury, Arun R.: Demographic Effects in Facial Recognition and Their Dependence on Image Acquisition: An Evaluation of Eleven Commercial Systems.  IEEE Transactions on Biometrics, Behavior, and Identity Science, 1(1):32–41, 2019.
  * [De22] Deng, Jiankang; Guo, Jia; Yang, Jing; Xue, Niannan; Kotsia, Irene; Zafeiriou, Stefanos: ArcFace: Additive Angular Margin Loss for Deep Face Recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):5962–5979, oct 2022.
  * [dFPM22] de Freitas Pereira, Tiago; Marcel, Sébastien: Fairness in Biometrics: A Figure of Merit to Assess Biometric Verification Systems.  IEEE Transactions on Biometrics, Behavior, and Identity Science, 4(1):19–29, 2022.
  * [Fi88] Fitzpatrick, T B: The validity and practicality of sun-reactive skin types I through VI.  Arch Dermatol, 124(6):869–871, June 1988.
  * [Fo11] Founds, Andrew; Orlans, Nick; Genevieve, Whiddon; Watson, Craig: NIST Special Databse 32 - Multiple Encounter Dataset II (MEDS-II), 07 2011.
  * [Ho22] Howard, John J.; Laird, Eli J.; Sirotin, Yevgeniy B.; Rubin, Rebecca E.; Tipton, Jerry L.; Vemury, Arun R.: Evaluating Proposed Fairness Models for Face Recognition Algorithms, 2022.
  * [Ki06] King, Davis: Dlib C++ library, 2006.
  * [Mo19] Monk, Ellis: Monk Skin Tone Scale, 2019.
  * [Mo20] Morales, Aythami; Fierrez, Julian; Vera-Rodriguez, Ruben; Tolosana, Ruben: SensitiveNets: Learning Agnostic Representations with Application to Face Images, 2020.
  * [MR17] Mirjalili, Vahid; Ross, Arun: Soft biometric privacy: Retaining biometric utility of face images while perturbing gender.  In: 2017 IEEE International Joint Conference on Biometrics (IJCB).  pp. 564–573, 2017.
  * [OR15] Othman, Asem; Ross, Arun: Privacy of Facial Soft Biometrics: Suppressing Gender But Retaining Identity.  In (Agapito, Lourdes; Bronstein, Michael M.; Rother, Carsten, eds): Computer Vision - ECCV 2014 Workshops.  Springer International Publishing, Cham, pp. 682–696, 2015.
  * [RT06] Ricanek, K.; Tesafaye, T.: MORPH: a longitudinal image database of normal adult age-progression.  In: 7th International Conference on Automatic Face and Gesture Recognition (FGR06).  pp. 341–345, 2006.
  * [SBB03] Sim, T.; Baker, S.; Bsat, M.: The CMU pose, illumination, and expression database.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12):1615–1618, 2003.
  * [Se22] Serna, Ignacio; Morales, Aythami; Fierrez, Julian; Obradovich, Nick: Sensitive loss: Improving accuracy and fairness of face representations with discrimination-aware deep learning.  Artificial Intelligence, 305:103682, 2022.
  * [Te20a] Terhörst, Philipp; Fährmann, Daniel; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan: Beyond Identity: What Information Is Stored in Biometric Face Templates?, 2020.
  * [Te20b] Terhörst, Philipp; Kolf, Jan Niklas; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan: Post-Comparison Mitigation of Demographic Bias in Face Recognition Using Fair Score Normalization, 2020.
  * [Te20c] Terhörst, Philipp; Tran, Mai Ly; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan: Comparison-Level Mitigation of Ethnic Bias in Face Recognition.  In: 2020 8th International Workshop on Biometrics and Forensics (IWBF).  pp. 1–6, 2020.
  * [WZD21] Wang, Mei; Zhang, Yaobin; Deng, Weihong: Meta Balanced Network for Fair Face Recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2021.
  * [Xi] Xie, Weidi: Weidixie/Keras-VGGFACE2-RESNET50.
  * [Xu18] Xu, Depeng; Yuan, Shuhan; Zhang, Lu; Wu, Xintao: FairGAN: Fairness-aware Generative Adversarial Networks, 2018.