P_{k}^{d},P_{j}^{d}\right\rangle=\delta_{kj}\frac{|\mathcal{S}^{d-1}|}{\nu_{d}(k)\,|\mathcal{S}^{d-2}|},\quad\forall
k,j\in\mathbb{N}_{0}.$

In view of (2.1) and (2.2) we have

$Q(t)=\sum_{j=0}^{\beta}\frac{(c_{j,\,d}(\beta))^{2}}{\nu_{d}(j)}P_{j}^{\,d}(t)-\psi_{d}^{2}(\beta),\quad
t\in[-1,1].$

We calculate for $k\leq\beta$ with $P_{0}^{d}\equiv 1$

$\displaystyle\int_{-1}^{1}P_{k}^{\,d}(t)Q(t)(1-t^{2})^{(d-3)/2}$
$\displaystyle=\sum_{j=0}^{\beta}\frac{(c_{j,\,d}(\beta))^{2}}{\nu_{d}(j)}\left\langle
P_{k}^{d},P_{j}^{d}\right\rangle-\int_{-1}^{1}P_{k}^{\,d}(t)\psi_{d}^{2}(\beta)(1-t^{2})^{(d-3)/2}\text{d}t$
$\displaystyle=\frac{|\mathcal{S}^{d-1}|}{|\mathcal{S}^{d-2}|}\sum_{j=0}^{\beta}\left(\frac{c_{j,\,d}(\beta)}{\nu_{d}(j)}\right)^{2}\delta_{kj}-\psi_{d}^{2}(\beta)\left\langle
P_{k}^{d},P_{0}^{d}\right\rangle$
$\displaystyle=\frac{|\mathcal{S}^{d-1}|}{|\mathcal{S}^{d-2}|}\left[\left(\frac{c_{k,\,d}(\beta)}{\nu_{d}(k)}\right)^{2}-\delta_{k0}\psi_{d}^{2}(\beta)\right].$

Thereby we obtain

$\lambda_{k}=\frac{|\mathcal{S}^{d-2}|}{|\mathcal{S}^{d-1}|}\int_{-1}^{1}P_{k}^{\,d}(t)Q(t)(1-t^{2})^{(d-3)/2}\text{d}t=\left(\frac{c_{k,\,d}(\beta)}{\nu_{d}(k)}\right)^{2}-\delta_{k0}\psi_{d}^{2}(\beta),\quad
k\leq\beta.$

Evidently is $\lambda_{k}=0$ for $k>\beta$ by the orthogonality of the
Legendre polynomials. Consider the case $k=0$. With $\nu_{d}(0)=1$ we already
know that

$\lambda_{k}=(c_{0,\,d}(\beta))^{2}-\psi_{d}^{2}(\beta)=(c_{0,\,d}(\beta)-\psi_{d}(\beta))(c_{0,\,d}(\beta)+\psi_{d}(\beta)).$

If $\beta$ is odd, then $\psi_{d}(\beta)=c_{0,\,d}(\beta)=0$ due to (2.1) and
Proposition A.8. Thus let $\beta$ be even. Then the Funk–Hecke-Theorem for
$1\in\mathcal{H}_{0}(\mathcal{S}^{d-1})$, equation (A.5), and the
orthogonality of the Legendre polynomials yield

$\displaystyle\psi_{d}(\beta)$
$\displaystyle=\mathbb{E}({b}^{\top}\vspace{-0.05cm}U)^{\,\beta}=\frac{1}{|\mathcal{S}^{d-1}|}\int_{\mathcal{S}^{d-1}}({b}^{\top}\vspace{-0.05cm}\omega)^{\,\beta}\,\text{d}\sigma(\omega)=\frac{|\mathcal{S}^{d-2}|}{|\mathcal{S}^{d-1}|}\int_{-1}^{1}t^{\,\beta}(1-t^{2})^{(d-3)/2}\,\text{d}t$
$\displaystyle=\frac{|\mathcal{S}^{d-2}|}{|\mathcal{S}^{d-1}|}\sum_{j=0}^{\beta}c_{j,\,d}(\beta)\int_{-1}^{1}P_{j}^{\,d}(t)(1-t^{2})^{(d-3)/2}\,\text{d}t=\frac{|\mathcal{S}^{d-2}|}{|\mathcal{S}^{d-1}|}\sum_{j=0}^{\beta}c_{j,\,d}(\beta)\left\langle
P_{j}^{d},P_{0}^{d}\right\rangle$
$\displaystyle=\frac{|\mathcal{S}^{d-2}|}{|\mathcal{S}^{d-1}|}\sum_{j=0}^{\beta}c_{j,\,d}(\beta)\frac{\delta_{j\,0}|\mathcal{S}^{d-1}|}{\nu_{d}(0)\,|\mathcal{S}^{d-2}|}=c_{0,\,d}(\beta).$

  2. ii)

Let $\\{\phi_{k,1},\ldots,\phi_{k,\nu_{d}(k)}\\}$ be an orthonormal basis of
$\mathcal{H}_{k}(\mathcal{S}^{d-1})$. Then
$\bigcup_{k=0}^{\infty}\\{\phi_{k,1},\ldots,\phi_{k,\nu_{d}(k)}\\}$ is an
orthonormal basis of $L^{2}(\mathcal{S}^{d-1},\text{d}\sigma)$ and we get the
unique series expansion for $f\in L^{2}(\mathcal{S}^{d-1},\text{d}\sigma)$

$f\overset{L^{2}}{=}\sum_{k=0}^{\infty}\Psi_{k},$ (C.1)

with $\Psi_{k}=\sum_{j=1}^{\nu_{d}(k)}\left\langle
f,\phi_{k,j}\right\rangle_{L^{2}}\phi_{k,j}\in\mathcal{H}_{k}(\mathcal{S}^{d-1})$,
compare with (A.3). Set $a_{k,j}=\left\langle
f,\phi_{k,j}\right\rangle_{L^{2}}$ for all $j\in\\{1,\ldots,\nu_{d}(k)\\}$,
$k\in\mathbb{N}_{0}$. It follows for $x\in\mathcal{S}^{d-1}$

$\displaystyle K_{\beta}f(x)$
$\displaystyle=\frac{1}{|\mathcal{S}^{d-1}|}\int_{\mathcal{S}^{d-1}}\rho_{\beta}(\omega,x)f(\omega)\,\text{d}\sigma(\omega)=\frac{1}{|\mathcal{S}^{d-1}|}\left\langle\,\rho_{\beta}(\cdot,x),f\right\rangle_{L^{2}}$
$\displaystyle=\frac{1}{|\mathcal{S}^{d-1}|}\sum_{k=0}^{\infty}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}\left\langle\,\rho_{\beta}(\cdot,x),\phi_{k,j}\right\rangle_{L^{2}}\overset{i)}{=}\sum_{k=0}^{\infty}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}\lambda_{k}\phi_{k,j}(x)\mathds{1}_{\\{k\,\leq\,\beta\\}}$
$\displaystyle=\sum_{k=1}^{\beta}\lambda_{k}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}\phi_{k,j}(x).$

Hence, $K_{\beta}$ is a finite-rank operator, and it has the representation

$K_{\beta}f(x)=\sum_{k=1}^{\beta}\lambda_{k}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}\phi_{k,j}(x),\quad
x\in\mathcal{S}^{d-1}.$ (C.2)

  3. iii)

Let $\lambda\in\mathbb{C}$,
$\lambda\notin\\{0\\}\cup\\{\lambda_{k}\;|\;k=1,\ldots,\beta\\}$, and consider
the equation $\lambda f-K_{\beta}f=0$ für $f\in
L^{2}(\mathcal{S}^{d-1},\text{d}\sigma)$ as in (C.1). Observe for
$i\in\\{1,\ldots,\nu_{d}(m)\\}$, $m\in\\{1,\ldots,\beta\\}$

$\displaystyle\lambda\left\langle f,\phi_{m,i}\right\rangle_{L^{2}}$
$\displaystyle=\left\langle
K_{\beta}f,\phi_{m,i}\right\rangle_{L^{2}}=\sum_{k=1}^{\beta}\lambda_{k}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}\left\langle\phi_{k,j},\phi_{m,i}\right\rangle_{L^{2}}$
$\displaystyle=\sum_{k=1}^{\beta}\lambda_{k}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}\delta_{mk}\delta_{ij}=\sum_{k=1}^{\beta}\lambda_{k}\delta_{mk}a_{k,i}=\lambda_{m}a_{m,i}=\lambda_{m}\left\langle
f,\phi_{m,i}\right\rangle_{L^{2}}.$

Since $\lambda\neq\lambda_{m}$, it immediately follows that $\left\langle
f,\phi_{m,i}\right\rangle_{L^{2}}=0$. Hence, $\left\langle
f,\phi_{m,i}\right\rangle_{L^{2}}=0$ for all $i\in\\{1,\ldots,\nu_{d}(m)\\}$,
$m\in\\{1,\ldots,\beta\\}$ and therewith $K_{\beta}f=0$. We conclude $\lambda
f=0$ and, since $\lambda\neq 0$, it follows that $f=0$. This, in turn, means
that $\lambda$ is a resolvent point.

  4. iv)

With $f\in L^{2}(\mathcal{S}^{d-1},\text{d}\sigma)$ like in (C.1) and by (C.2)
we conclude

$\langle
K_{\beta}f,f\rangle_{L^{2}}=\sum_{k=1}^{\beta}\lambda_{k}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}\langle\phi_{k,j},f\rangle_{L^{2}}=\sum_{k=1}^{\beta}\lambda_{k}\sum_{j=1}^{\nu_{d}(k)}a_{k,j}^{2}\geq
0,$

since all eigenvalues are non-negative.

∎

Proof of Proposition 2.4.

###### Proof.

We prove the claim only for the case of $\beta$ odd as the other case can be
done analogously. Since a centred Gaussian process is defined by the
covariance kernel, we merely have to show that the covariance kernels of
$Z_{\beta}(\cdot)$ and the process on the right-hand side, denoted by
$Y(\cdot)$, coincide. Clearly, after some calculations we first see that
$Y(\cdot)$ is a centred Gaussian process. Furthermore,

$\displaystyle\mathbb{E}Y(b)Y(c)$
$\displaystyle=|\mathcal{S}^{d-1}|\sum_{\begin{subarray}{c}k,m=1\\\
k,m\textrm{
odd}\end{subarray}}^{\beta}\sqrt{\lambda_{k}\lambda_{m}}\sum_{j=1}^{\nu_{d}(k)}\sum_{i=1}^{\nu_{d}(m)}\phi_{k,j}(b)\,\phi_{m,i}(c)\,\mathbb{E}N_{k,j}N_{m,i}$
$\displaystyle=|\mathcal{S}^{d-1}|\sum_{\begin{subarray}{c}k,m=1\\\
k,m\textrm{
odd}\end{subarray}}^{\beta}\sqrt{\lambda_{k}\lambda_{m}}\sum_{j=1}^{\nu_{d}(k)}\sum_{i=1}^{\nu_{d}(m)}\phi_{k,j}(b)\,\phi_{m,i}(c)\,\delta_{km}\delta_{ij}$
$\displaystyle=|\mathcal{S}^{d-1}|\sum_{\begin{subarray}{c}k=1\\\ k\textrm{
odd}\end{subarray}}^{\beta}\lambda_{k}\sum_{j=1}^{\nu_{d}(k)}\phi_{k,j}(b)\,\phi_{k,j}(c)=\rho_{\beta}(b,c),$

where the last equality follows from Mercer’s theorem, see [41], Theorem 2.10,
which is applicable due to our previously obtained results. Note that the
eigenvalues in (2.4) equal zero for even indices, if $\beta$ is odd, for the
occurring constants $c_{k,\,d}(\beta)$ are zero in these cases. Compare also
with Proposition A.8. ∎

Proof of Theorem 3.1.

###### Proof.

Lemma B.2 yields by Le Cams first Lemma, see [29] Proposition 5.2.1, that
$\mathbb{P}^{(n)}$ and $\mathbb{A}^{(n)}$ are mutually contiguous.
Straightforward calculations show for $b\in\mathcal{S}^{d-1}$ under
$\mathbb{P}^{(n)}$

  1. i)

$\displaystyle\lim_{n\rightarrow\infty}\mbox{Cov}\left(Z_{n,\beta}(b),\log
L_{n}\right)=S^{*}_{\beta}(b)$,

  2. ii)

for $l\in\mathbb{N}$, $a_{1},\ldots,a_{l}\in\mathcal{S}^{d-1}$ the
distribution of $(Z_{n,\beta}(a_{1}),\ldots,Z_{n,\beta}(a_{l}),\log
L_{n})^{\top}$ converges to

$\mbox{N}_{l+1}\left(\left(0,\ldots,0,-\frac{\tau^{2}}{2}\right)^{\top},\left(\begin{array}[]{cc}\Sigma_{a_{1},\ldots,a_{l}}&z\\\
z^{\top}&\tau^{2}\end{array}\right)\right),$

where
$\Sigma_{a_{1},\ldots,a_{l}}=\left(\rho_{\beta}\left(a_{j_{1}},a_{j_{2}}\right)\right)_{1\leq
j_{1},j_{2}\leq l}$ is a $l\times l$-matrix,
$z=\left(S^{*}_{\beta}(a_{1}),\ldots,S^{*}_{\beta}(a_{l})\right)^{\top}$ is a
$l$-dimensional vector, and $\tau$ is defined in Lemma B.2.

Le Cam’s third lemma shows that, under $\mathbb{A}^{(n)}$ the finite-
dimensional distributions of $Z_{n,\beta}$ converge weakly to the
corresponding distributions of the shifted Gaussian process
$Z_{\beta}+S^{*}_{\beta}$. Since $Z_{n,\beta}$ is tight under
$\mathbb{P}^{(n)}$ and $\mathbb{A}^{(n)}$ is contiguous to $\mathbb{P}^{(n)}$
we have tightness of $Z_{n,\beta}$ and
$Z_{n,\beta}\stackrel{{\scriptstyle\mathcal{D}}}{{\longrightarrow}}Z_{\beta}+S^{*}_{\beta}$
under $\mathbb{A}^{(n)}$. ∎

Proof of Theorem 5.1.

###### Proof.

For the first statement, we use [6], Theorem 7.2, and verify the two
conditions therein for the case of the approximate Bahadur slope, compare with
[5], Section 6. First of all, due to the strong law of large numbers we have

$\frac{1}{n}\sum_{j=1}^{n}({b}^{\top}\vspace{-0.05cm}U_{j})^{\,\beta}-\psi_{d}(\beta)\overset{\mathcal{\mathbb{P}_{\kappa}}\textrm{-f.s.}}{\rightarrow}\gamma_{\kappa}(b),\quad
b\in\mathcal{S}^{d-1}.$

Hence we immediately obtain

$\frac{\widetilde{T}_{n,\,\beta}}{\sqrt{n}}\overset{\mathcal{\mathbb{P}_{\kappa}}}{\rightarrow}\underset{b\in\mathcal{S}^{d-1}}{\max}\gamma_{\kappa}^{2}(b),\quad\kappa>0.$

Under $H_{0}$, Corollary 2.6 yields

$\widetilde{T}_{n,\,\beta}=\sqrt{T_{n,\,\beta}}\overset{\mathcal{D}}{\longrightarrow}\underset{b\in\mathcal{S}^{d-1}}{\max}|\,Z_{\beta}(b)\,|.$

With $F(t)=\mathbb{P}(\,\max_{b\in\mathcal{S}^{d-1}}|\,Z_{\beta}(b)\,|<t)$,
$t\in\mathbb{R}$, an application of [28], Corollary 3.2, gives

$\displaystyle\lim_{t\to\infty}\frac{\log(1-F(t))}{t^{2}}$
$\displaystyle=\lim_{t\to\infty}\frac{\log\mathbb{P}\left(\,\max_{b\in\mathcal{S}^{d-1}}|\,Z_{\beta}(b)\,|\geq
t\right)}{t^{2}}$
$\displaystyle=\lim_{t\to\infty}\frac{\log\mathbb{P}\left(\,\|\,Z_{\beta}(\cdot)\,\|_{\infty}\geq
t\right)}{t^{2}}=-\frac{1}{2\underset{b\in\mathcal{S}^{d-1}}{\max}\rho_{\beta}(b,b)}.$

Thus, the approximate Bahadur slope is

$c_{\widetilde{T}_{\beta}}^{\;a}(\kappa)=\frac{\underset{b\in\mathcal{S}^{d-1}}{\max}\gamma_{\kappa}^{2}(b)}{\underset{b\in\mathcal{S}^{d-1}}{\max}\rho_{\beta}(b,b)},\quad\kappa>0.$

For the second statement, we utilize again Theorem 7.2 in [6] and prove the
second condition for sufficiently small $\kappa>0$ with the help of [37],
Lemma 2.1 and Lemma 2.2. Let us consider, once again, the
$C(\mathcal{S}^{d-1},\mathbb{R})$-valued random elements
$W_{j}(b)=({b}^{\top}\vspace{-0.05cm}U_{j})^{\,\beta}-\psi_{d}(\beta)$,
$b\in\mathcal{S}^{d-1}$, which are centered under $H_{0}$ and due to their
compact support fulfill the integrability condition of [37], Lemma 2.1. Hence,
with $F_{n}(t)=\mathbb{P}_{0}\left(\widetilde{T}_{n,\,\beta}<t\right)$,
$t\in\mathbb{R}$, we obtain for sufficiently small $\epsilon>0$

$\displaystyle\lim_{n\to\infty}\frac{\log(1-F_{n}(\sqrt{n}\epsilon))}{n}$
$\displaystyle=\lim_{n\to\infty}\frac{\log\mathbb{P}_{0}\left(\widetilde{T}_{n,\,\beta}\geq\sqrt{n}\epsilon\right)}{n}$
$\displaystyle=\lim_{n\to\infty}\frac{\log\mathbb{P}_{0}\left(\,\left\|\frac{1}{n}\sum_{j=1}^{n}W_{j}(\cdot)\,\right\|_{\infty}\geq\epsilon\right)}{n}=-\frac{\epsilon^{2}}{2\underset{b\in\mathcal{S}^{d-1}}{\sup}\mathbb{E}W_{1}^{2}(b)}+o(\epsilon^{2}).$

According to the proof of Theorem 2.1, $W_{1}(\cdot)$ defines the covariance
kernel $\rho_{\beta}$ of the Gaussian process $Z_{\beta}(\cdot)$, so that
indeed

$\lim_{n\to\infty}\frac{\log(1-F_{n}(\sqrt{n}\epsilon))}{n}=-\frac{\epsilon^{2}}{2\underset{b\in\mathcal{S}^{d-1}}{\max}\rho_{\beta}(b,b)}+o(\epsilon^{2})$

for sufficiently small $\epsilon>0$. Due to the assumed condition (5.1), it
follows that

$\displaystyle\left|\gamma_{\kappa}(b)\right|$
$\displaystyle=\left|\int_{\mathcal{S}^{d-1}}({b}^{\top}\vspace{-0.05cm}x)^{\,\beta}f(x\,|\,\kappa)\,d\sigma(x)-\psi_{d}(\beta)\right|=\left|\int_{\mathcal{S}^{d-1}}({b}^{\top}\vspace{-0.05cm}x)^{\,\beta}\left(f(x\,|\,\kappa)-f(x\,|\,0)\right)\,d\sigma(x)\right|$
$\displaystyle\leq||f(\cdot\,|\,\kappa)-f(\cdot\,|\,0)||_{L^{1}}\overset{\kappa\rightarrow
0^{+}}{\longrightarrow}0.$

Lebesgue’s dominated convergence theorem shows that $\gamma_{\kappa}\in
C(\mathcal{S}^{d-1};\mathbb{R})$ for each $\kappa>0$, and the last calculation
justifies the uniform convergence of $\gamma_{\kappa}$ against $0$ on
$\mathcal{S}^{d-1}$ for $\kappa\rightarrow 0^{+}$. This implies
$\lim_{\kappa\rightarrow
0^{+}}\underset{b\in\mathcal{S}^{d-1}}{\max}\gamma_{\kappa}^{2}(b)=0$. Thus,
the exact Bahadur slope is

$c_{\widetilde{T}_{\beta}}(\kappa)=\frac{\underset{b\in\mathcal{S}^{d-1}}{\max}\gamma_{\kappa}^{2}(b)}{\underset{b\in\mathcal{S}^{d-1}}{\max}\rho_{\beta}(b,b)}+o\left(\underset{b\in\mathcal{S}^{d-1}}{\max}\gamma_{\kappa}^{2}(b)\right)$

for sufficiently small $\kappa>0$. Finally, notice that for each
$b\in\mathcal{S}^{d-1}$, we have

$\displaystyle\rho_{\beta}(b,b)$
$\displaystyle=\eta_{\beta}(b,b)-\psi_{d}^{2}(\beta)=\sum_{j=0}^{\beta}\frac{(c_{j,\,d}(\beta))^{2}}{\nu_{d}(j)}P_{j}^{\,d}({b}^{\top}\vspace{-0.05cm}b)-\psi_{d}^{2}(\beta)=\sum_{j=0}^{\beta}\frac{(c_{j,\,d}(\beta))^{2}}{\nu_{d}(j)}-\psi_{d}^{2}(\beta)$
$\displaystyle=\frac{(c_{0,\,d}(\beta))^{2}}{\nu_{d}(0)}+\sum_{j=1}^{\beta}\lambda_{j}\nu_{d}(j)-\psi_{d}^{2}(\beta)=\sum_{j=1}^{\beta}\lambda_{j}\nu_{d}(j).$

Here, the second and third equality follow from (2.1) and Remark A.6,
respectively. ∎

J. Borodavka,
Steinbuch Centre for Computing,
Karlsruhe Institute of Technology (KIT),
Zirkel 2, D-76131 Karlsruhe.
E-mail<EMAIL_ADDRESS>
B. Ebner,
Institute of Stochastics,
Karlsruhe Institute of Technology (KIT),
Englerstr. 2, D-76128 Karlsruhe.
E-mail<EMAIL_ADDRESS>


# Interactive Garment Recommendation with User in the Loop

Federico Becattini  University of SienaItaly<EMAIL_ADDRESS>,
Xiaolin Chen  e-mail<EMAIL_ADDRESS>School of Software, Shandong
UniversityChina ,  Andrea Puccia  University of FlorenceItaly
<EMAIL_ADDRESS>,  Haokun Wen  School of Computer Science and
Technology, Harbin Institute of Technology (Shenzhen)China
<EMAIL_ADDRESS>,  Xuemeng Song  School of Computer Science and
Technology, Shandong UniversityChina<EMAIL_ADDRESS>,  Liqiang Nie  School
of Computer Science and Technology, Harbin Institute of Technology
(Shenzhen)China<EMAIL_ADDRESS>and  Alberto Del Bimbo  University of
FlorenceFlorenceItaly<EMAIL_ADDRESS>
###### Abstract.

Recommending fashion items often leverages rich user profiles and makes
targeted suggestions based on past history and previous purchases. In this
paper, we work under the assumption that no prior knowledge is given about a
user. We propose to build a user profile on the fly by integrating user
reactions as we recommend complementary items to compose an outfit. We present
a reinforcement learning agent capable of suggesting appropriate garments and
ingesting user feedback so to improve its recommendations and maximize user
satisfaction. To train such a model, we resort to a proxy model to be able to
simulate having user feedback in the training loop. We experiment on the
IQON3000 fashion dataset and we find that a reinforcement learning-based agent
becomes capable of improving its recommendations by taking into account
personal preferences. Furthermore, such task demonstrated to be hard for non-
reinforcement models, that cannot exploit exploration during training.

Iterative recommendation, fashion, garment recommendation

††copyright: acmcopyright††journal: TOMM††journalyear: 2022††journalvolume:
1††journalnumber: 1††article: 1††publicationmonth: 1††price: 15.00††doi:
10.1145/3531017††ccs: Information systems Recommender systems

##  1\. Introduction

With the recent flourishing of the e-commerce fashion industry, the Internet
has accumulated plenty of clothing data, making it increasingly difficult for
users to seek their desired fashion items. Accordingly, fashion
recommendation, which aims to recommend suitable items for users, has
attracted considerable interest from both academia and industry due to its
substantial economic value. In fact, a surge of researches are dedicated to
exploiting fashion recommendation (Guan et al., 2022a; Yu et al., 2018).
However, in many cases, it is intractable for users to find the ideal items to
exactly convey their intent. Therefore, to allow the users to flexibly provide
feedback, interactive fashion recommendation merits our specific attention.

As a matter of fact, various methods (Sevegnani et al., 2022; Wu et al.,
2022b; Pal et al., 2023) have been presented for interactive fashion
recommendation, where the modification of the given visual image is regarded
as user feedback provided as a textual description. Despite the great success
achieved by these efforts, they mainly work on single-turn modifications, and
cannot further refine the retrieved items in multi-turn. Meanwhile, recent
years have witnessed remarkable progress of refinement learning techniques in
modeling dynamic evolution (Wu et al., 2022a; Yanai et al., 2022), which
propels us to explore the potential of incorporating refinement learning
techniques in the context of interactive fashion recommendation.

In this work, we aim to investigate the interactive fashion recommendation
task, which allows users to express their feedback in a loop until the system
recommends satisfactory fashion items. Fig. 1 shows an overview of the
iterative recommendation system. When in operation, the system is initialized
with a top garment given by the user. The system will then recommend bottom
garments that are compatible with the top and such that the resulting outfit
is satisfactory for the user.

Figure 1. Compatible outfit recommendation through iterative feedback. The
user presents a top garment to the system, which generates a recommendation,
namely a bottom garment to be paired with the top. The user then provides
feedback rating, according to personal preferences towards the generated
outfit. The recommendation system integrates such feedback and proposes a new
bottom to the user, attempting to maximize the feedback. The system does not
have any prior knowledge about the user.

Accurately fulfilling such interactive recommendation task is non-trivial. On
the one hand, user feedback plays an essential role by providing personalized
scores for different items and allowing continuous updating of the
recommendation during the interaction. However, exploiting the user feedback
for training is practically impossible due to the intractable size of data
collections. According to this, some mechanism that subtly simulates the user
behavior and provides a sufficiently accurate personalized score for each
system proposal is needed. On the other hand, the ultimate goal of providing
such interactivity is making the system converge towards more and more
satisfactory proposals. In this task, the system must keep track of all
previously recommended items, by retaining an internal state that is updated
after every iteration. The internal state has the responsibility to condense
information in order to profile the user’s feelings and provide appropriate
recommendations at each timestep based on the current state. Training the
system to make it able to capture the dynamic evolution of the dialogue and
recommend the appropriate item is a tough challenge.

To address the aforementioned challenges, we propose a novel interactive
system for fashion recommendation which consists of two key components: a
_User feedback model_ and a _Recommendation agent model_. In more detail, for
the first component, we exploit the GP-BPR (Song et al., 2019b) compatibility
estimator as a proxy model of the user behavior. It estimates personalized
outfit compatibility for each proposal and provides user feedback in the
training loop. For the second component, we employ reinforcement learning. A
reinforcement learning agent performs garment recommendation in three steps:
_initialization_ , _recommendation_ , and _update_. Starting from the internal
state, based on its learned policy, it determines the best action for the
recommendation, i.e., it selects the garment that is the most likely to be
compatible with the input and the user. We use $Q$-learning to predict the
best action. Extensive experiments have validated the effectiveness of our
proposed system.

Our main contributions can be summarized as follows:

  * •

To the best of our knowledge, we are among the first to implement interactive
fashion recommendation in a multi-turn manner, integrating feedback to
maximize user satisfaction.

  * •

We present a novel interactive fashion recommendation system, which employs
GP-BPR as a proxy model to provide personalized scores for different items at
training time. In addition, the proposed system utilizes a reinforcement
learning agent to conduct the recommendation by keeping track of all
previously suggested items.

  * •

Extensive experiments conducted on a real-world dataset demonstrate the
effectiveness of our proposed system.

##  2\. Related work

Our work is closely related to studies on fashion recommendation, feedback-
guided fashion search, and reinforcement learning.

###  2.1. Fashion Recommendation

Traditional recommendation systems utilizing collaborative filtering or
content-based filtering are unsuitable in the fashion domain due to the
sparsity of purchase records (Sha et al., 2016). Instead, the mainstream
literature has leveraged the rich information of fashion items to achieve
promising fashion recommendations based on fashion compatibility modeling,
which estimates whether two or several fashion items go well together. For
example, Song et al (Song et al., 2017) proposed a multimodal compatibility
modeling scheme, whereby the Bayesian Personalized Ranking optimization (Zhao
and Wang, 2019) is adopted to model the compatibility between fashion items.
Later, Vasileva et al. (Vasileva et al., 2018) considered the fashion item
category information when studying the compatibility for the outfit.
Additionally, Han et al. (Han et al., 2017b) regarded the outfit as a
predefined ordered sequence of fashion items and employed the Bi-LSTM network
to uncover outfit compatibility. Although this method can analyze the
compatibility relationship of an outfit from the global perspective, it is
questionable to represent an outfit as a sequence of ordered items.
Considering this, the current mainstream methods model each outfit as a graph
and turn to the graph neural networks (Niepert et al., 2016) to fulfill this
task. For example, Cui et al. (Cui et al., 2019) proposed node-wise graph
neural networks towards fashion compatibility modeling. In this method, each
outfit is represented as a graph, where each node denotes a specific fashion
item. In addition, Guan et al. (Guan et al., 2022b) developed a hierarchical
outfit compatibility modeling based on item-level graph and attribute-level
graph, which can perform compatibility modeling more comprehensively.
Similarly, in (Becattini et al., 2023b), the authors combine the effectiveness
of representing outfits using graphs and a transformer architecture, modifying
its self-attention mechanisms by leveraging a graph-based message-passing
mechanism. Memory augmented neural networks have also been recently used for
fashion recommendation. Such architectures can learn to store relevant items
such as pairs of compatible garments (De Divitiis et al., 2023, 2021b;
Becattini et al., 2023a) or disentangled prototypes representing fashion
attributes (Zhao et al., 2017; De Divitiis et al., 2021a; Scaramuzzino et al.,
2023) and can also be used to manipulate features (Pal et al., 2023).

Although these studies have achieved significant success, they mainly focus on
fashion compatibility modeling to recommend a compatible item to go well with
the given items, while overlooking the users’ individual demands and
preferences. Beyond these studies, in this work, we explore the users’
feedback information in the recommendation loop, so as to recommend the
tailored fashion item.

###  2.2. Feedback-guided Fashion Search

Another line of work close to ours is feedback-guided fashion search (Vo et
al., 2019), where the users can utilize a modification text as feedback to
refine the reference image to search for the ideal image. Since the most
promising application scenarios for this task lie in the fashion domain, most
existing efforts (Chen et al., 2020; Wen et al., 2021; Lee et al., 2021; Gu et
al., 2021; Wen et al., 2023; Zhang et al., 2021; Baldrati et al., 2022; Pal et
al., 2023) focus on tackling this task based on fashion-oriented datasets (Wu
et al., 2021; Guo et al., 2018; Han et al., 2017a). Many ingenious composition
manners have been proposed to properly compose multimodal queries to capture
users’ search demands. For example, Vo et al. (Vo et al., 2019) first proposed
a gating and residual model to compose the multimodal query features, where
the reference image is adaptively preserved and transformed to meet the users’
ideal image. Following that, Chen et al. (Chen et al., 2020) utilized the
attention mechanism to fuse the multimodal query representations. Later, Wen
et al. (Wen et al., 2021) utilized two fine-grained multimodal composition
modules from both local and global perspectives, which can model the diverse
modification demands more precisely. Recently, Baldrati et al. (Baldrati et
al., 2022) resorted to adopting the pre-trained large model CLIP in this task,
and brought out its potential to achieve cutting-edge performance.

Whereas, these efforts focus on the single-turn iterative fashion search, and
cannot further refine the retrieved image in multi-turn. While in this work,
we allow users to express their feedback in a loop until the system recommends
satisfactory fashion items.

A few works have recently addressed multi-turn feedback, often including free
text to precisely describe how recommendations should be refined, thus
addressing the task as a sort of dialogue between the user and the recommender
system (Yuan and Lam, 2021; Pal et al., 2023). Such systems are therefore
required to include a natural language understanding framework, resulting in
complex and slow architectures. Differently from these approaches, we do not
attempt to follow some direct manipulation command or to implement language-
based interactions but we rather strive to maximize the satisfaction of the
user by modeling the provided feedback.

###  2.3. Reinforcement Learning

As an active research topic, reinforcement learning works on training an agent
to derive the optimal policy by interacting dynamically (Wu et al., 2022a),
which has exhibited prominent success in a plethora of tasks, such as the
person re-identification (Wu et al., 2022a) and basketball games modeling
(Yanai et al., 2022). For example, Wu et al. (Wu et al., 2022a) proposed a
temporal complementarity-guided reinforcement learning approach for the image-
to-video person re-identification task, which employs reinforcement learning
to accumulate complementary information. In addition, Chen et al. (Yanai et
al., 2022) presented a deep reinforcement learning based framework for
modeling basketball games, where reinforcement learning is employed to
integrate the discrete actions and continuous actions of basketball players.
In particular, various studies have also emerged in conducting recommendations
based on reinforcement learning (Zheng et al., 2018; Xin et al., 2020). For
example, Zheng et al. (Zheng et al., 2018) conducted the online personalized
news recommendation with a reinforcement learning framework, which can capture
the dynamic nature of news characteristics and user preferences. Besides, Xin
et al. (Xin et al., 2020) proposed self-supervised reinforcement learning for
sequential recommendation tasks, where reinforcement learning is deployed to
incorporate reward-driven properties.

In a sense, these studies have curated the superiority of reinforcement
learning in modeling dynamic evolution, which inspires us to resort to
reinforcement learning for modeling user engagement and thus fulfill the
requirements of an iterative recommendation system.

##  3\. Problem Statement and Application Context

Here we outline our vision of an application context for an interactive
recommendation framework. Typically, a recommendation system to be effective
must have a form of user engagement so that suggestions can be personalized.
In fact, a classic recommendation system uses data about a user’s past
behavior and preferences to make personalized suggestions. To gather this
information, the system first builds a user profile based on data collected
from the user’s past interactions. This may include items that the user has
viewed, liked, or purchased, as well as any ratings or feedback that the user
has provided. Once the system has built a user profile, it can use this
information to make recommendations in a variety of scenarios. For example, if
the user is actively looking for something, the system can leverage the user’s
profile to make targeted suggestions. On the other hand, if the system wants
to attract a passive user, it may use a call to action to encourage the user
to engage with the recommendation system and discover personalized
suggestions. When the user is not known, i.e. when no user profile has been
collected, the recommendations can still be refined based on how the user
interacts in the current session. In this case, user engagement is
fundamental. This may involve prompting the user to provide ratings or
feedback on the suggestions that the system makes or asking the user to
specify their preferences or interests. By collecting active feedback from the
user, the system can continuously improve the accuracy and relevance of its
recommendations. To summarize, a recommendation system can be deployed in two
main scenarios: when a user is actively looking for something and when the
system itself wants to attract a passive user. In this paper we are interested
in the latter.

In particular, we pose the question of how to attract customers with targeted
recommendations when no information is known about the user. We imagine a
scenario where unknown and uncooperative users may observe a smart shop window
or advertising board and no explicit feedback can be gathered. The objective
is to make an effective call to action and stimulate the user to enter the
shop or interact with the system itself, falling back to a classic
recommendation system with an active user. To reach this goal we assume to
have access to some form of implicit feedback, which might not be actively
provided by the user. Some preliminary studies on the matter have been
recently carried out exploiting computer vision (Bigi et al., 2020; Becattini
et al., 2021; Becattini et al., 2022b, a). In these approaches, a vision-based
approach is used to estimate the degree of interest or appreciation of a user,
based on their behaviors such as posture or facial (micro) expressions. We
imagine our proposed model to be framed in a scenario where vision-based
modules are capable of effectively estimating human feedback without actively
engaging them, but just by observing their body language. In this paper,
however, we leverage a proxy model to estimate human reactions so to provide
feedback to the recommendation model, without the need to have real humans in
the loop, which would make training unfeasible.

Figure 2. Overview of the proposed approach. The system is initialized with
the visual feature of the top garment presented by the user. The reinforcement
learning agent, based on its policy, proposes an initial recommendation. The
user then provides a feedback score, which is used to update the internal
state of the agent. Based on such updated state, the agent proposes a new
bottom, attempting to maximize the feedback score at every iteration. The
Q-Function $Q$ selects the best action among the action space $A$. The
function is approximated with a three-layer feedforward neural network, that
projects the internal state of the agent into a distribution over all possible
actions. The best action (i.e. the most compatible bottom) is selected by
taking the most likely one according to the estimated likelihood.

##  4\. Interactive Garment Recommendation Model

We propose a system that provides garment recommendations through a dialog
with the user, where the user provides feedback for the proposed items and the
system adjusts the next recommendation accordingly, with the intent of
providing more satisfactory proposals. We model feedback as a numeric
evaluation, in order to maintain the system as generic as possible. Such score
could be explicitly provided by the user through a digital signage display
directly in the store, provided via a web interface in online shops, or could
even be implicitly estimated from user reactions using computer vision
techniques (Becattini et al., 2021; Becattini et al., 2022b). By including the
user feedback in the recommendation loop, the system is able to recommend
personalized items without any prior knowledge of the user. As the model
proposes a garment, the user evaluates the recommendation, thus the model is
just optimized to maximize such feedback. Fig. 2 shows an overview of the
iterative recommendation system.

The system operates in three steps as follows:

  * •

Initialization: let $t$ be a top garment and let’s assume that the user is
looking for a compatible bottom $b$. The visual feature $f_{t}=F(t)$ of $t$ is
extracted with a convolutional model $F$ and used to initialize the internal
state of the reinforcement learning agent. Keeping track of the top garment in
the agent’s state allows the model to propose suitable bottoms.

  * •

Recommendation: starting from its internal state, the agent selects the best
action $a\in A$ according to some learned policy. Each action corresponds to a
different bottom garment being proposed to the user. The action space $A$ is
composed of the set of possible recommendable bottom garments. Performing the
best action therefore corresponds to generating a probability distribution
over the set of candidate bottoms and selecting the garment with the highest
probability.

(1) $a^{*}=argmax_{i}P(b_{i}|t,u)$

where $b_{i}$ is the i-th bottom in the collection, $t$ is the top and $u$ the
user.

  * •

Updating: after a bottom $b*$ has been recommended, the model integrates the
feedback from the user $p_{tb*}$ and updates its internal state $s$. The
updating step takes the current internal state and modifies it by adding the
bottom feature $f_{b}*=F(b*)$ of the proposed bottom, weighted by the feedback
(compatibility score) provided by the user. In this way, the system keeps
track of all the recommended bottoms as a weighted sum of bottom features
$f_{b}$:

(2) $s=s+p_{tb*}F(b^{*})$

The Recommendation and Updating steps are repeated iteratively either for a
finite number of steps or until the user is satisfied. To avoid repetitions in
the Recommendation step, candidate bottoms that have already been proposed are
removed from the action space. The procedure is detailed in Algorithm 1.

Algorithm 1 Iterative recommendation algorithm. The system iteratively
recommends garments to the user and integrates feedback scores on the fly.

1:Initialization:

2: Let $t$ be a top garment.

3: Let $u$ be a user looking for a compatible bottom for $t$.

4: Visual feature extraction: $f_{t}=F(t)$, where $F$ is a convolutional
neural network.

5: Agent internal state initialization: $s=f_{t}$.

6:Recommendation:

7: Generate probability distribution $P(b_{i}|t,u)$ for each candidate bottom
$b_{i}$.

8: Select the action with the highest probability:

9: $a^{*}=\text{argmax}_{i}P(b_{i}|t,u)$

10: Recommend the corresponding bottom $b^{*}$

11:Update:

12: Let $p_{tb*}$ be the feedback score given by user $u$ for the top $t$ and
the recommended bottom $b*$.

13: Integrate user feedback score and update internal state:

14: $s\leftarrow s+p_{tb*}F(b^{*})$

15:Repeat Recommendation and Update steps until max iter or user satisfaction.

16:Output: Recommended bottoms based on user preferences.

The recommendation system is implemented as a reinforcement learning agent.
The agent has an internal state $s\in\mathcal{R}^{N}$, which has the same
dimension as the features extracted from garment images with the convolutional
model $F$. The agent can perform actions $a$, that are selected from an action
space $A$. Any action corresponds to selecting a bottom garment from a set of
possible candidates to be recommended to the user. The agent policy is learned
by a Q-function that outputs the best action given the current internal state
$s$. This is obtained according to the Bellman optimality equations for the
Action-Value function (Sutton and Barto, 2018). In practice, the Q-function
has been approximated by a neural network with three fully connected layers,
as shown in Fig. 2. The first two layers have 2048 neurons, while the third
has a dimensionality equal to the cardinality of the action space, i.e. the
number of possible bottoms. All intermediate layers have a ReLU activation.
The network determines the best action by choosing the highest Action-Value
function $Q(s,a)$ for a set of $n$ possible actions, given a state $s$. As a
reward, we use the difference between the user scores in two consecutive
recommendations to enforce the model to cause positive feedback that are
monotonically increasing through time.

Including user feedback in the training loop is challenging. Ideally, for each
user, we want to obtain a reward after each recommendation. While this is
naturally obtained at inference time in real applications, it is unfeasible at
training time, with thousands of samples and hundreds of users. Even assuming
to have a dataset of outfits labeled with personalized scores, it remains
impossible to ask each user to label any combination of top and bottom. To
overcome this limitation, we included a proxy model that replaces the user,
providing compatibility scores for any top-bottom pair and any user in the
training set. From the solutions proposed in literature (Song et al., 2017;
Zhao and Wang, 2019; Vasileva et al., 2018; Song et al., 2019b) we have used
the state of the art GP-BPR model (Song et al., 2019b) as the proxy-model of
our system but, in principle, any personalized compatibility estimator could
be used.

Using a proxy-model for training does not pose any limitations at test time.
In fact, the recommendation system learns to optimize the recommendations
based on the provided compatibility scores, without any knowledge of the
source of such feedback.

##  5\. Training the system

In the following, we discuss training for the different components of our
system. However, before describing the training phase, we discuss the pre-
processing steps that are needed to make processing effective.

###  5.1. Preprocessing

Figure 3. We used K-means to group similar bottom garments before performing
recommendations. These are two examples of clusters obtained from clustering
images in the IQON3000 dataset. TO perform recommendation the model recommends
a whole cluster by suggesting the garment that is closest to the centroid of
the cluster.

The rationale behind the need of pre-processing stems from the fact that the
training dataset could include redundant items, namely variations of the same
garments (e.g. blue jeans, white skirts, etc.). Since we strive to propose
diverse recommendations to iteratively discover a garment that meets the needs
of a user, we do not want to propose sequences of similar items. To this end,
we group garments by similarity, and propose only one representative per
cluster, allowing the user to fine-tune its search within the corresponding
cluster afterwards. This also makes training easier, by reducing the
complexity of the problem.

As a first step, we extract features using an Imagenet-pretrained ResNet model
as in (Song et al., 2019b) and then compress the features by training an
autoencoder. This yields a compact visual representation of the items, on
which to run a clustering algorithm. The autoencoder takes as input the
2048-dimensional feature extracted by the last layer of ResNet and gradually
projects it into smaller feature spaces of dimensions 1024, 512, 128 and 64
using fully connected layers with ReLU activations. The opposite is done to
reconstruct the features using transposed convolutions. We then run K-means on
the 64-dimensional embeddings obtained with the autoencoder. As the number of
clusters we set $K=1000$. An example of obtained clusters is shown in Fig. 3.
In our framework we perform recommendations using cluster medoids, i.e. we
consider only the closest element to the centroid for each cluster as
recommendation candidates.

###  5.2. User Proxy Model

GP-BPR proposed by (Song et al., 2019b) aims to conduct the personalized
compatibility modeling for personalized clothing matching, consisting of two
pivotal components: _general compatibility modeling_ and _personal preference
modeling_. The general compatibility modeling component aims to calculate the
general compatibility between a top garment and a bottom garment, while the
personal preference modeling component devotes to modeling the user’s
preference towards the bottom garment for a given to-be-matched top garment.

General Compatibility Modeling. The general compatibility modeling component
of GP-BPR aims to project the fashion garments into a latent space where the
compatibility can be well reflected by distance metrics. In particular, this
component considers both the visual and contextual representations of each
garment. Notably, each garment is associated with an image and certain
contextual information, like the brief description and category metadata.
Here, we take the visual representation leaning of the top garment as an
example. Given the $i$-th top $v_{i}^{t}$, the representation can be
calculated as follows,

(3)
$\begin{split}\begin{cases}\mathbf{h}^{t}_{i1}=s(\mathbf{W}_{1}^{t}{\mathbf{v}_{i}^{t}}+\mathbf{b}_{1}^{t}),\\\
\mathbf{h}^{t}_{ik}=s(\mathbf{W}_{k}^{t}{\mathbf{h}^{t}_{i(k-1)}}+\mathbf{b}_{k}^{t}),k=2,\cdots,K,\end{cases}\end{split}$

where $\mathbf{h}^{t}_{ik}$ refers to the hidden representation,
$\mathbf{W}_{k}^{t}$ and $\mathbf{b}_{k}^{t}$, $k=1,\cdots,K$, are the weight
matrices and biases. $s(\cdot)$ is the non-linear activation function in the
multi-layer perceptron. We treat the output of the $K$-th layer as the latent
visual representation for the top garment (_i.e.,_
$\mathbf{\tilde{v}}_{i}^{t}={\mathbf{h}^{t}_{iK}}\in\mathbb{R}^{D_{v0}}$),
where ${D_{v0}}$ is the dimension of the latent space. Similarly, we can
obtain the contextual representation for the top $t_{i}$, as well as the
visual and contextual representations for the bottom $b_{j}$ as
$\mathbf{\tilde{c}}_{i}^{t}$, $\mathbf{\tilde{v}}_{j}^{b}$, and
$\mathbf{\tilde{c}}_{j}^{b}$, respectively. Thereafter, the general
compatibility score can be calculated as follows,

(4)
$s_{ij}=\phi(\mathbf{\tilde{v}}_{i}^{t\top}){\mathbf{\tilde{v}}_{j}^{b}}+(1-\phi)({\mathbf{\tilde{c}}_{i}^{t}})^{\top}\mathbf{\tilde{v}}_{j}^{b},$

where $\phi$ is the trade-off parameter, and $s_{ij}$ represents the general
compatibility score between the top $t_{i}$ and bottom $b_{j}$. Note that the
contextual information can be easily filtered out if textual meta-data is not
available for the garments under consideration by simply setting $\phi$ to
zero (i.e., discarding the second part of Eq. 4

Personal Preference Modeling. The personal preference modeling component of
GP-BPR resorts to matrix factorization to model user preference. To be
specific, the component firstly calculates the user preference $c_{mj}$ via
the latent overall preference factors. Thereafter, the component also captures
the latent content-based preference factors, which can be integrated into the
matrix factorization framework as follows,

(5)
$c_{mj}=\alpha+{\beta_{m}}+{\beta_{j}}+{\bm{\gamma}_{m}^{\top}}{\bm{\gamma}_{j}}+\eta(\bm{\xi}_{m}^{v})^{\top}{\bm{\xi}_{j}^{v}}+(1-\eta)(\bm{\xi}_{m}^{c})^{\top}{\bm{\xi}_{j}^{c}},$

where $\bm{\gamma}_{m}$ and $\bm{\gamma}_{j}$ are the latent factors of user
$u_{m}$ and bottom $b_{j}$, respectively. $\bm{\xi}_{m}^{v}$ and
$\bm{\xi}_{j}^{v}$ refer to the latent contextual factors of user $u_{m}$ and
bottom $b_{j}$, while $\bm{\xi}_{m}^{c}$ and $\bm{\xi}_{j}^{c}$ denote the
latent contextual factors of user $u_{m}$ and bottom $b_{j}$, respectively.
$\eta$ is the non-negative tradeoff parameter.

Ultimately, following the BPR loss, the GP-BPR framework has reached the final
objective function for the personalized compatibility modeling as follows,

(6)
$\mathcal{L}=\sum_{(m,i,j,k)\in\mathcal{D}}{-ln(\sigma(p_{ij}^{m}-p_{ik}^{m}))}+\frac{\lambda}{2}||\bm{\Theta}_{F}||_{F}^{2},$

where $\mathcal{D}$ is the training set, and $(m,i,j,k)$ indicates that to
match the given top $t_{i}$ and make a proper outfit, the user $u_{m}$ prefers
the bottom $b_{j}$ to $b_{k}$. $p_{ij}^{m}$ denotes the preference of the user
$u_{m}$ towards the bottom $b_{j}$ for the top $t_{i}$, where $p_{ik}^{m}$ is
the preference of the user $u_{m}$ towards the bottom $b_{k}$ for the top
$t_{i}$. $\lambda$ stands for the non-negative hyperparameter and the last
term is designed to avoid the overfitting problem. $\bm{\Theta}_{F}$ refers to
the model parameters.

Figure 4. Unnormalized score distribution of GP-BPR on the validation set of
IQON. We perform a min-max normalization to bound scores in [0, 1].

GP-BPR Score Normalization. In this work, we rely on the GP-BPR model as a
proxy to provide compatibility scores for user-outfit pairs. The model
provides unnormalized and unconstrained compatibility scores, yet since we use
such a score in our feedback loop, we normalize it for better training
stability. Furthermore, having normalized compatibility scores, e.g. in [0, 1]
allows us to easily replace the proxy with a human user at inference time.

To normalize GP-BPR’s scores, we compute the distribution of its compatibility
scores on the validation set of the dataset we use for our experiments
(IQON3000, see Sec. 6.1 for details). We show the distribution in Fig. 4. We
store statistics to perform min-max normalization at inference time. As min
and max values, we used the 5th and 95th percentile of the distribution to
filter outliers.

###  5.3. Training the Interaction

To train the network that approximates the Q-function, we rely on the Deep
Q-Learning algorithm (Fan et al., 2020), which is based on simulating the
behavior of the agent. The training process undergoes the following steps: (i)
the simulation is initialized by selecting a top garment and a user at random;
(ii) the agent acts following a training policy and, at every step,
information is gathered regarding current state, chosen action, reward and
next state; (iii) previously observed instances are sampled from a Replay
Memory (Lin, 1992) in order to compute the cost function necessary to apply a
gradient descent step and update the neural network.

The role of the training policy is fundamental for training since it
guarantees the right compromise between exploration and exploitation. The
agent, in fact, must explore the state space and the action space to learn an
effective Q-function for each state-action pair. At the same time, actions
that allow the agent to accumulate rewards by simulating its real behavior
must be chosen. For this reason we adopt an $\epsilon$-greedy training policy
where, with probability $\epsilon$ a random action is chose (exploration)
while with probability $1-\epsilon$ the action that maximises the Q-function
is chosen (exploitation).

The value of $\epsilon$ balances exploration and exploitation. During training
we used an exponential decaying $\epsilon$ in order to converge to a greedy
policy while guaranteeing exploration during an initial training phase. At
each epoch $i$ we update $\epsilon$ in the following way:

(7) $\epsilon=\epsilon_{end}+(\epsilon_{start}-\epsilon_{end})\cdot
e^{\frac{-i}{\epsilon_{decay}}}$

where $\epsilon_{start}$ and $\epsilon_{end}$ are respectively the initial and
ending $\epsilon$ values and $\epsilon_{decay}$ regulates the decaying step.
We use $\epsilon_{start}=0.9$, $\epsilon_{end}=0.25$ and
$\epsilon_{decay}=200$.

##  6\. Experimental setting

Table 1. Detailed statistics of the IQON3000 dataset. Category | Number | Category | Number
---|---|---|---
Outerwear | 35, 765 | Top | 119, 895
Bottom | 77, 813 | Shoes | 106, 598
One Piece | 25, 816 | Accessories | 306, 448

###  6.1. Dataset and metrics

In this work, we adopted the IQON3000 dataset (Song et al., 2019a), which
contains $308,747$ outfits created by $3,568$ users with $672,335$ fashion
items. To be specific, items in IQON3000 span six common categories: _Coat,
Top, Bottom, One Piece, Shoes and Accessories_ , and as for each item, there
is the corresponding visual image, categories, attributes, and description.
Besides, each outfit is associated with its price and number of likes. For
each outfit in the dataset, there are also examples of negative garments that
would not pair well with the others in the outfit. More detailed information
about the IQON3000 dataset is summarized in Tab. 1. In this work, we only
retain data regarding top and bottom pairs, leaving the development of a
recommendation system for a whole outfit including different categories as
future work. In particular, we focus on recommending bottom garments to be
paired with a given top, although it should be noted that swapping tops and
bottoms is trivial.

The experimental setting is the following. We organize our experiments in
episodes, corresponding to a recommendation session. At the beginning of each
episode, we assume to have a user id and a top id, corresponding to the
garment for which the user wants to find a compatible bottom. Our model can
then perform a sequence of $N$ recommendation steps, incorporating user
feedback at the end of every iteration. Ideally, the model should be able to
maximize the feedback, i.e. the satisfaction of the user. In our experiments
we use $N=10$.

To evaluate our experiments, we resort to several metrics, as defined as
follows. To compare relative improvements in user feedback we leverage raw GP-
BPR scores since they provide accurate enough estimates of how a given user
(observed at training time) can respond to a top-bottom pairing. Ideally, one
would want monotonically increasing scores, indicating that the model is
suggesting increasingly better items.

We then introduce two metrics to quantify the quality of the recommendations,
Higher-than-Negative (HN) and Higher-than-Positive (HP). These two metrics
refer to the compatibility score (as provided by GP-BPR) obtained with the
triplet composed of the given user, the given top and a candidate bottom. In
particular, we define HP as 1 if at least one of the suggested bottoms obtains
a score which is higher than the ground truth bottom, and zero otherwise.
Vice-versa, we define HN as 1 if at least one candidate bottom is considered
to be better than the negative bottom associated with the outfit, and zero
otherwise. In simple words, the HN metric indicates whether the suggested item
is better than a negative, random bottom. This metric acts as a control
experiment, indicating whether the system is suggesting random items or not.
On the other hand, HP indicates whether we are able to suggest at least one
item which is even better than the positive reference given by the ground
truth. Note that the ground truth bottom does not necessarily achieve the
highest score among all possible bottoms. This is because outfits in the
dataset are composed of garments that are considered compatible by the user,
without excluding the possibility that the user might prefer another item to
improve the overall outfit.

We also report variants of these metrics, namely HN@T and HP@T, indicating if
there is at least one recommended item that is better than the associated
positive element (in case of HP) or negative element (in case of HN) within
the first T recommendations.

We also report two additional metrics. First, we compute the overall number of
proposed bottoms across the whole test set. This metric is important to assess
if the model has the tendency to recommend always the same outfits and
therefore has been affected by biases in the data. Ideally, the model should
be able to recommend a large amount of items, depending on the input data and
the feedback of the user. Finally, we also report the overall amount of
recommendations that obtain a score higher than the negative reference in the
dataset.

##  7\. Results

Figure 5. Average GP-BPR scores obtained by the recommendation system at
different iterations. At first, the system suggests a garment without knowing
any prior information about the user. Therefore, such a recommendation cannot
take into account personal preferences. As the user starts providing feedback,
the model is able to suggest on average a better item to complement the given
top for the given user.

We evaluated the behavior of the system, taking the score of the proxy model,
GP-BPR, at different recommendation steps. Each iteration depends on the
result of the previous one, including also the personalized feedback
evaluating the recommended outfit. In our experiments, we always limit the
number of iterative recommendations to 10 for simplicity. However, this is not
a limit posed by the model, which can recommend garments indefinitely. In a
real scenario, we could expect the loop to terminate whenever the user is
satisfied with the recommendation. We report in Fig. 5 the results for our
reinforcement learning agent. We also report the results obtained by a similar
agent that has been trained without exploration, i.e. using $\epsilon=0$
during the whole training phase (see Eq. 7). In addition, we use two
baselines: LSTM and random. The former is a simple LSTM model with a hidden
state of dimension 2048 (same as the image feature), that is initialized with
the top garment feature and receives as input at every timestep the feedback
score, lifted to the hidden dimension with a linear layer. A final linear
layer selects the bottom garment to be proposed. This model is not trained
with reinforcement learning but it is trained end-to-end with vanilla SGD. The
random baseline instead acts as a lower bound and simply draws a random bottom
at every iteration. All results are averaged across every test sample.

The reinforcement learning agent, on average obtains a compatibility score
that increases during the first five iterations and eventually saturates to
high values. This indicates that the model is indeed capable of inferring user
preferences on the fly and proposing garments that are increasingly more
suitable for the given bottom and user. Using the model without exploration
instead, the recommendations tend to have oscillating yet similar scores,
meaning that the model is not able to discover personalized outfits based on
previous recommendations. This is due to the fact that during training the
agent learns to propose a small set of garments that are good on average, thus
limiting the personalization of the recommendation set. However, on average,
both the models are consistently better than the random and LSTM baselines,
confirming that even the model without exploration is capable of capturing
general preferences, although without being able to personalize the
recommendations. In particular, it is interesting to observe that the LSTM
model does not manage to produce meaningful recommendations and obtains on
average very similar results to random. We found that training an LSTM-based
model for this task has convergence issues since it always predicts the same
bottoms, regardless of the user and the top. This once again confirms the
importance of the exploration term in reinforcement learning agents, which
allows a model to learn to recommend diverse items and take into account both
the user and the top.

Figure 6. Trends of individual episodes. For each plot, we select a random
user and fix a random top and we show the G-BPR scores of the recommended
bottoms. The increasing trend is well observable in every plot.

To provide a better analysis of the behavior of the model, we analyze a few
individual samples, randomly selected from the test set. Fig. 6 shows four
different samples relative to different users with different top garments. The
increasing trend of the recommendation score is clearly observable in all
samples. For User 2 and User 4, the curves are similar to the average one of
Fig. 5, while User 1 and User 2 reach the highest scoring recommendation in
the final steps. Interestingly, we can also observe that there is a bias in
the scores provided by the proxy model. The bias is given by the combination
of user and top, which leads to different ranges of scores from one sample to
another. For example, some users always provided low scores in the training
set and there are top garments that are on average well appreciated or
disliked, regardless of the paired bottom. However, we can see that such bias
is consistent within an episode and that different episodes exhibit similar
increasing trends.

We then inspect the Higher-than-Negative and Higher-than-Positive metrics.
Tab. 2 reports the results for the four methods, i.e. indicating the
percentage of episode in which at least one recommendation is considered to be
better than the positive ground truth (HP) or the negative reference (HN),
averaged over the whole test set. Interestingly, our reinforcement learning
agent is capable of suggesting at least one bottom that is more satisfying
than the ground truth 65% of the time. The baselines instead obtain lower
results. In particular, the LSTM baseline performs worse than random, since it
always proposes the same bottoms. We attribute the higher results of the
random baseline to the fact that by chance the baseline could select garments
that obtain a higher score than the ones selected by the LSTM. The
reinforcement learning agent also consistently recommends bottoms that are
better than the negative reference.

To see how the recommendations evolve through time, Fig. 7 shows the HN@T and
HP@T metrics, for T=1,…,10, i.e. throughout the whole episode. Results are
averaged across all episodes in the test set. As expected both metrics
increase monotonically for all methods, with the reinforcement learning agent
obtaining the best scores for every iteration. This means that the agent is
capable of modeling the preferences of the users earlier and more effectively
than the others.

| RL Agent | No Exploration | LSTM | Random
---|---|---|---|---
HN | 0.96 | 0.93 | 0.92 | 0.91
HP | 0.70 | 0.65 | 0.63 | 0.59
Table 2. Higher-than-Negative and Higher-than-Positive metrics. These metrics
indicate the percentage of episodes where the model is able to recommend at
least one garment which is better than the negative reference (HN) and the
positive ground truth (HP).

Figure 7. HN@T and HP@T, for various T. We consider a recommended bottom to be
a hit if there is at least one item better than the negative/positive ground
truth items in the first T recommendations.

Finally, we report statistics about the recommendations in Tab. 3. In
particular, we report the absolute number of bottom garments that are
recommended when evaluating the model on the whole test set (10
recommendations for each sample). Interestingly, the reinforcement learning
agent recommends a large number of bottoms (658 out of a total of 1500),
almost 12 times more than its counterpart without exploration (56 out of
1500). This once again confirms the importance of exploration during training.
For the LSTM model, the variability is even worse. In fact, for every episode,
the model always suggests the same 10 bottoms. On the other hand, the random
baseline achieves the highest variability, which however is not informative
since garments are drawn casually among the 1500 candidates.

The table also reports the average number of recommendations per episode that
are better than the negative bottom in the dataset. Interestingly, more than
half of the recommendations of the reinforcement learning agent are better
than the negative sample.

| RL Agent | No Exploration | LSTM | Random
---|---|---|---|---
# Proposed bottoms | 658 | 56 | 10 | 1500
# Score ¿ Neg Bottom | 5.3 | 4.4 | 2.7 | 3.8
Table 3. Statistics for the proposed method and baselines regarding the
overall number of proposed garments when evaluating the whole test set (each
episode lasts 10 iterations) and average number of recommended garments per
episode, which is better than the negative reference bottom.

###  7.1. Qualitative Results

We now provide a qualitative analysis of the results of the proposed approach.
In Fig. 8 we show the results for four different episodes. For each episode,
we fix a user and a top garment and we depict the positive bottom as well as
its GP-BPR unnormalized compatibility score. The user is fixed by simply
considering an annotated outfit in the test set for the given user (so that we
have a ground truth reference) and we propose the top garment to our model.
The user is then used to condition GP-BPR and get the estimated feedback at
every iteration. At first, the recommendation agent suggests a bottom based on
general compatibility, without any personalization. It can be seen that these
samples obtain a compatibility that is usually lower than the one of the
ground truth. As the model keeps suggesting and observing the feedback, the
scores improve, until a bottom with a sufficiently high score is suggested.
The highest scoring recommendation is highlighted with a blue border in Fig.
8. Interestingly, a garment with a higher score than the ground truth is
always obtained in these examples. Furthermore, such a garment is not
necessarily similar to the ground truth. This is due to the fact that
different outfits can be suitable for the user and that the one annotated in
the dataset might not be the overall preferred combination for that user. It
is also important to notice the high variability of garments proposed by the
model, as would be expected given the results in Tab. 3.

To be sure that the model is actually personalizing the recommendations, we
show in Fig. 9 and Fig. 10 the sequence of recommendations obtained by the
reinforcement learning agent respectively with and without exploration. In
this case, we fix the top garment and select two different users at random. It
can be seen that the agent with exploration (Fig. 9) starts with a
recommendation that only depends on the bottom and then starts to diversify
its outputs, based on the feedback of the user. Differently, the agent trained
without exploration (Fig. 10) is less capable of personalizing the outputs. In
the first example, all recommended items are the same for both users, whereas
in the second one, we can note two interesting facts. First, the
recommendation starts with the same outfit in all episodes, even if the top or
the user change. Second, despite there being diversity from the prior episode
and the two sequences are not the same, most of the recommended bottoms are
the same but in a different order. This suggests that the agent without
exploration is less inclined to change its recommendations based on user
feedback and that there is a bias towards a small subset of bottoms.

Figure 8. Qualitative results. We fix a given user and a top and suggest a
sequence of bottom garments. For each recommended garment, we also show the
unnormalized score provided by the proxy model GP-BPR. Highlighted in blue is
the highest-scoring recommendation. Figure 9. Recommendations for the same
top, but for different users. The results are obtained with the reinforcement
learning agent. It can be seen that the sequence of recommended bottoms
changes based on the feedback of the user. Figure 10. Recommendations for the
same top, but for different users. The results are obtained with the
reinforcement learning agent without exploration. In this case, the agent does
not manage to diversity the recommendations with user preferences but only
suggests bottoms that are suitable with the top. Garments are also often
repeated for different tops.

##  8\. Conclusion and Future Works

We have presented a system for personalized recommendations of compatible
garments. In our framework, the model proposes a garment and waits for user
feedback before providing the next recommendation. The aim is to maximize the
satisfaction of the user, attempting to model user preferences without any
prior knowledge about the user. We found reinforcement learning, and including
an exploration term during training, to be fundamental to achieving our goal.
Of particular importance when defining our training loop was to define a proxy
model to approximate the feedback of the user, and therefore overcome the
requirement of having actual humans in the loop. As directions for future
work, we intend to bring the paradigm of interactive garment recommendation
closer to real use with human subjects, possibly fine-tuning the model
directly in stores in an online learning fashion, to continuously improve by
observing real feedback. We also intend to combine our approach with the
recent developments in the field of vision and language, that is offering the
possibility to the user to interact verbally with the model to perform
explicit actions such as changing colors or defining a specific type of
garment.

## References

  * (1)
  * Baldrati et al. (2022) Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. 2022.  Effective conditioned and composed image retrieval combining CLIP-based features. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 21434–21442.
  * Becattini et al. (2022a) Federico Becattini, Giuseppe Becchi, Andrea Ferracani, Alberto Del Bimbo, Liliana Lo Presti, Giuseppe Mazzola, Marco La Cascia, Federico Cunico, Andrea Toaiari, Marco Cristani, et al. 2022a.  I-MALL An Effective Framework for Personalized Visits. Improving the Customer Experience in Stores. In _Proceedings of the 1st Workshop on Multimedia Computing towards Fashion Recommendation_. 11–19.
  * Becattini et al. (2023a) Federico Becattini, Lavinia De Divitiis, Claudio Baecchi, and Alberto Del Bimbo. 2023a.  Fashion recommendation based on style and social events.  _Multimedia Tools and Applications_ (2023), 1–16.
  * Becattini et al. (2022b) Federico Becattini, Federico Palai, and Alberto Del Bimbo. 2022b.  Understanding Human Reactions Looking at Facial Microexpressions With an Event Camera.  _IEEE Transactions on Industrial Informatics_ 18, 12 (2022), 9112–9121.
  * Becattini et al. (2021) Federico Becattini, Xuemeng Song, Claudio Baecchi, Shi-Ting Fang, Claudio Ferrari, Liqiang Nie, and Alberto Del Bimbo. 2021\.  PLM-IPE: A Pixel-Landmark Mutual Enhanced Framework for Implicit Preference Estimation.  In _ACM Multimedia Asia_. 1–5.
  * Becattini et al. (2023b) Federico Becattini, Federico Maria Teotini, and Alberto Del Bimbo. 2023b.  Transformer-based Graph Neural Networks for Outfit Generation.  _IEEE Transactions on Emerging Topics in Computing_ (2023).
  * Bigi et al. (2020) Wolmer Bigi, Claudio Baecchi, and Alberto Del Bimbo. 2020\.  Automatic interest recognition from posture and behaviour. In _Proceedings of the 28th ACM International Conference on Multimedia_. 2472–2480.
  * Chen et al. (2020) Yanbei Chen, Shaogang Gong, and Loris Bazzani. 2020\.  Image Search with Text Feedback by Visiolinguistic Attention Learning. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 2998–3008.
  * Cui et al. (2019) Zeyu Cui, Zekun Li, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019.  Dressing as a Whole: Outfit Compatibility Learning Based on Node-wise Graph Neural Networks. In _The World Wide Web Conference_. ACM, 307–317.
  * De Divitiis et al. (2021a) Lavinia De Divitiis, Federico Becattini, Claudio Baecchi, and Alberto Del Bimbo. 2021a.  Garment recommendation with memory augmented neural networks. In _International conference on pattern recognition_. Springer, 282–295.
  * De Divitiis et al. (2021b) Lavinia De Divitiis, Federico Becattini, Claudio Baecchi, and Alberto Del Bimbo. 2021b.  Style-based outfit recommendation. In _2021 International Conference on Content-Based Multimedia Indexing (CBMI)_. IEEE, 1–4.
  * De Divitiis et al. (2023) Lavinia De Divitiis, Federico Becattini, Claudio Baecchi, and Alberto Del Bimbo. 2023.  Disentangling features for fashion recommendation.  _ACM Transactions on Multimedia Computing, Communications and Applications_ 19, 1s (2023), 1–21.
  * Fan et al. (2020) Jianqing Fan, Zhaoran Wang, Yuchen Xie, and Zhuoran Yang. 2020\.  A theoretical analysis of deep Q-learning. In _Learning for dynamics and control_. PMLR, 486–489.
  * Gu et al. (2021) Chunbin Gu, Jiajun Bu, Zhen Zhang, Zhi Yu, Dongfang Ma, and Wei Wang. 2021\.  Image Search with Text Feedback by Deep Hierarchical Attention Mutual Information Maximization. In _Proceedings of the International ACM Conference on Multimedia_. ACM, 4600–4609.
  * Guan et al. (2022a) Weili Guan, Xuemeng Song, Haoyu Zhang, Meng Liu, Chung-Hsing Yeh, and Xiaojun Chang. 2022a.  Bi-directional Heterogeneous Graph Hashing towards Efficient Outfit Recommendation. In _Proceedings of the ACM International Conference on Multimedia_. ACM, 268–276.
  * Guan et al. (2022b) Weili Guan, Haokun Wen, Xuemeng Song, Chun Wang, Chung-Hsing Yeh, Xiaojun Chang, and Liqiang Nie. 2022b.  Partially Supervised Compatibility Modeling.  _IEEE Trans. Image Process._ 31 (2022), 4733–4745.
  * Guo et al. (2018) Xiaoxiao Guo, Hui Wu, Yu Cheng, Steven Rennie, Gerald Tesauro, and Rogério Schmidt Feris. 2018\.  Dialog-based Interactive Image Retrieval. In _Proceedings of the Conference on Neural Information Processing Systems_. 676–686.
  * Han et al. (2017a) Xintong Han, Zuxuan Wu, Phoenix X. Huang, Xiao Zhang, Menglong Zhu, Yuan Li, Yang Zhao, and Larry S. Davis. 2017a.  Automatic Spatially-Aware Fashion Concept Discovery. In _Proceedings of the IEEE International Conference on Computer Vision_. IEEE, 1472–1480.
  * Han et al. (2017b) Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry S. Davis. 2017b.  Learning Fashion Compatibility with Bidirectional LSTMs. In _Proceedings of the International ACM Conference on Multimedia_. ACM, 1078–1086.
  * Lee et al. (2021) Seungmin Lee, Dongwan Kim, and Bohyung Han. 2021.  CoSMo: Content-Style Modulation for Image Retrieval With Text Feedback. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. Computer Vision Foundation / IEEE, 802–812.
  * Lin (1992) Long-Ji Lin. 1992\.  Self-improving reactive agents based on reinforcement learning, planning and teaching.  _Machine learning_ 8 (1992), 293–321.
  * Niepert et al. (2016) Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. 2016\.  Learning Convolutional Neural Networks for Graphs. In _Proceedings of the International Conference on Machine Learning_. JMLR.org, 2014–2023.
  * Pal et al. (2023) Anwesan Pal, Sahil Wadhwa, Ayush Jaiswal, Xu Zhang, Yue Wu, Rakesh Chada, Pradeep Natarajan, and Henrik I Christensen. 2023.  FashionNTM: Multi-turn Fashion Image Retrieval via Cascaded Memory.  _arXiv preprint arXiv:2308.10170_ (2023).
  * Scaramuzzino et al. (2023) Giovanna Scaramuzzino, Federico Becattini, and Alberto Del Bimbo. 2023.  Attribute disentanglement with gradient reversal for interactive fashion retrieval.  _Pattern Recognition Letters_ 172 (2023), 203–212.
  * Sevegnani et al. (2022) Karin Sevegnani, Arjun Seshadri, Tian Wang, Anurag Beniwal, Julian J. McAuley, Alan Lu, and Gerard Medioni. 2022.  Contrastive Learning for Interactive Recommendation in Fashion.  _CoRR_ abs/2207.12033 (2022).
  * Sha et al. (2016) Dandan Sha, Daling Wang, Xiangmin Zhou, Shi Feng, Yifei Zhang, and Ge Yu. 2016\.  An Approach for Clothing Recommendation Based on Multiple Image Attributes. In _Web-Age Information Management International Conference_. Springer, 272–285.
  * Song et al. (2017) Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, and Jun Ma. 2017\.  NeuroStylist: Neural Compatibility Modeling for Clothing Matching. In _Proceedings of the International ACM Conference on Multimedia_. ACM, 753–761.
  * Song et al. (2019a) Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, and Liqiang Nie. 2019a.  GP-BPR: Personalized Compatibility Modeling for Clothing Matching. In _Proceedings of the ACM International Conference on Multimedia_. ACM, 320–328.
  * Song et al. (2019b) Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, and Liqiang Nie. 2019b.  GP-BPR: Personalized compatibility modeling for clothing matching. In _Proceedings of the 27th ACM International Conference on Multimedia_. 320–328.
  * Sutton and Barto (2018) Richard S Sutton and Andrew G Barto. 2018.  _Reinforcement learning: An introduction_.  MIT press.
  * Vasileva et al. (2018) Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David A. Forsyth. 2018.  Learning Type-Aware Embeddings for Fashion Compatibility. In _Proceedings of the European Conference on Computer Vision_. Springer, 405–421.
  * Vo et al. (2019) Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, and James Hays. 2019.  Composing Text and Image for Image Retrieval - an Empirical Odyssey. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 6439–6448.
  * Wen et al. (2021) Haokun Wen, Xuemeng Song, Xin Yang, Yibing Zhan, and Liqiang Nie. 2021.  Comprehensive Linguistic-Visual Composition Network for Image Retrieval. In _Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval_. ACM, 1369–1378.
  * Wen et al. (2023) Haokun Wen, Xian Zhang, Xuemeng Song, Yinwei Wei, and Liqiang Nie. 2023.  Target-Guided Composed Image Retrieval. In _Proceedings of the ACM International Conference on Multimedia_. ACM, 915–923.
  * Wu et al. (2021) Hui Wu, Yupeng Gao, Xiaoxiao Guo, Ziad Al-Halah, Steven Rennie, Kristen Grauman, and Rogério Feris. 2021.  Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 11307–11317.
  * Wu et al. (2022a) Wei Wu, Jiawei Liu, Kecheng Zheng, Qibin Sun, and Zhengjun Zha. 2022a.  Temporal Complementarity-Guided Reinforcement Learning for Image-to-Video Person Re-Identification. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. IEEE, 7309–7318.
  * Wu et al. (2022b) Yaxiong Wu, Craig Macdonald, and Iadh Ounis. 2022b.  Multi-Modal Dialog State Tracking for Interactive Fashion Recommendation. In _Proceedings of the ACM Conference on Recommender Systems_. ACM, 124–133.
  * Xin et al. (2020) Xin Xin, Alexandros Karatzoglou, Ioannis Arapakis, and Joemon M. Jose. 2020.  Self-Supervised Reinforcement Learning for Recommender Systems. In _Proceedings of the International ACM SIGIR conference on research and development in Information Retrieval_. ACM, 931–940.
  * Yanai et al. (2022) Chen Yanai, Adir Solomon, Gilad Katz, Bracha Shapira, and Lior Rokach. 2022.  Q-Ball: Modeling Basketball Games Using Deep Reinforcement Learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_. AAAI Press, 8806–8813.
  * Yu et al. (2018) Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, and Zheng Qin. 2018.  Aesthetic-based Clothing Recommendation. In _Proceedings of the World Wide Web Conference on World Wide Web_. ACM, 649–658.
  * Yuan and Lam (2021) Yifei Yuan and Wai Lam. 2021.  Conversational fashion image retrieval via multiturn natural language feedback. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 839–848.
  * Zhang et al. (2021) Gangjian Zhang, Shikui Wei, Huaxin Pang, and Yao Zhao. 2021\.  Heterogeneous Feature Fusion and Cross-modal Alignment for Composed Image Retrieval. In _Proceedings of the International ACM Conference on Multimedia_. ACM, 5353–5362.
  * Zhao et al. (2017) Bo Zhao, Jiashi Feng, Xiao Wu, and Shuicheng Yan. 2017\.  Memory-augmented attribute manipulation networks for interactive fashion search. In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_. 1520–1528.
  * Zhao and Wang (2019) Haifeng Zhao and Xingjun Wang. 2019.  Bi-Group Bayesian Personalized Ranking from Implicit Feedback. In _Proceedings of the International Conference on Computer Science and Software Engineering_. ACM, 35–39.
  * Zheng et al. (2018) Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, and Zhenhui Li. 2018.  DRN: A Deep Reinforcement Learning Framework for News Recommendation. In _Proceedings of the World Wide Web Conference on World Wide Web_. ACM, 167–176.


# Complex Frequency Domain Linear Prediction:
A Tool to Compute Modulation Spectrum of Speech

###### Abstract

Conventional Frequency Domain Linear Prediction (FDLP) technique models the
squared Hilbert envelope of speech with varied degrees of approximation which
can be sampled at the required frame rate and used as features for Automatic
Speech Recognition (ASR). Although previously the complex cepstrum of the
conventional FDLP model has been used as compact frame-wise speech features,
it has lacked interpretability in the context of the Hilbert envelope. In this
paper, we propose a modification of the conventional FDLP model that allows
easy interpretability of the complex cepstrum as temporal modulations in an
all-pole model approximation of the power of the speech signal. Additionally,
our “complex” FDLP yields significant speed-ups in comparison to conventional
FDLP for the same degree of approximation.

Index Terms: Automatic Speech Recognition, Frequency Domain Linear Prediction,
Modulation Spectrum

##  1 Introduction

“Message is carried in changes in vocal tract shape, which modulate spectral
components of speech” - Homer Dudley.

Human beings have learnt to communicate by modulating the power of speech
signals, predominantly below 30 Hz with an energy peak around 4-5 Hz [1, 2,
3]. Concurrently, human hearing has evolved to maintain a symbiotic
relationship with human speech production. In prior studies, human listeners
reported severe reduction in intelligibility when important speech modulations
were selectively deleted [4, 5]. Not surprisingly, related studies on machine
recognition of speech has exhibited similar results [4, 6].

Traditionally, speech modulations have been obtained by band-pass filtering
the time trajectories of the power of speech signal in different frequency
sub-bands [2, 5, 7, 8]. Eventual development of tools to mathematically model
the power trajectories of speech signal such as Frequency Domain Linear
Prediction (FDLP) led to the use of the complex cepstrum of the FDLP model as
compact frame-wise representation of speech [9, 10]. A connection between
complex cepstrum and the modulation spectrum, although intuitively obvious,
remains to be established. In this work, we formally establish this
connection, show its fundamental limitation when interpreted as temporal
modulations and propose a complex valued FDLP modeling of the power of speech
signal to resolve this limitation.

##  2 Frequency Domain Linear Prediction (FDLP)

Linear Prediction (LP) is used to perform discrete time-series analysis where
the current signal value is predicted as a weighted combination of its past
values [11] as in Eq. 1

$x[n]=\sum_{k=1}^{p}\alpha_{k}x[n-k]+Gu[n]$ (1)

, where $x$ is the discrete time signal, $n$ denotes discrete time points, $p$
is the model order, $G$ is the model gain and $u$ is the input to the LP
model.

One of the foremost properties of LP analysis of a signal is that it results
in an all-pole transfer function whose frequency response fits the power
spectrum of the original signal to varying levels of approximation based on
the filter order. An analogous application of linear prediction was used by
Marios et al. [9, 12] where this property of linear prediction was utilized in
a dual fashion resulting in Theorem 1.

###### Theorem 1

Linear prediction of the Discrete Cosine Transform (DCT) of a signal results
in an all-pole transfer function whose time response approximates the even-
symmetrized squared Hilbert envelope of the signal.

A proof of this statement can be found in [12]. Hilbert envelope is the
magnitude of the analytic signal of the original signal and, as the name
suggests, is one of several ways to compute the envelope of a signal (Figure
1). For the remainder of the paper, we will be referring to this technique as
conventional FDLP.

Figure 1: A sample speech waveform of the word “hello” and its Hilbert
envelope are shown, together with half of the FDLP fit to the even symmetric
squared Hilbert envelope with model order 50.

###  2.1 Modulation Spectrum from Conventional FDLP

Temporal modulations of a signal are the frequency components in its time
envelope. Therefore, the frequency components in the all-pole approximation of
the Hilbert envelope after any acceptable transformation is a logical
definition of modulation spectrum obtained from a FDLP model. Since
conventional FDLP fits the even-symmetrized Hilbert envelope, for computing
modulations by direct Fourier transform of the FDLP response we need to only
preserve the first half of the all-pole approximation.

Computing the modulation spectrum in the aforementioned way, requires two fast
Fourier transforms - firstly to compute the time response of the all-pole
transfer function and secondly to compute the modulation spectrum. Each
transform requires $\mathcal{O}(N\log N)$ computations, N being the number of
data points in the original signal and thus are computationally expensive
operations.

Marios et al. [9, 10] used the complex cepstrum [13] of the FDLP model as
computationally superior and compact features from the FDLP model - a
connection to the modulation spectrum was suggested but not elucidated.
Considering $\mathcal{F}$ to be the discrete time Fourier transform, the
complex cepstrum of an FDLP model is defined as

$\displaystyle c[f]$ $\displaystyle=$ $\displaystyle\mathcal{F}^{-1}\log
H(e^{j\tau})$ (2)

, where $f$ denotes discrete frequency points determined by the original
signal length $T$ and $\tau=\frac{2\pi t}{T}$ represents the “normalized
radian time” where $t$ represents the original un-normalized time. As a
result, $\tau=0$ and $\tau=2\pi$ normalized radian time denotes the beginning
and end of the even-symmetric signal respectively. $H(e^{j\tau})$ is the time
response of the FDLP all-pole transfer function. There exists a
computationally efficient way to compute the complex cepstrum directly from
the all-pole model coefficients $\\{\alpha_{k},k=1,2,\dots p\\}$ [14].

Practically, we can only compute the Discrete Fourier Transform (DFT) of a
signal on a digital device, for which $|\mathcal{F}^{-1}\log
H(e^{j\tau})|=|\mathcal{F}\log H(e^{j\tau})|$ (see Appendix A). This bridges
the gap between complex cepstrum and modulation spectrum and leads to a
judicious proposition to compute the modulation spectrum from a conventional
FDLP model using the cepstral recursion.

###### Proposition 1

The magnitude of the complex cepstrum $|c[f]|$ of a conventional FDLP model
obtained from a signal gives the modulation spectrum of the even-symmetrized
signal.

Proposition 1 justifies the use of the complex cepstrum as ASR features in
[9], and eventually in the definition of modulation vectors in [15, 16] to
compute sub-band based robust modulation features for ASR.

####  2.1.1 The Main drawback of Proposition 1

Consider an amplitude modulated signal

$x(t)=\left(1-\gamma\cos(2\pi f^{(m)}t+\phi)\right)\sin(2\pi f^{(c)}t)$ (3)

with carrier frequency $f^{(c)}$ Hz, modulation frequency $f^{(m)}$ Hz,
modulation depth $\gamma$ and modulation phase $\phi$. Unlike the direct
computation of modulation from half of the FDLP response, $|c[f]|$ captures
modulations in the even symmetrized FDLP response, making them real valued
numbers in the DCT domain and sensitive to the phase of the modulation as can
be seen in Figure 2.

Figure 2: $|c[f]|$ for conventional FDLP models obtained from a 1 second long
modulated signal $x(t)$ with $f^{(c)}=1000$ Hz, $f^{(m)}=5$ Hz and modulation
depth $\gamma=0.1$. The modulation phase shift $\phi$ takes four different
values. The correct modulation spectrum is obtained only for $\phi=0$ degrees.
When $\phi=90$ degrees, the 5 Hz modulation is not detected at all.

This results in ambiguity in the meaning of each of these modulation
coefficients. Although a DCT contains the same information as Fourier
transform, the representation of a 5 Hz modulation, as shown in Figure 2, is
not unique.

##  3 Complex Frequency Domain Linear Prediction

A sensible solution to this problem is to obtain all-pole approximations of
envelopes of the original signal instead of the even-symmetrized signal. We
revisit the property that linear prediction of a signal gives an all-pole
transfer function whose frequency response fits the power spectrum of the
signal and consider Theorem 2.

######  Theorem 2 (Complex FDLP)

Linear prediction of the inverse Fourier transform of a signal results in an
all-pole transfer function whose time response approximates the the power of
the signal.

A proof of this theorem is straight-forward since the power spectrum of
$\mathcal{F}^{-1}(x[n])$ is given by
$|\mathcal{F}(\mathcal{F}^{-1}x[n])|^{2}=x[n]^{2}$.

The primary differences between conventional FDLP and complex FDLP are

  * •

Complex FDLP, as the name suggests, executes linear prediction with a complex
valued signal $\mathcal{F}^{-1}(x[n])$ leading to a complex transfer function

  * •

From properties of polynomials, the poles of conventional FDLP are conjugate
symmetric which results in the even-symmetric time response. The poles of the
complex FDLP filter are not conjugate symmetric and $\tau=0$, $\tau=2\pi$
denote the start and end of the original non-tampered signal ( see Figure 3 ).

  * •

Unlike the conventional FDLP, the complex cepstrum of the complex FDLP model
is complex valued and since $|\mathcal{F}^{-1}\log
H(e^{j\tau})|=|\mathcal{F}\log H(e^{j\tau})|$, the magnitude of the
modulations can be explicitly computed.

  * •

In addition, since the complex FDLP fits the original non-symmetric signal,
$|\mathcal{F}\log H(e^{j\tau})|$ is insensitive to the phase of the
modulation.

Figure 3:  (Top) Conventional FDLP model fit to the even-symmetric Hilbert
envelope of a one second sample of speech with model order 20. The poles can
be seen to be conjugate symmetric, i.e. every pole at $\tau=\theta$ has a
conjugate pole at $\tau=2\pi-\theta$
(Bottom) Complex FDLP fit to the power of the same one second speech signal
with model order 10. The poles do not have conjugate symmetry.

Proposition 2 follows.

###### Proposition 2

The magnitude of the complex cepstrum $|c[f]|$ of a complex FDLP model
obtained from a signal gives the modulation spectrum of the signal.

Unlike conventional FDLP, figure 4 shows that $|c[f]|$ from complex FDLP model
captures the value of the correct modulation and is insensitive to the phase
of modulation111A Python implementation of complex FDLP based modulation
spectrum is available at https://github.com/sadhusamik/fdlp_spectrogram..

Figure 4: $|c[f]|$ for complex FDLP models obtained from a 1 second long
modulated signal $x(t)$ with $f^{(c)}=1000$ Hz, $f^{(m)}=5$ and modulation
depth $\gamma=0.1$. The modulation phase shift $\phi$ takes four different
values. Complex FDLP is capable of capturing the true magnitude of the
modulations at all phase shifts.

###  3.1 Combination of Modulations

Although our technique is capable of capturing isolated modulations,
applications would require accurate computation of the entire modulation
spectrum of a signal. Consider a signal with co-occurring modulations

$x(t)=\left(1-\sum_{i=1}^{N}\gamma_{i}\cos(2\pi
f^{(m)}_{i}t+\phi_{i})\right)\sin(2\pi f^{(c)}t)$ (4)

with carrier frequency $f^{(c)}$ Hz, modulation frequencies
$f^{(m)}_{1},f^{(m)}_{2},\dots,f^{(m)}_{N}$ Hz with modulation depths
$\gamma_{1},\gamma_{2},\dots,\gamma_{N}$ and phases
$\phi_{1},\phi_{2},\dots,\phi_{N}$ respectively.

Figure 5:
Signal 1 (Two modulations): $N=2$, $f^{(m)}_{1}=7$ Hz, $f^{(m)}_{2}=10$ Hz,
modulation depths $\gamma_{1}=0.05,\gamma_{2}=0.1$, 5(a) shows the power of
the signal and its FDLP response and 5(b) shows the FDLP derived modulation
spectrum.
Signal 2 (Three modulations): $N=3$, $f^{(m)}_{1}=2$ Hz, $f^{(m)}_{2}=5$
Hz,$f^{(m)}_{3}=8$ Hz, modulation depths
$\gamma_{1}=0.05,\gamma_{2}=0.1,\gamma_{1}=0.05$ and carrier frequency
$f_{c}=1000$ Hz. 5(c) shows the power of the signal and its FDLP response and
5(d) shows the FDLP derived modulation spectrum.
The modulation spectrum of both signals are independent of the phases of the
modulations.

With two simple examples, Figures 5(b) and 5(d) show that for composite
modulations of varying modulation depth, the complex FDLP processing is able
to capture the true modulation spectrum. Minor harmonics are generated due to
the non-linear log transformation of the approximated envelope.

###  3.2 A Computational Benefit of Complex FDLP

As is evident from Figure 3, complex FDLP requires estimation of half the
number of poles for the same approximation power - cutting the model order by
half. On the contrary, computation of complex numbers is more resource
intensive in comparison to real numbers - resulting in two conflicting
influences on the total computational time. In an empirical study with 5000
examples of 1.5 second long speech signals using 300 order conventional FDLP
and 150 order complex FDLP the observed average computation times are
approximately $12.5$ and $10.5$ ms respectively on a 2 GHz Intel Core i5
processor - a $16\%$ reduction in computation time.

FDLP based features, namely LP-TRAPS [9], modulation vectors [15], FDLP-
spectrogram [14] require per frame cepstral computations in several frequency
sub-bands over one utterance span and using complex FDLP yields significant
computational speed-ups. In the following section we look at the ASR
performance of FDLP-spectrogram derived with complex FDLP.

###  3.3 FDLP-spectrograms [14]

FDLP-spectrograms compute FDLP approximations to the power in different
frequency sub-bands over 1.5 second Hanning windowed speech. Spectrograms of
an arbitrary length speech signal is obtained by overlap-add of 1.5 second
long spectrograms. Additionally these spectrograms are designed to preserve
speech related modulations exploiting the FDLP machinery. Table 1 shows the
results of End-to-end ASR models trained with espnet2 recipes from ESPnet [17]
using complex FDLP based FDLP-spectrograms.

Data set |  | Mel-spectrogram
---
WER %
| FDLP-spectrogram
---
WER %
REVERB [18] | 5.6 | 4.8
CHiME 4 [19] | 12.6 | 12.4
Table 1: ESPnet end-to-end ASR performances are shown for two different speech
data sets. REVERB training data set consists of simulated reverberated speech
and CHiME 4 consists of simulated noisy speech. Evaluation is done with real
multi-channel reverberated and noisy speech data respectively. FDLP
spectrograms are computed with 50 filters and FDLP model order 80 whereas Mel-
spectrogram features are the default log-filterbank energy features computed
by ESPnet. The ASR models are trained without speed perturbation and
SpecAugment.

##  4 Conclusions

In this work, we propose a complex FDLP model that allows us to explicitly
define and compute speech modulations from an all-pole approximation of the
speech power trajectory. We show two-fold advantage of complex FDLP over
conventional FDLP models, namely

  * •

An un-ambiguous interpretation of the magnitude of the complex cepstrum of the
complex FDLP model as speech modulations - a feature that cannot be attributed
to the conventional FDLP model.

  * •

Improvement in computational time of all-pole model fits to the speech power
envelope over conventional FDLP.

In conclusion, we showed the end-to-end ASR performance of complex-FDLP based
FDLP-spectrogram features on two speech data sets.

##  5 Acknowledgements

This work was supported by the Center of Excellence in Human Language
Technologies, The Johns Hopkins University and by a gift from Google Research.

##  6 Appendix A

Consider

  1. 1.

A discrete time signal $x[n]$ with $N$ samples

  2. 2.

A Discrete Fourier Transform (DFT) pair
$x[n]\overset{\mathcal{F}}{\longleftrightarrow}X[k]$

Then

$\displaystyle|\mathcal{F}^{-1}(x[n])|$ $\displaystyle=$
$\displaystyle\left|\frac{1}{\sqrt{N}}\sum_{n=0}^{N-1}x[n]e^{\frac{j2\pi
kn}{N}}\right|$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{N}}\left|\sum_{n=0}^{N-1}x[n]\cos{\frac{2\pi
kn}{N}}+j\sum_{n=0}^{N-1}x[n]\sin{\frac{2\pi kn}{N}}\right|$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{N}}\sqrt{\left(\sum_{n=0}^{N-1}x[n]\cos{\frac{2\pi
kn}{N}}\right)^{2}+\left(\sum_{n=0}^{N-1}x[n]\sin{\frac{2\pi
kn}{N}}\right)^{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{N}}\left|\sum_{n=0}^{N-1}x[n]\cos{\frac{2\pi
kn}{N}}-j\sum_{n=0}^{N-1}x[n]\sin{\frac{2\pi kn}{N}}\right|$ $\displaystyle=$
$\displaystyle\left|\frac{1}{\sqrt{N}}\sum_{n=0}^{N-1}x[n]e^{\frac{-j2\pi
kn}{N}}\right|$ $\displaystyle=$ $\displaystyle|\mathcal{F}(x[n])|$

## References

  * [1] N. Ding, A. D. Patel, L. Chen, H. Butler, C. Luo, and D. Poeppel, “Temporal modulations in speech and music,” _Neuroscience & Biobehavioral Reviews_, vol. 81, pp. 181–187, 2017.
  * [2] H. Hermansky, “The modulation spectrum in the automatic recognition of speech,” in _1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings_. IEEE, 1997, pp. 140–147.
  * [3] S. Greenberg, J. Hollenback, and D. Ellis, “Insights into spoken language gleaned from phonetic transcription of the switchboard corpus,” in _Proc. ICSLP_ , vol. 96, 1996, pp. 24–27.
  * [4] N. Kanedera, T. Arai, H. Hermansky, and M. Pavel, “On the importance of various modulation frequencies for speech recognition,” in _Fifth European Conference on Speech Communication and Technology_ , 1997.
  * [5] R. Drullman, J. M. Festen, and R. Plomp, “Effect of reducing slow temporal modulations on speech reception,” _The Journal of the Acoustical Society of America_ , vol. 95, no. 5, pp. 2670–2680, 1994.
  * [6] T. Arai, M. Pavel, H. Hermansky, and C. Avendano, “Intelligibility of speech with filtered time trajectories of spectral envelopes,” in _Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP’96_ , vol. 4. IEEE, 1996, pp. 2490–2493.
  * [7] S. Greenberg and B. Kingsbury, “The modulation spectrogram: in pursuit of an invariant representation of speech,” in _1997 IEEE International Conference on Acoustics, Speech, and Signal Processing_ , vol. 3, 1997, pp. 1647–1650 vol.3.
  * [8] R. Drullman, J. M. Festen, and R. Plomp, “Effect of temporal envelope smearing on speech reception,” _The Journal of the Acoustical Society of America_ , vol. 95, no. 2, pp. 1053–1064, 1994.
  * [9] M. Athineos, H. Hermansky, and D. P. Ellis, “LP-TRAP: Linear predictive temporal patterns,” 2004.
  * [10] M. Athineos and D. Ellis, “Frequency-domain linear prediction for temporal features,” in _2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721)_ , 2003, pp. 261–266.
  * [11] J. Makhoul, “Linear prediction: A tutorial review,” _Proceedings of the IEEE_ , vol. 63, no. 4, pp. 561–580, 1975.
  * [12] M. Athineos, “Linear prediction of temporal envelopes for speech and audio applications,” Ph.D. dissertation, 2007. [Online]. Available: https://www.proquest.com/dissertations-theses/linear-prediction-temporal-envelopes-speech-audio/docview/304861691/se-2?accountid=11752
  * [13] A. Oppenheim and R. Schafer, “Homomorphic analysis of speech,” _IEEE Transactions on Audio and Electroacoustics_ , vol. 16, no. 2, pp. 221–226, 1968\.
  * [14] S. Sadhu and H. Hermansky, “Radically Old Way of Computing Spectra: Applications in End-to-End ASR,” in _Proc. Interspeech 2021_ , 2021, pp. 1424–1428.
  * [15] S. Sadhu, R. Li, and H. Hermansky, “M-vectors: sub-band based energy modulation features for multi-stream automatic speech recognition,” in _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2019, pp. 6545–6549.
  * [16] S. Sadhu and H. Hermansky, “Modulation vectors as robust feature representation for asr in domain mismatched conditions.” in _INTERSPEECH_ , 2019, pp. 3441–3445.
  * [17] S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno, N.-E. Y. Soplin, J. Heymann, M. Wiesner, N. Chen _et al._ , “ESPnet: End-to-end speech processing toolkit,” _Proc. Interspeech 2018_ , pp. 2207–2211, 2018.
  * [18] K. Kinoshita, M. Delcroix, T. Yoshioka, T. Nakatani, E. Habets, R. Haeb-Umbach, V. Leutnant, A. Sehr, W. Kellermann, R. Maas _et al._ , “The REVERB challenge: A common evaluation framework for dereverberation and recognition of reverberant speech,” in _2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics_. IEEE, 2013, pp. 1–4.
  * [19] E. Vincent, S. Watanabe, A. A. Nugraha, J. Barker, and R. Marxer, “An analysis of environment, microphone and data simulation mismatches in robust speech recognition,” _Computer Speech & Language_, vol. 46, pp. 535–557, 2017\.