# quantum Gromov-Hausdorff propinquity convergence of Christensen-Ivan quantum
metrics on AF algebras

Clay Adams  ,  Konrad Aguilar  Department of Mathematics and Statistics,
Pomona College, 610 N. College Ave., Claremont, CA 91711
<EMAIL_ADDRESS>https://aguilar.sites.pomona.edu ,  Esteban Ayala  ,
Evelyne Knight  and  Chloe Marple

###### Abstract.

We provide convergence in the quantum Gromov-Hausdorff propinquity of
Latrémolière of some sequences of infinite-dimensional Leibniz compact quantum
metric spaces of Rieffel given by AF algebras and Christensen-Ivan spectral
spaces. The main examples are convergence of Effros-Shen algebras and UHF
algebras.

###### Key words and phrases:

Gromov–Hausdorff propinquity, quantum metric spaces, Effros–Shen algebras, UHF
algebras, spectral triples

###### 2000 Mathematics Subject Classification:

Primary: 46L89, 46L30, 58B34.

The second author is supported by NSF grant DMS-2316892

###### Contents

  1. 1 Introduction and Background
  2. 2 Finite-dimensional approximations and associated continuous fields of $L$-seminorms
  3. 3 Convergence of sequences of Effros-Shen algebras and UHF algebras

##  1\. Introduction and Background

The first example of convergence of sequences of infinite-dimensional quantum
metric spaces was established by Rieffel [22], where he showed that the
quantum tori converged with respect their parameters that defined their anti-
commutation relation. This was accomplished by the introduction of the theory
of quantum metric spaces and a noncommutative analogue to the Gromov-Hausdorff
distance both introduce by Rieffel in [21, 22], respectively. This introduced
an new field of study known as noncommutative metric geometry, which has its
roots from work of Connes in [6, 5] and Gromov [9].

Since the introduction of Rieffel’s noncommutative analogue to the Gromov-
Hausdorff distance, there has been much progress in developing noncommutative
analogues of the Gromov-Hausdorff distance to capture the C*-algebraic
structure of the quantum metric space [19, 11, 24, 15, 14]. In particular, in
[13], Latrémolière proved convergence of the quantum tori in this stronger
sense. Moreoever, the Gromov-Hausdorff propinquity of Latrémolière first
introduced in [15] has been adapted to capture more structure such as module
structure [12] and spectral triple structure [18, 17].

Another example of convergence of infinite-dimensional quantum metric spaces
appeared in [2], where it was shown that the Effros-Shen algebras [8] are
continuous in Gromov-Hausdorff propinquity with respect to their natural
parameter space of irrationals in $(0,1)$ with the usuual topology. It was
also show that UHF algebras are continuous with respect to their natural
parameter space of multiplicity sequences metrized by the Baire space. This
was accomplished by introducing new quantum metrics on AF algebras equipped
with faithful tracial state motivated by work of Christensen and Ivan in [4].
Now, in [4], Christensen and Ivan did introduce quantum metrics on these
infinite-dimensional algebras, but at the time it wasn’t clear how to provide
convergence of these algebras in any noncommutative analogue to the Gromov-
Hausdorff distance, which is one reason why the quantum metrics of [2] were
introduced which are not defined using spectral triples. But, in an effort, to
bring the realms of noncommutative geometry and noncommutative metric geometry
closer, it is important to provide the convergence results of these infinite-
dimensional algebras of [2] with the quantum metrics induced by the spectral
triples of [4], which is exactly what is accomplished in this article.

The main hurdles to overcome in proving this arise from two issues that were
circumvented by the quantum metrics introduced in [2]. First, the spectral
triples of [4] are constructed using equivalence constants which are only
provided by existence and not explicitly given, which cause an issue when
providing continuous fields of $L$-seminorms as it is difficult to control
these non-explicit constants. Second, providing continuous fields of
$L$-seminorms provided by faithful tracial states is difficult when relying on
convergence in various operator norms given by different GNS represenations
for each spectral triple rather than a fixed C*-norm. The first issue is
overcome by an application of [1, Lemma 3.9], and the second issue is overcome
by a generalization of [1, Lemma 3.9]. Both of these issues are overcome in
Section 2, and we apply these results in the last section to provide
convergence of these infinite-dimensional algebras using quantum metrics
induced by Christensen-Ivan spectral triples. We only define what we mean by a
Leibniz compact quantum metric space as things can get quite overwhelming as
more definitions are provided, but for references regarding quantum metric
spaces, propinquity and propinquity in the context of AF algebras see [23, 15,
2, 16].

######  Definition 1.1.

[21, 15] Let ${\mathcal{A}}$ be a unital C*-algebra with norm
$\|\cdot\|_{\mathcal{A}}$ and unit $1_{\mathcal{A}}$. Let
$L:{\mathcal{A}}\rightarrow[0,\infty)$ be a seminnorm (possibly taking value
$\infty$) such that $\mathrm{dom}(L)=\\{a\in{\mathcal{A}}:L(a)<\infty\\}$ is a
dense *-subalgebra of ${\mathcal{A}}$. If

  1. (1)

$L(a)=L(a^{*})$ for every $a\in{\mathcal{A}}$,

  2. (2)

$\\{a\in{\mathcal{A}}:L(a)=0\\}={\mathds{C}}1_{\mathcal{A}}$,

  3. (3)

$L(ab)\leqslant\|a\|_{\mathcal{A}}L(b)+\|b\|_{\mathcal{A}}L(a)$ for every
$a,b\in{\mathcal{A}}$,

  4. (4)

the metric on the state space $S({\mathcal{A}})$ of ${\mathcal{A}}$ defined
for every $\phi,\psi\in S({\mathcal{A}})$ by

$mk_{L}(\phi,\psi)=\sup\\{|\phi(a)-\psi(a)|:a\in{\mathcal{A}},L(a)\leqslant
1\\}$

metrizes the weak* topology,

then we call $L$ an $L$-seminorm and $({\mathcal{A}},L)$ a Leibniz compact
quantum metric space.

##  2\. Finite-dimensional approximations and associated continuous fields of
$L$-seminorms

In what follows, we use various results from the beginning of [4, Section 2]
with some slightly different notation.
${\mathcal{A}}=\overline{\cup_{n\in{\mathds{N}}}A_{n}}^{\|\cdot\|_{A}}$ be a
unital AF algebra, where ${\mathcal{A}}_{0}={\mathds{C}}1_{\mathcal{A}}$
equipped with a faithful tracial state $\tau$. Let $H_{\tau}$ denote the
associated GNS Hilbert space with inner product defined for every $a,b\in
H_{\tau}$ by

$\langle a,b\rangle_{\tau}=\tau(b^{*}a)$

and associated norm $\|a\|_{\tau}=\sqrt{\langle a,a\rangle_{\tau}}$. Since
$\tau$ is faithful, we can canonically view ${\mathcal{A}}$ as a subspace (not
necessarily closed) of $H_{\tau}$. Let

$\pi_{\tau}:{\mathcal{A}}\longrightarrow B(H_{\tau})$

be the associated GNS representation such that $\pi_{\tau}(a)(b)=ab$ for every
$a,b\in{\mathcal{A}}$.

Let $n\in{\mathds{N}}$, since ${\mathcal{A}}_{n}$ is finite dimensional, we
have that ${\mathcal{A}}_{n}$ is a closed subspace of $H_{\tau}$. Let

$P^{\tau}_{n}:H_{\tau}\rightarrow{\mathcal{A}}_{n}$

denote the orthogonal projection of $H_{\tau}$ onto ${\mathcal{A}}_{n}$ and
define $Q^{\tau}_{n}=P^{\tau}_{n}-P^{\tau}_{n-1}$. Let

$E^{\tau}_{n}:{\mathcal{A}}\rightarrow{\mathcal{A}}_{n}$

denote the restriction of $P^{\tau}_{n}$ to ${\mathcal{A}}$, and by [2,
Theorem 3.5], we have that $E^{\tau}_{n}$ is the unique $\tau$-preserving
conditional expectation onto ${\mathcal{A}}_{n}$.

Next, since ${\mathcal{A}}_{n+1}$ is finite dimensional, there exists a sharp
$c^{\tau}_{n+1}>0$ such that

(2.1) $\|a\|_{\mathcal{A}}\leqslant c^{\tau}_{n+1}\cdot\|a\|_{\tau}$

for every $a\in{\mathcal{A}}_{n+1}$. Note that $c^{\tau}_{n+1}\geqslant 1$
since $\|\cdot\|_{\tau}\leqslant\|\cdot\|_{\mathcal{A}}$ on ${\mathcal{A}}$.

We now prove a crucial fact about these constants.

######  Proposition 2.1.

Let $(\tau^{n})_{n\in{\mathds{N}}}$ be a sequence of faithful tracial states
on ${\mathcal{A}}$ and let $\tau$ be a faithful tracial state on
${\mathcal{A}}$. If $(\tau^{n})_{n\in{\mathds{N}}}$ converges to $\tau$ in the
weak* topology, then for every $N\in{\mathds{N}}$, the sequence
$(c^{\tau^{n}}_{N})_{n\in{\mathds{N}}}$ converges to $c^{\tau}_{N}$ in the
usual topology on ${\mathds{R}}$.

###### Proof.

This is just [1, Proposition 3.10] applied to [1, Proposition 3.6] since norm
$\|\cdot\|_{\tau}$ is a Frobenius-Rieffel norm. ∎

Let $(\beta(n))_{n\in{\mathds{N}}}$ be a summable sequence of positive reals.
Set

$a^{\tau}_{\beta,n+1}=\frac{c^{\tau}_{n+1}}{\beta_{n+1}}.$

Next, we state a main result from [4].

######  Theorem 2.2.

 [4, Theorem 2.1] Let $(\beta(n))_{n\in{\mathds{N}}}$ be a summable sequence
of positive reals. Using the above setting, we have that

$D^{\tau}_{\beta}=\sum_{n=1}^{\infty}a^{\tau}_{\beta,n}Q^{\tau}_{n}$

defines an unbounded self-adjoint operator on $H_{\tau}$. Furthermore, if we
define

$L^{\tau}_{\beta}(a)=\|[D^{\tau}_{\beta},\pi_{\tau}(a)]\|_{B(H_{\tau})}$

for every $a\in{\mathcal{A}}$ such that $[D^{\tau}_{\beta},\pi_{\tau}(a)]$
extends to a bounded operator on $H_{\tau}$ denoted by
$[D^{\tau}_{\beta},\pi_{\tau}(a)]$, and set $L^{\tau}_{\beta}(a)=\infty$ if
not, then

$({\mathcal{A}},L^{\tau}_{\beta})$

is a Leibniz compact quantum metric space, and for every $n\in{\mathds{N}}$,
$({\mathcal{A}}_{n},L^{\tau}_{\beta})$ is a Leibniz compact quantum metric
space such that
$\mathrm{dom}(L^{\tau}_{\beta}(a))\cap{\mathcal{A}}_{n}={\mathcal{A}}_{n}$.

The following fact is stated after [3, Expression (4.5)], but we provide a
proof here.

######  Proposition 2.3.

Using the setting of Theorem 2.2, we have for every $n\in{\mathds{N}}$ and for
every $a\in{\mathcal{A}}_{n}$ that

$L^{\tau}_{\beta}(a)=\|[D^{\tau}_{\beta},\pi_{\tau}(a)]\|_{B({\mathcal{A}}^{\tau}_{n})},$

where ${\mathcal{A}}_{n}^{\tau}={\mathcal{A}}_{n}$ but for
$B({\mathcal{A}}^{\tau}_{n})$ we are considering bounded operators with
respect to the norm $\|\cdot\|_{\tau}$ on ${\mathcal{A}}_{n}$.

###### Proof.

Let $n\in{\mathds{N}}$. Let $a\in{\mathcal{A}}_{n}$. By definition, we have
that

$\|[D^{\tau}_{\beta},\pi_{\tau}(a)]\|_{B({\mathcal{A}}_{n})}\leqslant
L^{\tau}_{\beta}(a).$

Next, let $k\in\\{1,2,\ldots,n\\}$. We have since $\pi_{\tau}(a)$ commutes
with $P^{\tau}_{n}$ by the proof of [4, Theorem 2.1] or the proof of Step 1 of
[2, Theorem 3.5]. Moreover, $P_{n}P_{k}=P_{k}P_{n}=P_{k}$ and
$P_{n}P_{k-1}=P_{k-1}P_{n}=P_{k-1}$ by construction. Thus

$\displaystyle P^{\tau}_{n}[Q^{\tau}_{k},\pi_{\tau}(a)]P^{\tau}_{n}$
$\displaystyle=P^{\tau}_{n}(Q^{\tau}_{k}\pi_{\tau}(a)-\pi_{\tau}(a)Q^{\tau}_{k})P^{\tau}_{n}$
$\displaystyle=P^{\tau}_{n}((P^{\tau}_{k}-P^{\tau}_{k-1})\pi_{\tau}(a)-\pi_{\tau}(a)(P^{\tau}_{k}-P^{\tau}_{k-1}))P^{\tau}_{n}$
$\displaystyle=(P^{\tau}_{k}-P^{\tau}_{k-1})\pi_{\tau}(a)P^{\tau}_{n}-P^{\tau}_{n}\pi_{\tau}(a)(P^{\tau}_{k}-P^{\tau}_{k-1})$
$\displaystyle=(P^{\tau}_{k}-P^{\tau}_{k-1})P^{\tau}_{n}\pi_{\tau}(a)-\pi_{\tau}(a)P^{\tau}_{n}(P^{\tau}_{k}-P^{\tau}_{k-1})$
$\displaystyle=(P^{\tau}_{k}-P^{\tau}_{k-1})\pi_{\tau}(a)-\pi_{\tau}(a)(P^{\tau}_{k}-P^{\tau}_{k-1})$
$\displaystyle=Q^{\tau}_{k}\pi_{\tau}(a)-\pi_{\tau}(a)Q^{\tau}_{k}$
$\displaystyle=[Q^{\tau}_{k},\pi_{\tau}(a)]$

Thus

$\displaystyle[D^{\tau}_{\beta},\pi_{\tau}(a)]$
$\displaystyle=\sum_{k=1}^{n}a^{\tau}_{\beta,k}[Q^{\tau}_{k},\pi_{\tau}(a)]$
$\displaystyle=\sum_{k=1}^{n}a^{\tau}_{\beta,k}P^{\tau}_{n}[Q^{\tau}_{k},\pi_{\tau}(a)]P^{\tau}_{n}$
$\displaystyle=P^{\tau}_{n}[D^{\tau}_{\beta},\pi_{\tau}(a)]P^{\tau}_{n}.$

Hence, since $(P^{\tau}_{n})^{2}=P^{\tau}_{n}$ and $P^{\tau}_{n}$ is
contractive with respect to $\|\cdot\|_{\tau}$ and
$P^{\tau}_{n}(h)\in{\mathcal{A}}_{n}$ for every $h\in H_{\tau}$, we have

$\displaystyle L^{\tau}_{\beta}(a)$
$\displaystyle=\|P^{\tau}_{n}[D^{\tau}_{\beta},\pi_{\tau}(a)]P^{\tau}_{n}\|_{B(H_{\tau})}$
$\displaystyle=\sup\left\\{\|P^{\tau}_{n}[D^{\tau}_{\beta},\pi_{\tau}(a)]P^{\tau}_{n}(h)\|_{\tau}:h\in
H_{\tau},\|h\|_{\tau}\leqslant 1\right\\}$
$\displaystyle=\sup\left\\{\|P^{\tau}_{n}[D^{\tau}_{\beta},\pi_{\tau}(a)](P^{\tau}_{n})^{2}(h)\|_{\tau}:h\in
H_{\tau},\|h\|_{\tau}\leqslant 1\right\\}$
$\displaystyle=\sup\left\\{\|P^{\tau}_{n}[D^{\tau}_{\beta},\pi_{\tau}(a)]P^{\tau}_{n}(P^{\tau}_{n}(h))\|_{\tau}:h\in
H_{\tau},\|h\|_{\tau}\leqslant 1\right\\}$
$\displaystyle=\sup\left\\{\|[D^{\tau}_{\beta},\pi_{\tau}(a)](P^{\tau}_{n}(h))\|_{\tau}:h\in
H_{\tau},\|h\|_{\tau}\leqslant 1\right\\}$
$\displaystyle\leqslant\sup\left\\{\|[D^{\tau}_{\beta},\pi_{\tau}(a)]\|_{\tau}:h\in{\mathcal{A}}_{n},\|h\|_{\tau}\leqslant
1\right\\}$
$\displaystyle=\|[D^{\tau}_{\beta},\pi_{\tau}(a)]\|_{B({\mathcal{A}}_{n})}.$

Therefore
$\|[D^{\tau}_{\beta},\pi_{\tau}(a)]\|_{B({\mathcal{A}}_{n})}\leqslant
L^{\tau}_{\beta}(a)\leqslant\|[D^{\tau}_{\beta},\pi_{\tau}(a)]\|_{B({\mathcal{A}}_{n})}$
as desired. ∎

With this we can provide finite-dimensional approximations, which has been
conveniently already proven in [3].

######  Theorem 2.4.

 [3, Theorem 4.8] For every $n\in{\mathds{N}}$, it holds that

${\mathsf{\Lambda}}(({\mathcal{A}},L^{\tau}_{\beta}),({\mathcal{A}}_{n},L^{\tau}_{\beta}))\leqslant\sum_{k=n}^{\infty}\beta_{k},$

where ${\mathsf{\Lambda}}$ is the quantum Gromov-Hausdorff propinquity of
[15].

The main examples of AF algebras in this article, Effros-Shen algebras and UHF
algebras, are given in the setting of inductive limits of finite-dimensional
C*-algebras. Thus, we introduce notation to prove results in this setting.

Let $({\mathcal{B}}_{n},\alpha_{n})_{n\in{\mathds{N}}}$ be an inductive
sequence of C*-algebras (see [20, Section 6.1]) such that:

  1. (1)

${\mathcal{B}}_{0}={\mathds{C}}$ and
${\mathcal{B}}_{n}=\bigoplus_{k=1}^{n_{n}}{\mathfrak{M}}_{d_{n,k}}({\mathds{C}})$
for all $n\in{\mathds{N}}\setminus\\{0\\}$, where
$d_{n,k}\in{\mathds{N}}\setminus\\{0\\}$ for each
$n\in{\mathds{N}}\setminus\\{0\\}$ and $k\in\\{1,2,\ldots,n_{n}\\}$;

  2. (2)

$\alpha_{n}:{\mathcal{B}}_{n}\rightarrow{\mathcal{B}}_{n+1}$ is a unital
*-monomorphism for all $n\in{\mathds{N}}$;

  3. (3)

the inductive limit ${\mathcal{A}}=\underrightarrow{\lim}\
({\mathcal{B}}_{n},\alpha_{n})_{n\in{\mathds{N}}}$ is equipped with a faithful
tracial state $\tau$.

For each $n\in{\mathds{N}}$, let
$\alpha^{(n)}:{\mathcal{B}}_{n}\rightarrow{\mathcal{A}}$ be the canonical
unital *-monomorphism satisfying

(2.2) $\alpha^{(n+1)}\circ\alpha_{n}=\alpha^{(n)},$

and if for each $k\in\\{1,2,\ldots,n-1\\}$, we define

$\alpha_{k,n}=\alpha_{n}\circ\alpha_{n-1}\circ\cdots\alpha_{k},$

then inductively, we have

(2.3) $\alpha^{(n+1)}\circ\alpha_{k,n}=\alpha^{(k)}$

Note that
${\mathcal{A}}=\overline{\cup_{n\in{\mathds{N}}}\alpha^{(n)}({\mathcal{B}}_{n})}^{\|\cdot\|_{\mathcal{A}}}$
and
$\alpha^{(n)}({\mathcal{B}}_{n})\subseteq\alpha^{(n+1)}({\mathcal{B}}_{n+1})$
and $\alpha^{(0)}({\mathcal{B}}_{0})={\mathds{C}}1_{\mathcal{A}}$ (see [20,
Section 6.1]). So, for each $n\in{\mathds{N}}$, set

${\mathcal{A}}_{n}=\alpha^{(n)}({\mathcal{B}}_{n}).$

As above, for each $n\in{\mathds{N}}$, let

$E^{\tau}_{n}:{\mathcal{A}}\rightarrow{\mathcal{A}}_{n}$

denote the unique $\tau$-preserving faithful conditional expectation onto
${\mathcal{A}}_{n}.$ For each $n\in{\mathds{N}}$, let

(2.4) $\tau_{n}=\tau\circ\alpha^{(n)},$

which is a faithful tracial state on ${\mathcal{B}}_{n}$ and let
$\pi_{\tau_{n}}$ denote the associated GNS representation. Let
$k\in\\{0,1,\ldots,n+1\\}$ let

$E^{\tau_{n+1}}_{n+1,k}:{\mathcal{B}}_{n+1}\rightarrow\alpha_{k,n}({\mathcal{B}}_{k})$

be the unique $\tau_{n+1}$-preserving faithful conditional expectation onto
$\alpha_{k,n}({\mathcal{B}}_{k})$. Define

$Q^{\tau_{n+1}}_{n+1,k}=E^{\tau_{n+1}}_{n+1,k}-E^{\tau_{n+1}}_{n+1,k-1}$

and let

$D^{\tau_{n+1}}_{\beta}=\sum_{k=1}^{n+1}a^{\tau}_{\beta,k}Q^{\tau_{n+1}}_{n+1,k}.$

For every $a\in{\mathcal{B}}_{n+1}$, define

(2.5)
$L^{\tau_{n+1}}_{\beta}(a)=\|[D^{\tau_{n+1}}_{\beta},\pi_{\tau_{n+1}}(a)]\|_{B({\mathcal{B}}^{\tau_{n+1}}_{n+1})}.$

By finite dimensionality, we have that

$({\mathcal{B}}_{n+1},L^{\tau_{n+1}}_{\beta})$

is a Leibniz compact quantum metric space. We can now add to Proposition 2.3
in the inductive limit setting.

######  Theorem 2.5.

Let $n\in{\mathds{N}}$. It holds that

$L^{\tau}_{\beta}\circ\alpha^{(n)}(a)=\sup\\{\|[D^{\tau}_{\beta},\pi_{\tau}(\alpha^{(n)}(a))](\alpha^{(n)}(b))\|_{\tau}:b\in{\mathcal{B}}_{n},\|b\|_{\tau_{n}}\leqslant
1\\}=L^{\tau_{n}}_{\beta}(a)$

for every $a\in{\mathcal{B}}_{n}$.

###### Proof.

Let $n\in{\mathds{N}}$ and let $a\in{\mathcal{B}}_{n}$, then by Proposition
2.3

$\displaystyle L^{\tau}_{\beta}\circ\alpha^{(n)}(a)$
$\displaystyle=L^{\tau}_{\beta}(\alpha^{(n)}(a))$
$\displaystyle=\|[D^{\tau}_{\beta},\pi_{\tau}(\alpha^{(n)}(a))]\|_{B({\mathcal{A}}^{\tau}_{n})}$
$\displaystyle=\sup\\{\|[D^{\tau}_{\beta},\pi_{\tau}(\alpha^{(n)}(a))](c)\|_{\tau}:c\in{\mathcal{A}}_{n},\|c\|_{\tau}\leqslant
1\\}.$

Consider $c\in{\mathcal{A}}_{n}$, then there exists a unique
$b\in{\mathcal{B}}_{n}$ such that $\alpha^{(n)}(b)=c$. We have

$\displaystyle\|c\|_{\tau}^{2}=\tau(c^{*}c)=\tau\left(\alpha^{(n)}(b)^{*}\alpha^{(n)}(b)\right)=\tau\left(\alpha^{(n)}(b^{*}b)\right)=\tau_{n}(b^{*}b)=\|b\|^{2}_{\tau_{n}}.$

Hence

$L^{\tau}_{\beta}\circ\alpha^{(n)}(a)=\sup\\{\|[D^{\tau}_{\beta},\pi_{\tau}(\alpha^{(n)}(a))](\alpha^{(n)}(b))\|_{\tau}:b\in{\mathcal{B}}_{n},\|b\|_{\tau_{n}}\leqslant
1\\}$

Let $b\in{\mathcal{B}}_{n}$. Let $k\in\\{0,1,\ldots,n\\}$. Then a similar
argument as the beginning of the proof of [1, Proposition 3.5] provides

$E^{\tau}_{k}\circ\alpha^{(n)}=\alpha^{(n)}\circ E^{\tau_{n}}_{n,k}$

and

$E^{\tau}_{k-1}\circ\alpha^{(n)}=\alpha^{(n)}\circ E^{\tau_{n}}_{n,k-1}$

by Expression (2.3). Next, we have

$\displaystyle[Q^{\tau}_{k},\pi_{\tau}(\alpha^{(n)}(a))](\alpha^{(n)}(b))$
$\displaystyle\quad=((E^{\tau}_{k}-E^{\tau}_{k-1})\pi_{\tau}(\alpha^{(n)}(a))-\pi_{\tau}(\alpha^{(n)}(a))(E^{\tau}_{k}-E^{\tau}_{k-1}))(\alpha^{(n)}(b))$

Now

$\displaystyle\pi_{\tau}(\alpha^{(n)}(a))(E^{\tau}_{k}-E^{\tau}_{k-1})(\alpha^{(n)}(b))$
$\displaystyle=\pi_{\tau}(\alpha^{(n)}(a))(\alpha^{(n)}(E^{\tau_{n}}_{n,k}(b))-\alpha^{(n)}(E^{\tau_{n}}_{n,k-1}(b)))$
$\displaystyle=\alpha^{(n)}(a)(\alpha^{(n)}(E^{\tau_{n}}_{n,k}(b))-\alpha^{(n)}(E^{\tau_{n}}_{n,k-1}(b)))$
$\displaystyle=\alpha^{(n)}(aE^{\tau_{n}}_{n,k}(b)-aE^{\tau_{n}}_{n,k-1}(b))$

and similarly

$(E^{\tau}_{k}-E^{\tau}_{k-1})\pi_{\tau}(\alpha^{(n)}(a))(\alpha^{(n)}(b))=\alpha^{(n)}(E^{\tau_{n}}_{n,k}(ab)-E^{\tau_{n}}_{n,k-1}(ab)).$

Thus

$\displaystyle[Q^{\tau}_{k},\pi_{\tau}(\alpha^{(n)}(a))](\alpha^{(n)}(b))$
$\displaystyle\quad=\alpha^{(n)}\left(E^{\tau_{n}}_{n,k}(ab)-E^{\tau_{n}}_{n,k-1}(ab)-(aE^{\tau_{n}}_{n,k}(b)-aE^{\tau_{n}}_{n,k-1}(b))\right).$

However,

$\displaystyle
E^{\tau_{n}}_{n,k}(ab)-E^{\tau_{n}}_{n,k-1}(ab)-(aE^{\tau_{n}}_{n,k}(b)-aE^{\tau_{n}}_{n,k-1}(b))$
$\displaystyle\quad=E^{\tau_{n}}_{n,k}(\pi_{\tau_{n}}(a)(b))-E^{\tau_{n}}_{n,k-1}(\pi_{\tau_{n}}(a)(b))$
$\displaystyle\quad\quad-(\pi_{\tau_{n}}(a)(E^{\tau_{n}}_{n,k}(b))-\pi_{\tau_{n}}(a)(E^{\tau_{n}}_{n,k-1}(b)))$
$\displaystyle\quad=(E^{\tau_{n}}_{n,k}-E^{\tau_{n}}_{n,k-1})(\pi_{\tau_{n}}(a)(b))-\pi_{\tau_{n}}(a)((E^{\tau_{n}}_{n,k}-E^{\tau_{n}}_{n,k-1})(b))$
$\displaystyle\quad=Q^{\tau_{n}}_{n,k}(\pi_{\tau_{n}}(a)(b))-\pi_{\tau_{n}}(a)((Q^{\tau_{n}}_{n,k})(b))$
$\displaystyle\quad=(Q^{\tau_{n}}_{n,k}(\pi_{\tau_{n}}(a))-\pi_{\tau_{n}}(a)(Q^{\tau_{n}}_{n,k}))(b)$
$\displaystyle\quad=[Q^{\tau_{n}}_{n,k},\pi_{\tau_{n}}(a)](b).$

Hence

$[D^{\tau}_{\beta},\pi_{\tau}(\alpha^{(n)}(a))](\alpha^{(n)}(b))=\alpha^{(n)}([D^{\tau_{n}}_{\beta},\pi_{\tau_{n}}(a)](b))$

and so as above

$\|[D^{\tau}_{\beta},\pi_{\tau}(\alpha^{(n)}(a))](\alpha^{(n)}(b))\|_{\tau}=\|[D^{\tau_{n}}_{\beta},\pi_{\tau_{n}}(a)](b)\|_{\tau_{n}}.$

Therefore

$L^{\tau}_{\beta}\circ\alpha^{(n)}(a)=\sup\\{\|[D^{\tau}_{\beta},\pi_{\tau}(\alpha^{(n)}(a))](\alpha^{(n)}(b))\|_{\tau}:b\in{\mathcal{B}}_{n},\|b\|_{\tau_{n}}\leqslant
1\\}=L^{\tau_{n}}_{\beta}(a)$

of Expression (2.5) as desired. ∎

Now that we have an expression for the L-seminorms on the terms of a given
inductive sequence, we would like to show that these form a continuous field
of L-seminorms with respect to weak* convergence of the faithful tracial
state. However, since the norms defining our L-seminorms are operator norms
this takes some care, which is why we need some tools from metric geometry.
The following result might be known in metric geometry, but we cannot find a
proof and so we provide one here. The following result also serves as a
generalization of [1, Lemma 3.9].

######  Lemma 2.6.

Let $(X,d)$ be a metric space. Let $(C_{n})_{n\in{\mathds{N}}}$ be a sequence
of compact subsets of $X$ that converges in the Hausdorff distance with
respect to $d$, $Haus_{d},$ to a compact $C\subseteq X$. Let
$C^{\prime}\subseteq X$ be a compact set such that
$C\cup(\cup_{n\in{\mathds{N}}}C_{n})\subseteq C^{\prime}$. Let
$(f_{n})_{n\in{\mathds{N}}}$ be a sequence of real-valued continuous functions
on $X$ and let $f:X\rightarrow{\mathds{R}}$ be continuous.

If $(f_{n})_{n\in{\mathds{N}}}$ converges uniformly to $f$ on $C^{\prime}$,
then $(\sup_{x\in C_{n}}f_{n}(x))_{n\in{\mathds{N}}}$ converges to $\sup_{x\in
C}f(x)$ in the usual topology on ${\mathds{R}}.$

###### Proof.

Let $\varepsilon>0$. By uniform convergence, there exists $\delta>0$ such that
for every $a,b\in C^{\prime}$ and $n\in{\mathds{N}}$, we have

$|f_{n}(a)-f_{n}(b)|<\varepsilon/2.$

Let $N\in{\mathds{N}}$ such that for every $n\geqslant N$

$Haus_{d}(C_{n},C)<\delta/3$

and

$|\sup_{x\in C}f_{n}(x)-\sup_{x\in C}f(x)|<\varepsilon/2$

by [1, Lemma 3.9].

Let $n\geqslant N$. By compact, there exists $x^{\prime}\in C$ such that

$\sup_{x\in C}f_{n}(x)=f_{n}(x^{\prime}).$

Now consider $\sup_{x\in C_{n}}f_{n}(x).$ Assume by way of contradiction that
$|\sup_{x\in C}f_{n}(x)-\sup_{x\in C_{n}}f_{n}(x)|>\varepsilon/2.$ Assume
first that

$\sup_{x\in C}f_{n}(x)-\sup_{x\in C_{n}}f_{n}(x)>\varepsilon/2$
$f_{n}(x^{\prime})-\sup_{x\in C_{n}}f_{n}(x)>\varepsilon/2$ $\sup_{x\in
C_{n}}f_{n}(x)<f_{n}(x^{\prime})-\varepsilon/2.$

Hence

$f_{n}(x)<f_{n}(x^{\prime})-\varepsilon/2$

for every $x\in C_{n}$. Now there exists $x\in C_{n}$ such that
${\mathsf{d}}(x,x^{\prime})<\delta$ by definition of the Hausdorff distance.
Hence

$|f_{n}(x)-f_{n}(x^{\prime})|<\varepsilon/2.$

And so

$f_{n}(x^{\prime})-\varepsilon/2<f_{n}(x)<f_{n}(x^{\prime})-\varepsilon/2,$

contradiction.

On the other hand, if $\sup_{x\in C_{n}}f_{n}(x)-\sup_{x\in
C}f_{n}(x)>\varepsilon/2$. Then

$\varepsilon/2<\sup_{x\in C_{n}}f_{n}(x)-f_{n}(x^{\prime}).$

Now, by compact, there exists $z\in C_{n}$ such that $\sup_{x\in
C_{n}}f_{n}(x)=f_{n}(z)$. Hence

$f_{n}(x^{\prime})<f_{n}(z)-\varepsilon/2$

and so

$\sup_{x\in C}f_{n}(x)<f_{n}(z)-\varepsilon/2.$

Thus

$f_{n}(x)<f_{n}(z)-\varepsilon/2$

for every $x\in C$. This leads to a similar contradiction. Hence,

$|\sup_{x\in C_{n}}f_{n}(x)-\sup_{x\in C}f_{n}(x)|\leqslant\varepsilon/2.$

Finally,

$\displaystyle|\sup_{x\in C_{n}}f_{n}(x)-\sup_{x\in C}f(x)|$
$\displaystyle\leqslant|\sup_{x\in C_{n}}f_{n}(x)-\sup_{x\in
C}f_{n}(x)|+|\sup_{x\in C}f_{n}(x)-\sup_{x\in C}f(x)|$
$\displaystyle\leqslant\varepsilon/2+|\sup_{x\in C}f_{n}(x)-\sup_{x\in
C}f(x)|<\varepsilon/2+\varepsilon/2=\varepsilon.\qed$

Before providing continuous fields of $L$-seminorms, we need one more result
so that we can satisfy the hypothesis of the previous Lemma.

######  Proposition 2.7.

Let $\overline{{\mathds{N}}}={\mathds{N}}\cup\\{\infty\\}.$ Let
${\mathcal{B}}$ be a finite-dimensional C*-algebra and let
$(\tau^{n})_{n\in\overline{{\mathds{N}}}}$ be a sequence of faithful tracial
states on ${\mathcal{B}}$ such that $(\tau^{n})_{n\in{\mathds{N}}}$ weak*
converges to $\tau_{\infty}$. For each $n\in\overline{{\mathds{N}}}$, define
$C_{n}=\\{b\in{\mathcal{B}}:\|b\|_{\tau^{n}}\leqslant 1\\}$.

It holds that $(C_{n})_{n\in{\mathds{N}}}$ converges to $C_{\infty}$ in the
Hausdorff distance with respect to $\|\cdot\|_{\mathcal{B}}$.

###### Proof.

Since ${\mathcal{B}}$ is finite dimensional there exist
$N\in{\mathds{N}},m_{1},m_{2},\ldots,m_{N}\in{\mathds{N}}$ and a *-isomorphism
$\alpha:\oplus_{k=1}^{N}M_{m_{k}}({\mathds{C}})\rightarrow{\mathcal{B}}$ onto
${\mathcal{B}}$. Set $\oplus_{k=1}^{N}M_{n_{k}}({\mathds{C}})={\mathcal{A}}$.
For each $n\in\overline{{\mathds{N}}}$, define that
$\sigma^{n}=\tau^{n}\circ\alpha$. We have that
$(\sigma^{n})_{n\in{\mathds{N}}}$ is s sequence of faithful tracial states
that weak* converges to $\sigma_{\infty}$. Let $n\in\overline{{\mathds{N}}}$,
since $\sigma^{n}$ is a faithful tracial state there exist
$\mu^{n}_{1},\mu^{n}_{2},\ldots,\mu^{n}_{N}\in(0,\infty)$ such that
$\sum_{k=1}^{N}\mu^{n}_{k}=1$ and

$\sigma^{n}((a_{1},a_{2},\ldots,a_{N}))=\sum_{k=1}^{N}\frac{\mu^{n}_{k}}{m_{k}}\mathrm{Tr}(a_{k})$

for every $(a_{1},a_{2},\ldots,a_{N})$. By weak* convergence, we have that
$((\mu^{n}_{1},\mu^{n}_{2},\ldots,\mu^{n}_{N}))_{n\in{\mathds{N}}}$ converges
to $(\mu^{\infty}_{1},\mu^{\infty}_{2},\ldots,\mu^{\infty}_{N})$ in the
product topology on ${\mathds{R}}^{N}$.

Define $D_{n}=\\{a\in{\mathcal{A}}:\|a\|_{\sigma^{n}}\leqslant 1\\}$. Let
$a\in D_{\infty}$. Now since $\sigma^{n}$ is faithful we may define

$y=\left(\frac{\sqrt{\mu^{\infty}_{1}}}{\sqrt{\mu^{n}_{1}}}a_{1},\frac{\sqrt{\mu^{\infty}_{2}}}{\sqrt{\mu^{n}_{2}}}a_{2},\ldots,\frac{\sqrt{\mu^{\infty}_{N}}}{\sqrt{\mu^{n}_{N}}}a_{N}\right).$

Thus

$\displaystyle\|y\|_{\sigma^{n}}^{2}$ $\displaystyle=\sigma^{n}(y^{*}y)$
$\displaystyle=\sigma^{n}\left(\frac{\mu^{\infty}_{1}}{\mu^{n}_{1}}a_{1}^{*}a_{1},\frac{\mu^{\infty}_{2}}{\mu^{n}_{2}}a_{2}^{*}a_{2},\ldots,\frac{\mu^{\infty}_{N}}{\mu^{n}_{N}}a_{N}^{*}a_{N}\right)$
$\displaystyle=\sum_{k=1}^{N}\frac{\mu^{n}_{\infty}}{m_{k}}\mathrm{Tr}(a_{k}^{*}a_{k})$
$\displaystyle=\|a\|_{\sigma^{\infty}}^{2}\leqslant 1.$

Thus $y\in D_{n}$. Next,

$\displaystyle\|a-y\|_{\mathcal{A}}$
$\displaystyle=\max\\{\|a_{1}-y_{1}\|_{M_{n_{1}}({\mathds{C}})},\|a_{2}-y_{2}\|_{M_{n_{2}}({\mathds{C}})},\ldots,\|a_{N}-y_{N}\|_{M_{n_{N}}({\mathds{C}})}\\}$

Consider $k\in\\{1,2,\ldots,N\\}$. We have that since the operator norm is
bounded by the Frobenius norm

$\displaystyle\|a_{k}-y_{k}\|_{M_{n_{k}}({\mathds{C}})}$
$\displaystyle=\left\|a_{k}-\frac{\sqrt{\mu^{\infty}_{k}}}{\sqrt{\mu^{n}_{k}}}a_{k}\right\|_{M_{n_{k}}({\mathds{C}})}$
$\displaystyle=\left|1-\frac{\sqrt{\mu^{\infty}_{k}}}{\sqrt{\mu^{n}_{k}}}\right|\cdot\|a\|_{M_{n_{k}}({\mathds{C}})}$
$\displaystyle\leqslant\left|1-\frac{\sqrt{\mu^{\infty}_{k}}}{\sqrt{\mu^{n}_{k}}}\right|\cdot\sqrt{\mathrm{Tr}(a_{k}^{*}a_{k})}.$

However, as

$\sum_{k=1}^{N}\frac{\mu^{\infty}_{k}}{m_{k}}\mathrm{Tr}(a_{k}^{*}a_{k})=\|a\|_{\sigma^{\infty}}^{2}\leqslant
1,$

we have that
$\frac{\mu^{n}_{\infty}}{m_{k}}\mathrm{Tr}(a_{k}^{*}a_{k})\leqslant 1$, and so

$\sqrt{\mathrm{Tr}(a_{k}^{*}a_{k})}\leqslant\frac{\sqrt{m_{k}}}{\sqrt{\mu^{\infty}_{k}}}$

and thus

$\|a_{k}-y_{k}\|_{M_{n_{k}}({\mathds{C}})}\leqslant\left|1-\frac{\sqrt{\mu^{\infty}_{k}}}{\sqrt{\mu^{n}_{k}}}\right|\cdot\frac{\sqrt{m_{k}}}{\sqrt{\mu^{\infty}_{k}}}$

Hence

$\|a-y\|_{\mathcal{A}}\leqslant\max\left\\{\left|1-\frac{\sqrt{\mu^{\infty}_{k}}}{\sqrt{\mu^{n}_{k}}}\right|\cdot\frac{\sqrt{m_{k}}}{\sqrt{\mu^{\infty}_{k}}}:k\in\\{1,2,\ldots,N\\}\right\\}.$

By a symmetric argument, we have that

$\displaystyle Haus_{\|\cdot\|_{\mathcal{A}}}(D_{n},D_{\infty})$
$\displaystyle\leqslant\max\Bigg{\\{}\max\left\\{\left|1-\frac{\sqrt{\mu^{\infty}_{k}}}{\sqrt{\mu^{n}_{k}}}\right|\cdot\frac{\sqrt{m_{k}}}{\sqrt{\mu^{\infty}_{k}}}:k\in\\{1,2,\ldots,N\\}\right\\}\
,$ $\displaystyle\quad\quad\quad\quad\quad\
\max\left\\{\left|1-\frac{\sqrt{\mu^{n}_{k}}}{\sqrt{\mu^{\infty}_{k}}}\right|\cdot\frac{\sqrt{m_{k}}}{\sqrt{\mu^{n}_{k}}}:k\in\\{1,2,\ldots,N\\}\right\\}\Bigg{\\}}$

by definition of the Hausdorff distance. Thus as
$(\mu^{n}_{k})_{n\in{\mathds{N}}}$ converges to $\mu^{\infty}_{k}$ for each
$k\in\\{1,2,\ldots,N\\}$, we have that
$\lim_{n\to\infty}Haus_{\|\cdot\|_{\mathcal{A}}}(D_{n},D_{\infty})=0.$ By
construction of $\sigma^{n}$ and since $\alpha$ is a *-isomorphism, the proof
is complete. ∎

We use these results to provide continuous fields of $L$-seminorms.

######  Theorem 2.8.

Let $m\in\overline{{\mathds{N}}}={\mathds{N}}\cup\\{\infty\\}$. Let
$(\tau^{n})_{n\in\overline{{\mathds{N}}}}$ be a sequence of faithful tracial
states on ${\mathcal{A}}$. If $(\tau^{n}_{m})_{n\in{\mathds{N}}}$ of
Expression (2.4) weak* converges to $\tau^{\infty}_{m}$ on
${\mathcal{B}}_{m}$, then for every $a\in{\mathcal{B}}_{m}$, we have
$(L^{\tau^{n}_{m}}_{\beta}(a))_{n\in{\mathds{N}}}$ of Expression (2.5)
converges to $L^{\tau^{\infty}_{m}}_{\beta}(a)$ in the usual topology on
${\mathds{R}}$.

###### Proof.

Let $a\in{\mathcal{B}}_{m}$. Let $n\in\overline{{\mathds{N}}}$, define

$f_{n}:{\mathcal{B}}_{m}\rightarrow{\mathds{R}}$

by
$f_{n}(b)=\|[D^{\tau^{n}_{m}}_{\beta},\pi_{\tau^{n}_{m}}(a)](b)\|_{\tau^{n}_{m}}$.
Note that $f_{n}$ is continuous with respect to
$\|\cdot\|_{{\mathcal{B}}_{m}}$ by finite dimensionality.

Define

$C_{n}=\\{b\in{\mathcal{B}}_{n}:\|b\|_{\tau^{n}_{m}}\leqslant 1\\},$

which is compact with respect to $\|\cdot\|_{{\mathcal{B}}_{m}}$ by finite
dimensionality.

Next, we verify that all the $C_{n}$’s are contained in one compact set. By
finite dimensional, there exists a sharp $\nu_{n}>0$ such that

$\|\cdot\|_{{\mathcal{B}}_{m}}\leqslant\nu_{n}\cdot\|\cdot\|_{\tau^{n}_{m}}.$

By [1, Proposition 3.10], we have that $(\nu_{n})_{n\in{\mathds{N}}}$
converges to $\nu_{\infty}$. Hence
$r=\sup_{n\in\overline{{\mathds{N}}}}\nu_{n}<\infty$. Now, let $b\in C_{n}$,
then $\|\cdot\|_{\tau^{n}_{m}}\leqslant 1$, and so

$\|b\|_{{\mathcal{B}}_{m}}\leqslant\nu_{n}\cdot\|\cdot\|_{\tau^{n}_{m}}\leqslant\nu_{n}\leqslant
r.$

Hence $b\in\\{b\in{\mathcal{B}}_{m}:\|b\|_{{\mathcal{B}}_{m}}\leqslant r\\}$.
Set $C^{\prime}=\\{b\in{\mathcal{B}}_{m}:\|b\|_{{\mathcal{B}}_{m}}\leqslant
r\\}$. We have that $C^{\prime}$ is compact by finite dimensionality and that
$C_{n}\subseteq C^{\prime}$ for every $n\in\overline{N}$.

Next, by Proposition 2.1 and by a similar argument to [1, Proposition 3.6], we
have that $(f_{n})_{n\in{\mathds{N}}}$ converges uniformly to $f_{\infty}$ on
any compact subset of $({\mathcal{B}}_{m},\|\cdot\|_{{\mathcal{B}}_{m}})$
including $C^{\prime}$. Finally, we have that $(C_{n})_{n\in{\mathds{N}}}$
converges to $C_{\infty}$ in the Hausdorff distance with respect to
$\|\cdot\|_{{\mathcal{B}}_{m}}$ by weak* convergence by Proposition 2.7.
Therefore by Lemma 2.6, we have that

$(\sup_{b\in
C_{n}}f_{n}(b))_{n\in{\mathds{N}}}=(L^{\tau^{n}_{m}}_{\beta}(a))_{n\in{\mathds{N}}}$

converges to $\sup_{b\in
C_{\infty}}f_{\infty}(b)=L^{\tau^{\infty}_{m}}_{\beta}(a)$ in the usual
topology on ${\mathds{R}}.$ ∎

##  3\. Convergence of sequences of Effros-Shen algebras and UHF algebras

We will now provide our main convergence results. But first, we need notation
for each of these applications. We begin with the Effros-Shen algebras which
were first defined in [8].

Let $\theta\in{\mathds{R}}$ be irrational. There exists a unique sequence of
integers $(r^{\theta}_{n})_{n\in{\mathds{N}}}$ with $r^{\theta}_{n}>0$ for all
$n\in{\mathds{N}}\setminus\\{0\\}$ such that

$\theta=\lim_{n\to\infty}r_{0}^{\theta}+\cfrac{1}{r^{\theta}_{1}+\cfrac{1}{r^{\theta}_{2}+\cfrac{1}{r^{\theta}_{3}+\cfrac{1}{\ddots+\cfrac{1}{r^{\theta}_{n}}}}}}.$

When $\theta\in(0,1)$, we have that $r^{\theta}_{0}=0$. The sequence
$(r^{\theta}_{n})_{n\in{\mathds{N}}_{0}}$ is the continued fraction expansion
of $\theta$ [10].

For each $n\in{\mathds{N}}$, define

$p_{0}^{\theta}=r_{0}^{\theta},\quad p_{1}^{\theta}=1\quad\text{ and }\quad
q_{0}^{\theta}=1,\quad q_{1}^{\theta}=r^{\theta}_{1},$

and set

$p_{n+1}^{\theta}=r^{\theta}_{n+1}p_{n}^{\theta}+p_{n-1}^{\theta}$

and

$q_{n+1}^{\theta}=r^{\theta}_{n+1}q_{n}^{\theta}+q_{n-1}^{\theta}.$

The sequence $\left(p_{n}^{\theta}/q_{n}^{\theta}\right)_{n\in\mathbb{N}_{0}}$
of convergents $p^{\theta}_{n}/q^{\theta}_{n}$ converges to $\theta$. In fact,
for each $n\in{\mathds{N}}$,

$\frac{p_{n}^{\theta}}{q_{n}^{\theta}}=r_{0}^{\theta}+\cfrac{1}{r^{\theta}_{1}+\cfrac{1}{r^{\theta}_{2}+\cfrac{1}{r^{\theta}_{3}+\cfrac{1}{\ddots+\cfrac{1}{r^{\theta}_{n}}}}}}.$

We now define the terms for the inductive sequence that form the Effros-Shen
algebras. Let ${\mathcal{B}}_{\theta,0}=\mathbb{C}$ and, for each
$n\in\mathbb{N}_{0}$, let

${\mathcal{B}}_{\theta,n}=M_{q_{n}^{\theta}}({\mathds{C}})\oplus
M_{q_{n-1}^{\theta}}({\mathds{C}})$

and for each $n\in{\mathds{N}}$, set
${\mathcal{A}}_{\theta,n}=\alpha^{(n)}({\mathcal{B}}_{\theta,n})$.

These form an inductive sequence with the maps

(3.1) $\alpha_{\theta,n}:a\oplus
b\in{\mathcal{B}}_{\theta,n}\mapsto\mathrm{diag}\left(a,\ldots,a,b\right)\oplus
a\in{\mathcal{B}}_{\theta,n+1},$

where there are $r^{\theta}_{n+1}$ copies of $a$ on the diagonal in the first
summand of ${\mathcal{B}}_{\theta,n+1}$. This is a unital *-monomorphism by
construction. For $n=0$,

$\alpha_{\theta,0}:\lambda\in{\mathcal{B}}_{\theta,0}\mapsto\mathrm{diag}(\lambda,\ldots,\lambda)\oplus\lambda\
\in{\mathcal{B}}_{\theta,1}.$

The Effros–Shen algebra associated to $\theta$ is the inductive limit (see
[20, Section 6.1])

${\mathcal{A}}_{\theta}=\underrightarrow{\lim}\
({\mathcal{B}}_{\theta,n},\alpha_{\theta,n})_{n\in{\mathds{N}}}$

by [8].

There exists a unique faithful tracial state $\tau^{\theta}$ on
${\mathcal{A}}_{\theta}$ such that for each
$n\in{\mathds{N}}\setminus\\{0\\}$, $\tau_{\theta,n}$ (see Expression (2.4))
is defined for each $(a,b)\in{\mathcal{B}}_{\theta,n}$ by

$\tau_{\theta,n}(a,b)=t(\theta,n)\frac{1}{q_{n}^{\theta}}\mathrm{Tr}(a)+(1-t(\theta,n))\frac{1}{q_{n-1}^{\theta}}\mathrm{Tr}(b),$

where

$t(\theta,n)=(-1)^{n-1}q_{n}^{\theta}(\theta
q_{n-1}^{\theta}-p_{n-1}^{\theta})\in(0,1)$

(see [2, Lemma 5.5]).

For each $n\in{\mathds{N}}$, define

(3.2)
$\beta^{\theta}_{n}=\frac{1}{\dim({\mathcal{A}}_{\theta,n})}=\frac{1}{(q_{n}^{\theta})^{2}+(q_{n-1}^{\theta})^{2}},$

and note that $(\beta^{\theta}_{n})_{n\in{\mathds{N}}}$ is summable by [10].
Finally, for each $n\in{\mathds{N}}$, define

$a^{\tau_{\theta}}_{n}=\frac{c^{\tau^{\theta}}_{n}}{\beta^{\theta}_{n}}$

where $c^{\tau^{\theta}}_{n}$ is given by Expression (2.1).

######  Theorem 3.1.

The map

$\theta\in(0,1)\setminus{\mathds{Q}}\longmapsto({\mathcal{A}}_{\theta},L^{\tau_{\theta}}_{\beta_{\theta}})$

is continuous with respect to the quantum Gromov-Hausdorff propinquity of [15]
where $L^{\tau_{\theta}}_{\beta_{\theta}}$ is given by Theorem 2.2.

###### Proof.

Note that for every $\theta\in(0,1)\setminus{\mathds{Q}}$ there exists a
summable sequence of positive reals $(\beta_{n})_{n\in{\mathds{N}}}$ such that
$\beta^{\theta}_{n}\leqslant\beta_{n}$ for every $n\in{\mathds{N}}$ (see the
beginning of the proof of [2, Theorem 5.14]. Now, let
$(\theta^{n})_{n\in{\mathds{N}}}$ be a sequence in
$(0,1)\setminus{\mathds{Q}}$ that converges to some
$\theta\in(0,1)\setminus{\mathds{Q}}$ with respect to the usual topology on
${\mathds{R}}$. Let $\varepsilon>0$. Choose $N_{1}\in{\mathds{N}}$ such that
$\sum_{k=n}^{\infty}\beta_{n}<\varepsilon/3$ for every $n\geqslant N_{1}$. Now
choose $N_{2}\in{\mathds{N}}$ such that $N_{2}\geqslant N_{1}$ and
$q^{\theta_{n}}_{k}=q^{\theta}_{k}$ for every $n\geqslant N_{1}$ and
$k\in\\{1,2,\ldots,N_{1}\\}$ which is possible by [2, Proposition 5.10]. Thus,
for every $n\geqslant N_{2}$, we have
${\mathcal{B}}_{\theta_{n},k}={\mathcal{B}}_{\theta,k}$ and
$\alpha_{\theta_{n},k}=\alpha_{\theta,k}$ for every $k\leqslant N_{1}$. Now by
[2, Lemma 5.5], we have that $(\tau_{\theta_{l},N_{1}})_{l\geqslant N_{2}}$
converges to $\tau_{\theta,N_{1}}$ in the weak* topology. Thus, by the same
proof as [2, Lemma 5.13], we have that there exists $N_{3}\geqslant N_{2}$
such that

${\mathsf{\Lambda}}(({\mathcal{B}}_{N_{1}},L^{\tau_{\theta_{n},N_{1}}}_{\beta^{\theta}_{n}}),({\mathcal{B}}_{N_{1}},L^{\tau_{\theta,N_{1}}}_{\beta_{\theta}}))<\varepsilon/3$

by Theorem 2.8. Let $n\geqslant N_{3}$. By Theorem 2.4 and Theorem 2.5 and the
triangle inequality, we have

$\displaystyle{\mathsf{\Lambda}}(({\mathcal{A}}_{\theta_{n}},L^{\tau^{\theta_{n}}}_{\beta^{\theta_{n}}}),({\mathcal{A}}_{\theta},L^{\tau^{\theta}}_{\beta^{\theta}}))$
$\displaystyle\quad\leqslant{\mathsf{\Lambda}}(({\mathcal{A}}_{\theta_{n}},L^{\tau^{\theta_{n}}}_{\beta^{\theta_{n}}}),({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta_{n}}}_{\beta^{\theta_{n}}}))$
$\displaystyle\quad\quad+{\mathsf{\Lambda}}(({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta_{n}}}_{\beta^{\theta_{n}}}),({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta}}_{\beta^{\theta}}))$
$\displaystyle\quad\quad\quad+{\mathsf{\Lambda}}(({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta}}_{\beta^{\theta}}),({\mathcal{A}}_{\theta},L^{\tau^{\theta}}_{\beta^{\theta}}))$
$\displaystyle\leqslant\sum_{k=N_{1}}^{\infty}\beta^{\theta_{n}}_{k}+{\mathsf{\Lambda}}(({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta_{n}}}_{\beta^{\theta_{n}}}),({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta}}_{\beta^{\theta}}))+\sum_{k=N_{1}}^{\infty}\beta^{\theta_{n}}_{k}$
$\displaystyle\leqslant\sum_{k=N_{1}}^{\infty}\beta_{k}+{\mathsf{\Lambda}}(({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta_{n}}}_{\beta^{\theta_{n}}}),({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta}}_{\beta^{\theta}}))+\sum_{k=N_{1}}^{\infty}\beta_{k}$
$\displaystyle<\varepsilon/3+{\mathsf{\Lambda}}(({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta_{n}}}_{\beta^{\theta_{n}}}),({\mathcal{A}}_{\theta_{n},N_{1}},L^{\tau^{\theta}}_{\beta^{\theta}}))+\varepsilon/3$
$\displaystyle=2\varepsilon/3+{\mathsf{\Lambda}}(({\mathcal{B}}_{N_{1}},L^{\tau_{\theta_{n},N_{1}}}_{\beta^{\theta}_{n}}),({\mathcal{B}}_{N_{1}},L^{\tau_{\theta,N_{1}}}_{\beta_{\theta}}))$
$\displaystyle<2\varepsilon/3+\varepsilon/3=\varepsilon$

as desired. ∎

Next, we move to the UHF case.

######  Definition 3.2.

The _Baire space_ ${\mathscr{N}}$ is the set
$({\mathds{N}}\setminus\\{0\\})^{\mathds{N}}$ endowed with the metric
$\mathsf{d}$ defined, for any two $(x(n))_{n\in{\mathds{N}}}$,
$(y(n))_{n\in{\mathds{N}}}$ in ${\mathscr{N}}$, by

$d_{\mathscr{N}}\left((x(n))_{n\in{\mathds{N}}},(y(n))_{n\in{\mathds{N}}}\right)=\begin{cases}0\
\ \ \ \text{ if $x(n)=y(n)$ for all $n\in{\mathds{N}}$},\\\ \\\
2^{-\min\left\\{n\in{\mathds{N}}:x(n)\not=y(n)\right\\}}\ \ \ \text{
otherwise}\text{.}\end{cases}$

Next, we define UHF algebras in a way that suits our needs. Given
$(\beta(n))_{n\in{\mathds{N}}}\in{\mathscr{N}}$, let

$\boxtimes\beta(n)=\begin{cases}1&\text{ if }n=0,\\\
\prod_{j=0}^{n-1}(\beta(j)+1)&\text{ otherwise}.\end{cases}$

For each $n\in{\mathds{N}}$, define a unital *-monomorphism by

$\mu_{\beta,n}:a\in{\mathfrak{M}}_{\boxtimes\beta(n)}({\mathds{C}})\longmapsto\mathrm{diag}(a,a,\ldots,a)\in{\mathfrak{M}}_{\boxtimes\beta(n+1)}({\mathds{C}}),$

where there are $\beta(n)+1$ copies of $a$ in $\mathrm{diag}(a,a,\ldots,a)$.
Set $\mathsf{uhf}((\beta(n))_{n\in{\mathds{N}}})=\underrightarrow{\lim}\
({\mathfrak{M}}_{\boxtimes\beta(n)}({\mathds{C}}),\mu_{\beta,n})_{n\in{\mathds{N}}}$.
The map

$(\beta(n))_{n\in{\mathds{N}}}\in{\mathscr{N}}\longmapsto\mathsf{uhf}((\beta(n))_{n\in{\mathds{N}}})$

is a surjection onto the class of all UHF algebras up to *-isomorphism by [7,
Chapter III.5].

For each $n\in{\mathds{N}}$, let

$\gamma_{\beta}(n)=\frac{1}{\dim({\mathfrak{M}}_{\boxtimes\beta(n)}({\mathds{C}}))},$

and let

$\rho_{\beta}$

be the unique faithful tracial state on $uhf((\beta(n))_{n\in{\mathds{N}}})$.
We now state our result for continuity of UHF algebras with respect to the
Baire space.

######  Theorem 3.3.

The map

$\beta\in{\mathscr{N}}\longmapsto(\mathsf{uhf}(\beta),L^{\rho_{\beta}}_{\gamma_{\beta}})$

is continuous with respect to the quantum Gromov-Hausdorff propinquity of [15]
where $L^{\rho_{\beta}}_{\gamma_{\beta}}$ is given by Theorem 2.2.

###### Proof.

This follows similarly as the proof of Theorem 3.1 since convergence in the
Baire space is equivalent to convergence of irrationals by [2, Proposition
5.10]. ∎

## References

  * [1] Konrad Aguilar, Stephan Ramon Garcia, Elena Kim, and Frédéric Latrémolière, _The strongly Leibniz property and the Gromov-Hausdorff propinquity_ , J. Math. Anal. Appl. 529 (2024), no. 1, Paper No. 127572, 22. MR 4620742
  * [2] Konrad Aguilar and Frédéric Latrémolière, _Quantum ultrametrics on AF algebras and the Gromov-Hausdorff propinquity_ , Studia Math. 231 (2015), no. 2, 149–193, arXiv: 1511.07114. MR 3465284
  * [3] Jacopo Bassi, Roberto Conti, Carla Farsi, and Frédéric Latrémolière, _Isometry groups of inductive limits of metric spectral triples and gromov-hausdorff convergence_ , (2023), 38 pages, ArXiv: 2302.09117.
  * [4] Erik Christensen and Cristina Ivan, _Spectral triples for AF $C^{*}$-algebras and metrics on the Cantor set_, J. Operator Theory 56 (2006), no. 1, 17–46. MR 2261610
  * [5] A. Connes, _Compact metric spaces, Fredholm modules, and hyperfiniteness_ , Ergodic Theory Dynam. Systems 9 (1989), no. 2, 207–220. MR 1007407
  * [6] by same author, _Noncommutative geometry_ , Academic Press, Inc., San Diego, CA, 1994\. MR 1303779
  * [7] Kenneth R. Davidson, _$C^{*}$ -algebras by example_, Fields Institute Monographs, vol. 6, American Mathematical Society, Providence, RI, 1996. MR 1402012
  * [8] E. G. Effros and C. L. Shen, _Approximately finite $C^{\ast}$-algebras and continued fractions_, Indiana Univ. Math. J. 29 (1980), no. 2, 191–204.
  * [9] Mikhael Gromov, _Groups of polynomial growth and expanding maps_ , Inst. Hautes Études Sci. Publ. Math. (1981), no. 53, 53–73. MR 623534
  * [10] G. H. Hardy and E. M. Wright, _An introduction to the theory of numbers_ , sixth ed., Oxford University Press, Oxford, 2008.
  * [11] D. Kerr, _Matricial quantum Gromov-Hausdorff distance_ , J. Funct. Anal. 205 (2003), no. 1, 132–167, math.OA/0207282.
  * [12] F. Latrémolière, _The modular Gromov-Hausdorff propinquity_ , Submitted (2016), 67 pages, ArXiv: 1608.04881.
  * [13] Frédéric Latrémolière, _Convergence of fuzzy tori and quantum tori for the quantum Gromov-Hausdorff propinquity: an explicit approach_ , Münster J. Math. 8 (2015), no. 1, 57–98. MR 3549521
  * [14] by same author, _The dual Gromov-Hausdorff propinquity_ , J. Math. Pures Appl. (9) 103 (2015), no. 2, 303–351. MR 3298361
  * [15] by same author, _The quantum Gromov-Hausdorff propinquity_ , Trans. Amer. Math. Soc. 368 (2016), no. 1, 365–411. MR 3413867
  * [16] by same author, _Quantum metric spaces and the Gromov-Hausdorff propinquity_ , Noncommutative geometry and optimal transport, Contemp. Math., vol. 676, Amer. Math. Soc., Providence, RI, 2016, pp. 47–133. MR 3578737
  * [17] by same author, _Convergence of spectral triples on fuzzy tori to spectral triples on quantum tori_ , Comm. Math. Phys. 388 (2021), no. 2, 1049–1128. MR 4334254
  * [18] by same author, _The Gromov-Hausdorff propinquity for metric spectral triples_ , Adv. Math. 404 (2022), no. part A, Paper No. 108393, 56. MR 4411527
  * [19] H. Li, _$C^{\ast}$ -algebraic quantum Gromov-Hausdorff distance_, (2003), ArXiv: math.OA/0312003.
  * [20] Gerard J. Murphy, _$C^{*}$ -algebras and operator theory_, Academic Press, Inc., Boston, MA, 1990. MR 1074574
  * [21] M. A. Rieffel, _Metrics on states from actions of compact groups_ , Doc. Math. 3 (1998), 215–229. MR 1647515
  * [22] by same author, _Gromov-Hausdorff distance for quantum metric spaces_ , vol. 168, 2004, Appendix 1 by Hanfeng Li, Gromov-Hausdorff distance for quantum metric spaces. Matrix algebras converge to the sphere for quantum Gromov-Hausdorff distance, pp. 1–65. MR 2055927
  * [23] by same author, _Compact quantum metric spaces_ , Operator algebras, quantization, and noncommutative geometry, Contemporary Math, vol. 365, American Mathematical Society, 2005, ArXiv: 0308207, pp. 315–330.
  * [24] W. Wu, _Non-commutative metric topology on matrix state spaces_ , Proc. Amer. Math. Soc. 134 (2006), no. 2, 443–453, ArXiv: math.OA/0410587.


# DiffClone: Enhanced Behaviour Cloning in Robotics with Diffusion-Driven
Policy Learning

Sabariswaran Mani
Department of E&ECE
IIT Kharagpur, India
<EMAIL_ADDRESS>
Sreyas Venkataraman*
Department of Mathematics
IIT Kharagpur, India
<EMAIL_ADDRESS>
Abhranil Chandra*
David R. Cheriton School of Computer Science
University of Waterloo, Waterloo, Canada
<EMAIL_ADDRESS>
Yash Sirvi*
Department of CSE
IIT Kharagpur, India
<EMAIL_ADDRESS>
Adyan Rizvi*
Department of Mathematics
IIT Kharagpur, India
<EMAIL_ADDRESS>
Soumojit Bhattacharya*
Department of E&ECE
IIT Kharagpur, India
<EMAIL_ADDRESS>
Aritra Hazra
Department of CSE
IIT Kharagpur, India
<EMAIL_ADDRESS>

###### Abstract

Robot learning tasks are extremely compute-intensive and hardware-specific.
Thus the avenues of tackling these challenges, using a diverse dataset of
offline demonstrations that can be used to train robot manipulation agents, is
very appealing. The Train-Offline-Test-Online (TOTO) Benchmark provides a
well-curated open-source dataset for offline training comprised mostly of
expert data and also benchmark scores of the common offline-RL and behaviour
cloning agents. In this paper, we introduce DiffClone, an offline algorithm of
enhanced behaviour cloning agent with diffusion-based policy learning, and
measured the efficacy of our method on real online physical robots at test
time. This is also our official submission to the Train-Offline-Test-Online
(TOTO) Benchmark Challenge organized at NeurIPS 2023. We experimented with
both pre-trained visual representation and agent policies. In our experiments,
we find that MOCO finetuned ResNet50 performs the best in comparison to other
finetuned representations. Goal state conditioning and mapping to transitions
resulted in a minute increase in the success rate and mean-reward. As for the
agent policy, we developed DiffClone, a behaviour cloning agent improved using
conditional diffusion.

##  1 Introduction

Gathering data from robots to learn manipulation policies is typically
expensive and time-consuming. Nonetheless, by utilizing pre-collected data
that is readily accessible, these concerns may be mitigated. Offline data from
various robotics hardware, increases the diversity of the dataset and also its
size as more data leads to better training of the current models. This
improves the efficacy of the learning techniques and improves generalization
and robustness upon transferring the offline policies to online real-world
situations. Algorithms like Implicit Q-Learning[Kostrikov et al. (2021)],
Decision Transformers[Chen et al. (2021)], TD3-BC[Fujimoto and Gu (2021)] have
shown success in solving offline RL problems. Offline benchmark datasets
enable and make robotics research more accessible, and helps gauge and
compare, results and metrics fairly as was already the case in computer vision
and NLP [Russakovsky et al. (2015) Wang et al. (2019) Dasari et al. (2022)].

The Train Offline, Test Online Benchmark provides a robust offline robot
learning dataset to promote equitable research and a solid benchmark to
compare the advancement of offline algorithms and visual representations that
can effectively utilize diverse real-world data fairly.

The task at hand is to solve planning tasks involving complex manipulation
like pouring and scooping on a Franka-Emica Panda arm using previously
collected datasets consisting of over 1.26 million images of robot actions in
1895 trajectories of scooping data and 1003 trajectories of pouring data. The
dataset consists of RGB and depth images, along with the joint states of the
arm, the actions, and a sparse reward for each time step [Zhou et al. (2023)].
We leverage this offline data to train a visual representation model and an
agent policy for each of the tasks without any on-policy data gathering and
fine-tuning. The agents are then tested in a real-robot setup where they are
evaluated on their ability to generalize and adapt to previously unseen
objects, positions, and other such out-of-distribution settings.

Leveraging this dataset with particularly high-quality expert demonstrations,
we propose a framework to improve vanilla behavior cloning agents using
diffusion policy. We name our agent DiffClone (1): Enhanced Behaviour Cloning
in Robotics with Diffusion-Driven Policy Learning.

In DiffClone, we start by selectively sub-sampling trajectories to create a
sub-set of "expert" data. This involves choosing trajectories with the highest
rewards, ensuring the dataset captures optimal behaviour. Following this, we
employ a Momentum Contrast (MoCo) model, fine-tuned on our datasets, as our
visual-encoder backbone. This model processes images to extract relevant
states. Once these states are obtained, we normalize them across the dataset
to enhance the stability of the policy we intend to learn. Finally, we
implement a behaviour cloning agent using a CNN-based Diffusion Policy
[Ronneberger et al. (2015) Chi et al. (2023)]. We chose this strategy over
other offline RL alternatives, motivated by our success in generating an
expert dataset that accurately represents the distribution of the given
trajectories. The "expert" dataset’s quality and representativeness allowed us
to use behaviour cloning techniques that more effectively and accurately
replicated the desired optimal behaviour and policy [Kumar et al. (2022)].

The rest of the paper is organized as follows, Section 2 elaborates on the
background of the problem statement by breaking it down into two sub-problems,
and discusses the prevalent methods for solving them. Section 3 introduces our
approach to the problem, and provides a detailed step-by-step explanation.
Section 4 documents our experiments and the results obtained. Finally, we
conclude in Section 5.

##  2 Background and Preliminaries

We breakdown our approach into two parts namely the pre-trained visual encoder
backbone, and the RL/BC method which we use as a decision-making agent. In the
next subsections, we will look at an overview of the methods benchmarked in
TOTO [Zhou et al. (2023)] and then proceed to describe our method which
improves upon the baseline.

###  2.1 Visual Encoders

One of the core problems that arise when learning a policy from images is the
difficulty in learning due to the high dimensions of the image. One of the
solutions to this is to train an end-to-end model which learns good
representations for the task at hand, but this method requires a considerable
amount of data and compute, and may not always be feasible. Thus, we often use
pre-trained representations trained on a large dataset such as ImageNet, which
reduces dimensionality without much loss in important task-related
information. We can then fine-tune them on our dataset, or utilise them in a
zero-shot manner. The following subsections describe the pre-trained
representations we have considered and utilised.

####  2.1.1 Bootstrap Your Own Latent (BYOL)

Bootstrap Your Own Latent (BYOL) Grill et al. (2020) is a self-supervised
learning approach aimed at learning representation $y_{\theta}$ which can be
used for downstream tasks. The method involves two neural networks: the online
and target networks. The online network is defined by a set of weights
$\theta$ and is comprised of three stages: an encoder $f_{\theta}$, a
projector $g_{\theta}$, and a predictor $q_{\theta}$. The target network has
the same architecture as the online network but uses a different set of
weights $\xi$. The target network provides the regression targets to train the
online network, and its parameters $\xi$ are an exponential moving average of
the online parameters $\theta$.

The main mathematical equation of BYOL is the loss function, which minimizes a
similarity loss between $q_{\theta}(z_{\theta})$ and $sg(z^{\prime}_{\xi})$,
where $\theta$ are the trained weights, $\xi$ are an exponential moving
average of $\theta$, and $sg$ means stop-gradient. In a more simplified form,
the BYoL objective can be written as:

$L_{BYOL}(\theta)=E[Z_{\theta}P_{\theta}-Z^{\prime}_{\xi}]^{2}_{F}$

where $Z_{\theta}$ and $Z^{\prime}_{\xi}$ are the representations of the
online and target networks respectively, $P_{\theta}$ is the predictor of the
online network, and $E$ denotes the expectation Richemond et al. (2023).

At the end of training, everything but $f_{\theta}$ is discarded, and
$y_{\theta}$ is used as the image representation. This method is benchmarked
using a Resnet-50 architecture trained on the ImageNet dataset as the encoder.

####  2.1.2 Momentum Contrast (MoCo)

Momentum Contrast (MoCo) He et al. (2020) too is a self-supervised learning
algorithm that uses a contrastive loss for learning visual representations.
MoCo frames the unsupervised learning process as a dictionary look-up task
from recent mini-batches. Each image or view is assigned a key, represented by
an encoder network. The learning process trains encoders to perform dictionary
look-up, where an encoded "query" should be similar to its matching key and
dissimilar to others. Within each mini-batch, one image is treated as a
positive sample, while the others serve as negative samples.

The dictionary in MoCo is maintained as a queue of data samples. The encoded
representations of the current mini-batch are enqueued, and the oldest are
dequeued. This approach decouples the dictionary size from the mini-batch
size, allowing the dictionary to be large. Moreover, as the dictionary keys
come from the preceding several mini-batches, a slowly progressing key encoder
is proposed. This is implemented as a momentum-based moving average of the
query encoder, ensuring that the encoder network gradually updates itself over
time by taking a weighted average of its current state and its previous state.

The core objective of MoCo is to minimize the Euclidean distance between the
same data points in query and key encodings while maximizing the distance
between all different data points. This approach effectively enhances the
learning of distinct and robust visual features from unlabeled data. The
contrastive loss used in MoCo is based on the InfoNCE loss van den Oord et al.
(2019), which is formulated as:

$L_{MoCo}=-\frac{1}{N}\sum_{i=1}^{N}\log\frac{\exp(q_{i}^{T}k_{i+}/\tau)}{\exp(q_{i}^{T}k_{i+}/\tau)+\sum_{j=1}^{K}\exp(q_{i}^{T}k_{i-}^{j}/\tau)}$
(1)

where $q_{i}$ is the query representation, $k_{i+}$ is the positive key
representation, $k_{i-}^{j}$ are the negative key representations, $N$ is the
number of queries, $K$ is the number of negative samples, and $\tau$ is the
temperature parameter. This loss function forces the positive pairs to come
closer and negative pairs are pushed further apart, thereby learning a
representation that distinguishes different objects.

MoCo has shown competitive results in various tasks, including image
classification, object detection, and semantic segmentation.

In our experiments MoCo outperformed BOYL, and was the visual encoder that we
finally used in DiffClone.

###  2.2 Agents for Policy Learning

This part of the algorithm learns the policy, which maps the observation to
the corresponding action. Here we consider behavioral cloning, and other
offline RL algorithms to efficiently learn the action given a representation
of the state of the robot arm.

####  2.2.1 Imitation Learning via Behaviour Cloning

Imitation Learning via Behaviour Cloning (BC) [Pomerleau (1988)] is a machine
learning approach where an agent learns to perform tasks by mimicking expert
demonstrations. The objective in BC is to learn a policy $\pi$ that maps
states $s$ to actions $a$. This is often achieved through supervised learning
by minimizing the discrepancy between the agent’s actions and the expert’s
actions. Mathematically, this can be formulated as:

$\min_{\pi}\sum_{(s,a)\in D}L(\pi(s),a)$ (2)

where $D$ represents the dataset of expert state-action pairs, $\pi(s)$
denotes the action predicted by the policy for state $s$, $a$ is the
corresponding expert action, and $L$ is a loss function, such as mean squared
error, that measures the difference between the predicted action $\pi(s)$ and
the expert action $a$. The policy is trained to replicate the expert behaviour
as closely as possible. This in turn poses a major bottleneck, behaviour
cloning is very brittle to generalization to new situations and can’t handle
the variability in the demonstrations.

We experimented with Behaviour Cloning, fine-tuned on each visual
representation baseline to act as the agent. A quasi-open loop approach is
used to predict action sequences of n-steps where n is a hyperparameter. In
the TOTO-baseline, the value of n is kept at 50, and we used the same setup in
our experiments.

####  2.2.2 Visual Imitation via Nearest Neighbors (VINN)

Visual Imitation through Nearest Neighbors (VINN) [Pari et al. (2021)] is a
framework for visual imitation learning that decouples representation learning
from behaviour learning. The VINN framework consists of two decoupled parts:
training an encoding network on offline visual data using BYOL and querying
against the provided demonstrations for a non-parametric locally weighted
Nearest-Neighbor Regression based action prediction.

The nearest neighbors of the encoded input are found from the set of
demonstration embeddings. The algorithm implicitly assumes that a similar
observation must result in a similar action. Thus, once the k nearest
neighbors of the query are found, the next action is set as a weighted average
of the actions associated with those k nearest neighbors. This is done by
performing nearest neighbors search based on the distance between embeddings,
and then setting the action as the Euclidean kernel weighted average of those
examples’ associated actions:

$\hat{a}=\frac{\sum_{i=1}^{k}\exp(-\|e-e^{(i)}\|^{2})\cdot
a^{(i)}}{\sum_{i=1}^{k}\exp(-\|e-e^{(i)}\|^{2})}$ (3)

where $\hat{a}$ is the predicted action, $e$ is the encoded input, $e^{(i)}$
is the $i$th nearest neighbor, and $a^{(i)}$ is the action associated with the
$i$th nearest neighbor.

####  2.2.3 Offline RL Methods

We also experimented with two standard offline reinforcement learning methods,
namely Implicit Q-Learning (IQL) [Kostrikov et al. (2021)] and Decision
Transformer(DT) [Chen et al. (2021)].

IQL (Implicit Q-Learning): IQL is an offline reinforcement learning approach
that estimates Q-values for state-action pairs without explicit policy
optimization. It minimizes the Bellman residual while implicitly regularizing
to avoid overestimation of unseen pairs. The objective is given by:

$\min_{Q}\mathbb{E}_{(s,a,r,s^{\prime})\sim\mathcal{D}}\left[\left(Q(s,a)-\left(r+\gamma\max_{a^{\prime}}Q(s^{\prime},a^{\prime})\right)\right)^{2}\right]$
(4)

where $\mathcal{D}$ represents the dataset of transitions.

Decision Transformer: This method applies transformers to reinforcement
learning, framing return-conditioned policy optimization as sequence modeling.
The model predicts actions given a target return and past states and actions.
The formulation is:

$\text{maximize }\mathbb{E}_{(G,s,a)\sim\mathcal{D}}[\log P(a|G,s)]$ (5)

where $P$ models the probability and $\mathcal{D}$ is the dataset.

##  3 DiffClone: The Proposed Framework

Behaviour cloning methods being a reward-free setup and being more data-
efficient than RL methods, directly learn to mimic expert demonstrations,
which might be better suited for these complex planning-based manipulation
tasks without the need for exploration. Offline RL methods are also harder to
train and optimize, particularly in sparse reward setups like robot
manipulation. As a result in complex test scenarios having different objects,
positions, and lighting conditions than the training scenarios, reward-free
learning methods like BC tend to perform better particularly if the dataset is
diverse and well-made with abundant optimal trajectories covering a variety of
scenarios [Kumar et al. (2022)], all of which are satisfied by the TOTO
dataset.

We tried several experiments with the baseline visual encoders and the agent
policies mentioned in the previous section. The experiments indeed justified
our hypothesis, we found that Behaviour Cloning agents give superior results
in comparison to other offline RL Methods such as IQL and Decision Transformer
in complex manipulation tasks. This claim was further validated by the
benchmark results of the TOTO paper and thus we focused on methods and
architectures that would further improve the results of Behaviour Cloning.

Typical methods in improving behaviour cloning are done by either looking more
into the future (increasing horizon length) or by using better sequence
modeling architectures such as RNNs, LSTMs etc. Diffusion models have proven
successful in capturing complex distributions and efficiently preserving the
multi-modality of the distributions they model. These claims were validated by
the results achieved on various standard tasks using diffusion policy-based BC
as mentioned in the work by Chi et al. (2023). We found that diffusion-policy
performs better than other architectures for BC in our simulator environment
as well.

###  3.1 Data Preprocessing

We use a MoCo finetuned ResNet50 [He et al. (2015)], as a visual backbone
which is provided by the authors of the TOTO paper.

For behaviour cloning, we found that using only the best trajectories(high-
reward) trajectories led to more rewards and a better policy than using the
whole dataset. Therefore, we restrict our data to only high-reward
trajectories. Further after passing the image through our visual encoder, we
append the current joint-states of the arm to get the observation. State
normalization is used for its capacity to enhance the stability of the taught
policy and increase performance in several offline reinforcement learning
benchmark tasks [Fujimoto and Gu (2021)]. The mean and variance of the states
are calculated from dataset D. These parameters are then utilized for both
training and testing. Therefore, we normalize the observation using the
parameters, as mentioned above, before passing it to the agent.

###  3.2 Diffusion Policy for Robot Behaviour

The Diffusion Policy [Chi et al. (2023)] introduced a novel approach for
generating robot behaviour using a conditional denoising diffusion process.
This method significantly outperforms existing robot learning methods across
various benchmarks.

Figure 1: Diffusion Policy: A generative model that takes input the latest
$T_{o}$ observations $O_{t}$ and predicts $T_{a}$ subsequent actions $A_{t}$,
at each time step $t$. In the CNN variant, it uses Feature-wise Linear
Modulation (FiLM) for conditioning at each convolution layer Perez et al.
(2017). The Transformer-based approach Vaswani et al. (2023) passes
observation embeddings through a causally masked decoder with multi-head
cross-attention.

  * •

Denoising Diffusion Probabilistic Models (DDPMs): These models form the basis
of the diffusion policy, expressed as:

$x_{t-1}=\alpha_{t}x_{t}-\gamma_{t}\epsilon_{\theta}(x_{t},t)+\mathcal{N}(0,\sigma_{t}^{2}\mathbf{I}),$
(6)

where $x_{t}$ represents the state at timestep $t$, $\alpha_{t}$,
$\gamma_{t}$, and $\sigma_{t}$ are time-dependent coefficients, and
$\epsilon_{\theta}$ is the noise prediction network [Ho et al. (2020)].

  * •

Training Process: The training involves minimizing the mean squared error
between the actual and predicted noise, formulated as:

$L=\text{MSE}(\epsilon_{k},\epsilon_{\theta}(x_{0}+\epsilon_{k})).$ (7)

  * •

Adaptation for Visuomotor Policy Learning: The formulation for visuomotor
policy learning modifies the DDPM to represent robot actions and conditions
the denoising process on input observations. This is expressed as:

$A_{t_{k}-1}=\alpha(A_{t_{k}}-\gamma\epsilon_{\theta}(O_{t},A_{t_{k}},k)+\mathcal{N}(0,\sigma^{2}\mathbf{I})),$
(8)

where $A_{t_{k}}$ denotes the action at time step $k$, and $O_{t}$ represents
the input at time $t$.

  * •

Noise Schedule: The noise schedule, critical for capturing the characteristics
of action signals, is defined by the Square Cosine Schedule [Nichol and
Dhariwal (2021)].

The Diffusion Policy method, leveraging Denoising Diffusion Probabilistic
Models (DDPMs), excels in action gradient optimization through an iterative
refinement process, crucial for robotic control tasks. This iterative
refinement is mathematically represented in the equation above. The process
utilizes the gradient of the log probability of the action given an
observation,

$A_{t-1}=A_{t}-\eta\nabla_{A_{t}}\log p_{\theta}(A_{t}|O_{t}),$

where $\eta$ is the step size. This method is particularly effective in
handling multimodal action distributions, a common challenge in robotic tasks.
The approach incorporates elements of Stochastic Langevin Dynamics,

$A_{new}=A_{old}+\eta\nabla_{A_{old}}\log
p(A_{old}|O)+\mathcal{N}(0,2\eta\mathbf{I}),$

blending exploration and exploitation to refine action choices, demonstrating
significant improvements in action prediction and execution in complex, multi-
modal environments.

We use the policy’s capacity to forecast action sequences in high-dimensional
spaces and receding-horizon control to accomplish robust execution. This
architecture enables the policy to consistently adjust its actions in a
closed-loop fashion while ensuring temporal action consistency, so striking a
balance between long-horizon planning and responsiveness.

Overall, the Diffusion Policy utilizes the effectiveness of DDPMs in
visuomotor policy learning, and the action gradient-guided exploration of
state space to achieve the best-performing agent, demonstrating remarkable
improvements over existing offline and imitation methods, in handling
multimodal action distributions and ensuring robustness and stability in
training.

This Diffusion Policy is used on the offline TOTO dataset with a MoCo-
finetuned ResNet50 as a visual backbone along with the data pre-processing and
augmentations mentioned above as our final agent policy. We call it DiffClone.

Algorithm 1 DiffClone: Our Proposed Framework

  1. 1.

Prepare previously-collected data-set (here i is trajectory number and j is
time-step in trajectory) D =$\\{x_{ij}$(image),$o_{ij}$(joint-
state),$a_{ij}$(action)$\\}$

  2. 2.

Initialize visual encoder $\phi,$ agent $\theta$, and set the horizon number
to be H

  3. 3.

Pass each $x_{i}$ through the visual encoder $\phi$, to get image embeddings
$e_{ij}$

  4. 4.

Concatenate $o_{ij}$ to corresponding image embedding $e_{ij}$ to get
observation $s_{ij}$

  5. 5.

Perform state normalization over D for observations and actions

  6. 6.

For training: For $m=1$ to maximum number of epochs

    1. (a)

Sample mini-batch of K transitions from D = $\\{x_{i},o_{i},s_{i},a_{i}\\}$

    2. (b)

For $i=1$ to $K$

      1. i.

We have $s_{i}$ as the current observation, and
$a_{i}=\\{a_{i,j}|j=0...H-1\\}$ where $a$ denotes the next H actions taken
from observation $s_{i}$

      2. ii.

Sample random noise_level_t from (1, T)

      3. iii.

$a^{\prime}=$ add_noise($a_{i},\,$noise_level_t)

      4. iv.

$\gamma=\epsilon_{\theta}(a^{\prime}\,,$noise_level_t,$\,s_{i})$ Predict added
noise

      5. v.

$L_{\theta}\leftarrow\,$compute_loss($\gamma,\,a^{\prime}-a_{i}\,$)

      6. vi.

Update $\theta$

  7. 7.

For inference: For sample s=$\\{x,o\\}$ in $D_{test}$

    1. (a)

Pass $x$ through the visual encoder $\phi$ to get embedding $e$

    2. (b)

Concatenate $o$ to $e$ to obtain $s$, where s is current observation

    3. (c)

$a=$ initialise_with_noise()

    4. (d)

For t in reversed(time_steps(0,T)):

      1. i.

$\gamma=\epsilon_{\theta}(a\,,$noise_level_t,$\,s)$

      2. ii.

$a=$ remove_noise($a,\,\gamma,\,$noise_level_t)

Figure 2: Schematic Model of our proposed DiffClone Framework

##  4 Experiments and Results

We tried multiple experiments and ablations with the visual representation and
agent policy, before arriving at our final agent- DiffClone. The following
sections give a brief overview of our initial experimentation, followed by our
results using DiffClone.

Figure 3: Agent learning policy gradually using DiffClone

###  4.1 Experiments with Visual Representation

To further highlight the information of the state of the robot and the
dynamics of successive steps, we tried to map the delta of successive
embeddings to the delta of their successive states using an MLP, and this
served as a contrastive objective. This gave a small increase in rewards and
retained the success rate with behaviour cloning. We also tried adding a small
objective of goal conditioning, where we approximated the goal to the final
state of the arm and then tried mapping each embedding to its corresponding
goal state. We also attempted to use the depth images given to extract a mask
using off-the-shelf models. These experiments did not lead to significant
improvement in rewards or success rates.

###  4.2 Experiments with Agent Policy

We tried various offline RL algorithms such as Q-learning, implicit
Q-Learning, and trajectory prediction transformers, but these did not yield
better results than the ones we got from direct behaviour cloning. We then
focused on experimenting with various sequence modeling architectures such as
RNNs, LSTMs, and Diffusion Policy. Diffusion policy gave significantly better
results than plain behaviour cloning in simulation. In the next section, we
will discuss the ablations and hyper-parameter tuning we did with diffusion
policy to arrive at our best submission.

We performed ablation studies by varying architecture and hyper-parameters
such as de-noising steps, sub-sampling period, horizon length, open loop-
closed loop inference, and the architecture used for the DDPM. These
experiments are discussed in the following section.

###  4.3 Experiments with Diffusion Policy

####  4.3.1 Architectural Choice

Inspired by Motion Diffusion Models [Tevet et al. (2022)], a transformer
encoder architecture was used for the noise prediction network. We first
concatenated the current observation with the sequence of actions and used an
Attention mechanism to learn the conditional mappings and the temporal change
across the sequence of actions. This method did not successfully learn the
diversity of the actions, resulting in a mode collapse. As a possible
solution, we could have experimented with a Diversity Loss across batches, but
we were unable to experiment further with the due to compute constraints.
Also, Transformer encoders are known to be very sensitive to hyperparameters.
Hence, we shifted to 1D-Conditional U-Net architectures for the same task,
which was able to capture the distribution. We experimented with directly
predicting the denoised sample instead of the added noise, but it didn’t give
any improvement. We also could not experiment with training the encoder +
diffusion policy in an end-to-end fashion due to compute limitations. This
approach may give better results.

Model | Mean Reward | Success Rate(%)
---|---|---
DiffClone (U-Net based) (Ours) | 51 | 92
DiffClone (Transformer based) (Ours) | 5.6 | 24
MoCo + BC (TOTO) | 20.33 | 68
Table 1: Our simulation evaluation on Pouring Model | Mean Reward | Success Rate(%)
---|---|---
DiffClone (130 Epochs) (Ours) | 7.833 | 33.33
DiffClone (2000 Epochs) (Ours) | 6.417 | 33.33
MoCo + BC (TOTO) | 22.86 | 72.2
DiffClone (Subsampling 3) (Ours) | 0 | 0
Table 2: Real Robot Testing on Pouring Model | Mean Reward | Success Rate(%)
---|---|---
DiffClone (130 Epochs) (Ours) | 6.91 | 58.30
DiffClone (600 Epochs) (Ours) | 2 | 25
MoCo + BC (TOTO) | 7.42 | 83.3
Table 3: Real Robot Testing on Scooping

####  4.3.2 Hyperparameters

We used the DDPM scheduler from Diffusers Library [von Platen et al. (2022)]
during training with 50 time steps. When the same scheduling is used during
inference, it may lead to inconsistent and broken motion of the robot arm due
to high latency. We can use DDIM with fewer time steps to resolve this issue
and improve latency.

Sub-sampling of actions from training data proved to be an influential factor
in training stability. Higher sub-sampling rates (> 3) resulted in the loss of
information, which led to sub-optimal policies and mode collapse in some
instances. On the other hand, lower sub-sampling rates increase the number of
data points used in training, thus allowing us to cover the entire dataset,
but as a trade-off resulted in increased training. We found that in
simulation, a sub-sampling period of 1 gives the best results.

We use a prediction horizon length of sixteen but recalculate after every
eight action steps. This is a trade-off between goal-aware long-horizon
trajectories and the responsiveness of the policy, as mentioned above in the
methodology. We have added the hyperparameter table in the appendix.

Our algorithm performed extremely well and significantly better than TOTO-
baselines in our evaluation in simulation but failed to do so in real-world
testing. This may be due to the fact that they do not generalize well or are
very sensitive to hyper-parameters that need to be readjusted for efficient
and successful real-world implementation. We plan to explore and re-evaluate
this concern as a part of our future work.

##  5 Conclusion

In this paper, we introduce DiffClone, a diffusion-based behavior cloning
agent that performs complex robot manipulation tasks from offline data. Our
method captures complex distributions and efficiently preserves its multi-
modality, thus solving the offline RL problem in an efficient and robust
fashion. To this end, we adopted a state-of-the-art diffusion-policy-based
approach along with a MoCo fine-tuned Resnet50 visual backbone. In our
evaluation, we found that our policies achieved high scores compared to the
established baselines in the simulation. At the same time, we observed that
diffusion policies are very susceptible to changes in hyper-parameters, such
as the number of de-noising time steps and sub-sampling period. In our future
work, we plan to implement DDIM for improved latency during inference and
explore regularisation methods such as KL regularisation to enable a more
robust transfer to real-time environments. We plan to further explore methods
for the efficient transfer of our algorithm from simulation to the real world.
The code of our work is open-sourced at the following link:
https://github.com/sirabas369/DiffClone.git, the project website is as
follows: https://sites.google.com/view/iitkgp-nips23toto/home, it also
contains a few working videos of our trained policies.

###  5.1 Acknowledgement

We would like to take this opportunity to thank the organisers of TOTO
benchmark challenge for giving us an opportunity to explore this paradigm and
for open-sourcing their code and dataset. We would also like to thank the
authors of Diffusion Policy, for open-sourcing their implementation of the
CNN-based diffusion policy, and Dr. P. P. Chakrabarti (Professor, Dept. of
CSE, IIT Kharagpur), for his invaluable inputs and guidance.

## References

  * Kostrikov et al. (2021) Ilya Kostrikov, Ashvin Nair, and Sergey Levine.  Offline reinforcement learning with implicit q-learning, 2021.
  * Chen et al. (2021) Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch.  Decision transformer: Reinforcement learning via sequence modeling, 2021.
  * Fujimoto and Gu (2021) Scott Fujimoto and Shixiang Shane Gu.  A minimalist approach to offline reinforcement learning, 2021.
  * Russakovsky et al. (2015) Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  Imagenet large scale visual recognition challenge, 2015.
  * Wang et al. (2019) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.  Glue: A multi-task benchmark and analysis platform for natural language understanding, 2019.
  * Dasari et al. (2022) Sudeep Dasari, Jianren Wang, Joyce Hong, Shikhar Bahl, Yixin Lin, Austin Wang, Abitha Thankaraj, Karanbir Chahal, Berk Calli, Saurabh Gupta, David Held, Lerrel Pinto, Deepak Pathak, Vikash Kumar, and Abhinav Gupta.  Rb2: Robotic manipulation benchmarking with a twist, 2022.
  * Zhou et al. (2023) Gaoyue Zhou, Victoria Dean, Mohan Kumar Srirama, Aravind Rajeswaran, Jyothish Pari, Kyle Hatch, Aryan Jain, Tianhe Yu, Pieter Abbeel, Lerrel Pinto, Chelsea Finn, and Abhinav Gupta.  Train offline, test online: A real robot learning benchmark, 2023.
  * Ronneberger et al. (2015) Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation, 2015.
  * Chi et al. (2023) Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song.  Diffusion policy: Visuomotor policy learning via action diffusion, 2023.
  * Kumar et al. (2022) Aviral Kumar, Joey Hong, Anikait Singh, and Sergey Levine.  When should we prefer offline reinforcement learning over behavioral cloning?, 2022.
  * Grill et al. (2020) Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko.  Bootstrap your own latent: A new approach to self-supervised learning, 2020.
  * Richemond et al. (2023) Pierre H. Richemond, Allison Tam, Yunhao Tang, Florian Strub, Bilal Piot, and Felix Hill.  The edge of orthogonality: A simple view of what makes byol tick, 2023.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning, 2020.
  * van den Oord et al. (2019) Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding, 2019.
  * Pomerleau (1988) Dean A. Pomerleau.  Alvinn: An autonomous land vehicle in a neural network.  In D. Touretzky, editor, _Advances in Neural Information Processing Systems_ , volume 1. Morgan-Kaufmann, 1988.
  * Pari et al. (2021) Jyothish Pari, Nur Muhammad Shafiullah, Sridhar Pandian Arunachalam, and Lerrel Pinto.  The surprising effectiveness of representation learning for visual imitation, 2021.
  * He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition, 2015.
  * Perez et al. (2017) Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville.  Film: Visual reasoning with a general conditioning layer, 2017.
  * Vaswani et al. (2023) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2023.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models, 2020.
  * Nichol and Dhariwal (2021) Alex Nichol and Prafulla Dhariwal.  Improved denoising diffusion probabilistic models, 2021.
  * Tevet et al. (2022) Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H. Bermano.  Human motion diffusion model, 2022.
  * von Platen et al. (2022) Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, and Thomas Wolf.  Diffusers: State-of-the-art diffusion models, 2022.

##  Appendix A Appendix

###  A.1 Hyperparameters

Hyperparameter | Value
---|---
Batch Size | 128
Prediction Horizon | 16
Execution Horizon | 8
Sub-sample Period | 1
Denoising time-steps | 50
Action Dimension | 7
Learning rate | 1e-4
Table 4: Hyperparameters used for the MoCo DiffClone model

###  A.2 Other Experiments

Model | Mean Reward | Success Rate(%)
---|---|---
MoCo + BC (TOTO) | 20.33 | 68
$\delta$ observations -$\delta$ actions(transformer encoder) | 15 | 36
$\delta$ observations -$\delta$ actions(MLP ) | 19 | 72
$\delta$ observations -$\delta$ joint$\\_$states(MLP) | 31.33 | 68
$\delta$ observations -$\delta$ joint$\\_$states(transformer encoder) | 11 | 32
Table 5: Our simulation experiments on Pouring

###  A.3 Training and Test Plots

Figure 4: Pouring Training Loss Plot Figure 5: Pouring Simulation Test Loss
Plot Figure 6: Scooping Training Loss Plot