# A simple electronic ladder model harboring $\mathbb{Z}_{4}$ parafermions

Botond Osváth  Department of Physics of Complex Systems, ELTE Eötvös Loránd
University, H-1117, Budapest, Hungary  Gergely Barcza  Wigner Research Centre
for Physics, H-1525, Budapest, Hungary  MTA–ELTE Lendület ”Momentum” NewQubit
Research Group, Pázmány Péter, Sétány 1/A, 1117 Budapest, Hungary  Örs Legeza
Wigner Research Centre for Physics, H-1525, Budapest, Hungary  Institute for
Advanced Study, Technical University of Munich, Lichtenbergstrasse 2a, 85748
Garching, Germany  Balázs Dóra  Department of Theoretical Physics, Institute
of Physics, Budapest University of Technology and Economics, Műegyetem rkp.
3., H-1111 Budapest, Hungary  László Oroszlány  Department of Physics of
Complex Systems, ELTE Eötvös Loránd University, H-1117, Budapest, Hungary
Wigner Research Centre for Physics, H-1525, Budapest, Hungary
<EMAIL_ADDRESS>
###### Abstract

Parafermions are anyons with the potential for realizing non-local qubits that
are resilient to local perturbations. Compared to Majorana zero modes,
braiding of parafermions implements an extended set of topologically protected
quantum gates. This, however, comes at the price that parafermionic zero modes
can not be realized in the absence of strong interactions whose theoretical
description is challenging. In the present work, we construct a simple lattice
model for interacting spinful electrons with parafermionic zero energy modes.
The explicit microscopic nature of the considered model highlights new
realization avenues for these exotic excitations in recently fabricated
quantum dot arrays. By density matrix renormalization group calculations, we
identify a broad range of parameters, with well-localized zero modes, whose
parafermionic nature is substantiated by their unique $8\pi$ periodic
Josephson spectrum.

Introduction. As outlined at the turn of the century, key requirements for the
realization of a quantum computer are qubits with long decoherence times and a
universal set of robust quantum gates acting on them DiVincenzo (2000).
Topological quantum computers, where quantum information is encoded in non-
local quasiparticles, promise to address these issues at the hardware level
Nayak  _et al._ (2008); Aasen  _et al._ (2016). Majorana fermions Kitaev
(2001) and their fractionalized counterparts, parafermions Fendley (2012), are
such excitations in topological superconductors Alicea (2012); Alicea and
Fendley (2016). Parafermions are classified by a $\mathbb{Z}_{d}$ index with
$d=2$ corresponding to Majorana fermions. The characteristic non-Abelian
exchange statistics of parafermionic excitations can be exploited to implement
topologically protected quantum gates. Braiding of $d>2$ parafermion zero
modes implements an extended set of quantum operations compared to Majorana
zero modes Hutter and Loss (2016).

Blueprints for realizing parafermions rely on electron-electron interactions
Alicea and Fendley (2016). Furthermore, some of these proposals require
combining superconductivity and fractional quantum Hall edge modes, whose
realization is challenging as the magnetic field needed for stabilizing the
quantum Hall phase is detrimental for conventional superconductors Clarke  _et
al._ (2013); Lindner  _et al._ (2012). Several recent theoretical works have
proposed platforms for realizing $\mathbb{Z}_{4}$ parafermion zero modes
circumventing this conundrum. These approaches rely on strongly interacting
quantum spin Hall (QSH) insulators coupled to superconductors and do not
require a substantial external magnetic field Zhang and Kane (2014a); Orth
_et al._ (2015). In these proposals, interactions open a gap in the edge
states while preserving time-reversal symmetry. Parafermion modes emerge at
interfaces between edge regions gapped by interactions and proximity-coupled
superconducting sections. The $\mathbb{Z}_{4}$ parafermionic modes manifest a
four-fold degenerate ground state and are characterized by $8\pi$-periodic
fractional Josephson effect. Interestingly, this effect can be realized
without an extended region of strong interactions. In recent proposals, it was
suggested that coupling an impurity quantum spin to the Josephson junction
formed at a QSH edge can exhibit $8\pi$-periodicity Peng  _et al._ (2016);
Vinkler-Aviv  _et al._ (2017). Weak interactions in a constriction of
topological insulators can also stabilize $8\pi$ Josephson signal Fleckenstein
_et al._ (2019). The existence of parafermionic zero modes was also
investigated in spinless Klinovaja and Loss (2014a) and spinful Klinovaja and
Loss (2014b) nano-wire setups.

All the studies above regarding the physical implementation of parafermionic
phases rely on the bosonization technique, thus neglecting high energy or
lattice scale effectsGiamarchi (2004). Alternatively, lattice models can be
used to study the rich landscape defined by parafermionic excitations. Density
matrix renormalization group (DMRG) methods White (1992) have been applied to
investigate dynamical excitations in parafermionic chains Kaskela and Lado
(2021). There is a natural mapping between $\mathbb{Z}_{4}$ parafermionic and
spinful fermionic degrees of freedom since they both encode a four-dimensional
local Hilbert space. The algebraic structure of this mapping has recently been
thoroughly explored Chew  _et al._ (2018). The fermionized version of
parafermion chains has also been investigated, yielding an electronic lattice
model with exotic components, such as three-particle interactions as well as
occupation and spin-dependent hopping Calzona  _et al._ (2018); Teixeira and
Dias da Silva (2021).

Here, we follow a more realistic approach inspired by experimentally feasible
proposals utilizing the helical QSH edge modes. We construct a lattice model
for spinful electrons hosting $\mathbb{Z}_{4}$ parafermionic zero modes.
Utilizing the DMRG approach we explore its rich phase diagram and elucidate
the requirements for the stability of the anyonic phase. The simplicity of our
model highlights new realization avenues of parafermionic zero modes in highly
tunable quantum dot arrays.

The model. In the following, we introduce a ladder Hamiltonian acting on
spinful electrons which captures the essential properties of the edge states
of two-dimensional topological insulators without explicitly treating the
insulating bulk. We build on previous works where a similar lattice model was
applied for modelling free fermions Creutz (1999); Gholizadeh  _et al._
(2018); Sticlet  _et al._ (2013) and helical Majorana modes subject to
interactions Grover  _et al._ (2014); Li  _et al._ (2020). In the proposed
model each electronic site has local spin degrees of freedom denoted by
${\uparrow}$ and ${\downarrow}$ while the left and right leg of the ladder
will be referred to simply by the labels $\mathrm{L}$ and $\mathrm{R}$
respectively. We write the Hamiltonian of the system as the sum of three
parts:

$H=H_{\text{kin}}+H_{\text{sc}}+H_{\text{int}}\,.$ (1)

The first term describes the kinetic contributions, capturing propagation
along the legs and hopping across the rungs of the ladder,

$\displaystyle{\displaystyle H}_{\text{kin}}$
$\displaystyle=\sum_{m}c_{m}^{\dagger}\left(-\mu_{m}s_{0}\otimes\zeta_{0}+ts_{0}\otimes\zeta_{x}\right)c_{m}$
(2)
$\displaystyle-\frac{t}{2}\sum_{m}c_{m+1}^{\dagger}\left(is_{z}\otimes\zeta_{z}+s_{0}\otimes\zeta_{x}\right)c_{m}+\text{h.c.}\,.$

Here, $s_{\alpha}$ and $\zeta_{\alpha}$ are Pauli matrices acting on the spin
and leg degrees of freedom respectively, and
$c^{\dagger}_{m}=\begin{bmatrix}c^{\dagger}_{m,\mathrm{L},{\uparrow}}&c^{\dagger}_{m,\mathrm{R},{\uparrow}}&c^{\dagger}_{m,\mathrm{L},{\downarrow}}&c^{\dagger}_{m,\mathrm{R},{\downarrow}}\end{bmatrix}$
where $c^{\dagger}_{m,\zeta,s}$ denotes the creation operator of an electron
with spin projection $s\in\\{{\uparrow},{\downarrow}\\}$ on-site $m$ of leg
$\zeta\in\\{\mathrm{L},\mathrm{R}\\}$. $t$ serves as an overall energy scale
for the system, while $\mu_{m}$ is a site-dependent potential. A visual
representation of the different considered hopping processes is shown in Fig.
1 (a).

Figure 1: (a) Sketch of the kinetic term, detailing all single particle
hopping processes along the ladder. (b) Configuration of a finite size system
used for obtaining the phase diagram and excitation spectrum. Red shade
denotes the region of superconductivity with pair potential strength $\Delta$.
Blue color signals a region with interactions of strength $V$, homogeneous
potential $\mu$, and superconducting term with pair potential
$\Delta^{\prime}$. The green dashed cut denotes the interface at which the
entanglement entropy is obtained for the phase diagram calculations.

For low energies, the kinetic term (2) describes the propagation of helical
particles which can be made more explicit if we take the Fourier transform of
(2) along the ladder, by introducing operators
$c_{k}=\sum_{m}\mathrm{e}^{\mathrm{i}mk}c_{m}$. Thus the kinetic term, for
uniform values of the chemical potential $\mu_{m}=\mu$ becomes
$H_{\text{kin}}=\sum_{k}c^{\dagger}_{k}\mathcal{H}(k)c_{k}$. For low energies,
this term can be analyzed by the envelope function approximation at $k=0$
yielding

$\mathcal{H}(k)\approx-\mu s_{0}\otimes\zeta_{0}+kts_{z}\otimes\zeta_{z}\,.$
(3)

Crucially there are no terms mixing the two legs, thus for low-energy helical
particles, the two legs are decoupled just as they would be for two spatially
separated edges of a large two-dimensional topological insulator.

The second term in the Hamiltonian (1) describes proximity to an $s$-wave
superconductor,

$H_{\text{sc}}=\sum_{m,\zeta}\Delta_{m,\zeta}\left[c_{m,\zeta,{\uparrow}}^{\dagger}c_{m,\zeta,{\downarrow}}^{\dagger}+\text{h.c.}\right],$
(4)

with a site and leg-dependent pair potential $\Delta_{m,\zeta}$.

The last term, $H_{\text{int}}$, captures a particular, albeit simple form of
interaction among charge carriers in the system inspired by the bosonized
calculation of Orth and coworkersOrth  _et al._ (2015):

$H_{\text{int}}=\sum_{m,\zeta}V_{m,\zeta}\left[c_{m,\zeta,{\uparrow}}^{\dagger}c_{m,\zeta,{\downarrow}}c_{m+1,\zeta,{\uparrow}}^{\dagger}c_{m+1,\zeta,{\downarrow}}+\text{h.c.}\right]\,.$
(5)

Introducing
$c^{\dagger}_{m,\zeta}=\begin{bmatrix}c^{\dagger}_{m,\zeta,{\uparrow}}&c^{\dagger}_{m,\zeta,{\uparrow}}\end{bmatrix}$
and rewriting the interaction term with operators
$S^{\alpha}_{m,\zeta}=c^{\dagger}_{m,\zeta}s_{\alpha}c_{m,\zeta}$ we get

$H_{\text{int}}=\sum_{m,\zeta}\frac{V_{m,\zeta}}{2}\left[S^{x}_{m,\zeta}S^{x}_{m+1,\zeta}-S^{y}_{m,\zeta}S^{y}_{m+1,\zeta}\right],$
(6)

that is, this term describes an anisotropic symmetric exchange coupling of
strength $V_{m,\zeta}$ between electron spins localized to neighboring sites
along a leg. The relevance of local anisotropy and anisotropic exchange
interaction in this setting was recognised in the context of impurity spins at
the QSH edge Peng  _et al._ (2016); Vinkler-Aviv  _et al._ (2017).

An important aspect of this model is that it can be used to effectively
circumvent fermion doubling Montvay and Münster (1994), without breaking time-
reversal symmetry albeit at the cost of dismissing charge conservation. For
instance in a configuration depicted in Fig. 1 (b) a fixed, large value of
$\Delta$ gaps the whole red region thus allowing the exploration of effects of
arbitrary local interactions on the helical states localized in the blue
region.

Figure 2: Phase diagram for model setup shown in Fig. 1(b) as a function of
interaction strength $V$ for $N_{1}=N_{3}=20$ and $N_{2}=100$ and
$\Delta/t=1.0$. In subfigure (a) the chemical potential $\mu$, while in (b)
the pair potential $\Delta^{\prime}$ is varied.

Phase diagram. We explore the phase diagram of the proposed model (1) with
DMRG Fishman  _et al._ (2020); Legeza  _et al._ (2020); SM  as it is an ideal
tool for characterizing quasi-one-dimensional systems. Quantum phase
transitions can be detected by the anomaly of entanglement measures Wootters
(1998); Osborne and Nielsen (2002); Osterloh  _et al._ (2002); Legeza and
Sólyom (2006), in practice we measured the half system von Neumann entropy
Mund  _et al._ (2009) for the geometry shown in Fig. 1 (b). The obtained
entanglement entropy maps are shown in Fig. 2. For $\Delta^{\prime}=0$ three
separate phases are clearly discernible see Fig. 2 (a). Allowing for finite
$\Delta^{\prime}$ shown in Fig. 2 (b), for small enough interactions any
finite value of $\Delta^{\prime}$ gaps the system thus pushing the middle
region of the system where interactions are active (bluish part in Fig. 1 (b))
to the same superconducting phase as the rest. A hallmark of this is the
marked drop in entanglement entropy as $\Delta^{\prime}$ is increased. For
large enough interaction strength, the two phases observed for
$\Delta^{\prime}/t=0$ remain stable even for large $\Delta^{\prime}$.

Excitation spectrum. The phase with moderate interaction strength is endowed
with one key indicator of the presence of $\mathbb{Z}_{4}$ parafermions, in
the form of zero energy excitation. This is evident in Fig. 3 where we show
the low energy many-body excitation spectrum of our model for
$\Delta^{\prime}/t=\mu/t=0$ as the function of the interaction strength $V$.
For weak interaction, up to around $V/t=1.5$, the system has a well-defined
ground state with even fermion parity, and the first excited state is a doubly
degenerate odd state. In the thermodynamic limit, the spectrum of this phase
shows a metallic character with a vanishing excitation gap SM . For
intermediate interaction strengths, beyond the reach of perturbation theory, a
phase with a fourfold degenerate ground state emerges with a considerable gap
in the excitation spectrum. This spectral structure is found to be independent
of the size of the system SM . At around $V/t=3$ a second phase transition is
observed. For stronger interaction strengths, the fourfold degeneracy is
lifted and the gap increases linearly with $V$.

The phase at moderate interaction strengths with fourfold degeneracy shall be
the focus for the remainder of our work.

Figure 3: Excitation spectrum of the considered model as the function of the
interaction strength $V$. For $N_{i}=20$ and $\mu/t=0$ in the configuration
depicted in Fig. 1 (b). The two lowest energy even and odd parity states are
marked by $|e_{i}\rangle$ and $|o_{i}\rangle$ with $i\in\\{1,2\\}$. Figure 4:
Different local quantities evaluated in the fourfold degenerate subspace with
$V/t=2.2$ and $\Delta^{\prime}=\mu=0$ for a setup depicted in Fig. 1 (b) with
$N_{1}=N_{3}=20$ and $N_{2}=40$. (a) and (b) depicts the expectation value of
the local spin momentum $\langle e_{1}|S^{y}_{m,\zeta}|e_{1}\rangle$ and
$\langle e_{2}|S^{y}_{m,\zeta}|e_{2}\rangle$ for the even subspace. Up to
numerical precision, states in the odd subspace, $|o_{1}\rangle$ and
$|o_{1}\rangle$, show the same pattern. (c) depicts the matrix element
$\sum_{s}\left|\langle o_{1}|c_{m,\zeta,s}|e_{1}\rangle\right|^{2}$ of the
local annihilation operator $c_{m,\zeta,s}$ between states of the even and odd
subspace with the same magnetization pattern.

Local characteristics. The four-fold degeneracy of the ground state is
resilient against fluctuations of the electrostatic potential. The expectation
value of the local electron density
$n_{m,\zeta}=\sum_{s}c^{\dagger}_{m,\zeta,s}c_{m,\zeta,s}$, as confirmed by
our calculations, is the same for all four states. The matrix element between
two distinct states is exactly zero.

On the contrary, breaking time-reversal symmetry with a local magnetic field
breaks the degeneracy of the ground state. For instance, as can be observed in
Fig. 4 (a) and (b) the expectation value of $S^{y}_{m\zeta}$ is sharply
localized to the region with interactions and it differs in sign for the two
states in the even subspace $|e_{1}\rangle$ and $|e_{2}\rangle$. The other
matrix elements related to the local spin operators were found to be
negligible. Similar behaviour is observed in the odd subspace. This
magnetization pattern exhibited by the ground state manifold is a clear
consequence of the considered interaction (6) favouring an anisotropic spin
configuration.

Zero energy single particle excitations, in the ground state manifold, are
exponentially localized to the interfaces of the superconducting and
interacting regions, as depicted in Fig. 4 (c). This is again an indicator for
two parafermionic zero modes attached to the boundary of the interacting
region. However in order to clearly distinguish these zero-energy excitations
from time-reversal invariant pair of Majorana bound statesHaim  _et al._
(2014) further analysis is necessary.

Josephson spectrum. The Josephson spectrum, depicted in Fig. 5 decisively
reveals the presence of parafermionic zero modes in the considered model. The
evolution of the energy of the localized four modes as the function of phase
bias $\varphi$ in between two superconducting terminals can be used to
characterize anyonic excitationsFu and Kane (2009). In particular, time-
reversal invariant Majorana modes show a $4\pi$ periodic modulationZhang and
Kane (2014b); Zhang  _et al._ (2013), while $\mathbb{Z}_{4}$ parafermions
exhibit $8\pi$ periodicity Alicea and Fendley (2016).

The introduced model allows for the isolation of the Josephson current passing
through two parafermionic excitations localized at the boundary of the
interacting region SM . In Fig. 5 we consider a short junction of $N=8$ sites,
thus the degeneracy is slightly lifted. At $\varphi=0$ time-reversal symmetry
protects the twofold degeneracy of states with odd parity while no such
restrictions apply for even states. Initializing the system in
$|\psi_{1}\rangle$ we can continuously follow its progression as the phase
bias $\varphi$ is tuned. At $\varphi=\pi$ as the consequence of the joint
manifestation of the parity anomaly and time-reversal symmetry, the spectrum
exhibits two distinct degeneracies Vinkler-Aviv  _et al._ (2017). Continuing
further a crossing between $\varphi=\pi$ and $\varphi=3\pi/2$ is protected by
local parity of the junction. Arriving at $\varphi=2\pi$ the system evolves
smoothly in $|\psi_{2}\rangle$. We need a further three cycles, that is a
total of $8\pi$ shift of the phase bias to recover the initial state.

Figure 5: Schematic representation of a phase bias induced Josephson current
crossing two parafermionic zero modes (left panel) and Josephson spectrum
(right panel) of a junction with $N=8$ sites and interaction strength
$V/t=2.2$.

_New experimental avenues._ The introduced ladder model implies a new bottom-
up approach for realizing parafermionic excitations in quantum dot arrays.
Ladder-like geometries consisting of fourHendrickx  _et al._ (2021) and more
recently eight quantum dotsHsiao  _et al._ (2023) have been fabricated where
hopping, spin-orbit coupling and interactions could be controlled with high
accuracy. Quantum dot arrays in conjunction with topological superconductivity
have been studied theoreticallyLiu  _et al._ (2022) and realized
experimentallyDvir  _et al._ (2023); Bordin  _et al._ (2023). Crucially in
these systems the relevant energy scales for superconducting pair potential,
hopping, and spin-orbit coupling can be tuned and are roughly in the same
typical energy scale of 10-100 meV, thus providing a versatile platform for
implementing the proposed model. Ultracold atomic ladders Li  _et al._ (2013)
in combination with spin-orbit coupling mimicking mechanismsLin  _et al._
(2011); Galitski and Spielman (2013); Valdés-Curiel  _et al._ (2021) could
also provide an alternative route for the realization of our model. In line
with frameworks employing the QSH phase layered van der Waals materials may
provide a fresh realization pathway as all necessary ingredients can be found
in these systems. The QSH effect has been observed in WSe2Ugeda  _et al._
(2018), superconductivity was measured in twisted bilayer grapheneCao  _et
al._ (2018) and in 2H-NbS2 samplesDevarakonda  _et al._ (2020). Anisotropic
exchange interaction, crucial for our model, can be also realized using
magnetic van der Waals materials Telford  _et al._ (2023).

Conclusions. We constructed a lattice model with explicit time-reversal
symmetry which is capable of describing the edge states of a topological
insulator in the presence of superconductivity and interaction. We identified
the different phases in the model, in particular, we found a parameter regime
where the model hosts parafermionic excitations. Our model is thus capable of
capturing beyond Majorana physics, therefore it serves as an important
stepping stone for simulating topological quantum information processing
devices. Our model highlights new realization avenues for these exotic
excitations in quantum dot arrays.

Acknowledgements. Fruitful discussions with András Pályi, János Asbóth, Yuval
Oreg, Jelena Klinovaja, Daniel Loss, Elsa Prada, Miles Stoudenmire, Gergely
Markó, and Alberto Cortijo are greatly acknowledged. This work was supported
by the Ministry of Innovation and Technology and the National Research,
Development and Innovation Office of Hungary (NKFIH) within the National
Quantum Technology Program (Grant No. 2022-2.1.1- NL-2022-00004) and project
K-115608, K142179. L.O. acknowledges the support of the Bolyai Research
Scholarship of MTA and the Bolyai Plusz Scholarship of ÚNKP. Ö.L. has been
supported by NKFIH through Grant Nos. K134983 and TKP2021-NVA-04, by the Hans
Fischer Senior Fellowship programme funded by the Technical University of
Munich - Institute for Advanced Study, and by the Center for Scalable and
Predictive methods for Excitation and Correlated phenomena (SPEC), funded as
part of the Computational Chemical Sciences Program by the U.S. Department of
Energy(DOE), Office of Science, Office of Basic Energy Sciences, Division of
Chemical Sciences, Geosciences, and Biosciences at Pacific Northwest National
Laboratory. We acknowledge KIFÜ for awarding us access to computational
resources based in Hungary.

## References

  * DiVincenzo (2000) D. P. DiVincenzo, Fortschritte der Physik 48, 771 (2000).
  * Nayak  _et al._ (2008) C. Nayak, S. H. Simon, A. Stern, M. Freedman, and S. Das Sarma, Rev. Mod. Phys. 80, 1083 (2008).
  * Aasen  _et al._ (2016) D. Aasen, M. Hell, R. V. Mishmash, A. Higginbotham, J. Danon, M. Leijnse, T. S. Jespersen, J. A. Folk, C. M. Marcus, K. Flensberg, and J. Alicea, Phys. Rev. X 6, 031016 (2016).
  * Kitaev (2001) A. Y. Kitaev, Physics-Uspekhi 44, 131 (2001).
  * Fendley (2012) P. Fendley, Journal of Statistical Mechanics: Theory and Experiment 2012, P11020 (2012).
  * Alicea (2012) J. Alicea, Reports on Progress in Physics 75, 076501 (2012).
  * Alicea and Fendley (2016) J. Alicea and P. Fendley, Annual Review of Condensed Matter Physics 7, 119 (2016).
  * Hutter and Loss (2016) A. Hutter and D. Loss, Physical Review B 93, 125105 (2016).
  * Clarke  _et al._ (2013) D. J. Clarke, J. Alicea, and K. Shtengel, Nature communications 4, 1348 (2013).
  * Lindner  _et al._ (2012) N. H. Lindner, E. Berg, G. Refael, and A. Stern, Phys. Rev. X 2, 041002 (2012).
  * Zhang and Kane (2014a) F. Zhang and C. L. Kane, Phys. Rev. Lett. 113, 036401 (2014a).
  * Orth  _et al._ (2015) C. P. Orth, R. P. Tiwari, T. Meng, and T. L. Schmidt, Physical Review B 91, 081406 (2015).
  * Peng  _et al._ (2016) Y. Peng, Y. Vinkler-Aviv, P. W. Brouwer, L. I. Glazman, and F. von Oppen, Phys. Rev. Lett. 117, 267001 (2016).
  * Vinkler-Aviv  _et al._ (2017) Y. Vinkler-Aviv, P. W. Brouwer, and F. von Oppen, Phys. Rev. B 96, 195421 (2017).
  * Fleckenstein  _et al._ (2019) C. Fleckenstein, N. T. Ziani, and B. Trauzettel, Phys. Rev. Lett. 122, 066801 (2019).
  * Klinovaja and Loss (2014a) J. Klinovaja and D. Loss, Phys. Rev. Lett. 112, 246403 (2014a).
  * Klinovaja and Loss (2014b) J. Klinovaja and D. Loss, Phys. Rev. B 90, 045118 (2014b).
  * Giamarchi (2004) T. Giamarchi,  _Quantum Physics in One Dimension_ (Oxford University Press, Oxford, 2004).
  * White (1992) S. R. White, Phys. Rev. Lett. 69, 2863 (1992).
  * Kaskela and Lado (2021) V. Kaskela and J. L. Lado, Phys. Rev. Res. 3, 013095 (2021).
  * Chew  _et al._ (2018) A. Chew, D. F. Mross, and J. Alicea, Phys. Rev. B 98, 085143 (2018).
  * Calzona  _et al._ (2018) A. Calzona, T. Meng, M. Sassetti, and T. L. Schmidt, Physical Review B 98, 201110 (2018).
  * Teixeira and Dias da Silva (2021) R. L. R. C. Teixeira and L. G. G. V. Dias da Silva, Phys. Rev. Res. 3, 033014 (2021).
  * Creutz (1999) M. Creutz, Phys. Rev. Lett. 83, 2636 (1999).
  * Gholizadeh  _et al._ (2018) S. Gholizadeh, M. Yahyavi, and B. Hetényi, Europhysics Letters 122, 27001 (2018).
  * Sticlet  _et al._ (2013) D. Sticlet, B. Dóra, and J. Cayssol, Phys. Rev. B 88, 205401 (2013).
  * Grover  _et al._ (2014) T. Grover, D. Sheng, and A. Vishwanath, Science 344, 280 (2014).
  * Li  _et al._ (2020) C. Li, H. Ebisu, S. Sahoo, Y. Oreg, and M. Franz, Phys. Rev. B 102, 165123 (2020).
  * Montvay and Münster (1994) I. Montvay and G. Münster,  _Quantum Fields on a Lattice_, Cambridge Monographs on Mathematical Physics (Cambridge University Press, 1994).
  * Fishman  _et al._ (2020) M. Fishman, S. R. White, and E. M. Stoudenmire, “The ITensor software library for tensor network calculations,” (2020), arXiv:2007.14822 .
  * Legeza  _et al._ (2020) Ö. Legeza, L. Veis, and T. Mosoni, “QC-DMRG-Budapest, a program for model and ab initio DMRG calculations,” (2020).
  * (32) See further details in the Supplementary Material.
  * Wootters (1998) W. K. Wootters, Phys. Rev. Lett. 80, 2245 (1998).
  * Osborne and Nielsen (2002) T. J. Osborne and M. A. Nielsen, Phys. Rev. A 66, 032110 (2002).
  * Osterloh  _et al._ (2002) A. Osterloh, L. Amico, G. Falci, and R. Fazio, Nature 416, 608 (2002).
  * Legeza and Sólyom (2006) O. Legeza and J. Sólyom, Phys. Rev. Lett. 96, 116401 (2006).
  * Mund  _et al._ (2009) C. Mund, O. Legeza, and R. M. Noack, Phys. Rev. B 79, 245130 (2009).
  * Haim  _et al._ (2014) A. Haim, A. Keselman, E. Berg, and Y. Oreg, Phys. Rev. B 89, 220504 (2014).
  * Fu and Kane (2009) L. Fu and C. L. Kane, Phys. Rev. B 79, 161408 (2009).
  * Zhang and Kane (2014b) F. Zhang and C. L. Kane, Phys. Rev. B 90, 020501 (2014b).
  * Zhang  _et al._ (2013) F. Zhang, C. L. Kane, and E. J. Mele, Phys. Rev. Lett. 111, 056402 (2013).
  * Hendrickx  _et al._ (2021) N. W. Hendrickx, W. I. Lawrie, M. Russ, F. van Riggelen, S. L. de Snoo, R. N. Schouten, A. Sammak, G. Scappucci, and M. Veldhorst, Nature 591, 580 (2021).
  * Hsiao  _et al._ (2023) T.-K. Hsiao, P. C. Fariña, S. Oosterhout, D. Jirovec, X. Zhang, C. van Diepen, W. Lawrie, C.-A. Wang, A. Sammak, G. Scappucci, _et al._ , arXiv preprint arXiv:2307.02401 (2023).
  * Liu  _et al._ (2022) C.-X. Liu, G. Wang, T. Dvir, and M. Wimmer, Phys. Rev. Lett. 129, 267701 (2022).
  * Dvir  _et al._ (2023) T. Dvir, G. Wang, N. van Loo, C.-X. Liu, G. P. Mazur, A. Bordin, S. L. Ten Haaf, J.-Y. Wang, D. van Driel, F. Zatelli, _et al._ , Nature 614, 445 (2023).
  * Bordin  _et al._ (2023) A. Bordin, X. Li, D. van Driel, J. C. Wolff, Q. Wang, S. L. ten Haaf, G. Wang, N. van Loo, L. P. Kouwenhoven, and T. Dvir, arXiv preprint arXiv:2306.07696 (2023).
  * Li  _et al._ (2013) X. Li, E. Zhao, and W. Vincent Liu, Nature communications 4, 1523 (2013).
  * Lin  _et al._ (2011) Y.-J. Lin, K. Jiménez-García, and I. B. Spielman, Nature 471, 83 (2011).
  * Galitski and Spielman (2013) V. Galitski and I. B. Spielman, Nature 494, 49 (2013).
  * Valdés-Curiel  _et al._ (2021) A. Valdés-Curiel, D. Trypogeorgos, Q.-Y. Liang, R. Anderson, and I. Spielman, Nature communications 12, 593 (2021).
  * Ugeda  _et al._ (2018) M. M. Ugeda, A. Pulkin, S. Tang, H. Ryu, Q. Wu, Y. Zhang, D. Wong, Z. Pedramrazi, A. Martín-Recio, Y. Chen, _et al._ , Nature communications 9, 3401 (2018).
  * Cao  _et al._ (2018) Y. Cao, V. Fatemi, S. Fang, K. Watanabe, T. Taniguchi, E. Kaxiras, and P. Jarillo-Herrero, Nature 556, 43 (2018).
  * Devarakonda  _et al._ (2020) A. Devarakonda, H. Inoue, S. Fang, C. Ozsoy-Keskinbora, T. Suzuki, M. Kriener, L. Fu, E. Kaxiras, D. C. Bell, and J. G. Checkelsky, Science 370, 231 (2020).
  * Telford  _et al._ (2023) E. J. Telford, D. G. Chica, M. E. Ziebel, K. Xie, N. S. Manganaro, C.-Y. Huang, J. Cox, A. H. Dismukes, X. Zhu, J. P. S. Walsh, T. Cao, C. R. Dean, and X. Roy, Advanced Physics Research n/a, 2300036 (2023).


††thanks: These authors contributed equally††thanks: These authors contributed
equally

# Linearized Integrated Microwave Photonic Circuit for Filtering and Phase
Shifting

Gaojian Liu  Nonlinear Nanophotonics Group, MESA+ Institute of Nanotechnology,
University of Twente, Enschede, Netherlands  China Academy of Space Technology
(Xi’an), Xi’an, China  Kaixuan Ye  Nonlinear Nanophotonics Group, MESA+
Institute of Nanotechnology,
University of Twente, Enschede, Netherlands  Okky Daulay  Nonlinear
Nanophotonics Group, MESA+ Institute of Nanotechnology,
University of Twente, Enschede, Netherlands  Qinggui Tan  China Academy of
Space Technology (Xi’an), Xi’an, China  Hongxi Yu  China Academy of Space
Technology (Xi’an), Xi’an, China  David Marpaung<EMAIL_ADDRESS>Nonlinear Nanophotonics Group, MESA+ Institute of Nanotechnology,
University of Twente, Enschede, Netherlands

###### Abstract

Photonic integration, advanced functionality, reconfigurability, and high RF
performance are key features in integrated microwave photonic systems that are
still difficult to achieve simultaneously. In this work, we demonstrate an
integrated microwave photonic circuit that can be reconfigured for two
distinct RF functions, namely, a tunable notch filter and a phase shifter. We
achieved $>$50 dB high-extinction notch filtering over 6-16 GHz and 2$\pi$
continuously tunable phase shifting over 12-20 GHz frequencies. At the same
time, we implemented an on-chip linearization technique to achieve a spurious-
free dynamic range of more than 120 $\rm{dB}\cdot\rm{Hz}^{4/5}$ for both
functions. Our work combines multi-functionality and linearization in one
photonic integrated circuit, and paves the way to reconfigurable RF photonic
front-ends with very high performance.

## Introduction

The development of cognitive and intelligent radio frequency (RF) systems
calls for RF front-ends with programmable multiple functions Baylis2014solving
; Islam2020diode . Microwave photonic (MWP) circuits with high bandwidth and
flexible reconfigurability can play an important role in those modern RF
systems capmany2007microwave ; marpaung2019integrated ; Zhu2020Broadband . A
number of approaches have been proposed to achieve programmable RF photonic
circuits. Functionalities including filtering fandino2017monolithic ;
liu2020integrated ; daulay2020chip ; Chen2023Multiband ; tao2021hybrid , phase
shifting chew2022Inline ; porzi2018photonic ; McKay2019Brillouin ; lin2020high
; chew2019integrated , and beamforming zhu2020silicon ; Romero2022high ;
Lin2022low  have been realized in application-specific photonic integrated
circuits (PIC) that can achieve high performance. Another approach is to
create general-purpose integrated MWP circuits that can be configured for a
high number of functionalities. These circuits have been demonstrated in the
form of waveguide mesh or micro-disk resonator array zhuang2015programmable ;
perez2017multipurpose ; bogaerts2020 ; zhang2020photonic . Finally, cascaded
MWP systems that can simultaneously perform notch filtering and phase shifting
have also been demonstrated in Nguyen2016 ; Xue2012 .

To be relevant for real applications, MWP circuits need to be more than just
programmable. Sufficient RF performance such as low noise figure and high
spurious-free dynamic range (SFDR) is also required so that they can be placed
at the beginning of the RF front-end chain, where the broad bandwidth is most
needed.

However, to date, most demonstrated integrated MWP systems are plagued by poor
SFDR tao2021hybrid ; fandino2017monolithic ; Yao2022silicon ; LiuY2022tunable
. There is an opportunity to circumvent this problem by implementing
linearization techniques wu2019multi ; Wang2022high ; liu2021integrated . But
only very recently can the RF photonic functionality and linearization be
simultaneously achieved in the same system, for example in fiber-based phase
shifters bai2021large , or in a chip-based programmable filter
daulay2022ultrahigh . Extending this concept to the multi-functional circuit
will be important for the MWP field.

Fig. 1: Concept of the reconfigurable integrated MWP circuit with enhanced
dynamic range. (a) The micrograph of the fabricated chip. (b) Schematic of the
reconfigurable integrated MWP circuit. (c-d) The illustration of the achieved
functionalities.

In this work, we present a reconfigurable integrated MWP circuit that can
perform two distinct functionalities, i.e., a tunable notch filter and a
2$\pi$-range phase shifter, while maintaining an ultrahigh SFDR. The tunable
notch filter function can achieve rejection of more than 50 dB with a 3-dB
bandwidth of 315 MHz from 6 to 16 GHz. In the meantime, the tunable phase
shifter function can realize 2$\pi$ continuous tuning range from 12 to 20 GHz.
Moreover, both functions can provide a SFDR larger than 120
$\rm{dB}\cdot\rm{Hz}^{4/5}$. This work shows the important first step towards
high-performance integrated MWP subsystems with a large number of RF
functions.

## Results

### Reconfigurable Integrated MWP Circuit

The concept of our reconfigurable integrated MWP circuit is illustrated in
Fig. 1. It can serve as a notch filter or a phase shifter, with different
configurations. By manipulating the phases and amplitudes of multi-order
optical sidebands independently, both functionalities can also exhibit
ultrahigh dynamic range.

The multi-functional integrated MWP circuit is fabricated with the Si3N4
TriPleX process roeloffzen2018low ; worhoff2015triplex . Fig. 1(a) shows the
micrograph of the chip. It consists of two spectral de-interleavers, a tunable
attennuator, a phase shifter, and an array of all-pass ring resonators, as
illustrated in Fig. 1(b). The spectral de-interleavers are implemented with an
asymmetric Mach Zehnder interferometer (aMZI) loaded with 3 ring resonators
luo2010high , exhibiting flat-top complementary filter response with free
spectral range (FSR) of 160 GHz liu2021integrated . The array of all-pass ring
resonators (FSR of 50 GHz) between the two de-interleavers are with tunable
coupling ratio and round trip phase that can be controlled via thermo-optic
tuning. By applying different voltages to the microheaters, this chip can be
configured as a notch filter or a phase shifter while keeping an ultrahigh
SFDR, as indicated in Fig. 1(c) and (d).

The signal processing of both functionalities follows a similar procedure.
First, the de-interleaver separates the input optical spectrum into two paths.
The attenuator and phase shifter in the upper path change the relative phase
and amplitude between the two paths for wideband spectrum shaping. In the
lower path, an array of all-pass ring resonators performs narrowband phase and
amplitude tailoring with high precision. The signals in the two paths are then
recombined and directly coupled out of the chip, or coupled into the second
de-interleaver for further processing. By tailoring the amplitudes and phases
of the optical carrier and multi-order sidebands in such a manner, the
$3^{rd}$-order intermodulation distortion (IMD3) from beating products between
different optical sidebands can destructively interfere with each other,
leading to an ultrahigh dynamic range.

Fig. 2: High-rejection notch filter with ultrahigh dynamic range. (a) Measured
linearized RF notch response with rejection of 52 dB compared with the bench-
marked SSB RF notch filter. (b) The measured two-tone RF spectra at the output
of the photodetector for the linearized RF notch filter (blue) and single-
sideband (SSB) RF notch filter without linearization (red) with 28.7 dB
reduction of IMD3 power. (c) The measured spurious-free dynamic range (SFDR)
of the proposed linearized notch filter and SSB notch filter without
linearization at RF frequency of 9 GHz. The proposed linearized filter has a
record-high SFDR of 123.6 $\rm{dB}\cdot\rm{Hz}^{4/5}$. (d) The measured
rejection of the linearized notch filter and the SFDR at 9 GHz with the tuning
of the notch frequency.

### High-rejection notch filter with ultrahigh SFDR

We first configure this multi-functional MWP circuit into a high-rejection
notch filter with enhanced SFDR. A phase-modulated optical signal is sent to
the programmable photonic chip for spectral shaping. Because of the
nonlinearity of the phase modulator, the modulated optical signal consists of
multi-order sidebands, among which the $\pm~{}1^{st}$-order sidebands have a
$\pi$ phase difference. We use the spectral de-interleaver to first separate
the optical carrier and lower sidebands from the upper sidebands. Then, the
lower sidebands and optical carrier are attenuated to generate the asymmetric
double sideband (aDSB) spectrum and simultaneously satisfying the
linearization condition daulay2022ultrahigh . Meanwhile, an under-coupled ring
resonator imposes a shallow notch at the upper sideband to meet equal
amplitude and anti-phase conditions at the notch frequency. After that the
signal is recombined, and a high rejection MWP notch filter based on RF
interference is created marpaung2013si . Because of the destructive
interference between different IMD3 components, the linearity of the notch
filter can be significantly improved (See Supplementary Note for more
experimental details).

The performance of the linearized MWP notch filter is shown in Fig. 2. To
highlight the advantages of our filter, we compared it with a standard single
sideband (SSB) modulated MWP notch filter using the same ring response without
linearization. Our demonstrated notch filter has a notch rejection of 52 dB
and a 3-dB bandwidth of 315 MHz (at the center frequency of 12 GHz), as shown
in Fig. 2(a). We also performed the two-tone test to evaluate the linearity of
the demonstrated notch filter. The results shown in Fig. 2(b) exhibit an IMD3
suppression of 28.7 dB when the two-tone RF frequency is at 9 GHz with a
spacing of 10 MHz. The large suppression of the IMD3 makes the $5^{th}$-order
nonlinearity dominate, leading to SFDR improvement from 103.8
$\rm{dB}\cdot\rm{Hz}^{2/3}$ to 123.6 $\rm{dB}\cdot\rm{Hz}^{4/5}$ with a noise
floor of -164.5 dBm/Hz, as observed in Fig. 2 (c). Moreover, the filter keeps
high rejection and large SFDR during the tuning of the notch frequency as
shown in Fig. 2 (d) The filter shows a link gain of -26.3 dB and a noise
figure (NF) of 35.8 dB. This moderate gain and NF is because of the partially
destructive interference between two $1^{st}$-order sidebands (See
Supplementary Note for extended measurement results).

Fig. 3: 2$\pi$-range tunable phase shifter with ultrahigh dynamic range. (a)
The phase tuning of the proposed phase shifter over 2$\pi$. The phase response
is flat in the range of 12-20 GHz. (b) The measured two-tone RF spectra at the
output of the photodetector for the linearized RF phase shifter (blue) and
bench-marked RF phase shifter without linearization (red) with 20.0 dB
improvement of fundamental to IMD3 ratio. (c) The measured spurious-free
dynamic range (SFDR) of the proposed linearized phase shifter and bench-marked
phase shifter without linearization at RF frequency of 14 GHz. The proposed
phase shifter exhibits a record-high SFDR of 121.2
$\rm{dB}\cdot\rm{Hz}^{4/5}$.

### 2$\pi$-range phase shifter with ultrahigh SFDR

The chip can also be reconfigured as a 2$\pi$-range tunable phase shifter
while maintaining the ultrahigh dynamic range. To realize the phase shifting
function, we first use an external band pass filter (BPF) to remove the lower
sideband of the phase-modulated optical signal. We also attenuate the optical
carrier with the BPF by positioning the optical carrier at the transition band
for the linearization purpose. The preprocessed single-sideband (SSB) optical
signal is then coupled into the chip. We separate the optical carrier and the
$+1^{st}$-order sideband from the $+2^{nd}$-order sideband with the spectral
de-interleaver. By applying two over-coupled ring resonators at the optical
carrier, 2$\pi$-range phase shift is introduced with negligible amplitude
variation. The optical phase shift is then converted to the RF phase shift
when the optical carrier beats with the $+1^{st}$-order sideband at the
photodetector (See the Supplementary Note for the experimental setup). To
suppress the IMD3 and improve the linearity of the phase shifter, both
amplitude and phase conditions between the optical carrier and the
$+2^{nd}$-order sideband need to be satisfied (See the Supplementary Note for
the linearization method). On the one hand, we adjust the power ratio between
the optical carrier and $+2^{nd}$-order sideband by controlling the passband
location of the external BPF and the coupling coefficients of the ring
resonators. On the other hand, we tune the phase of the $+2^{nd}$-order
sideband with an optical phase shifter to satisfy the phase condition, while
the MWP phase shifter tuning over the 2$\pi$ range.

The performance of the demonstrated phase shifter is presented in Fig. 3. We
can realize continuous phase tuning over 2$\pi$ range by adjusting the
resonance frequencies of the two over-coupled rings. Because of the relatively
flat stopband response of the two cascaded ring resonators, the amplitude
fluctuation of the phase shifter is less than 1 dB over the whole 2$\pi$
range, as shown in Fig. 3 (a). To evaluate the linearity of the demonstrated
phase shifter, we also conducted a two-tone test and compared the results with
a phase shifter without linearization for the same phase shift (0.6 $\pi$ in
this case). As shown in Fig. 3(b), the linearized phase shifter exhibits an
improved fundamental to IMD3 ratio of 20 dB. As a result, the SFDR of the
phase shifter increases from 105.2 $\rm{dB}\cdot\rm{Hz}^{2/3}$ to 121.2
$\rm{dB}\cdot\rm{Hz}^{4/5}$, which are shown in Fig. 3(c). The noise floor of
both phase shifters is -146.8 dBm/Hz. The phase shifter exhibits a link gain
of -5.8 dB and NF of 33 dB. The NF of the proposed phase shifter is mainly
deteriorated by the phase noise to intensity noise conversion of the laser
when the optical carrier is attenuated by the external BPF.

## Discussion and Conclusion

In this work, we demonstrated a multi-functional integrated MWP circuit that
can be reconfigured between a high-rejection notch filter and a 2$\pi$-range
phase shifter. By vectorially manipulating the multi-order optical sidebands,
ultrahigh dynamic range over 120 $\rm{dB}\cdot\rm{Hz}^{4/5}$ is achieved in
both functions. The array of microring resonators and two spectral de-
interleavers lead to a versatile circuit, and could potentially unlock even
more functionalities apart from filter and phase shifting, for example, true
time delay. Moreover, the possibility to realize multiple different functions
in one circuit also opens the path to the cascaded integrated MWP system,
which would be a major leap towards integrated RF photonic front-ends that can
be directly applied in real RF environments.

Currently, the reconfigurable MWP circuit exhibits relatively low link gain
and high noise figure. In principle, the link gain can be improved by
increasing the optical power to a high power-handling modulator. The strategy
for noise figure reduction, on the other hand, is more intricate. The use of
the phase modulator in our experiments prevents NF reduction techniques such
as low-biasing a Mach-Zehnder intensity modulator daulay2022ultrahigh  to be
employed. Emulation of such technique using carrier processing by ring
resonator has been previously considered daulay2020chip  but the power builds
up in the high-Q ring used for carrier processing, preventing the effective NF
reduction. We believe that achieving simultaneous high link gain and low NF
together with the features demonstrated in this work will require an entirely
new topology potentially using a more complex interferometric modulator (for
example dual-drive or dual-parallel MZM) with on-chip linearization. Several
recent works have shown encouraging results in this direction
feng2022ultrahighlinearity ; moran2023linearization .

## Author Contribution

G.L. and K.Y. contributed equally in this work. D.M. and G.L. developed the
concept and proposed the physical system. O.D. designed the photonic circuits,
G.L. and K.Y. developed and performed numerical simulations. G.L. and K.Y.
performed the experiments with input from O.D.. D.M., G.L. and K.Y. wrote the
manuscript with input from Q.T., H.Y. D.M. led and supervised the entire
project.

## Funding Information

This project is funded by the European Research Council Consolidator Grant
(101043229 TRIFFIC) and Nederlandse Organisatie voor Wetenschappelijk
Onderzoek (NWO) projects (740.018.021 and 15702).

## References

  * (1) C. Baylis, M. Fellows, L. Cohen, and R. J. Marks II, “Solving the spectrum crisis: Intelligent, reconfigurable microwave transmitter amplifiers for cognitive radar,” _IEEE Microwave Magazine_ , vol. 15, no. 5, pp. 94–107, 2014.
  * (2) H. Islam, S. Das, T. Bose, and T. Ali, “Diode based reconfigurable microwave filters for cognitive radio applications: A review,” _IEEE Access_ , vol. 8, pp. 185 429–185 444, 2020.
  * (3) J. Capmany and D. Novak, “Microwave photonics combines two worlds,” _Nature photonics_ , vol. 1, no. 6, p. 319, 2007.
  * (4) D. Marpaung, J. Yao, and J. Capmany, “Integrated microwave photonics,” _Nature photonics_ , vol. 13, no. 2, pp. 80–90, 2019.
  * (5) D. Zhu and S. Pan, “Broadband cognitive radio enabled by photonics,” _J. Lightwave Technol._ , vol. 38, no. 12, pp. 3076–3088, Jun 2020.
  * (6) J. S. Fandiño, P. Muñoz, D. Doménech, and J. Capmany, “A monolithic integrated photonic microwave filter,” _Nature Photonics_ , vol. 11, no. 2, pp. 124–129, 2017.
  * (7) Y. Liu, A. Choudhary, D. Marpaung, and B. J. Eggleton, “Integrated microwave photonic filters,” _Advances in Optics and Photonics_ , vol. 12, no. 2, pp. 485–555, 2020.
  * (8) O. Daulay, R. Botter, and D. Marpaung, “On-chip programmable microwave photonic filter with an integrated optical carrier processor,” _OSA Continuum_ , vol. 3, no. 8, pp. 2166–2174, 2020.
  * (9) Y. Chen, Z. Fan, Y. Lin _et al._ , “A multiband microwave photonic filter based on a strongly coupled microring resonator with adjustable bandwidth,” _IEEE Photonics Journal_ , vol. 15, no. 1, pp. 1–6, 2023.
  * (10) Y. Tao, H. Shu, X. Wang _et al._ , “Hybrid-integrated high-performance microwave photonic filter with switchable response,” _Photonics Research_ , vol. 9, no. 8, pp. 1569–1580, 2021.
  * (11) S. X. Chew, S. Song, L. Li, L. Nguyen, and X. Yi, “Inline microring resonator based microwave photonic phase shifter with self-mitigation of rf power variations,” _Journal of Lightwave Technology_ , vol. 40, no. 2, pp. 442–451, 2022.
  * (12) C. Porzi, G. Serafino, M. Sans _et al._ , “Photonic integrated microwave phase shifter up to the mm-wave band with fast response time in silicon-on-insulator technology,” _Journal of Lightwave Technology_ , vol. 36, no. 19, pp. 4494–4500, 2018.
  * (13) L. McKay, M. Merklein, A. C. Bedoya _et al._ , “Brillouin-based phase shifter in a silicon waveguide,” _Optica_ , vol. 6, no. 7, pp. 907–913, 2019\.
  * (14) D. Lin, X. Xu, P. Zheng _et al._ , “A high-performance microwave photonic phase shifter based on cascaded silicon nitride microrings,” _IEEE Photonics Technology Letters_ , vol. 32, no. 19, pp. 1265–1268, 2020.
  * (15) S. X. Chew, D. Huang, L. Li _et al._ , “Integrated microwave photonic phase shifter with full tunable phase shifting range ($>$ 360°) and rf power equalization,” _Optics express_ , vol. 27, no. 10, pp. 14 798–14 808, 2019\.
  * (16) C. Zhu, L. Lu, W. Shan _et al._ , “Silicon integrated microwave photonic beamformer,” _Optica_ , vol. 7, no. 9, pp. 1162–1170, 2020.
  * (17) P. M.-C. Romero, D. P. López, D. P. Galacho _et al._ , “High-precision broadband silicon photonics beamformer,” in _2022 IEEE International Topical Meeting on Microwave Photonics (MWP)_ , 2022, pp. 1–4.
  * (18) D. Lin, S. Shi, P. Liu _et al._ , “Low loss silicon nitride 1×4 microwave photonic beamforming chip,” _Opt. Express_ , vol. 30, no. 17, pp. 30 672–30 683, Aug 2022.
  * (19) L. Zhuang, C. G. Roeloffzen, M. Hoekman, K.-J. Boller, and A. J. Lowery, “Programmable photonic signal processor chip for radiofrequency applications,” _Optica_ , vol. 2, no. 10, pp. 854–859, 2015.
  * (20) D. Pérez, I. Gasulla, L. Crudgington _et al._ , “Multipurpose silicon photonics signal processor core,” _Nature communications_ , vol. 8, no. 1, pp. 1–9, 2017.
  * (21) W. Bogaerts, D. Pérez, J. Capmany _et al._ , “Programmable photonic circuits,” _Nature_ , vol. 586, no. 7828, pp. 207–216, Oct 2020.
  * (22) W. Zhang and J. Yao, “Photonic integrated field-programmable disk array signal processor,” _Nature communications_ , vol. 11, no. 1, pp. 1–9, 2020.
  * (23) L. Nguyen, X. Yi, P. Bian _et al._ , “Distributed optical signal processing for microwave photonics subsystems,” _Optics Express, Vol. 24, Issue 5, pp. 4730-4739_ , vol. 24, pp. 4730–4739, 3 2016.
  * (24) X. Xue, X. Zheng, B. Zhou, and H. Zhang, “All-optical microwave bandpass filter and phase shifter using a broadband optical source and an optical phase modulator,” _Optics Letters, Vol. 37, Issue 10, pp. 1661-1663_ , vol. 37, pp. 1661–1663, 5 2012.
  * (25) Y. Yao, Y. Zhao, J. Dong, and X. Zhang, “Silicon integrated frequency-tunable microwave photonic bandpass filter,” _IET Optoelectronics_ , 2022.
  * (26) Y. Liu, Y. Chen, L. Wang _et al._ , “Tunable and reconfigurable microwave photonic bandpass filter based on cascaded silicon microring resonators,” _Journal of Lightwave Technology_ , vol. 40, no. 14, pp. 4655–4662, 2022\.
  * (27) R. Wu, T. Jiang, S. Yu, J. Shang, and W. Gu, “Multi-order nonlinear distortions analysis and suppression in phase modulation microwave photonics link,” _Journal of Lightwave Technology_ , vol. 37, no. 24, pp. 5973–5981, 2019.
  * (28) W. Wang, Y. Bai, S. Fu _et al._ , “High linearity microwave photonic up-conversion system based on parallel dual-drive mach zehnder modulators,” _Photonics_ , vol. 9, no. 4, 2022.
  * (29) G. Liu, O. Daulay, Y. Klaver _et al._ , “Integrated microwave photonic spectral shaping for linearization and spurious-free dynamic range enhancement,” _Journal of Lightwave Technology_ , 2021.
  * (30) Y. Bai, X. Song, Z. Su _et al._ , “Large dynamic range microwave photonic phase shifter based on multi-order sidebands optical spectrum vector process technique,” in _2021 Optical Fiber Communications Conference and Exhibition (OFC)_. IEEE, 2021, pp. 1–3.
  * (31) O. Daulay, G. Liu, K. Ye _et al._ , “Ultrahigh dynamic range and low noise figure programmable integrated microwave photonic filter,” _Nature communications_ , vol. 13, no. 1, p. 7798, 2022.
  * (32) C. G. Roeloffzen, M. Hoekman, E. J. Klein _et al._ , “Low-loss si3n4 triplex optical waveguides: Technology and applications overview,” _IEEE journal of selected topics in quantum electronics_ , vol. 24, no. 4, pp. 1–21, 2018.
  * (33) K. Worhoff, R. G. Heideman, A. Leinse, and M. Hoekman, “Triplex: a versatile dielectric photonic platform,” _Advanced Optical Technologies_ , vol. 4, no. 2, pp. 189–207, 2015.
  * (34) L.-W. Luo, S. Ibrahim, A. Nitkowski _et al._ , “High bandwidth on-chip silicon photonic interleaver,” _Optics express_ , vol. 18, no. 22, pp. 23 079–23 087, 2010.
  * (35) D. Marpaung, B. Morrison, R. Pant _et al._ , “Si 3 n 4 ring resonator-based microwave photonic notch filter with an ultrahigh peak rejection,” _Optics express_ , vol. 21, no. 20, pp. 23 286–23 294, 2013\.
  * (36) H. Feng, K. Zhang, W. Sun _et al._ , “Ultra-high-linearity integrated lithium niobate electro-optic modulators,” _Arxiv_ , pp. 1–16, 2022.
  * (37) L. Torrijos-Moran, C. Catala-Lahoz, D. Perez-Lopez _et al._ , “Linearization of a dual-parallel mach-zehnder modulator using optical carrier band processing,” _Journal of Lightwave Technology_ , pp. 1–7, 2023\.

## Supplementary Note: Experimental details

In the supplementary note, we present the experimental details of both the
notch filter and the phase shifter. We start from the experimental setups and
signal flows. Then, we derive the linearization conditions by manipulating the
amplitude and phase of the optical carrier and the multi-order sidebands.
Finally, we show extended measurement results of both experiments.

### Experimental Setups

The experimental setup for the notch filter is shown in Fig. S1. Light from
the low relative-intensity noise (RIN) CW laser (Pure Photonics PPCL550) is
set at 1550 nm and 18 dBm. The light is modulated with RF signal via a phase
modulator (EOSpace, 20 GHz). Because of the nonlinearity of the phase
modulator, higher-order sidebands would be generated along with the
$1^{st}$-order sidebands. The signal is amplified by a low noise erbium-doped
fiber amplifier (EDFA, Amonics) before injected into the chip for signal
processing. The on-chip deinterleaver separates the signal into two path, one
containing the optical carrier and lower-sidebands and the other one
containing upper sidebands. In the subsequent processing, a ring resonator is
set at the under-coupling state and creates a notch response at the
$1^{st}$-order upper sideband. In the meantime, the $1^{st}$-order lower
sideband and optical carrier is suppressed with the tunable attenuator. After
that, signals at two paths are recombined and sent to the photo detector
(EMCORE 20GHz) to be converted to the RF domain. Since the 1st-order upper and
lower sidebands at the notch frequency are equal in amplitude but have anti-
phase. A very deep notch can be formed at that frequency because of the
destructive interference in the RF domain. The filter response is measured
with the VNA (keysight P9375A). For the SFDR measurement, the phase modulator
is driven by a two-tone signal with a spacing of 10 MHz generated by RF signal
generators (Wiltron 69147A and Rohde-Schwarz SMP02), and the converted RF
signal is measured with an RF spectrum analyzer (Keysight N9000B).

Fig. S1: Experiment setup and signal flow for demonstrating linearized notch
filter.

The setup and signal flow of the phase shifter experiment is shown in Fig. S2.
Unlike the setup of the notch filter experiment in Fig. S1, we use an external
bandpass filter (EXFO XTM-50) to filter out the low sidebands and also
introduce 9 dB attenuation to the optical carrier for linearization (as
discussed later). Then, the signal is amplified by a low-noise EDFA (Amonics)
and injected into the chip. We further use the on-chip deinterleaver to
spatially separate the 2nd-order upper sideband from the optical carrier and
the 1st-order upper sideband into two paths. For the path with the optical
carrier, we apply two cascaded all-pass ring resonators at the over-coupled
state to introduce phase shift to the optical carrier. On the other hand, for
the path with the 2nd-order upper sideband, we use the thermal-optic phase
shifter to create the opposite phase to the 2nd-order upper sideband and use
the tunable attenuator to adjust the power ratio between the optical carrier
and $2^{nd}$-order sideband for the best IMD3 suppression. Finally, the
signals at two paths recombine and are converted back to the RF domain with
the photo detector (APIC, 40 GHz), and the converted RF signal is measured
with a VNA (keysight P9375A). The settings of the SFDR measurement of the
phase shifter is the same as the notch filter.

Fig. S2: Experiment setup and signal flow for demonstrating linearized phase
shifter.

### Linearization Method

Phase-modulated optical spectrum consists of multi-order optical sidebands
which would generate intermodulation distortions in RF domain after
photodetection. We manipulate the phase and amplitude of these multi-order
optical sidebands to achieve selective destructive interference among the IMD3
terms to improve the linearity of the integrated MWP systems. As the
linearization condition of the notch filter has been extensively explained in
daulay2022ultrahigh , here we only derive the linearization condition of the
phase shifter.

We adopt a two-tone analysis to investigate the main contributors of IMD3
terms and the requirements to suppress them. When the optical carrier is
modulated by a two-tone RF signal at frequencies $\rm\omega_{1}$ and
$\rm\omega_{2}$ with a phase modulator, the optical field after the PM can be
expressed as:

$E_{\rm p}(t)=\sqrt{P_{in}}\
e^{j\omega_{c}t}\sum_{n=-\infty}^{+\infty}\sum_{k=-\infty}^{+\infty}J_{n}(m)J_{k}(m)e^{j(n\omega_{1}+k\omega_{2})t}$
(1)

Where $P_{in}$ is the input power, $\omega_{c}$ is the optical carrier
frequency, $J_{n}$ is the $n^{th}$-order Bessel function of the first kind,
and $m$ is the modulation index, which can be expressed as: $m=\pi\cdot
V_{RF}/V_{\pi}$.

To simplify the analysis, we assume that the two-tone RF signal at frequencies
$\rm\omega_{1}$ and $\rm\omega_{2}$ are small signals, we only need to
consider up to the 2nd-order sidebands. We assume the attenuation to the
optical carrier and $2^{nd}$-order sideband is $\alpha_{0}$ and $\alpha_{2}$
respectively. Then the simplified optical field after the chip can be written
as:

$E_{\mathrm{out}}=\sqrt{P_{i}}\cdot
e^{j\omega_{c}t}\left\\{\begin{array}[]{l}\alpha_{0}\left[J_{0}^{2}e^{j\Delta\varphi_{0}}-J_{1}^{2}e^{j\left[\left(\omega_{1,2}-\omega_{2,1}\right)t+\Delta\varphi_{0}\right]}\right]\\\
+\left[J_{0}J_{1}e^{j\omega_{1,2}t}-J_{1}J_{2}e^{j\left[\left(2\omega_{1,2}-\omega_{2,1}\right)t\right]}\right]\\\
+\alpha_{2}J_{0}J_{2}e^{j\left(2\omega_{1,2}t+\Delta\varphi_{2}\right)}\end{array}\right\\}$
(2)

The photo-current of the RF signal detected from the processed optical
spectrum can be expressed as

$\displaystyle I_{PD}(t)$ $\displaystyle=R_{PD}\left|E_{p}(t)\right|^{2}$ (3)
$\displaystyle=I_{1}\cos\omega_{1,2}t+I_{3}\cos\left(2\omega_{1,2}-\omega_{2,1}\right)t$

where $R_{PD}$ is the responsivity of the photo detector, $I_{1}$ and $I_{3}$
are the amplitude coefficients for fundamental signal and IMD3 components,
which can be written as

$\displaystyle
i_{\mathrm{RF}}(t)\propto\alpha_{0}J_{0}^{3}J_{1}\cdot\cos\left[\omega_{1,2}t-\Delta\varphi_{0}\right]$
(4) $\displaystyle
i_{IMD3}(t)\propto\alpha_{2}J_{0}^{2}J_{1}J_{2}\cdot\cos\left[\left(2\omega_{1,2}-\omega_{1,2}\right)t+\Delta\varphi_{2}\right]-\alpha_{0}\left(J_{0}^{2}J_{1}J_{2}+J_{0}J_{1}^{3}\right)\cdot\cos\left[\left(2\omega_{1,2}-\omega_{1,2}\right)t-\Delta\varphi_{0}\right]$

To minimize the IMD3 terms and maximize the fundamental RF signal, the
attenuation and phase shift imposed to the optical carrier and $+2^{nd}$-order
optical sideband should satisfy the condition:

$\left\\{\begin{array}[]{l}\Delta\varphi_{0}=-\Delta\varphi_{2}\\\
\alpha_{2}J_{0}^{2}J_{1}J_{2}-\alpha_{0}\left(J_{0}^{2}J_{1}J_{2}+J_{0}J_{1}^{3}\right)=0\end{array}\right.$
(5)

If we take small signal approximation and apply Taylor expansion to the Bessel
function of the first kind, then the condition in (5) can be simplified as:

$\left\\{\begin{array}[]{l}\Delta\varphi_{0}=-\Delta\varphi_{2}\\\
\alpha_{2}m^{3}-3\alpha_{0}m^{3}=0\end{array}\right.$ (6)

This condition means that the phase shift added to $+2^{nd}$-order optical
sideband is the opposite of that of the optical carrier. Moreover, the
attenuation imposed to the electrical field of the optical carrier is 3 times
of that applied to the $+2^{nd}$-order optical sideband. This means that the
power attenuation at optical carrier is 9.54 dB larger than the power
attenuation of the $+2^{nd}$-order optical sideband, and the phase shift
imposed to the RF signal is $-\Delta\varphi_{0}$.

### Extended Measurement Results

Table S1: Measured Fundamental to IMD3 ratio and the SFDR of the notch filter at different two-tone RF frequencies. RF frequencies | Fundamental to IMD3 ratio | SFDR of the notch filter with linearization (without linearization)
---|---|---
GHz | dB | dB$\cdot$Hz4/5 (dB$\cdot$Hz2/3)
8 | 63.8 | 122.6 (102.8)
9 | 64 | 122.3 (103.7)
10 | 63.2 | 123.5 (102.5)
16 | 61.6 | 122.6 (103.5)

To further characterize the performance of our proposed linearized RF notch
filter, we extend our measurements of IMD3 suppression and SFDR when the two-
tone test frequency and the notch filter frequency are tuned separately. We
first fix the notch frequency at 12 GHz and performed two-tone test at 8 GHz,
9 GHz, 10 GHz, and 16 GHz. The results of the IMD3 suppression and the SFDR at
different frequencies are listed in Table S1. The IMD3 terms are greatly
suppressed and fundamental to IMD3 ratios of more than 61 dB can be achieved
in all these two-tone frequencies. Moreover, SFDR of more than 122 dB$\cdot\rm
Hz^{4/5}$ are observed with improvements around 20 dB compared with
nonlinearized states.

We also measured SFDR at a specific frequency (in this case 9 GHz), when the
notch response is tuned at 6 GHz, 12 GHz, and 15GHz. The results are listed in
Table S2. In all these frequencies the improved SFDR are more than 122
dB$\cdot\rm Hz^{4/5}$. These results further demonstrate that the proposed
linearized notch filter has a high linearity and large SFDR in a wide
frequency range.

Table S2: Measured SFDR of the notch filter at different notch frequencies. Notch frequencies | 6 GHz | 12 GHz | 15 GHz
---|---|---|---
SFDR | 123.2 dB$\cdot$Hz4/5 | 122.3 dB$\cdot$Hz4/5 | 123.6 dB$\cdot$Hz4/5

To further characterize the performance of our proposed linearized RF phase
shifter, we extend our measurements of IMD3 suppression and SFDR when the two-
tone test frequency and the phase shifting are tuned. We performed two-tone
test at 16 GHz, 14 GHz with phase shifting of -130∘, -20∘ respectively. The
results of the IMD3 suppression and SFDR are shown in Fig. S3. It is clear
that the IMD3 terms are greatly suppressed and the SFDRs keep enhanced close
to 120 dB$\cdot\rm Hz^{4/5}$ in all combination of two-tone frequency and
phase shifting.

Fig. S3: IMD3 suppression and SFDR measurements of the phase shifter at
various two-tone frequencies and phase shifts.


2021

# Calibration Error Estimation Using Fuzzy Binning

Geetanjali Bihani<EMAIL_ADDRESS>Julia Taylor Rayz<EMAIL_ADDRESS>Computer and Information Technology, Purdue University, , West Lafayette,
47904, Indiana, USA

###### Abstract

Neural network-based decisions tend to be overconfident, where their raw
outcome probabilities do not align with the true decision probabilities.
Calibration of neural networks is an essential step towards more reliable deep
learning frameworks. Prior metrics of calibration error primarily utilize
crisp bin membership-based measures. This exacerbates skew in model
probabilities and portrays an incomplete picture of calibration error. In this
work, we propose a Fuzzy Calibration Error metric (FCE) that utilizes a fuzzy
binning approach to calculate calibration error. This approach alleviates the
impact of probability skew and provides a tighter estimate while measuring
calibration error. We compare our metric with ECE across different data
populations and class memberships. Our results show that FCE offers better
calibration error estimation, especially in multi-class settings, alleviating
the effects of skew in model confidence scores on calibration error
estimation. We make our code and supplementary materials available at:
https://github.com/bihani-g/fce

###### keywords:

Language Models, Calibration, Fine-tuning, Fuzzy theory, Classification,
Natural Language Processing

.

##  1 Introduction

Neural network-based decision-making systems have evolved rapidly in the
recent decade. Within the domain of natural language processing, deep learning
has shaped the current evolution in language modeling. These neural network-
based language models are trained on large text corpora and can be fine-tuned
across a wide range of NLP tasks and further improved using synthetic semantic
enhancement schemes bihani2021low , yielding state-of-the-art performance
chen2019bert ; devlin2018bert ; radford2019language ; yang2019xlnet . Ideally,
a neural model should output reliable and confident prediction probabilities.
But recent works have shown that neural networks are unreliable and output
highly overconfident predictions, resulting in over-estimation of the model’s
confidence in decisions guo2017calibration ; kong_calibrated_2020 ;
jiang_how_2021 . This leads to model miscalibration, i.e. a lack of alignment
between a model’s decision probabilities and its actual likelihood of
correctness. This lack of calibration can severely impact the trustworthiness
of a model’s decisions.

A widely adopted measure of the degree of miscalibration is Expected
Calibration Error (ECE) naeini2015obtaining , used to measure neural network
reliability ovadia2019can ; huang2020tutorial ; tack2020csi . The highly
overconfident output prediction probabilities of neural networks result in a
left-skewed probability distribution nixon_measuring_nodate . Since ECE
utilizes a fixed-width crisp binning scheme, this skew results in higher
probability bins largely contributing to the calibration error estimation,
while lower probability bins are ignored nixon_measuring_nodate ;
roelofs2022mitigating ; ding2020revisiting . To overcome these limitations,
prior works have proposed alternative binning strategies such as equal-
frequency binning roelofs2022mitigating , adaptive binning ding2020revisiting
, replacing binning with smoothed kernel density estimation zhang2020mix , and
more. Most calibration error estimation techniques rely on crisp binning,
which discards edge probabilities (probabilities that typically lie on the bin
edge) that could have contributed to a more accurate calibration error
estimation. Although some works have utilized fuzzification of prediction
probabilities for downstream NLP tasks bihani2022fuzzy , the calibration
impacts of such fuzzification are yet to be studied. We hypothesize that
fuzzifying the binning scheme would allow edge probabilities to contribute
toward more accurate calibration error estimation. Moreover, fuzzy binning
would increase the visibility of lower probability scores by allowing them to
have partial membership in higher probability bins, minimizing the skew
problem in calibration error estimation.

Towards testing this hypothesis, we propose a new metric for estimating
calibration error, i.e. Fuzzy Calibration Error (FCE), that utilizes fuzzy
binning instead of crisp binning to allow edge probability contributions and
minimize skew in calculating calibration error. We perform empirical
evaluation across different classification settings, comparing FCE with the
baseline calibration error estimation metric ECE.

Our results show that, unlike ECE, FCE better captures miscalibration in lower
probability bins and provides a tighter and less skewed estimate of
calibration error. These improvements are more visible in multi-class
settings, where the skew in confidence scores exacerbates the calibration
error estimation problem.

The contributions of this work are summarized as follows:

  * •

We propose Fuzzy Calibration Error (FCE) metric which uses fuzzy binning to
account for edge probabilities and minimize skew in calibration error
estimation

  * •

We perform empirical evaluation across a wide range of classification settings
and show the benefits of using FCE over ECE in minimizing the impacts of
probability skew on calibration error estimation

##  2 Background

###  2.1 Neural Network Calibration

Neural network calibration refers to the process of adjusting a neural network
model’s output probabilities to reflect the true probabilities of the events
it is predicting. With the increased application of neural network
architectures in high-risk real-world settings. their calibration has become
an extensively studied topic in recent years thulasidasan2019mixup ;
malinin2018predictive ; hendrycks_pretrained_2020 . Recent research has
focused on improving the calibration of neural networks, particularly in the
context of deep learning. Various methods have been proposed to achieve better
calibration, including temperature scaling guo2017calibration , isotonic
regression platt1999probabilistic , and histogram binning
zadrozny2001obtaining .

###  2.2 Expected Calibration Error

Expected calibration error (ECE) is a scalar measure of calibration error that
calculates the weighted average of the difference between the accuracy of a
model and its average confidence level over a set of bins defined by the
predicted probabilities. Estimation of expected accuracy from finite samples
is done by grouping predictions into $M$ interval bins (each of size
$\frac{1}{M}$), and the accuracy of each bin is calculated. Let $B_{m}$ be a
bin containing samples whose prediction confidence lies within the interval
$I_{m}=\left(\frac{m-1}{M},\frac{m}{M}\right]$. Then the accuracy of $B_{m}$,
where $y_{i}$ and $\hat{y}_{i}$ portray predicted and true class labels, is
calculated as shown in Eq. 1.

$\operatorname{acc}\left(B_{m}\right)=\frac{1}{\left|B_{m}\right|}\sum_{i\in
B_{m}}\mathbf{1}\left(\hat{y}_{i}=y_{i}\right)$ (1)

The average predicted confidence of $B_{m}$, is calculated as shown in Eq. 2,
where $\hat{p}_{i}$ refers to the prediction probability of the $i^{th}$
instance in $B_{m}$.

$\operatorname{conf}\left(B_{m}\right)=\frac{1}{\left|B_{m}\right|}\sum_{i\in
B_{m}}\hat{p}_{i}$ (2)

In an ideal scenario, for a perfectly calibrated model,
$\operatorname{acc}\left(B_{m}\right)=\operatorname{conf}\left(B_{m}\right)$
for all $m$ bins where $m\in\\{1,\ldots,M\\}$.

Finally, ECE is calculated as shown in Eq. 3, where $n$ is total number of
samples naeini2015obtaining .

$\mathrm{ECE}=\sum_{m=1}^{M}\frac{\left|B_{m}\right|}{n}\mid\operatorname{acc}\left(B_{m}\right)-\operatorname{conf}\left(B_{m}\right)$
(3)

##  3 Fuzzy Calibration Error

In this work, we propose Fuzzy Calibration Error (FCE), a metric that
transforms raw prediction probabilities into soft bin membership values for
calibration error estimation. This transformation has two benefits:

  1. 1.

Allows edge probability contributions when calculating calibration error

  2. 2.

Minimize probability skew effects by increasing visibility of lower
probability bins in calibration error estimation

To perform fuzzification, we utilize trapezoidal membership functions to map
raw softmax prediction probabilities to fuzzy bin membership values. The
difference between crisp and fuzzy binning of model prediction probabilities
is shown in Figure 1, with $M=3$ bins, and can be extended to any number of
bins where $M>3$. While ECE only allows for crisp membership within each bin,
FCE offers a more flexible binning approach, with partial memberships allowed
across multiple bins.

Figure 1: Crisp binning (Top left) and fuzzy binning (Bottom left) of
prediction probabilities, where the number of bins $M=3$. An example of the
difference in bin assignment based on $\hat{p_{i}}$ in crisp vs fuzzy binning
(Right).

Fuzzy Calibration Error ($FCE$) calculates the weighted average of the
difference between accuracy and average model confidence over a set of $M$
fuzzy bins. Estimation of expected accuracy from finite samples is done by
grouping predictions into $M$ fuzzy bins, and the accuracy of each bin is
calculated. Let $B_{m}$ be a bin containing samples whose prediction
confidence lies within the interval
$I_{m}=\left(\frac{m-1}{M},\frac{m}{M}\right]$. Then the accuracy for bin
$B_{m}$, where $y_{i}$ and $\hat{y}_{i}$ portray predicted and true class
labels, is calculated as shown in Eq. 4.

$\operatorname{acc}{{}_{fuzzy}}(B_{m})=\frac{1}{\lvert{\mu_{fuzzy}(B_{m})}\rvert}\sum_{i\in
B_{m}}\mu_{fuzzy}(B_{m})(\hat{y}_{i}=y_{i})$ (4)

Then, the average fuzzy predicted confidence of $B_{m}$, is calculated as
shown in Eq. 5.

$\operatorname{conf}{{}_{fuzzy}}(B_{m})=\frac{1}{\lvert{\mu_{fuzzy}(B_{m})}\rvert}\sum_{i\in
B_{m}}\mu_{fuzzy}(B_{m})\cdot\hat{p}_{i}$ (5)

Finally, FCE is calculated as shown in Eq. 6. Unlike ECE where the average is
taken over the number of samples in $B_{m}$ i.e., $n$, we take the average
over the total fuzzy membership in $B_{m}$ i.e.,
$\sum_{m=1}^{M}\mu_{fuzzy}(B_{m})$.

$FCE=\frac{1}{\sum_{m=1}^{M}\mu_{fuzzy}(B_{m})}\sum_{m=1}^{M}\lvert{\mu(B_{m})}\rvert\cdot\lvert{\operatorname{acc}{{}_{fuzzy}}}(B_{m})-{\operatorname{conf}{{}_{fuzzy}}}(B_{m})\rvert$
(6)

##  4 Experiments

To evaluate the impact of fuzzy binning on calibration error estimation, we
perform empirical evaluations across different classification settings. We
fine-tune large language models for text classification and measure their
calibration performance.

###  4.1 Experimental Setup

Datasets We consider three text classification datasets to run our analyses,
which vary in terms of class distributions, briefly described below.

  * •

20 Newsgroups (20NG): The 20 Newsgroups dataset mitchell1999twenty  is a
collection of newsgroup documents containing approximately $20,000$ documents
with an (almost) balanced class distribution across $20$ newsgroups/topics.

  * •

AGNews (AGN): The AG’s news topic classification dataset
Zhang2015CharacterlevelCN  is a collection of approximately $128,000$ news
articles, from $4$ sources. This dataset is widely used in clustering,
classification and information retrieval.

  * •

IMDb: The IMDb Movie reviews dataset maas-EtAl:2011:ACL-HLT2011  is a
collection of $50,000$ movie reviews from the Internet Movie Database (IMDb).
Each review is assigned either a positive or negative label, and the data is
widely used to train models for binary sentiment classification tasks.

We further simulate varying data resource settings to compare miscalibration
across different fine-tuning regimes. This is achieved by using a limited
portion of the training data to perform fine-tuning, and has been done in
prior works kim_bag_2023 .

Metrics To evaluate calibration across different fine-tuning setups, we use
ECE (refer to Eq. 3), FCE (refer to Eq. 6), and overconfidence (OF), described
below.

  * •

Overconfidence (OF): Overconfidence is the expectation of model prediction
probabilities $\hat{p}_{i}$ (confidence scores) over incorrect predictions and
is calculated as shown in Eq. 7.

$\operatorname{OF}=\frac{1}{\left|k\right|}\sum_{i\in incorrect}\hat{p}_{i}$
(7)

Here $k$ is the total number of incorrect predictions made by a given model.

Fine-tuning Setup We implement text classification using a fine-tuned BERT
devlin_bert_2019 . Since the focus of our work is not to create the most
accurate fine-tuned model but to compare the efficacy of ECE and FCE across
skewed prediction probabilities, we only fine-tune over one epoch and collect
miscalibrated prediction probabilities.

Figure 2: Variation in calibration error estimated using ECE and FCE across
different bin sizes (top to bottom) and class distributions (left vs right)

###  4.2 Results

Figure 3: Variation in model overconfidence (OF) across different sample sizes

Fuzzy binning in FCE better captures lower probability bins and edge
probabilities: While ECE bins are highly impacted by the leftward skew in
prediction probabilities, FCE yields a more uniformly distributed binning
scheme. This can be seen in Fig. 2, where the primary contributors of ECE
calculations are the higher probability bins, barely including lower
probability bins in calculations. On the other hand, FCE is more uniformly
spread across the probability range, better capturing lower probability bins
and offering immunity against highly skewed prediction probabilities.

Model overconfidence in multi-class classification settings is low but
continuously increasing: Refer to Fig. 3 to observe the changes in
overconfidence in model predictions. Although, a multi-class classification
dataset like 20 Newsgroups results in lower overconfidence in predictions in
limited data regimes, as compared to datasets with fewer classes, this
overconfidence increases as the number of samples during fine-tuning
increases. On the other hand, datasets with fewer classes i.e., i.e., AGNews
and IMDb output highly overconfident predictions in limited data regimes, but
this overconfidence plateaus as one keeps adding more samples.

| ECE | $\Delta_{ECE}$ | FCE | $\Delta_{FCE}$ |  |
---|---|---|---|---|---|---
Fine-tuning samples | AGNews |  |
100 | 15.41 | 2.36 | 32.50 | 0.00 |  |
1000 | 3.33 | 0.63 | 11.41 | 0.46 |  |
5000 | 0.71 | 0.41 | 7.77 | 0.71 |  |
10000 | 0.80 | 0.78 | 6.86 | 0.66 |  |
| IMDb |  |
100 | 5.00 | 1.71 | 22.50 | 0.00 |  |
1000 | 3.42 | 1.51 | 12.01 | 0.24 |  |
5000 | 1.49 | 0.23 | 7.41 | 0.82 |  |
10000 | 0.26 | 0.22 | 8.01 | 0.84 |  |
| 20 Newsgroups |  |
100 | 1.31 | 0.20 | 5.90 | 0.00 |  |
1000 | 29.21 | 4.47 | 38.83 | 0.27 |  |
5000 | 9.99 | 1.54 | 24.05 | 0.11 |  |
10000 | 2.28 | 1.30 | 16.18 | 0.39 |  |

  * 1

ECE, FCE, $\Delta_{ECE}$ and $\Delta_{FCE}$ values are scaled by a factor of
10.

  * •

Table 1: Variations in ECE and FCE across different fine-tuning settings.
Here, $\Delta$ calculates the average difference in estimated calibration
error when binning is performed using fewer bins ($M\in[2..7]$) versus more
bins ($M\in[8..15]$).

.

Figure 4: Binning of prediction probabilities across $M=15$ bins (model fine-
tuned on $n=5000$ samples)

Unlike ECE, FCE is not sensitive to the binning strategy and underlying data
used for training: ECE is a highly sensitive calibration error estimation
metric, and is easily influenced by slight changes in data and/or binning
strategies. Table 1 shows variations in $\Delta$, which calculates the average
difference in estimated calibration error when binning is performed using
fewer bins ($M\in[2..7]$) versus more bins ($M\in[8..15]$). While ECE displays
larger variations in calibration error estimation due to binning choices, FCE
is fairly immune to these choices and shows minimal $\Delta$ in most cases.
Further, Fig. 4 shows that the distribution of ECE across probability bins is
highly variable, and usually leftward skewed. On the other hand, FCE bins are
more evenly distributed and as shown in Table 1, output more conservative
calibration error estimates.

##  5 Conclusion

Overconfidence in neural networks lends to the problem of erroneous estimation
of calibration error. ECE, a widely adopted metric of measuring calibration
error across model decisions has recently come under scrutiny for being biased
towards high probability bins. To address this limitation, we propose a new
calibration error metric, i.e. Fuzzy Calibration Error (FCE). This metric
transforms raw model confidence scores into fuzzy bin memberships, allowing
more visibility of lower probability bins within the calibration error
calculations. Our results show that FCE offers a tighter estimate of
calibration error and the benefits of this metric are more prominent in multi-
class classification settings, where skew in model confidence largely affects
calibration error estimation using ECE.

Acknowledgments

This work was partially supported by the Department of Justice grant
#15PJDP-21-GK-03269-MECP.

## References

  *   * (1) Bihani, G., Rayz, J.T.: Low anisotropy sense retrofitting (laser): Towards isotropic and sense enriched representations. NAACL-HLT 2021, 81 (2021)
  * (2) Chen, Q., Zhuo, Z., Wang, W.: Bert for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909 (2019)
  * (3) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
  * (4) Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
  * (5) Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems 32 (2019)
  * (6) Guo, C., Pleiss, G., Sun, Y., Weinberger, K.: On calibration of modern neural networks. ICML 2017 (2017)
  * (7) Kong, L., Jiang, H., Zhuang, Y., Lyu, J., Zhao, T., Zhang, C.: Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data. ArXiv (2020). https://doi.org/10.18653/v1/2020.emnlp-main.102
  * (8) Jiang, Z., Araki, J., Ding, H., Neubig, G.: How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering. Transactions of the Association for Computational Linguistics 9, 962–977 (2021). https://doi.org/10.1162/tacl_a_00407
  * (9) Naeini, M.P., Cooper, G., Hauskrecht, M.: Obtaining well calibrated probabilities using bayesian binning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29 (2015)
  * (10) Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J., Lakshminarayanan, B., Snoek, J.: Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems 32 (2019)
  * (11) Huang, Y., Li, W., Macheret, F., Gabriel, R.A., Ohno-Machado, L.: A tutorial on calibration measurements and calibration models for clinical prediction models. Journal of the American Medical Informatics Association 27(4), 621–633 (2020)
  * (12) Tack, J., Mo, S., Jeong, J., Shin, J.: Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems 33, 11839–11852 (2020)
  * (13) Nixon, J., Dusenberry, M.W., Zhang, L., Jerfel, G., Tran, D.: Measuring Calibration in Deep Learning
  * (14) Roelofs, R., Cain, N., Shlens, J., Mozer, M.C.: Mitigating bias in calibration error estimation. In: International Conference on Artificial Intelligence and Statistics, pp. 4036–4054 (2022). PMLR
  * (15) Ding, Y., Liu, J., Xiong, J., Shi, Y.: Revisiting the evaluation of uncertainty estimation and its application to explore model complexity-uncertainty trade-off. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 4–5 (2020)
  * (16) Zhang, J., Kailkhura, B., Han, T.Y.-J.: Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning. In: International Conference on Machine Learning, pp. 11117–11128 (2020). PMLR
  * (17) Bihani, G., Rayz, J.T.: Fuzzy classification of multi-intent utterances. In: Explainable AI and Other Applications of Fuzzy Techniques: Proceedings of the 2021 Annual Conference of the North American Fuzzy Information Processing Society, NAFIPS 2021, pp. 37–51 (2022). Springer
  * (18) Thulasidasan, S., Chennupati, G., Bilmes, J.A., Bhattacharya, T., Michalak, S.: On mixup training: Improved calibration and predictive uncertainty for deep neural networks. Advances in Neural Information Processing Systems 32 (2019)
  * (19) Malinin, A., Gales, M.: Predictive uncertainty estimation via prior networks. Advances in neural information processing systems 31 (2018)
  * (20) Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., Song, D.: Pretrained Transformers Improve Out-of-Distribution Robustness. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2744–2751. Association for Computational Linguistics, Online (2020). https://doi.org/%****␣sn-article.bbl␣Line␣350␣****10.18653/v1/2020.acl-main.244
  * (21) Platt, J., et al.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10(3), 61–74 (1999)
  * (22) Zadrozny, B., Elkan, C.: Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In: Icml, vol. 1, pp. 609–616 (2001)
  * (23) Mitchell, T.: Twenty newsgroups data set. UCI Machine Learning Repository (1999)
  * (24) Zhang, X., Zhao, J.J., LeCun, Y.: Character-level convolutional networks for text classification. In: NIPS (2015)
  * (25) Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142–150. Association for Computational Linguistics, Portland, Oregon, USA (2011)
  * (26) Kim, J., Na, D., Choi, S., Lim, S.: Bag of Tricks for In-Distribution Calibration of Pretrained Transformers. arXiv. arXiv:2302.06690 [cs] (2023)
  * (27) Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-1423