# Sparse Expansions of Multicomponent Oxide Configuration Energy Using
Coherency & Redundancy

Luis Barroso-Luque  missing Department of Materials Science and Engineering,
University of California Berkeley, Berkeley CA, USA  Julia H Yang  Department
of Materials Science and Engineering, University of California Berkeley,
Berkeley CA, USA  Gerbrand Ceder  missing Department of Materials Science and
Engineering, University of California Berkeley, Berkeley CA, USA  Materials
Sciences Division, Lawrence Berkeley National Laboratory, Berkeley CA, USA

###### Abstract

Compressed sensing has become a widely accepted paradigm to construct high
dimensional cluster expansion models used for statistical mechanical studies
of atomic configuration in complex multicomponent crystalline materials.
However, strict sampling requirements necessary to obtain minimal coherence
measurements for compressed sensing to guarantee accurate estimation of model
parameters are difficult and in some cases impossible to satisfy due to the
inability of physical systems to access certain configurations. Nevertheless,
the dependence of energy on atomic configuration can still be adequately
learned without these strict requirements by using compressed sensing by way
of coherent measurements using redundant function sets known as frames. We
develop a particular frame constructed from the union of all occupancy-based
cluster expansion basis sets. We illustrate how using this highly redundant
frame yields sparse expansions of the configuration energy of complex oxide
materials that are competitive and often surpass the prediction accuracy and
sparsity of models obtained from standard cluster expansions.

Suggested keywords

###### pacs:

Valid PACS appear here

##  I Introduction

The Cluster Expansion Method (CEM), [1, 2, 3] has become a standard tool in
computational thermodynamics of multi-component crystalline systems as an
effective and efficient means to compute functions of atomic configuration.
The CEM is used to represent a coarse-grained lattice model of materials
properties as a function of the possible atomic configurations, i.e. the
occupations of crystallographic sites. The most common use of the CEM involves
the expansion of the formation energy of a particular multicomponent material
system. A cluster expansion is fitted using formation energies for a set of
representative structures calculated from first principles electronic
structure methods, most often calculated with density functional theory (DFT).
Well-converged cluster expansions fitted to DFT energies have been shown to
give very accurate predictions of formation energy to within a few meV/atom.
[4, 5, 3] A converged cluster expansion can subsequently be used in Monte
Carlo simulations to compute thermodynamic properties, such as configuration
entropy, [6, 7] phase diagrams, [8] ionic percolation,[9, 10] and short range
order.[11, 12]

The CEM involves the use of a discrete harmonic expansion in terms of basis
functions—known by practitioners as correlation functions—that act on groups
of symmetrically related clusters of crystallographic sites. The correlation
functions used in the CEM are constructed by taking symmetry adapted averages
of all possible $N$-fold products of site basis functions for all $N$ sites in
a crystal structure. Single site basis functions operate over the domain or
site space associated with each site in a structure. The site functions must
span the site space associated with each given site. In the CEM the site
functions constitute a basis for the space of functions acting over the
allowed occupations at a site, such that the number of site basis functions
must be equal to the number of allowed species (i.e. the size of the site
space). Additionally, in order for the resulting product basis to have a
cluster-like framework, where basis functions operate on different clusters of
sites, each set of site basis functions must include a constant function
$\phi_{0}\equiv 1$ as one of the functions used to span the corresponding
occupations for each site.[3]

Estimating the correlation function coefficients, known as Effective Cluster
Interactions (ECI), for a specific expansion is done using linear regression
usually with some form of regularization.[13, 3, 5] The values of a predefined
finite set of correlation functions—chosen within a spatial cutoff range and
maximum cluster size—are computed for each training structure to form a
truncated correlation matrix $\Pi_{S}=A\Pi$, where $\Pi$ is the full system
(infinite for bulk systems) correlation matrix, and $A$ is a sensing matrix
that truncates correlation functions and selects the correlation function
values for the finite set of training structures to be used. Columns of
$\Pi_{S}$ are the values for the included correlation functions evaluated for
each of the training structures, and are referred to as correlation vectors.
The several regression models that have been proposed in literature[4, 14, 15,
16] can be separated broadly into those dealing with over-determined linear
systems and those dealing with under-determined linear systems.

In over-determined systems more training structures are used than the number
of correlation functions used in the fit. The motivation for using an over-
determined system comes from several studies showing decreasing cross
validation errors with increasing number of training structures.[16, 17]
However this approach can be prone to over-fitting and lower prediction
accuracy [18] which has usually been addressed with $\ell_{2}$ regularization
(ridge regression),[14] or by using some form feature selection, such as step-
wise fitting,[17] evolutionary algorithms, [19, 20] or some other form of
$\ell_{p}$ regularization. [21, 22, 16] In contrast, for the case of under-
determined systems more correlation functions are included in the model than
structures used when fitting. In this case the main issue becomes degeneracy
of solutions, which is also addressed by performing feature selection via
$\ell_{1}$ or $\ell_{0}$ regularization in order to seek out sparse solutions
only. Although the regularized regression models for over-determined and
under-determined systems appear the same, the analysis of solutions is
mathematically distinct, and the guarantees for accurate recovery of
coefficients can be very different. [23] Here we focus on the case of using
under-determined systems for fitting cluster expansions, which can be analyzed
within the framework of Compressed Sensing (CS).[24]

For relatively small dimensional systems, such as binary or ternary alloys,
that allow accurate fits using only a small number of short range clusters of
low degree (number of sites in cluster), such that the number of total
correlation functions is manageable, practitioners have the option to choose
between using an over-determined system or an under-determined system
approach. However, for more complex high dimensional systems, there will often
be no such choice. Since the number of expansion terms that need to be
considered for an acceptable fit quickly becomes too large, computing DFT
energies for enough structures to obtain an over-determined system becomes
untenable. The reason for this is twofold. First off all, the number of terms
in a truncated expansion grows polynomially with the number of allowed species
at each site. Additionally, complex physical interactions may require longer
range and/or higher degree clusters. For example, these situations frequently
occur in the study of high entropy materials which has recently gained much
attention in the design of metallic alloys,[25] and ceramic materials.[26]
Although theoretically the CEM is well suited for the study of high entropy
materials, the large high dimensional configuration spaces involved render an
over-determined system approach prohibitive. For such scenarios, there is no
choice but to work with under-determined systems only, and hope that betting
on sparsity[27] applies favorably to cluster expansions of these high
dimensional multicomponent crystalline systems.

Fortunately, classical compressed sensing[24] in the CEM has already been
demonstrated to work well with some metallic alloy systems. [15, 21] Here we
go beyond previous work and apply the theory of compressed sensing with
coherency and redundancy to the realm of the CEM, and demonstrate how it
applies in much larger configuration spaces for ionic systems with multiple
sublattices and oxidation state assignment. In Section II we introduce the
main concepts of compressed sensing with coherence and redundancy and restate
the main signal recovery error bound [28] adapted to the domain of cluster
expansions. In Section III we lay out the details of cluster expansions using
site indicator basis functions. This is subsequently used as a building block
in the construction of a redundant function set, which we call the generalized
Potts frame, which we describe in Section IV. Finally, we demonstrate how the
generalized Potts frame yields sparse and accurate expansions by comparing
resulting fits of 3 different fluorinated lithium-transition metal oxide
materials with standard cluster expansions using site indicator functions and
orthogonal trigonometric/sinusoid basis functions. [29]

##  II Compressed Sensing with Coherency & Redundancy

The notion of classical compressed sensing (CS) that has been previously shown
to yield accurate cluster expansions of metallic alloys,[15, 21] relies
strictly on the concept of incoherence [24] in order to guarantee accurate
recovery of the underlying coefficients/ECI. The need for incoherent
measurements—which in the CEM correspond to correlation function values
evaluated for select structures—is clear when the goal is to accurately
recover the exact coefficients in the expansion of a function. This
requirement can be made intuitive by thinking of coherence as a measurement of
how similar correlation vectors are to each other. High incoherence (low
coherence), says that the set of correlation vectors used are more uniformly
spread out—they are closer to mutually orthogonal—in their span; and as a
result the portion of the energy represented by each, can be easily
distinguished. In the limit of zero coherence, the correlation matrix is
orthogonal, and in principle the portion of the energy for each correlation
vector can be identified exactly. For the cases with high coherence,
correlation vectors will be much more closely correlated and it is no longer
possible to distinguish which correlation vector a specific portion of the
energy comes from. Formally, the coherence of a measurement matrix $M$ is
defined as, [30]

$\mu(M)=\underset{i<j}{\max}\frac{|\langle
M_{i},M_{j}\rangle|}{||M_{i}||_{2}||M_{j}||_{2}}$ (1)

Where $M_{i}$ and $M_{j}$ are columns of matrix $M$. The normalized definition
given above bounds the coherence of a measurement matrix to values between
zero and 1, $0\leq M\leq 1$. Classical compressed sensing requires coherence
values to be as close to zero as possible in order to maximize the probability
of accurate reconstruction of coefficients/ECI.

The guarantees of classical compressed sensing depend on the coherence of the
measurement matrix $M$ being minimal. As suggested above, incoherent
measurements improve the chances of accurate recovery of ECI, specifically by
requiring a lower number of necessary training structures for a given level of
accuracy.[24] Furthermore, it is necessary for $M$ to satisfy the restricted
isometry property (RIP) with a small isometry constant $\delta$.[24] In broad
terms, the RIP is a condition which ensures that most of the possible sparse
solutions for the linear system lie outside the nullspace of the measurement
matrix $M$.[30]

In the case of the CEM, the measurement matrix $M=A\Pi$ is made up of the
values of a relatively small set of truncated correlation vectors evaluated
for each of the training structures for which DFT energies have been
calculated. The correlation vectors only include functions associated with
clusters within a physical predefined cutoff radius and cluster size. It has
been shown that matrices made up of random measurements, such as those
composed of Gaussian vectors on the unit hypersphere, satisfy the RIP with
overwhelming probability and lead to high levels of incoherence.[24] Random
measurements have been previously reported to lead to effective training
structure selection and resulting accurate under-determined fits of CE’s for
simple metallic alloys.[15, 21] Nevertheless, selecting training structures
such that the resultant measurement matrix has high incoherence and ideally
satisfies the RIP with a small constant, becomes difficult and in some cases
almost impossible for materials systems with more complex physics. This can be
generally understood as a result of physically imposed sampling constraints.
Usually the vast majority of all possible configurations will have high
energies that are complicated if not impossible to compute with first
principles methods. For example, systems with configurations that undergo
large structural relaxations can no longer be mapped back to the fixed
structure underlying the CE. As another example, certain cluster occupations
in ionic systems are difficult to access when very high electrostatic
repulsion exists. Finally, charge neutrality constraints in heterovalent ionic
systems restricts the possible configurations that can be sampled. These
phenomena complicate and in many cases prevent the possibility of obtaining
appropriate correlation matrices with minimal coherence required for classical
CS recovery, even with previously proposed methods, such as the aforementioned
use of uniformly random Gaussian vectors.

Fortunately, there is a variant of Compressed Sensing that has been shown to
give accurate reconstructions of sparse or compressible signals with only a
small set of coherent measurements.[28] We will show how this concept can be
used for learning sparse expansions of functions of configuration in
multicomponent crystalline materials. The essence of CS with coherent and
redundant measurements is not to recover the model coefficients as accurately
as possible, but rather to recover an approximation for the actual function as
accurately as possible by way of a redundant representation and possibly
highly coherent measurements. [28] For systems constructed using measurements
with high coherence and a redundant representation, solutions will always be
highly degenerate, but again the focus is on the accuracy of predictions
obtained using the recovered expansion and not the exact values of the fitted
coefficients. However, we will still seek only sparse solutions such that the
large degeneracy of solutions becomes manageable and the resulting expansion
can be used efficiently for subsequent predictions and simulations such as in
Monte Carlo studies. Figure 1 shows a schematic of the pertinent mathematical
objects, the corresponding domains and their relationships as it pertains to
CS. For our purposes we seek an accurate representation in the function domain
made up from a small number of coefficients but without much regard to whether
or not the obtained coefficients correspond to those of an underlying exact
expansion.

Coefficient Domain$J$$\Pi$Function Domain$H=\Pi J$$A$Sensing Domain$E=AH$
Figure 1: Schematic of different domains involved in Compressed Sensing. CS
with redundancy seeks to recover function $H$ in the function domain in the
center. Adapted from Candes et al.[28]

In what follows we only give a brief overview of CS with coherent and
redundant measurements adapted to our purposes. Further details and
accompanying proofs can be found in the original publication.[28]

The goal of CS, is to reconstruct a sparse representation or approximation of
a function by solving the following optimization problem.

$\hat{H}=\underset{\widetilde{H}}{\text{argmin}}||\Pi^{*}\widetilde{H}||_{1}\text{
subject to }||A\widetilde{H}-E||_{2}\leq\varepsilon$ (2)

Where $H$ is the function sought—in our case a function of atomic
configuration. $\Pi$ is a linear operator mapping coefficients $J$ to the
function $H$ as depicted in Figure 1. In the CEM, $\Pi$ is the matrix of all
correlation basis functions. $A$ is a sensing matrix used in selecting the
training data; and $E$ is the measured value of the function $H$ for the
training configurations. The measurements are of the form $E=AH+z$. Where $z$
represents an additive noise with some upper bound $||z||_{2}\leq\varepsilon$.

The main theorem of CS with coherency and redundancy gives the following error
bounds for an $s$-sparse reconstruction of $H$,

$||\hat{H}-H||_{2}\leq
C_{0}\varepsilon+C_{1}\frac{||\Pi^{*}H-(\Pi^{*}H)_{s}||_{1}}{\sqrt{s}}$ (3)

Where $\hat{H}$ is the $s$-sparse approximation to $H$. $(\Pi^{*}H)_{s}$
denotes a vector with the largest $s$ non-zero coefficients of $H$ and zeros
elsewhere. $C_{0}$ and $C_{1}$ are constants that depend only on the sensing
matrix $A$. The theorem holds if the measurement matrix satisfies a restricted
isometry property (D-RIP) adapted to the union of the span of $s$ columns of
$\Pi$ with isometry constant $\delta_{2d}<0.08$.[28] The D-RIP prevents
possible solutions from being highly distorted by the measurement matrix, and
similarly to the standard RIP, also prevents them from falling in its
nullspace.

The theorem from Equation 3 essentially says that the solution to the
optimization problem in Equation 2 gives accurate reconstructions of the
function $H$, when the coefficients $\Pi^{*}H$ are sparse and/or decay
rapidly—i.e. $H$ is compressible. These results suggest that if functions of
configuration for multicomponent materials are indeed compressible, meaning
they can be represented with only a small set of functions, then accurate
reconstructions can be obtained with coherent and redundant measurements.
Although a rigorous proof along with bounds on the compressibility of
functions of configuration would be of immense value, it is beyond the scope
of the present work. We do however provide numerical indication that such
functions are indeed favorably compressible. This result is not unanticipated
considering established knowledge regarding the physics of locality, the
success of atomic potentials with small numbers of multiple body terms, and
the general tenet of parsimony in physics. This translates to our intuition
that such expansions of atomic configuration should have low degree and small
associated cluster diameters, and as such we should be able to represent them
using a small number of terms. The crux is finding an optimal subspace spanned
by a small set of functions that allows an accurate and sparse representation.

##  III The Cluster Expansion with Site Occupancy Basis Functions.

Before introducing the generalized Potts frame, we give a brief exposition of
the process of constructing a basis in the CEM using a particular choice of
site basis functions. Although the original CEM was developed using
orthonormal site basis functions constructed from polynomials,[1, 2] any set
of functions can be used as long as they properly span the function space over
the corresponding site spaces. Here we will focus on one commonly used basis
set composed of site occupancy functions, [31] which we refer to as site
indicator functions. A site indicator function simply indicates whether a
given species occupies a site or not,

$\mathbf{1}_{\sigma_{j}}(\sigma_{i})=\begin{cases}1&\text{if
}\sigma_{i}=\sigma_{j}\\\ 0&\text{otherwise}\end{cases}$ (4)

Since the set of basis functions must include the constant function for the
CEM to yield basis functions that follow a cluster framework, the constant
function needs to replace an indicator function for one of the species at each
site. In other words, one of the total $m$ allowed species at each site will
not have an associated indicator function. [31] The sets of site basis
functions $\\{\phi_{j};j=0,\cdots,m-1\\}$ for the CEM in terms of site
indicator functions are given by,

$\phi_{j}(\sigma_{i})=\begin{cases}1&\text{if }j=0\\\
\mathbf{1}_{\sigma_{j}}(\sigma_{i})&\text{if }j>0\end{cases}$ (5)

The correlation functions are then constructed from symmetry adapted averages
of the following $N$-fold products of site basis functions for each site in a
structure,

$\Phi_{\bm{\alpha}}(\bm{\sigma})=\prod_{i=1}^{N}\phi_{\alpha_{i}}(\sigma_{i}),$
(6)

Since we can ignore all sites where the constant site basis function
$\phi_{0}\equiv 1$ is used, Equation 6, can be written in terms of site
indicator functions as,

$\Phi_{\bm{\alpha}}(\bm{\sigma})=\prod_{i\text{ s.t. }\alpha_{i}\neq
0}\mathbf{1}_{\sigma_{\alpha_{i}}}(\sigma_{i}),$ (7)

Where we notice that by construction, the functions in Equation 7 will
indicate whether a specific occupancy of a given cluster represented by
nonzero elements of the multi-index $\bm{\alpha}$ is present in a structure.
We can more briefly write Equation 7 as a cluster indicator function,

$\displaystyle\Phi_{\bm{\alpha}}(\bm{\sigma})$
$\displaystyle=\mathbf{1}_{\bm{\alpha}}(\bm{\sigma})$
$\displaystyle=\begin{cases}1&\text{if cluster }\bm{\alpha}\text{ is in
}\bm{\sigma}\\\ 0&\text{otherwise}\end{cases}$ (8)

Furthermore, since the final correlation functions are constructed from
averages of functions given by Equation 8 over symmetrically equivalent
clusters, the correlation functions represent (up to multiplicity constants)
the concentration of a specific occupancy of clusters for the different
crystallographic orbits,

$\langle\Phi(\bm{\sigma})\rangle_{\beta}=\frac{1}{N(\beta)}\sum_{\bm{\alpha}\in\beta}\mathbf{1}_{\bm{\alpha}}(\bm{\sigma})$
(9)

Where $\beta$ represents an orbit of labeled clusters (i.e. with each site
labeled with a particular indicator function), and $N(\beta)$ is the total
number of labeled clusters in the orbit $\beta$. The sum is carried over all
labeled clusters $\bm{\alpha}$ that are part of the orbit $\beta$.

We make two notable observations regarding CEM correlation functions with site
indicator basis functions. The first is that the basis sets in Equation 5 are
not orthogonal and as a result the resulting correlation functions in Equation
9 are not orthogonal either. This lack of orthogonality can further complicate
constructing highly incoherent measurement matrices compared to using
orthogonal/orthonormal basis sets.

The second observation is concerned with the set of clusters/orbits that are
selected by the correlation functions in Equation 9. The set of correlation
functions in Equation 9 indeed represents a basis for the function space over
all possible configurations $\bm{\sigma}$ and thus the set is linearly
independent.[31] As a result however, the included functions do not give the
concentrations of all possible occupied clusters. Namely, the correlation
functions in Equation 9 never include any occupied clusters that involve the
species that do not have an associated indicator function in the corresponding
site basis functions.

Notwithstanding these observations, cluster expansions using site indicator
functions formally constitute a basis for functions spaces over crystalline
configurations, and have been successfully used in the study of
configurational thermodynamics for some time. [31, 8, 10] However, as alluded
to before, the lack of orthogonality complicates obtaining incoherent
measurements to maximize accurate ECI estimation from classical CS.
Additionally, the choice of species left out in site basis sets is
mathematically meaningless, but can often lead to precarious interpretations
of fitted ECI. This begs the question whether we can do away with using proper
basis sets and seeking maximally incoherent measurements, yet still obtain
suitably sparse and accurate expansions by instead relying on redundancy by
including functions labeled for all possible occupations over the included
clusters.

##  IV The Generalized Potts Frame as a Redundant Representation.

We introduce a specific redundant set of functions that span the same function
space as correlation basis sets used in the CEM. The redundant set of
functions can be obtained simply by including cluster indicator functions of
the form in Equation 7 for all possible occupations. More formally we build
all the $N$-fold product functions from redundant site function sets that
include $\phi_{0}\equiv 1$ and site indicator functions for all allowed
species at each site. This results in $n+1$ total functions for a space of
dimension $n$, where the redundancy is the trivial linear relation,

$\phi_{0}(\sigma)-\sum_{i=1}^{n}\mathbf{1}_{\sigma_{i}}(\sigma)=0$ (10)

Carrying out the same operation of taking all possible $N$-fold products of
functions from each redundant set and taking symmetry adapted averages over
crystallographic orbits results in the set of symmetrized product functions
which give concentrations for any possible cluster occupancy. This resulting
set is clearly highly redundant since the total number of functions obtained
are of order $O\left((n+1)^{N}\right)$ and the dimension of the functions
space is of order $O(n^{N})$. In other words the combinatorics[32] for
identifying symmetrically equivalent clusters involve $n$ possible labels of
basis functions for each site for all clusters in distinct orbits compared
with the $n-1$ labels involved in a standard CEM basis. Nonetheless, the
resulting set of functions spans the same function space as a standard CEM
basis and therefore formally the set constitutes a frame.[33, 34] We provide a
derivation for a set of frame bounds in Appendix A, but make no effort in
optimizing them. The resulting frame evidently has a strong connection to the
well known Potts model.[35] Indeed it is a (normalized) generalization in both
spatial extent and interaction size of the original nearest neighbor pair
Potts model. [36] Hence we refer to the proposed frame as the generalized
Potts frame.

The generalized Potts frame can also be seen as the union of every possible
CEM correlation basis of site indicator functions. All possible CEM
correlation bases can be generated by cycling over the species that is not
indicated for in its corresponding site basis and building the corresponding
CEM correlation basis for every possible combination of site basis sets. We
make a special note that apart from the standard CEM basis sets, this includes
correlations of mixed products where symmetrically equivalent sites in the
underlying random structure can have distinct basis sets, i.e. a different
subset of species with associated indicator functions. Symbolically, the
generalized Potts frame is obtained after taking the corresponding symmetry
adapted averages for the product functions in the following set,

$\bigcup_{\sigma}\left\\{\Phi_{\bm{\alpha}};\;\forall\bm{\alpha}\;\text{ s.t.
}\bm{\alpha}\text{ is not in }\bm{\sigma}\right\\}$ (11)

Where $\bm{\alpha}$ represent labeled clusters and where we use each possible
symmetrically distinct configuration $\bm{\sigma}$ to represent the species at
each site that does not have an indicator function in its corresponding basis
set.

Though the generalized Potts frame is highly redundant, the motivation is that
by introducing more expansion functions than strictly necessary and using an
appropriate algorithm—such as for solving Equation 2—can yield both accurate
and sparse representations of functions of crystalline configurations without
the need of maximally incoherent measurements. An intuition for this can be
formed by picturing the union of all subspaces spanned by size $s$ subsets of
functions in the generalized Potts frame. If the function sought lies on any
of these subspaces or close enough, then the function can be accurately
represented by only $s$ terms.

$\bm{1}_{\leavevmode\hbox to6.06pt{\vbox
to6.06pt{\pgfpicture\makeatletter\hbox{\hskip 6.05722pt\lower-3.02861pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\pgfsys@color@rgb@stroke{1}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{-0.2pt}{0.0pt}\pgfsys@curveto{-0.2pt}{1.56221pt}{-1.4664pt}{2.82861pt}{-3.02861pt}{2.82861pt}\pgfsys@curveto{-4.59082pt}{2.82861pt}{-5.85722pt}{1.56221pt}{-5.85722pt}{0.0pt}\pgfsys@curveto{-5.85722pt}{-1.56221pt}{-4.59082pt}{-2.82861pt}{-3.02861pt}{-2.82861pt}\pgfsys@curveto{-1.4664pt}{-2.82861pt}{-0.2pt}{-1.56221pt}{-0.2pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{-3.02861pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.02861pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\pgfsys@color@rgb@stroke{1}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0}{0}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}(\cdot)$$\bm{1}_{\leavevmode\hbox
to6.06pt{\vbox to6.06pt{\pgfpicture\makeatletter\hbox{\hskip
3.02861pt\lower-6.05722pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{2.82861pt}{-3.02861pt}\pgfsys@curveto{2.82861pt}{-1.4664pt}{1.56221pt}{-0.2pt}{0.0pt}{-0.2pt}\pgfsys@curveto{-1.56221pt}{-0.2pt}{-2.82861pt}{-1.4664pt}{-2.82861pt}{-3.02861pt}\pgfsys@curveto{-2.82861pt}{-4.59082pt}{-1.56221pt}{-5.85722pt}{0.0pt}{-5.85722pt}\pgfsys@curveto{1.56221pt}{-5.85722pt}{2.82861pt}{-4.59082pt}{2.82861pt}{-3.02861pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{-3.02861pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-3.02861pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}(\cdot)$$\phi_{0}(\cdot)=1$
(a) Function space over single site occupations.

$\Phi_{\emptyset}(\cdot)=1$$\bm{1}_{\leavevmode\hbox to6.06pt{\vbox
to6.06pt{\pgfpicture\makeatletter\hbox{\hskip 6.05722pt\lower-3.02861pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{1,.5,0}\pgfsys@color@rgb@stroke{1}{.5}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{.5}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{-0.2pt}{0.0pt}\pgfsys@curveto{-0.2pt}{1.56221pt}{-1.4664pt}{2.82861pt}{-3.02861pt}{2.82861pt}\pgfsys@curveto{-4.59082pt}{2.82861pt}{-5.85722pt}{1.56221pt}{-5.85722pt}{0.0pt}\pgfsys@curveto{-5.85722pt}{-1.56221pt}{-4.59082pt}{-2.82861pt}{-3.02861pt}{-2.82861pt}\pgfsys@curveto{-1.4664pt}{-2.82861pt}{-0.2pt}{-1.56221pt}{-0.2pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{-3.02861pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.02861pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{1,.5,0}\pgfsys@color@rgb@stroke{1}{.5}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{.5}{0}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\leavevmode\hbox
to6.06pt{\vbox to6.06pt{\pgfpicture\makeatletter\hbox{\hskip
6.05722pt\lower-3.02861pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{1,.5,0}\pgfsys@color@rgb@stroke{1}{.5}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{.5}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{-0.2pt}{0.0pt}\pgfsys@curveto{-0.2pt}{1.56221pt}{-1.4664pt}{2.82861pt}{-3.02861pt}{2.82861pt}\pgfsys@curveto{-4.59082pt}{2.82861pt}{-5.85722pt}{1.56221pt}{-5.85722pt}{0.0pt}\pgfsys@curveto{-5.85722pt}{-1.56221pt}{-4.59082pt}{-2.82861pt}{-3.02861pt}{-2.82861pt}\pgfsys@curveto{-1.4664pt}{-2.82861pt}{-0.2pt}{-1.56221pt}{-0.2pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{-3.02861pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.02861pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{1,.5,0}\pgfsys@color@rgb@stroke{1}{.5}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{.5}{0}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}(\cdot)$$\bm{1}_{\leavevmode\hbox
to6.06pt{\vbox to6.06pt{\pgfpicture\makeatletter\hbox{\hskip
3.02861pt\lower-6.05722pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\pgfsys@color@rgb@stroke{1}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{2.82861pt}{-3.02861pt}\pgfsys@curveto{2.82861pt}{-1.4664pt}{1.56221pt}{-0.2pt}{0.0pt}{-0.2pt}\pgfsys@curveto{-1.56221pt}{-0.2pt}{-2.82861pt}{-1.4664pt}{-2.82861pt}{-3.02861pt}\pgfsys@curveto{-2.82861pt}{-4.59082pt}{-1.56221pt}{-5.85722pt}{0.0pt}{-5.85722pt}\pgfsys@curveto{1.56221pt}{-5.85722pt}{2.82861pt}{-4.59082pt}{2.82861pt}{-3.02861pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{-3.02861pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-3.02861pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\pgfsys@color@rgb@stroke{1}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0}{0}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\leavevmode\hbox
to6.06pt{\vbox to6.06pt{\pgfpicture\makeatletter\hbox{\hskip
3.02861pt\lower-6.05722pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\pgfsys@color@rgb@stroke{1}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{1,0.8,0.8}\pgfsys@color@rgb@fill{1}{0.8}{0.8}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{2.82861pt}{-3.02861pt}\pgfsys@curveto{2.82861pt}{-1.4664pt}{1.56221pt}{-0.2pt}{0.0pt}{-0.2pt}\pgfsys@curveto{-1.56221pt}{-0.2pt}{-2.82861pt}{-1.4664pt}{-2.82861pt}{-3.02861pt}\pgfsys@curveto{-2.82861pt}{-4.59082pt}{-1.56221pt}{-5.85722pt}{0.0pt}{-5.85722pt}\pgfsys@curveto{1.56221pt}{-5.85722pt}{2.82861pt}{-4.59082pt}{2.82861pt}{-3.02861pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{-3.02861pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-3.02861pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\pgfsys@color@rgb@stroke{1}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0}{0}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}(\cdot)$$\bm{1}_{\leavevmode\hbox
to6.06pt{\vbox to6.06pt{\pgfpicture\makeatletter\hbox{\hskip
3.02861pt\lower-6.05722pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{2.82861pt}{-3.02861pt}\pgfsys@curveto{2.82861pt}{-1.4664pt}{1.56221pt}{-0.2pt}{0.0pt}{-0.2pt}\pgfsys@curveto{-1.56221pt}{-0.2pt}{-2.82861pt}{-1.4664pt}{-2.82861pt}{-3.02861pt}\pgfsys@curveto{-2.82861pt}{-4.59082pt}{-1.56221pt}{-5.85722pt}{0.0pt}{-5.85722pt}\pgfsys@curveto{1.56221pt}{-5.85722pt}{2.82861pt}{-4.59082pt}{2.82861pt}{-3.02861pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{-3.02861pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-3.02861pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\leavevmode\hbox
to6.06pt{\vbox to6.06pt{\pgfpicture\makeatletter\hbox{\hskip
3.02861pt\lower-6.05722pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgffillcolor}{rgb}{0.8,0.8,1}\pgfsys@color@rgb@fill{0.8}{0.8}{1}\pgfsys@invoke{
}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@invoke{
}{}\pgfsys@moveto{2.82861pt}{-3.02861pt}\pgfsys@curveto{2.82861pt}{-1.4664pt}{1.56221pt}{-0.2pt}{0.0pt}{-0.2pt}\pgfsys@curveto{-1.56221pt}{-0.2pt}{-2.82861pt}{-1.4664pt}{-2.82861pt}{-3.02861pt}\pgfsys@curveto{-2.82861pt}{-4.59082pt}{-1.56221pt}{-5.85722pt}{0.0pt}{-5.85722pt}\pgfsys@curveto{1.56221pt}{-5.85722pt}{2.82861pt}{-4.59082pt}{2.82861pt}{-3.02861pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{-3.02861pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-3.02861pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}(\cdot)$ (b) Function
space over molecule configurations.

Figure 2: Binary symmetric diatomic molecule function space over site space
and functions space over configurations product space. (a) The two different
choices for site bases to construct a standard CE are colored red and blue,
and each set will also include the purple $\phi_{0}=1$ function. (b) The
generalized Potts frame includes all colored functions (blue/red/yellow). The
D-RIP in this case is adapted to the union of all colored planes in (b).

As a simple illustration consider a symmetric binary diatomic system. Figure
2(a) shows the corresponding site spaces and site indicator functions along
with the constant function $\phi_{0}$. Additionally, the resulting symmetrized
product bases are also shown in Figure 2(b). The basis functions in both cases
are colored to represent the possible CE bases. The union of all basis
functions from these CE bases corresponds to the generalized Potts frame for
this simple diatomic system. Again we highlight also the inclusion of products
of mixed site basis sets, which corresponds to the orange vector built from
the product of a red and blue basis function associated with each site
respectively. In this case the function space over configurations of the
binary diatomic molecule has 3 dimensions, however when a function lies close
to or on any of the highlighted planes that function can be well approximated
using only 2 terms.

##  V Applications to Multi-component Oxides

We demonstrate the performance of the generalized Potts frame by comparing
fits for three fluorinated lithium-transition metal oxide systems with fits
obtained using a standard indicator based CE, and a CE using orthogonal
sinusoid site basis functions.[29] These material systems are of interest as
novel cathode materials made with abundant elements,[37] however their
specific applications are not particularly relevant for the purpose of this
work. The configuration spaces for the materials considered increase in both
size and complexity (larger number of allowed species and number of
symmetrically distinct sites) as shown in Figure 3.

(a) 2-2 LiMnOF binary-binary rocksalt

(b) 3-2 LiMnTiOF ternary-binary rocksalt

(c) 5-3-2 LiMnOF quinary-ternary-binary spinel-like

Figure 3: (a) System 2-2: Li-Mn-O-F rocksalt system with binary (Li+/Mn3+)
cation sites and binary (O2-/F-) anion sites. (b) System 3-2: Li-Ti-Mn-O-F
rocksalt system with ternary (Li+/Mn3+/Ti4+) cation sites and binary (O2-/F-)
anion sites. (c) System 5-3-2: Li-Mn-O-F spinel-like system with quinary
(Li/Mn2+/Mn3+/Mn4+/vacancy) octahedral cation sites, ternary (Li/Mn2+/vacancy)
tetrahedral cation sites, binary (O2-/F-) anion sites.

The total number of expansion terms considered for each fit are listed as the
model size in Table 1. The total number of terms are obtained by using the
same cutoff radius for clusters with up to four sites. Functions that evaluate
to the same value (remain constant) for the training structures used in the
corresponding fit are subsequently removed from the final measurement matrix
used for each fit. Particularly, removal of constant functions was only
required for the 5-3-2 system, for which the total number of columns in the
measurement matrix ended up being $4194$ and $17350$ for the indicator basis
CEM based and Potts frame models respectively. Additionally, we include an
electrostatic energy term[8, 38] as an additional feature in every fit.
Finally, for the fits using the Potts frame, we remove one cluster indicator
function from each set associated with the same orbit; this is simply to do
away with the trivial linear relation in Equation 10 applied to cluster
concentrations. It is clear by construction that for the same spatial cutoffs,
the number of terms in the generalized Potts frame far exceeds the number of
terms in a standard cluster expansion, and this difference grows exponentially
with the configuration space complexity.

The number of training structures and test structures for each fit is also
listed in Table 1. The formation energy for each training/test structure was
computed with density functional theory using the Vienna Ab Initio Simulation
Package[39] using the projector augmented wave (PAW) method,[40] with
reciprocal space discretization of 25 k-points per Å and a plane wave energy
cutoff of 520 eV. We carry out spin-polarized calculations using the SCAN
meta-GGA exchange correlation [41] and pseudopotentials which include semi-
core states: Li_sv, Mn_pv, O, and F. Structures are converged to $10^{-6}$ eV
in total energy and 0.01 eV/Å on atomic forces.

System | Expansion Type | Training Set Size | Test Set Size | Model Size
---|---|---|---|---
2-2 | CEM | 112 | 337 | 121
Potts Frame | 112 | 337 | 520
3-2 | CEM | 195 | 456 | 312
Potts Frame | 195 | 456 | 1040
5-3-2 | CEM | 312 | 56 | 7030
Potts Frame | 312 | 56 | 23070
Table 1: Regression model and training/test data size specifications for the
three fluorinated lithium-transition metal oxide systems.

A total of 50 different fits for each system are computed by selecting a
random set of training structures that gives a full rank under-determined
system. The remaining structures are used as a test set. The regression
problem solved which is known as Basis Pursuit Denoising or the LASSO is
slightly different than the $\ell_{1}$-analysis problem given in Equation 2.
The corresponding optimization problem is,

$\bm{\hat{J}}=\underset{\bm{\widetilde{J}}}{\text{argmin}}\;||\bm{\widetilde{J}}||_{1}\text{
subject to }||A\Pi\bm{\widetilde{J}}-E||_{2}\leq\varepsilon$ (12)

The problem in Equation 12 is solved for each fit using the python package
scikit-learn[42] in a two-step process. First a hyperparameter cross
validation optimization search is done with the training set. Subsequently a
finer hyperparameter search is done centered at the previously obtained value
now optimizing for out-of-sample error with respect to the test set but
training only with the training set data. From the resulting fits, those with
sparsity (number of nonzero coefficients) above the third quartile of the set
are considered outliers and removed from the results.

Values for accuracy metrics and sparsity results for the fitted expansion
obtained from the set of fits for each of the 3 expansion types and for each
of the 3 materials systems are shown in boxplots in Figure 4. The average
prediction accuracy metrics given include cross validation root mean squared
error (CV RMSE) for the initial cross validation hyperparameter search, the
out of sample RMSE for the final fit (test structures only), and the full data
RMSE for both the training and test structures combined. The average sparsity
value for the resulting fits is also listed.

Figure 4 shows boxplots depicting the resulting fit statistics in terms of
cross validation, out of sample and full sample (both training and testing
structures) root mean squared errors, along with the corresponding model
sparsity values. The results depicted in Figure 4 show that for all systems,
the expansions resulting from the Potts frame tend to have either a lower
minimum, median and mean accuracy metrics, a lower spread in these accuracy
metrics or both. In all cases the accuracy metrics are either very competitive
(very close to standard cluster expansions), or better. As for the resulting
model sparsity, although considerably more terms are included in the Potts
frame fit, the resulting expansions have similar sparsity along with lower
spread in sparsity values. These results demonstrate the applicability of
using coherency and redundancy to obtain expansions that match and even exceed
accuracy and sparsity of those obtained with standard cluster expansion basis
sets.

Figure 4: Fit metric statistics for the three fluorinated lithium-transition
metal oxide systems tested using standard CEM with sinusoid site bases,
indicator site basis and generalized Potts frame. LiMnOF binary-binary (top),
LiMnTiOF ternary-binary (middle), LiMnOF quinary-ternary-binary (bottom).

The boxplots in Figure 4 provide a summary of the average values obtained for
each type of fit, however they do not allow to see which models have both high
accuracy and high sparsity (low number of terms). Tables 2 and 3 list the same
fit metrics as Figure 4 but for the single sparsest (smallest number of
nonzero coefficients) fit and the fit resulting in the highest accuracy in
terms of the full data RMSE for each materials system.

System | Expansion | CV RMSE | Out RMSE | Full RMSE | Sparsity
---|---|---|---|---|---
2-2 | Sinusoid | 34.10 | 25.26 | 26.36 | 7
Indicator | 23.16 | 19.37 | 18.79 | 19
Potts Frame | 23.82 | 22.31 | 21.5 | 13
3-2 | Sinusoid | 24.13 | 27.62 | 24.14 | 56
Indicator | 25.14 | 21.69 | 19.55 | 54
Potts Frame | 22.26 | 23.43 | 20.93 | 49
5-3-2 | Sinusoid | 131.76 | 158.16 | 68.80 | 77
Indicator | 45.52 | 49.99 | 52.12 | 29
Potts Frame | 44.58 | 40.51 | 39.65 | 69
Table 2: Fitted model accuracy metrics and sparsity of sparsest models. Cross validations root mean squared error (CV RMSE), out of sample root mean squared error (Out RMSE), and full dataset root mean squared error (Full RMSE) are in meV per formula unit (random structure primitive cell). System | Expansion | CV RMSE | Out RMSE | Full RMSE | Sparsity
---|---|---|---|---|---
2-2 | Sinusoid | 25.68 | 18.42 | 16.66 | 50
Indicator | 18.63 | 17.87 | 15.76 | 121
Potts Frame | 21.42 | 16.11 | 14.89 | 43
3-2 | Sinusoid | 23.10 | 18.88 | 16.44 | 105
Indicator | 28.02 | 17.89 | 15.07 | 310
Potts Frame | 23.09 | 17.77 | 16.06 | 72
5-3-2 | Sinusoid | 104.39 | 36.72 | 17.24 | 291
Indicator | 55.69 | 27.45 | 20.38 | 147
Potts Frame | 115.42 | 24.52 | 18.98 | 192
Table 3: Fitted model accuracy metrics and sparsity of most accurate models in
terms of the root means squared error on the whole dataset. Cross validations
root mean squared error (CV RMSE), Out of sample root mean squared error (Out
RMSE), and Full dataset root mean squared error (Full RMSE) are in meV/prim

From Table 2, we see that for the sparsest expansions obtained, the ones
fitted using the Potts frame result in the lowest error metrics with only a
few exceptions where the Potts frame fits are still of comparable accuracy.
Furthermore, the expansions based on the Potts frame result in the lowest or
second lowest sparsity for all three systems. The results for the most
accurate models in terms of full dataset RMSE in Table 3 show that the Potts
frame results in both sparse and accurate models. For the cases where one of
the CEM based fits results in a lower full RMSE, the sparsity of that model is
compromised and substantially worse than the corresponding Potts frame fit.
This behavior can also be observed in the results in Figure 4, where the box
plot for sparsity obtained with the Potts frame has a smaller interquartile
range than that of the CEM based fits (with the exception of the 5-3-2 system
where it is only slightly larger than that of the indicator based cluster
expansion).

Figure 5 shows the sorted magnitudes of the fitted coefficients for each
expansion type and materials system for both the sparsest models and most
accurate models obtained. Based on the adequacy of the resulting fit error
metrics and the fast decay of coefficients shown in Figure 5, we can conclude
that the configuration energy if not exactly sparse is highly
compressible—since signals or functions with power law decaying (or faster)
coefficients can be well approximated by a small subset of terms.[30]
Specifically, a series of coefficients obey a power law decay if the sorted
sequence satisfies the following,

$|c_{i}|\leq Ci^{-q}$ (13)

Where $c_{i}$ are the coefficients and $C,q>0$ are constants. For all
expansion types we see that the coefficient magnitude decay is faster than the
power law decay shown.

We also see that in the results in Figure 5, the coefficient decay of fits
using the Potts frame are, at worst, the second fastest decaying series for
both the sparsest and most accurate fits, such that expansions using the Potts
frame are arguably more reliable in yielding sparser models than standard
cluster expansions. This seems surprising considering that the total number of
terms in the under-determined system is exceedingly larger than that of the
standard cluster expansions. However, considering the geometry of the union of
$s$-dimensional subspaces as illustrated in Figure 2, we posit that indeed
functions of configurational energy lie close to one of these subspaces with
high probability, and we can therefore accurately represent the function with
a much smaller number of terms than the total considered in the under-
determined system. Additionally, in light of the Theorem in Equation 3, the
rapid decay of coefficients suggest that an $s$-sparse set of coefficients is
close to the real coefficients and as a result the second term in the function
approximation error is likely to be very small.

Figure 5: Sorted coefficient magnitudes for sparsest and most accurate model
(Full RMSE). LiMnOF binary-binary (top), LiMnTiOF ternary-binary (middle),
LiMnOF quinary-ternary-binary (bottom).

We can take the previous results as ex post facto evidence that configuration
energy is a compressible function and that CS with redundancy and coherent
measurements works well for fitting expansions of configuration energy.
Additionally, we observed that expansions fitted using the Potts frame also
tend to follow expectations driven by physical considerations. Figure 6 shows
the number of nonzero coefficients for each crystallographic orbit considered
for the most accurate indicator basis and Potts Frame based models with
respect to the number of nonzero coefficients obtained in the most accurate
sinusoid basis based model. We notice from the plots, that using the Potts
frame, despite having a much larger number of total coefficients associated
with each orbit, results in fits that set a similar number of nonzero
coefficients within each orbit and never exceeds 2 additional coefficients per
orbit when compared with the sinusoid CEM fit. The significance is that, not
only do we recover accurate and sparse models, but the models themselves also
have similar sparsity structure to the CEM based models. Figure 6 also shows
the root sum of squares or norm of the fitted ECI for each orbit. We observed
that for the most accurate models the fitted coefficient weight associated
with each orbit is less erratic than that of the indicator based cluster
expansion. Additionally, the fitted coefficients follow the well-known CEM
heuristic of coefficient decay with orbit size more fittingly than the
indicator based CEM. The aforementioned observations again demonstrate using
the Potts frame can result in fitted expansions that are not only accurate and
sparse, but also produce well-behaved coefficients that align with practices
and heuristics based on physical insights used in the field.

Figure 6: Number of nonzero coefficients relative to the sinusoid basis fit
for each orbit, and norm of coefficients for each orbit for the most accurate
models (Full RMSE). The vertical dotted lines separate the degree of orbit
(pairs/triplets/quadruplets). LiMnOF binary-binary (top), LiMnTiOF ternary-
binary (middle), LiMnOF quinary-ternary-binary (bottom).

##  VI Discussion & Conclusion

We have briefly introduced the concepts behind CS with redundant
representations and coherent measurements within the context of the cluster
expansion method, with the goal of addressing the obstacles preventing
adequate sampling of training structures to satisfy requirements of classical
CS when fitting CE models. We proposed the generalized Potts frame as a
redundant function set that can be used in lieu of standard CEM basis sets to
represent functions over configurations of multicomponent crystalline
materials. Additionally, we described the connections between the Potts frame
and the commonly used non-orthogonal indicator function CEM basis set, as well
as the essence of the Potts frame as a natural extension of the well known
Potts model. Furthermore, we demonstrated the capability of obtaining arguably
improved fitted expansions in terms of prediction accuracy, sparsity and
robustness of coefficients compared to standard CEM fits. We stopped short of
providing specific sensing and structure selection methods along with rigorous
proofs that ensure small D-RIP constants; however the results from
applications to complex lithium-transition metal oxide systems show promising
results and the possibility of obtaining high accuracy expansions with such
redundant representations.

Although this exposition of the power of using overcomplete frames in place of
basis sets is particular in implementation and application, the concept can
well be extended to many more problems in the growing space of novel methods
for learning property-structure relationships in Materials Science. To our
knowledge the concept has previously been used in the context of the CEM,
although not explicitly, in the development of the variational basis cluster
expansion and its connection to wavelets [43]—which can formally constitute a
redundant frame. As a matter of fact, the process of fitting a cluster
expansion for any heterovalent ionic material system using only a charge-
neutral configurations is itself an application of fitting an expansion using
a frame. The reason being that a set of CEM correlation functions used in the
space of functions over constrained configurations in heterovalent systems is
in itself an overcomplete or redundant representation, thus the set of CEM
correlation functions is rigorously a frame for the function space over such
constrained configurations. Additionally, the application of redundant frames
along with CS using coherent measurements may also be worthwhile in other
areas of computational materials science which could benefit from higher
representation flexibility and additional robustness, particularly in methods
where statistical learning methods are used to estimate basis expansions in
high dimensional spaces. The simple idea of using redundant sets to construct
redundant product functions can be directly applied in cluster expansions
beyond discrete species to include continuous vector spaces at each site such
as in the spin and atomic cluster expansion.[44, 45] Particularly in the
representation and learning of atomic potentials, function basis sets over
continuous vector spaces can be replaced with corresponding frames over those
spaces. Furthermore, when configurational degrees of freedom exist, frames
over continuous vector spaces can be combined with finite frames, such as a
generalized Potts frame; and so in such cases the resulting product basis of
an atomic cluster expansion becomes a tensor product of continuous and finite
frames.

With the advent of the high entropy paradigm, as more complex materials
systems are explored it will become increasingly necessary to develop
methodologies that can work aptly with the rapidly growing multidimensional
spaces and relatively fixed training data set sizes. The compressed sensing
and sparse representation framework has been developed particularly for these
situations in other fields and has previously been shown to be successful in
simple materials science applications.[15, 21] Continued exploration of
systems with complexity beyond these systems requires continued development of
methods to handle the increase in dimensionality. The generalized Potts Frame
developed and presented here represents one effort in said direction.

## Acknowledgements

This work was primarily funded by the U.S. Department of Energy, Office of
Science, Office of Basic Energy Sciences, Materials Sciences and Engineering
Division under Contract No. DE-AC02-05-CH11231 (Materials Project program
KC23MP). This material is also based upon work supported by the National
Science Foundation Graduate Research Fellowship under Grant No. DGE 1752814.

A portion of the research was performed using computational resources
sponsored by the Department of Energy’s Office of Energy Efficiency and
Renewable Energy and located at the National Renewable Energy Laboratory.

##  Appendix A Frame Bounds for the Generalized Potts Frame

###### Definition A.1

A countable sequence $\\{\Phi_{\gamma}\\}_{\gamma\in I}$ is said to be a frame
for a Hilbert space $\mathcal{H}$ if there exist frame bounds $A,B>0$ such
that, [33, 34]

$A||H||^{2}\leq\sum_{\gamma\in I}|\langle H,\Phi_{\gamma}\rangle|^{2}\leq
B||H||^{2}\;\;\;\;\forall H\in\mathcal{H}$

The definition above implies that the frame $\\{\Phi_{\gamma}\\}_{\gamma\in
I}$ spans $\mathcal{H}$.[33]

To derive frame bounds for the generalized Potts frame, we work with the
product-basis of cluster indicator functions without taking symmetry adapted
averages. The bounds obtained apply equally to the functions obtained from
symmetry adapted averages. The lower frame bound $A$ is obtained by splitting
up the sum of projections into the frame elements as follows,

$\displaystyle\sum_{\gamma\in I}|\langle H,\bm{1}_{\gamma}\rangle|^{2}$
$\displaystyle=\sum_{\gamma_{\max}}|\langle
H,\bm{1}_{\gamma_{\max}}\rangle|^{2}+\sum_{\gamma\in
I\setminus\\{\gamma_{\max}\\}}|\langle H,\bm{1}_{\gamma}\rangle|^{2}$
$\displaystyle=||H||^{2}+\sum_{\gamma\in
I\setminus\\{\gamma_{\max}\\}}|\langle H,\bm{1}_{\gamma}\rangle|^{2}$
$\displaystyle\geq||H||^{2}$

Where $\\{\alpha_{\max}\\}$ denotes the set of all maximal clusters, and thus
represents the canonical orthonormal basis for the Hilbert space
$\mathcal{H}$. The second sum over all smaller clusters is always greater than
or equal to zero, and so the obtained lower frame bound is $A=1$.

To obtain an upper frame bound we start by writing the Fourier expansion for
cluster indicator functions,

$\displaystyle\bm{1}_{\gamma}(\bm{\sigma})$
$\displaystyle=\sum_{\alpha}\langle\bm{1}_{\gamma},\Phi_{\alpha}\rangle\Phi_{\alpha}(\bm{\sigma})$
$\displaystyle=\sum_{\alpha}\prod_{i}\langle\bm{1}_{\gamma_{i}},\phi_{\alpha_{i}}\rangle\Phi_{\alpha}(\bm{\sigma})$
$\displaystyle=\sum_{\begin{subarray}{c}\alpha;\\\
\text{supp}(\alpha)\subseteq\text{supp}(\gamma)\end{subarray}}\frac{1}{n^{\\#\gamma}}\prod_{i}\phi_{\alpha_{i}}(\sigma_{\gamma_{i}})\Phi_{\alpha}(\bm{\sigma})$
$\displaystyle=\sum_{\begin{subarray}{c}\alpha;\\\
\text{supp}(\alpha)\subseteq\text{supp}(\gamma)\end{subarray}}\frac{1}{n^{\\#\gamma}}\Phi_{\alpha}(\bm{\sigma_{\gamma}})\Phi_{\alpha}(\bm{\sigma})$

Where $\text{supp}(\cdot)$ represents the support or indices of the nonzero
entries of $\alpha$ or $\gamma$, and $\\#\gamma$ represents the total number
of nonzero entries—i.e. the number of sites in a cluster. $n$ is the number of
species allowed at a site and $\bm{\sigma}_{\gamma}$ is any occupancy string
that includes the cluster represented by $\gamma$. Note the above expression
is for a system with a single type of site (i.e. with the same set of allowed
species in all sites). The general expression simply involves one over the
product of the different number of species for each site instead of the factor
$1/n^{\\#\gamma}$.

Using the expansion given above for cluster indicator functions, we obtain an
upper frame bound as follows,

$\displaystyle\sum_{\gamma\in I}|\langle H,\bm{1}_{\gamma}\rangle|^{2}$
$\displaystyle=\sum_{\gamma\in I}\left|\left\langle
H,\sum_{\begin{subarray}{c}\alpha;\\\
\text{supp}(\alpha)\subseteq\text{supp}(\gamma)\end{subarray}}\frac{1}{n^{\\#\gamma}}\Phi_{\alpha}(\bm{\sigma_{\gamma}})\Phi_{\alpha}(\bm{\sigma})\right\rangle\right|^{2}$
$\displaystyle=\sum_{\gamma\in
I}\left(\frac{1}{n^{\\#\gamma}}\right)^{2}\left|\sum_{\begin{subarray}{c}\alpha;\\\
\text{supp}(\alpha)\subseteq\text{supp}(\gamma)\end{subarray}}\hat{H}_{\alpha}\Phi_{\alpha}(\bm{\sigma_{\gamma}})\right|^{2}$
$\displaystyle=\sum_{S}\frac{1}{n^{2|S|}}\sum_{\begin{subarray}{c}\bm{\sigma}_{\gamma};\\\
\text{supp}(\gamma)=S\end{subarray}}\left|\sum_{\begin{subarray}{c}\alpha;\\\
\text{supp}(\alpha)\subseteq
S\end{subarray}}\hat{H}_{\alpha}\Phi_{\alpha}(\bm{\sigma_{\gamma}})\right|^{2}$
$\displaystyle=\sum_{S}\frac{1}{n^{|S|}}\sum_{\begin{subarray}{c}\alpha;\\\
\text{supp}(\alpha)\subseteq S\end{subarray}}\hat{H}^{2}_{\alpha}$
$\displaystyle=\sum_{\alpha}\left(\sum_{S\supseteq\text{supp}(\alpha)}\frac{1}{n^{|S|}}\right)\hat{H}_{\alpha}^{2}$
$\displaystyle\leq\left(\sum_{S}\frac{1}{n^{|S|}}\right)||H||^{2}$
$\displaystyle=\left(1+n^{-1}\right)^{N}||H||^{2}$

Where $\hat{H}_{\alpha}$ are the Fourier coefficients of $H$. The sets $S$
contain site indices and the sum over these contains all possible subsets of
site indices—i.e. all clusters of un-labeled sites. The upper frame bound
obtained is $B=\left(1+n^{-1}\right)^{N}$, where $N$ is the total number of
sites in the structure. The bound obtained is an improvement over the bound
$2^{N}$ given by the Cauchy-Schwarz inequality.

## References

  * Sanchez  _et al._ [1984] J. M. Sanchez, F. Ducastelle, and D. Gratias, Generalized cluster description of multicomponent systems, Physica A: Statistical Mechanics and its Applications 128, 334 (1984).
  * Sanchez [1993] J. M. Sanchez, Cluster expansions and the configurational energy of alloys, Physical Review B 48, 14013 (1993).
  * Van der Ven  _et al._ [2018] A. Van der Ven, J. Thomas, B. Puchala, and A. Natarajan, First-Principles Statistical Mechanics of Multicomponent Crystals, Annual Review of Materials Research 48, 27 (2018).
  * van de Walle and Ceder [2002] A. van de Walle and G. Ceder, Automating first-principles phase diagram calculations, Journal of Phase Equilibria 23, 348 (2002).
  * Sanchez [2017] J. M. Sanchez, Foundations and Practical Implementations of the Cluster Expansion, Journal of Phase Equilibria and Diffusion 38, 238 (2017).
  * Otto  _et al._ [2013] F. Otto, Y. Yang, H. Bei, and E. P. George, Relative effects of enthalpy and entropy on the phase stability of equiatomic high-entropy alloys, Acta Materialia 61, 2628 (2013).
  * Liu  _et al._ [2020] X. Liu, J. Zhang, J. Yin, S. Bi, M. Eisenbach, and Y. Wang, Monte Carlo simulation of order-disorder transition in refractory high entropy alloys: A data-driven approach, arXiv:2011.00698 [cond-mat] (2020), arXiv:2011.00698 [cond-mat] .
  * D. Richards  _et al._ [2016] W. D. Richards, Y. Wang, L. J. Miara, J. Chul Kim, and G. Ceder, Design of Li 1+2x Zn 1-x PS 4 , a new lithium ion conductor, Energy & Environmental Science 9, 3272 (2016).
  * Urban  _et al._ [2014] A. Urban, J. Lee, and G. Ceder, The Configurational Space of Rocksalt-Type Oxides for High-Capacity Lithium Battery Electrodes, Advanced Energy Materials 4, 1400478 (2014).
  * Ouyang  _et al._ [2020] B. Ouyang, N. Artrith, Z. Lun, Z. Jadidi, D. A. Kitchaev, H. Ji, A. Urban, and G. Ceder, Effect of Fluorination on Lithium Transport and Short-Range Order in Disordered-Rocksalt-Type Lithium-Ion Battery Cathodes, Advanced Energy Materials 10, 1903240 (2020).
  * Clément  _et al._ [2018] R. J. Clément, D. Kitchaev, J. Lee, and Gerbrand Ceder, Short-Range Order and Unusual Modes of Nickel Redox in a Fluorine-Substituted Disordered Rocksalt Oxide Lithium-Ion Cathode, Chemistry of Materials 30, 6945 (2018).
  * Kostiuchenko  _et al._ [2020] T. Kostiuchenko, A. V. Ruban, J. Neugebauer, A. Shapeev, and F. Körmann, Short-range order in face-centered cubic VCoNi alloys, Physical Review Materials 4, 113802 (2020).
  * Connolly and Williams [1983] J. W. D. Connolly and A. R. Williams, Density-functional theory applied to phase transformations in transition-metal alloys, Physical Review B 27, 5169 (1983).
  * Mueller and Ceder [2009] T. Mueller and G. Ceder, Bayesian approach to cluster expansions, Physical Review B 80, 024103 (2009).
  * Nelson  _et al._ [2013a] L. J. Nelson, G. L. W. Hart, F. Zhou, and V. Ozoliņš, Compressive sensing as a paradigm for building physics models, Physical Review B 87, 035125 (2013a).
  * Leong and Tan [2019] Z. Leong and T. L. Tan, Robust cluster expansion of multicomponent systems using structured sparsity, Physical Review B 100, 134108 (2019).
  * Ångqvist  _et al._ [2019] M. Ångqvist, W. A. Muñoz, J. M. Rahm, E. Fransson, C. Durniak, P. Rozyczko, T. H. Rod, and P. Erhart, ICET – A Python Library for Constructing and Sampling Alloy Cluster Expansions, Advanced Theory and Simulations 2, 1900015 (2019).
  * Fox [2016] J. Fox,  _Applied regression analysis and generalized linear models_ , third edition ed. (SAGE Publications, Thousand Oaks, California, 2016).
  * Blum  _et al._ [2005] V. Blum, G. L. W. Hart, M. J. Walorski, and A. Zunger, Using genetic algorithms to map first-principles results to model Hamiltonians: Application to the generalized Ising model for alloys, Physical Review B 72, 165113 (2005).
  * Hart  _et al._ [2005] G. L. W. Hart, V. Blum, M. J. Walorski, and A. Zunger, Evolutionary approach for determining first-principles hamiltonians, Nature Materials 4, 391 (2005).
  * Nelson  _et al._ [2013b] L. J. Nelson, V. Ozoliņš, C. S. Reese, F. Zhou, and G. L. W. Hart, Cluster expansion made easy with Bayesian compressive sensing, Physical Review B 88, 155105 (2013b).
  * Huang  _et al._ [2018] W. Huang, A. Urban, P. Xiao, Z. Rong, H. Das, T. Chen, N. Artrith, A. Toumar, and G. Ceder, An L$_0$L$_1$-norm compressive sensing paradigm for the construction of sparse predictive lattice models using mixed integer quadratic programming, arXiv:1807.10753 [cond-mat, physics:physics] (2018), arXiv:1807.10753 [cond-mat, physics:physics] .
  * Calafiore and El Ghaoui [2014] G. C. Calafiore and L. El Ghaoui,  _Optimization Models_ (Cambridge University Press, 2014).
  * Candes and Wakin [2008] E. J. Candes and M. B. Wakin, An Introduction To Compressive Sampling, IEEE Signal Processing Magazine 25, 21 (2008).
  * George  _et al._ [2019] E. P. George, D. Raabe, and R. O. Ritchie, High-entropy alloys, Nature Reviews Materials 4, 515 (2019).
  * Lun  _et al._ [2021] Z. Lun, B. Ouyang, D.-H. Kwon, Y. Ha, E. E. Foley, T.-Y. Huang, Z. Cai, H. Kim, M. Balasubramanian, Y. Sun, J. Huang, Y. Tian, H. Kim, B. D. McCloskey, W. Yang, R. J. Clément, H. Ji, and G. Ceder, Cation-disordered rocksalt-type high-entropy cathodes for Li-ion batteries, Nature Materials 20, 214 (2021).
  * Hastie  _et al._ [2001] T. Hastie, R. Tibshirani, and J. Friedman,  _The Elements of Statistical Learning: Data Mining, Inference, and Prediction_, Springer Series in Statistics (Springer-Verlag, New York, 2001).
  * Candès  _et al._ [2011] E. J. Candès, Y. C. Eldar, D. Needell, and P. Randall, Compressed sensing with coherent and redundant dictionaries, Applied and Computational Harmonic Analysis 31, 59 (2011).
  * van de Walle [2009] A. van de Walle, Multicomponent multisublattice alloys, nonconfigurational entropy and other additions to the Alloy Theoretic Automated Toolkit, Calphad Tools for Computational Thermodynamics, 33, 266 (2009).
  * Davenport  _et al._ [2012] M. A. Davenport, M. F. Duarte, Y. C. Eldar, and G. Kutyniok, Introduction to compressed sensing, in  _Compressed Sensing_, edited by Y. C. Eldar and G. Kutyniok (Cambridge University Press, Cambridge, 2012) pp. 1–64.
  * Zhang and Sluiter [2016] X. Zhang and M. H. F. Sluiter, Cluster Expansions for Thermodynamics and Kinetics of Multicomponent Alloys, Journal of Phase Equilibria and Diffusion 37, 44 (2016).
  * Hart and Forcade [2008] G. L. W. Hart and R. W. Forcade, Algorithm for generating derivative structures, Physical Review B 77, 224115 (2008).
  * Christensen [2008] O. Christensen,  _Frames and Bases: An Introductory Course_, Applied and Numerical Harmonic Analysis (Birkhäuser Basel, 2008).
  * Waldron [2018] S. Waldron,  _An Introduction to Finite Tight Frames_, Applied and Numerical Harmonic Analysis (Birkhäuser Basel, 2018).
  * Potts [1952] R. B. Potts, Some generalized order-disorder transformations, Mathematical Proceedings of the Cambridge Philosophical Society 48, 106 (1952).
  * Wu [1982] F. Y. Wu, The Potts model, Reviews of Modern Physics 54, 235 (1982).
  * Clément  _et al._ [2020] R. J. Clément, Z. Lun, and G. Ceder, Cation-disordered rocksalt transition metal oxides and oxyfluorides for high energy lithium-ion cathodes, Energy & Environmental Science 13, 345 (2020).
  * Seko and Tanaka [2014] A. Seko and I. Tanaka, Cluster expansion of multicomponent ionic systems with controlled accuracy: Importance of long-range interactions in heterovalent ionic systems, Journal of Physics: Condensed Matter 26, 115403 (2014).
  * Kresse and Furthmüller [1996] G. Kresse and J. Furthmüller, Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set, Computational Materials Science 6, 15 (1996).
  * Kresse and Joubert [1999] G. Kresse and D. Joubert, From ultrasoft pseudopotentials to the projector augmented-wave method, Physical Review B 59, 1758 (1999).
  * Sun  _et al._ [2015] J. Sun, A. Ruzsinszky, and J. P. Perdew, Strongly Constrained and Appropriately Normed Semilocal Density Functional, Physical Review Letters 115, 036402 (2015).
  * Pedregosa  _et al._ [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Research 12, 2825 (2011).
  * Sanchez [2010] J. M. Sanchez, Cluster expansion and the configurational theory of alloys, Physical Review B 81, 224202 (2010).
  * Drautz and Fähnle [2004] R. Drautz and M. Fähnle, Spin-cluster expansion: Parametrization of the general adiabatic magnetic energy surface with ab initio accuracy, Physical Review B 69, 104404 (2004).
  * Drautz [2019] R. Drautz, Atomic cluster expansion for accurate and transferable interatomic potentials, Physical Review B 99, 014104 (2019).