# Two-parameter landscape of transport efficiency in mesoscopic networks:
transitions from the Braess to normal regimes without a congestion relaxation

A.D. Lobanov<EMAIL_ADDRESS>Moscow Institute of Physics and
Technology, 9 Institutskiy per., Dolgoprudny, Moscow Region 141701, Russia.
A.D. Lobanova<EMAIL_ADDRESS>Moscow Institute of Physics and
Technology, 9 Institutskiy per., Dolgoprudny, Moscow Region 141701, Russia.
A.M. Pupasov-Maksimov<EMAIL_ADDRESS>Universidade Federal de Juiz de
Fora Department of Mathematics

###### Abstract

This paper deals with the Braess paradox in quantum transport. The scattering
matrix formalism is used to consider a two-parameter family of mesoscopic
conductors with the topology of the classical Braess transport network. The
study investigates the mutual influence of the congestion and smoothness of
the system on the Braess behavior. Both the Braess paradox and normal
transport regimes coexist within the two-parametric landscape under the same
congestion.

††preprint: APS/123-QED

##  I Introduction

The Braess paradox is an unusual behavior of a network in response to the
addition of ”extra” paths. Instead of improvements in transport efficiency,
”extra” paths impair network traffic. This phenomenon has been found in almost
all fields of physics. Its presence is recorded in road transport networks, in
electric circuits, hydraulic gas networks and even mechanical networks [1, 2,
3]. An analogue of this process in nuclear physics could be the nuclear
Overhauser effect [4]. Recently, such behaviour was predicted and observed
experimentally in a mesoscopic conducting network [5]. The effect implies a
weakening of the current in an extended network contrary to an expected
increase.

This study considers theoretical results [5, 6] within the framework of the
scattering matrix approach, using its equivalence with the non-equilibrium
Green functions method [7]. Further, we extend the model [5] to take into
account other possible parameters of a mesoscopic network. It was shown [5]
that releasing congestion considerably relaxes the paradoxical behavior of the
network. It happens because of allowing a larger number of conducting channels
to propagate in the region inside the structure. Here we demonstrate that
Braess behaviour can also be eliminated by smoothing the network whose
branches still maintain a congested transport regime. Smooth connections
correspond to adiabatic changes in structure width along the electron wave
function propagation. We construct a landscape of 2-3 branched transport
system properties according to two competing parameters - adiabaticity of the
system (smoothness) and congestion of additional branches. This construction
could lead to a better understanding of quantum transport.

Numerical calculations of the corresponding tight-binding model were done with
the Kwant package of Python 3 [8]. The paper is organized as follows. It
starts with the theoretical implementation of the model and the reproduction
of the main results of [5, 6]. Then a smoothed version of branched network is
introduced to provide calculations of the two-parameter landscape of transport
properties. The main results are discussed in conclusion.

Figure 1: We study 2-parametric family of mesoscopic systems. The first
parameter is the width $W_{3}$ of the third path. The second parameter is
responsible for continuous deformation from rectangular to ellipsoidal
contour. The rectangular shape is equivalent to the system from [5] in terms
of geometry. Figure 2: Absence of any Braess behavior in the classical circuit
without active elements or additional voltage drop. Increase of conductance
$\sigma_{3}$ leads to the corresponding increase of current when the bridge is
unbalanced ($\sigma_{1}\neq\sigma_{2}$). In the case of
$\sigma_{1}=\sigma_{2}$ the current does not depend on $\sigma_{3}$.

##  II Theoretical implementation of the model

Following [5], we compare transport properties of two mesoscopic samples with
2 and 3 ring structures (see Fig. 1 B, C). In the case of an analogous
classical electric circuit, Braess behavior is absent (see Fig. 2) since no
active elements (Zener diodes) or additional voltage drops are added to
simulate transportation costs analogous to the original transport network [3,
9]. It is obvious that the difference between the quantum problem and the
classical one is that the connection of two subsystems should be considered as
a perturbation, and this leads to their mutual influence and a change in
physical properties (spectrum, transmission coefficient, etc.). Note that in
classical networks (electrical, hydraulic), connections between elements can
also perturb the properties of the system; for example, the non-zero hydraulic
resistance of the tee or the bending of the pipe. In [10] such non-linearity
was used to cause the Braess paradox in microfluidic networks.

For computational purposes we downscale the linear size of the sample from [5]
by factor 10. The basis network takes a rectangular form with approximately 90
sites in horizontal direction and 60 sites in vertical direction. It is
connected to bottom (source) and top (drain) ohmic leads via two planar wires
(openings) of width $W_{0}=20$ sites. The left and right lateral wires of
width $W_{1}=W_{2}=10$ sites are narrower than the top and bottom horizontal
paths. Narrow lateral paths ensure congested transport mode of the electrons
propagation. The central wire directly connects the source to the drain and
creates a third alternative path of propagation (Fig. 1).

Using an analogy with hydrodynamics we also considered an analogue of smooth
bends in hydraulic networks to test its influence on the propagation of
transverse modes along the conductor. Notably, the width of the lateral wires
stays unchanged to maintain the congested regime which is necessary for the
Braess paradox.

The geometry of our network is defined by the following system of inequalities
(see also code [11])

$\displaystyle 1.2x_{l}^{n}+1.2y_{l}^{n}<30^{n}\quad\text{and}\quad
x_{s}^{n}+y_{s}^{n}>14^{n}$ (1)
$\displaystyle\text{or}\quad|x|<W_{3}\quad\text{and}\quad|y|<25.5\,,$ (2)

where

$x_{l}=\max(|x|-20,0),\qquad x_{s}=\max(|x|-28,0).$

Here, to modify the geometry of our network from Fig. 1, B, C to Fig. 1, A, D,
we introduce parameter $n$ which defines the geometry of external and internal
boundaries of lateral wires. Thus, the width of the central path $W_{3}$ and
the roundness parameter $n$ are used to define two-parametric family of
mesoscopic networks.

Figure 3: The dependence of conductivity on energy for the different profiles.
Here it is clearly seen that the rounding removes any hint of the presence of
Braess paradoxical behavior. Figure 4: The profile with no ”congestion” state.

To detect the Braess-like behavior, it is sufficient to compare the total
current through the systems, varying the width $W_{3}$ of the third path.
Therefore, we can use Landauer-Büttiker Formalism based on the scattering
matrix method [12, 13]. There are only two leads, and the current through the
lead ”a” reads

$I_{a}=\frac{e}{h}\int dE|S_{ab}(E)|^{2}\left(f_{b}(E)-f_{a}(E)\right)\,,$ (3)

where $S_{ab}(E)$ is the S-matrix, and

$f_{a,b}(E)=\frac{1}{1+\exp\left(\frac{E-\mu_{a,b}}{k_{B}T_{a,b}}\right)}\,,$
(4)

are Fermi distribution functions in the reservoirs ”a, b”. Under the DC bias,
$\Delta U_{ab}=U_{a}-U_{b}$, reservoirs ”a” and ”b” have different
electrochemical potentials,

$\mu_{a,b}=\mu_{0}+eU_{a,b}$ (5)

This approach is equivalent to the method based on non-equilibrium Green’s
functions [13, 14, 7]

$G_{i,j}^{R}(t,t^{\prime})=-i\Theta(t-t^{\prime})\sum_{\alpha}\int\frac{dE}{2\pi}\psi_{\alpha
E}(t,i)\psi_{\alpha E}^{*}(t^{\prime},j)$ (6)
$G_{i,j}^{<}(t,t^{\prime})=i\sum_{\alpha}\int\frac{dE}{2\pi}f_{\alpha}(E)\psi_{\alpha
E}(t,i)\psi_{\alpha E}^{*}(t^{\prime},j)$ (7)

which were used in [5]. Roughly speaking, one can solve the Dyson equation for
the wave function, instead of the Green function. From the wave function we
can extract scattering amplitudes and define all transport properties of the
system. Moreover, numerical calculations of wave functions require less memory
and operations.

The graph of the dependence of conductivity on energy (Fig. 3) clearly shows
that the conductivity of the two paths is higher; moreover, the difference
between them increases depending on the energy. It is also worth adding that
if you continue the graph to large values, you can see that the difference in
conductivities increases to a certain peak. Then, at this peak there is a
sharp drop in conductivity for the two paths, after which the results are
mirrored. This can be clearly seen in the models below. The results of the
probability density distribution for a given mode are shown next to it (Fig.
3). In fact, the last result (Fig. 3) varies greatly at different energies. It
gives us grounds to assume that a sharp increase in conductivity for three
paths simultaneously with a sharp drop for two is due to the fact that the
electrons ”go better” in the model with three paths for a given energy, and
”worse” in the two-path model.

Figure 5: 3D dependence of the total current on the width of the third channel
and adiabaticity.

##  III Two-parametric landscape of transport efficiency

In this section we investigate the interplay between parameters $W_{3}$ and
$n$ which results in a transition between the Braess paradox and normal
transport regime without relaxing the congestion of the network.

Below we first present some particular cases and plot the conductivity, as
well as the probability density distribution for a given mode (Fig. 3).

The results are dramatically different. The Braess behavior of the system has
disappeared. Also at all energies it can be seen that, as intuitively assumed,
a system with three paths wins in conductivity, relative to another system
with two paths. The picture of the probability density distribution shows that
the middle path is actively ”used” by electrons. However, the total current
here decreases three fold. There is no Braess behavior, and the new path
improves conductivity, but the value of the total current drops. The same
effect will be observed below, with other ”rounded” geometries.

Thus, it can be said that adiabaticity affects the conductivity of the system.
However, it is worth checking whether there is a place for the concept of
”congestion” in such a system, or whether such an effect is associated only
with adiabaticity.

To this end, we reproduced another result of [5], where all paths widen to a
state where there is no ”congestion” (Fig. 4), (Fig. 5).

As it was shown in [5], the ”broadening” of all system parameters, except for
the middle path, leads to the conductivity of three paths being almost always
higher than the conductivity of two. There are only several random peaks,
where the conductivity for the two paths is higher. These peaks could be
explained by fluctuations.

The resulting graph of the dependence of conductivity on energy is almost
identical, with the only difference being that the ”rounded” version has a
stronger negative peak, as here (Fig. 5). This means that adiabaticity affects
the results, but it is not the only reason for such Braess-like behavior. The
probability density distribution shows that the middle path is almost not
”congested”. These results lead to the idea that the concept of ”congestion”
is not unambiguous.

To show that it is not only the demand that affects here, we can give an
example of a situation where the source and drain are smaller than the central
wire (Fig. 5) Here, the demand will be minimal, however, according to the
energy conduction graph, there are still energies with Braess behavior.

Finally, we construct a landscape of transport efficiency of the considered
network. We plot a 3-D graph of the total current as a function of the width
of the central wire $W_{3}$ and the roundness parameter $n$ on Fig. 5 . To
test stability of the results we repeat calculations at different temperatures
and electro-chemical potentials.

It can be seen that in the case of a smooth (rounded) system, the total
current is increased as a function of $W_{3}$ thus avoiding the Braess
paradox. Moreover, increasing $n$ we obtain regions with Braess and normal
behaviour as a trivial consequence of a non-flat landscape.

##  IV Conclusion

To detect Braess-like behavior it is sufficient to calculate the total current
through the system, therefore the scattering matrix approach can be efficient.
Since the solution of Dyson equation for the non-equilibrium Green function
requires large computational resources, it is difficult to study transport
properties of a system under variation of several parameters. In our work we
demonstrated how such a study can be done with an alternative approach. First
of all, our two-parametric family of mesoscopic systems contains a region
already studied in [5, 6]. It is a region close to the BC segment of our
parameter space (see Fig. 5). The behaviour of our system in this region is in
accordance with the results of [5]. In [5] robustness of Braess-like behaviour
was proved by considering a large sample with many conducting channels
contributing to the total current and by varying the Fermi wavelength. Our
calculations also confirm that this Braess-like behaviour is not related to
phase-coherent or size effects. However, we also found regions in the
parameter space, where a mesoscopic sample with the same geometrical
congestion of lateral branches does not demonstrate Braess-like behavior.
Indeed, the total current increases along the AD segment in the upper part of
Fig. 5. Therefore we can conclude, that the Braess-like behavior observed in
[5, 6] is not related to geometrical congestion of the network.

Actually, it is a consequence of a nonlinear landscape of the total current as
a function of parameters of the model. Let us fix a point of the parameter
space and consider the corresponding Hamiltonian and wave functions
unperturbed. Small changes of parameters (width $W_{3}$ or smoothness of
boundaries) can be considered as a perturbation. Total current is the
expectation value of the current operator, therefore its derivative with
respect to the perturbation parameter can be calculated by the finite
temperature version of Hellmann–Feynman theorem [15]. In general, the sign of
this derivative can be arbitrary depending on the model. Only in the case of
an electrical current passing through a perfectly smooth narrow constriction
will such a derivative be non-positive. This is because the current is carried
by a finite number of quantized modes (analogous to those in a waveguide),
each of which contributes $2e^{2}/h$ to the conductance [16] the maximum
current is achieved. As a result, any perturbation will result in the decrease
of the total current. This decrease may correspond to Braess-like behavior
when one increases the width of conductor thus adding conducting channels,
while the source and drain leads stay unchanged.

In a more complicated model with 2-ring geometry we have shown that one can
relax Braess behavior without relaxing geometrical congestion of the network.
Finally, we can speculate that observing a 3-parameter family of mesoscopic
systems, one may find typical singular points in the level sets of the total
current landscape which lead to bifurcations in system behaviour and jumps in
its characteristics. These findings have the potential to advance the
development of built-in control mechanisms in mesoscopic networks.

## References

  * [1] John F Nash et al.  Equilibrium points in n-person games.  _Proceedings of the national academy of sciences_ , 36(1):48–49, 1950.
  * [2] Dietrich Braess.  Über ein paradoxon aus der verkehrsplanung.  _Unternehmensforschung_ , 12(1):258–268, 1968\.
  * [3] Joel E Cohen and Paul Horowitz.  Paradoxical behaviour of mechanical and electrical networks.  _Nature_ , 352(6337):699–701, 1991.
  * [4] David Neuhaus.  Nuclear overhauser effect.  _eMagRes_ , 2007.
  * [5] Marco Pala, Hermann Sellier, Benoit Hackens, Frederico Martins, Vincent Bayot, and Serge Huant.  A new transport phenomenon in nanostructures: a mesoscopic analog of the braess paradox encountered in road networks.  _Nanoscale research letters_ , 7(1):1–4, 2012a.
  * [6] Marco G Pala, Samuel Baltazar, Peng Liu, Hermann Sellier, Benoît Hackens, F Martins, Vincent Bayot, Xavier Wallart, Ludovic Desplanque, and Serge Huant.  Transport inefficiency in branched-out mesoscopic networks: An analog of the braess paradox.  _Physical Review Letters_ , 108(7):076802, 2012b.
  * [7] Thomas Kloss, Joseph Weston, Benoit Gaury, Benoit Rossignol, Christoph Groth, and Xavier Waintal.  Tkwant: a software package for time-dependent quantum transport.  _New Journal of Physics_ , 23(2):023025, 2021\.
  * [8] Christoph W Groth, Michael Wimmer, Anton R Akhmerov, and Xavier Waintal.  Kwant: a software package for quantum transport.  _New Journal of Physics_ , 16(6):063065, 2014\.
  * [9] Ladimer S Nagurney and Anna Nagurney.  Observation of the braess paradox in electric circuits.  _Europhysics Letters_ , 115:28004, 2016.
  * [10] Daniel J Case, Yifan Liu, István Z Kiss, Jean-Régis Angilella, and Adilson E Motter.  Braess’s paradox and programmable behaviour in microfluidic networks.  _Nature_ , 574(7780):647–652, 2019.
  * [11] A.D. Lobanov, A.D. Lobanova, and A.M. Pupasov-Maksimov, 2021.  URL https://github.com/LobanovAD/B.P.
  * [12] G. B. Lesovik and I. A. Sadovskyy.  Scattering matrix approach to the description of quantum electron transport.  _Phys. Usp._ , 54(10):1007–1059, 2011.
  * [13] Michael V Moskalets.  _Scattering matrix approach to non-stationary quantum transport_.  World Scientific, 2011.
  * [14] Benoit Gaury, Joseph Weston, Matthieu Santin, Manuel Houzet, Christoph Groth, and Xavier Waintal.  Numerical simulations of time-resolved quantum electronics.  _Physics Reports_ , 534(1):1–37, 2014.
  * [15] Marina Pons, Bruno Juliá-Díaz, Artur Polls, Arnau Rios, and Isaac Vidaña.  The hellmann–feynman theorem at finite temperature.  _American Journal of Physics_ , 88(6):503–510, 2020.
  * [16] JM Krans, JM Van Ruitenbeek, VV Fisun, IK Yanson, L De Jongh, and J.  The signature of conductance quantization in metallic point contacts.  _Nature_ , 375(6534):767–769, 1995.


# Image-Caption Encoding for Improving Zero-Shot Generalization

Eric Yang Yu  Christopher Liao  Sathvik Ravi  Theodoros Tsiligkaridis  Brian
Kulis

###### Abstract

Recent advances in vision-language models have combined contrastive approaches
with generative methods to achieve state-of-the-art (SOTA) on downstream
inference tasks like zero-shot image classification. However, a persistent
issue of these models for image classification is their out-of-distribution
(OOD) generalization capabilities. We first show that when an OOD datapoint is
misclassified, the correct class can be typically found in the Top-$K$
predicted classes. In order to steer the model prediction toward the correct
class within the top predicted classes, we propose the _Image-Caption Encoding
(ICE)_ method, a straightforward approach that directly enforces consistency
between the image-conditioned and caption-conditioned predictions at
evaluation time only. Intuitively, we take advantage of unique properties of
the generated captions to guide our local search for the correct class label
within the Top-$K$ predicted classes. We show that our method can be easily
combined with other SOTA methods to enhance Top-1 OOD accuracies by _0.5% on
average_ and _up to 3% on challenging datasets_. Our code:
https://github.com/Chris210634/ice

Machine Learning, ICML

##  1 Introduction

Figure 1: A demonstration for how our ICE method can be used to reclassify
correctly. In these examples, ICE is applied directly to a frozen pre-trained
CLIP-based model for zero-shot classification. Using the contexts given from
the generated captions, ICE is able to successfully influence the pretrained
model into predicting the correct classes.

There has been rapid progress in zero-shot image classification over the past
two years, thanks to advancements in vision-language (VL) pre-training such as
CLIP, ALIGN, and BLIP (Radford et al., 2021; Cohen, 1997; Li et al., 2022). At
a high level, these models use a pair of encoders that project visual and
textual inputs into a joint latent embedding space. As described in the CLIP
framework (Radford et al., 2021), zero-shot classification can be reformulated
as an image-to-text retrieval problem, where the class name closest to the
image in embedding space is predicted as the label. However, state-of-the-art
(SOTA) zero-shot classification lags behind in-distribution supervised fine-
tuning on all benchmarks. In many applications, in-distribution data is not
available during training, so fine-tuning on out-of-distribution (OOD) source
data in a way that generalizes to unseen data and labels remains a challenging
problem. Prior works in this area follow two general directions: (1) _Few-shot
OOD_ methods such as CoOp (Zhou et al., 2022b), CoCoOp (Zhou et al., 2022a),
and MaPLe (Khattak et al., 2023) fine-tune the VL model on generic few-shot
source data (e.g. ImageNet). The fine-tuning process is constrained to a
carefully selected subset of parameters to ensure generalization to target
datasets. (2) _Zero-shot_ methods, such as Menon & Vondrick (2022) and manual
prompt ensembling (Radford et al., 2021), focus on refining the zero-shot
prediction without additional fine-tuning. These methods do not require
additional data, but they typically either require a large closed-source LLM
or human-engineered prompts.

_Our goal is improve zero-shot classification by leveraging captioners, which
is previously under-explored._ Towards this goal, we first observe in Figure 2
that the Top-$K$ accuracy (the percentage of samples where the correct label
is within the $K$ classes with highest predicted scores, $K>1$) is
consistently higher than the Top-$1$ accuracy. The reason is that the Top-$K$
predicted classes form a strict superset of the Top-$1$ predicted classes when
$K>1$, and thus characterize a wider range of potentially correct classes. We
observe from Figure 2 that when an image is misclassified, the correct class
can usually be found within the Top-$5$ predicted classes. Thus, our
motivating question is: in order to improve Top-$1$ accuracy, _how can we
steer the model prediction toward the correct prediction within the Top- $K$
predicted classes using generated captions?_

Figure 2: A visualization of the Top-5 accuracies on misclassified Top-1
datapoints in each test dataset. Recall that correct Top-5 classifications
form a strict superset over the correct Top-1 classifications. We observe that
across all datasets, the true correct class can be found within the Top-5
predicted classes for most misclassified datapoints. Figure 3: A visualization
of Top-1 accuracies between CLIP, CoCa using image embeddings only, and CoCa
using caption embeddings only. We observe that while caption embeddings
generally underperform compared to standard CoCa, they still retain
competitive performance. We include more details on datasets and experiments
in Section 4.

Before we address this question, we first note that current SOTA zero-shot
image classification methods perform a nearest-neighbor search between image
and text CLIP embeddings (Radford et al., 2021). Recently, many works such as
CoCa (Yu et al., 2022), BLIP-2 (Li et al., 2023) and LLaVA (Liu et al., 2023)
extend CLIP with an additional text decoder. This text decoder is trained to
output a description of the image by cross-attending to _all_ image tokens
outputted by the image encoder. Consequently, the decoder output captures
fine-grained spatial information that may be absent from the image cls token.
Furthermore, the caption verbalizes the content of the image as discrete text
tokens, which can oftentimes be used to directly infer the image label. These
advantages are illustrated in Figure 3, where we use a spider plot to compare
the Top-$1$ zero-shot accuracies achieved by CLIP image embeddings, CoCa image
embeddings, and CoCa caption embeddings, across 15 datasets. We notice that
while the caption-only CoCa under-performs compared to standard CoCa, it is
still competitive and even surpasses CLIP on many datasets. This suggests that
the captions likely contain enough information about the image to supplement
the standard zero-shot prediction.

Figure 4: An overview of our _Image-Caption Encoding (ICE)_ method. Here, we
query a captioner and obtain the caption embedding using the text encoder. We
calculate the image and caption probability distributions over the classes by
passing the image embeddings, caption embeddings, and class embeddings through
the cosine similarity function $\theta$ and softmax operation. Then, we select
the Top-$K$ classes and perform a weighted sum of the image and caption
probabilities. The weight on the caption prediction $\lambda$ is adaptively
selected based on the relative confidence of the image and caption
predictions.

To leverage this additional information, we propose a novel zero-shot method
called _Image-Caption Encoding (ICE)_ , where we combine the information
encoded by both image embeddings and caption embeddings in order to make a
more informed decision at test time. As illustrated in Figure 4, ICE is a
zero-shot method with no training component and can be easily paired with
existing SOTA methods for improved downstream classification performance.

Although ICE draws inspiration from traditional ensembling techniques, there
are several key differences. First, ICE leverages the predictions obtained
from a _single_ model rather than those from several _different_ models.
Second, instead of aggregating predictions over all classes, we only consider
the Top-$K$ image predicted classes. Third, we incorporate a lightweight
confidence selection mechanism that sets the weight on the caption prediction
dynamically. Finally and most importantly, we exploit specific properties
induced within captions that are not present in the image embeddings for
standard zero-shot classification. We discuss these properties in-depth in
Section 3.4 and demonstrate specific examples of their non-trivial impact in
Section 4.3.

Our contributions are as follows:

$1.$ We extend the zero-shot classification literature by leveraging
captioners, which is a direction that is previously under-explored.

$2.$ We propose _Image-Caption Encoding (ICE)_ , a novel zero-shot
classification method that utilizes information from both images and captions
to make a decision at evaluation-time only.

$3.$ We provide experimental results for ICE paired with several different
SOTA baselines across 15 different OOD datasets. We show consistent
improvements of $0.5\%$ on average and up to $3\%$ on several challenging
datasets.

$4.$ We analyze the benefits and drawbacks of using ICE, and provide ablation
studies to analyze the effects of changing different parameters in our ICE
method.

##  2 Related Works

Multimodal foundational models. Many VL foundational models have emerged over
the past two years, including CLIP (Radford et al., 2021), ALIGN (Cohen,
1997), BLIP (Li et al., 2022), CoCa (Yu et al., 2022), BLIP-2 (Li et al.,
2023), and LLaVA (Liu et al., 2023). These models achieve SOTA on VL tasks by
using vast quantities of un-curated image-text data from the web. CLIP uses an
image encoder and a text encoder to project the two modalities into a joint
latent embedding space. Popular downstream applications include zero or few-
shot classification and image-text retrieval. The more recent models such as
CoCa, BLIP-2 and LLaVA improve CLIP by additionally training a text decoder to
explain the embedding space with a caption. In the current work, we leverage
this additional captioning capability to improve the CLIP zero-shot accuracy.

Robust fine-tuning. There is growing interest in fine-tuning multimodal
foundational models on limited training data such that the resulting model
achieves high accuracy even on domain-shifted data and data with labels not
seen during training. Many modern approaches rely on prompt tuning and
ensembling. CoOp (Zhou et al., 2022b) is a seminal work which treats the
prompt preceding the label names as soft learnable tokens. CoCoOp (Zhou et
al., 2022a) trains a meta-network to condition the prompt tokens on the image
embedding. MaPLe (Khattak et al., 2023) shows that learning a conditional
visual prompt jointly with textual prompts improves target accuracy. All three
works achieve impressive results on a diverse set of test datasets despite
only being trained on few-shot ImageNet data. ClipOOD (Shu et al., 2023) uses
an adaptive margin loss to optimize the visual encoder only, attaining good
results on domain generalization benchmarks. Our proposed method ICE is a
training-free approach that can be readily combined with the above fine-tuning
methods to yield higher accuracy on most target datasets.

Ensembling for robust classification. Ensembling methods leverage multiple
diverse predictions to form a robust final prediction; many recent works,
including ours, focus on discovering new sources of diversification. WiSE-FT
(Wortsman et al., 2022) calculates a weight space ensemble of the fine-tuned
and pre-trained models to increase robustness under distribution shifts in
target data, while inference time remains the same. Vogt-Lowell et al. (2023)
demonstrate that combining cross-entropy fine-tuning with stochastic weight
averaging improves domain generalization. Menon & Vondrick (2022) use GPT
descriptions to generate a more diverse set of text prototypes for zero-shot
classification. Radford et al. (2021) use an ensemble of 80 handcrafted
prompts to achieve the same goal. In our paper, while drawing inspiration from
ensembling, we introduce the novel use of captions as a unique source of
diversification, an avenue not yet explored by previous studies. Importantly,
our approach is designed to seamlessly integrate with other zero-shot methods,
as demonstrated in our experimental results.

##  3 Methodology

###  3.1 Preliminaries

Consider a dataset $\mathcal{D}\subset\mathcal{I}\times\mathcal{T}$ where
$\mathcal{I}$ is the image domain and $\mathcal{T}$ is the text domain, and
$(I_{i},T_{i})$ forms a corresponding image-text pair (i.e. $T_{i}$ is a
caption that describes $I_{i}$). In the CLIP framework (Radford et al., 2021),
there are two main architectural components: an image encoder
$f_{\mathcal{I}}:\mathcal{I}\rightarrow\mathbb{R}^{l}$ that maps images to a
shared latent space, and a text encoder
$f_{\mathcal{T}}:\mathcal{T}\rightarrow\mathbb{R}^{l}$ that maps text to the
same shared latent space. Both encoders are pre-trained using a contrastive
loss that pulls corresponding image-text embeddings close together in latent
space, and pushes non-corresponding image-text embeddings away from each
other. In our framework, we require an additional text decoder, pre-trained
using a next-token-prediction loss, to provide captioning functionality.

In image classification following the CLIP framework (Radford et al., 2021),
for an image vector $I$ and class labels vector
$y:=[y_{1},y_{2},\dots,y_{m}]^{\intercal}$, we first feed each class label
through a prompt skeleton to obtain class prompts (e.g. for class “cat” and a
prompt skeleton “A photo of a {}”, the resulting prompt is “A photo of a
cat”). Then, both the image and class prompt vectors pass through the image
and text encoders to obtain latent embeddings $\widetilde{I}$ and
$\widetilde{y}$, respectively. The predicted class label $\hat{y}$ is then
given by
$\operatorname*{arg\,max}_{i}\theta(\widetilde{I},\widetilde{y}_{i}),i\in\\{1,2,\dots,m\\}$,
where $\theta:\mathbb{R}^{l}\times\mathbb{R}^{l}\rightarrow\mathbb{R}$ is the
cosine similarity function.

Figure 5: A more detailed look at how our _Image-Caption Encoding (ICE)_
method works. In practice, instead of using a single caption for ICE, we use
the centroid of $\upsilon$ differently-prompted caption embeddings. Then,
using the centroid caption embedding, we adaptively select the $\lambda$
weight by comparing the standard deviations of the image prediction
probabilities and caption prediction probabilities, over the Top-$5$ classes.
The final ICE scores are then a $\lambda$-weighted sum between the two
probability distributions.

###  3.2 Image-Caption Encoding

Consider a text decoder
$f_{\mathcal{\phi}}:\mathcal{P}\times\mathcal{I}\rightarrow\mathcal{T}$ that
maps a prompt $p\in\mathcal{P}$ (e.g. ”a photo of”) and an image $I$ to a
caption $c\in\mathcal{T}$. We can feed caption $c$ back through the text
encoder to obtain $\widetilde{c}:=f_{\mathcal{T}}(c)$.

Using the Softmax function (Bridle, 1989), we obtain the class probabilities
for image $I$ and caption $c$, respectively, as

$\displaystyle
S^{I}:=\text{Softmax}\biggl{(}\Bigl{[}\theta(\widetilde{I},\widetilde{y}_{1}),\theta(\widetilde{I},\widetilde{y}_{2}),\dots,\theta(\widetilde{I},\widetilde{y}_{m})\Bigr{]}^{\intercal}\biggr{)}$
(1) $\displaystyle
S^{c}:=\text{Softmax}\biggl{(}\Bigl{[}\theta(\widetilde{c},\widetilde{y}_{1}),\theta(\widetilde{c},\widetilde{y}_{2}),\dots,\theta(\widetilde{c},\widetilde{y}_{m})\Bigr{]}^{\intercal}\biggr{)}.$

The indices corresponding to the $K$ classes with highest image-predicted
probability, and the final ICE prediction, are computed respectively as:

$\displaystyle\Omega^{I}_{K}:=\operatorname*{arg\,max}_{J\subset
M,|J|=K}\sum_{j\in J}S^{I}_{j}$
$\displaystyle\operatorname*{arg\,max}_{\omega\in\Omega^{I}_{K}}S^{I}_{\omega}+\lambda
S^{c}_{\omega}$ (2)

where $M=\\{1,2,\dots,m\\}$ and $\lambda$ is a scalar variable. In essence,
the ICE prediction remains anchored within the primary Top-$K$ classifications
as determined by the image class probabilities. When we incorporate the
caption scores tied to these Top-$K$ predictions, it reshapes the probability
landscape over the initial Top-$K$ image-determined classes. By selecting the
class with the highest probability from this refined distribution, we aim to
align closer with the true class while retaining the accuracy of previous
classifications. Intuitively, the caption probability distribution should
provide information about the image that is not clear or fully captured by the
image probability distribution, as detailed in Section 3.4, and their
aggregated prediction should provide a more reasonable downstream prediction.
Figures 4 and 5 contain a high-level overview and an in-depth visual
interpretation of our method, respectively.

###  3.3 Additional Modifications

As visualized in Figure 5, in our experiments, we use the centroid of a
diverse set of captions rather than a single caption for increased robustness.
That is, for a set of prompts $P\in\mathcal{P}^{\upsilon}$, we generate a set
of corresponding captions $C\in\mathcal{T}^{\upsilon}$, obtain their caption
embeddings
$\widetilde{C}:=\\{f_{\mathcal{T}}(c_{1}),f_{\mathcal{T}}(c_{2}),\dots
f_{\mathcal{T}}(c_{\upsilon})\\}$, and finally, their centroid
$\widetilde{\overline{c}}:=\frac{1}{\upsilon}\sum^{\upsilon}_{i}\widetilde{c}_{i}$.
The centroid $\widetilde{\overline{c}}$ is then used in place of $c$ in
Equations 1 and 2.

In addition, we dynamically compute the caption scalar variable
$\lambda\in\mathbb{R}_{+}$ as a function of the standard deviation of the
captions. That is, given some image $I$ and caption $c$, we compute $\lambda$
as

$\displaystyle\lambda=\xi\frac{\sigma(S^{c}_{K})}{\max(||[\sigma(S^{I}_{K}),\sigma(S^{c}_{K})]||_{2},\epsilon)}$
(3)

where $\xi$ is a constant, $\epsilon$ is a small constant, $S^{I}_{K}$ and
$S^{c}_{K}$ are the Top-$K$ probabilities for $S^{I}$ and $S^{c}$ respectively
(i.e. the probabilities whose indices are specified by $\Omega^{I}_{K}$), and
$\sigma$ is the standard deviation operator. Intuitively, the standard
deviation of the Top-$K$ highest probabilities of the image and caption
distributions correspond to the model confidence about their respective
predictions. On one hand, when the model confidence for the image prediction
is high and the caption confidence is low, then the caption probabilities
should not influence the image probabilities as much. On the other hand, when
the image confidence is low but the caption confidence is high, the caption
probabilities should more heavily influence the image probabilities. In the
event when both image and caption confidences are high or low, the default
weighting would be relatively equal. The constant $\xi$ can specify how much
the caption probabilities should affect the image probabilities overall.

###  3.4 Caption Properties

In general, the zero-shot accuracy obtained using only the caption embeddings
is significantly (on average about 5 %, see Table 1) lower than the zero-shot
accuracy obtained using image embeddings, with the notable exception of
aircraft fine-grained classification. Caption-only zero-shot classification is
often unreliable, since the caption does not always correspond to one of the
label choices. For example, a picture containing a teddy bear on top of a bed
might be captioned as “a picture of a teddy bear”, ignoring the bed in the
background. However, if teddy-bear is not one of the labels, and the correct
label is “bed”, the caption does not provide useful information for the
classification problem. For this reason, the optimal hyperparameters for ICE
place a greater emphasis on the prediction from the image embedding
$\widetilde{I}$. Nonetheless, for the caption embedding to contribute to a
higher aggregate accuracy, we only require that the caption-predicted
probabilities $S^{c}_{K}$ be not completely correlated with $S^{I}_{K}$. In
other words, _the caption sometimes contains extra information that nudges the
prediction in the correct direction_. We list here a few intuitions for why
using captions can improve overall classification accuracy:

$1.$ The text decoder cross-attends to all output image tokens from the vision
encoder, while the image prediction only uses the output cls token. The image
token matrix contains spatial information that may be pertinent to the target
task.

$2.$ The text decoder was pretrained to caption images with a language-
modeling loss. Consequently, it exhibits some rudimentary reasoning ability
based on learning relationships between certain concepts. For example, the
text decoder has learned that the painting “the starry night” is authored by
Vincent van Gogh. This correspondence is learnt by the weights of the decoder
and may be useful for some classification tasks. In our experiments, we found
that the caption prediction is much better than the image prediction on
aircraft classification. This is likely because the correspondence between
fine-grained visual concepts and the aircraft model name is learned by the
text decoder.

$3.$ The caption effectively isolates visual concepts; this is an inherent
property of textual data. For example, a caption that reads “a rough red
blanket” effectively isolates the texture, color and content of the image. In
our experiments, we found that captions on the EuroSAT dataset often isolate
the land-use information from the geographical information, e.g. “a photo of
agricultural land in China”. The caption explicitly separates useful
information (agricultural land) from information that is irrelevant to the
classification problem (China). Consequently, a classifier trained on captions
is less likely to learn domain-specific spurious correlations, especially in
the few-shot setting.

We provide concrete examples of these discussed intuitions in our empirical
analysis in Section 4.3.

##  4 Experiments

In our experiments in Table 1, we analyze the impact of combining ICE with
different zero-shot (ZS) baselines across a suite of benchmarks. We show that
our method consistently improves ZS baselines without requiring additional
training. In addition, we analyze several data points to show how ICE improves
over the base method that it is paired with. All implementation details can be
found in Appendix A. To demonstrate the general applicability of our ICE
method, we also present results using three different recent multimodal models
that contain a decoder component: CoCa (Yu et al., 2022), BLIP-2 (Li et al.,
2023), and LLaVA (Liu et al., 2023). These results are included in Table 3 in
the Appendix.

#### Datasets.

In line with prior work, our datasets are split into two categories: _cross-
dataset evaluation_ and _domain generalization_. For cross-dataset
generalization, each evaluation dataset has mostly non-overlapping classes and
unrelated data distributions for zero-shot classification. For domain
generalization, the evaluation datasets are domain-shifted variations of the
ImageNet dataset and share the same classes as ImageNet. We evaluate our
method on 11 cross-dataset generalization datasets covering a wide range of
image recognition tasks. These include two generic objects datasets, ImageNet
(Russakovsky et al., 2014) and Caltech101 (Fei-Fei et al., 2004); five fine-
grained datasets, OxfordPets (Parkhi et al., 2012), StanfordCars (Krause et
al., 2013), Flowers102 (Nilsback & Zisserman, 2008)), Food101 (Bossard et al.,
2014), and FGVCAircraft (Maji et al., 2013); a scene categorization dataset,
SUN397 (Xiao et al., 2010); an action recognition dataset, UCF101 (Soomro et
al., 2012); a describable textures dataset, DTD (Cimpoi et al., 2013), and a
satellite images dataset, EuroSAT (Helber et al., 2017). In addition, we
consider four domain generalization datasets, each applying a different
distribution shift to the source ImageNet dataset. These include an extension
of the ImageNet dataset, ImageNetV2 (Recht et al., 2019); a black and white
sketches dataset, ImageNet-Sketch (Wang et al., 2019), a naturally adversarial
dataset, ImageNet-A (Hendrycks et al., 2019), and a dataset containing
different renditions (e.g. cartoons, graffiti, plush objects, etc.) of the
ImageNet classes, ImageNet-R (Hendrycks et al., 2020).

###  4.1 Zero-Shot Classification

|  | Cross-dataset Evaluation Targets | Domain Generalization Targets
---|---|---|---
|  INet |  Caltech |  Pets |  Cars |  Flowers |  Food |  Aircraft |  SUN |  DTD |  EuroSAT |  UCF |  Average |  INet-V2 |  INet-Sketch |  INet-A |  INet-R |  Average
Zero-shot(Image) | 75.1 | 97.6 | 93.8 | 92.7 | 77.3 | 87.5 | 36.8 | 73.6 | 57.2 | 58.5 | 73.4 | 74.8 | 67.5 | 63.5 | 53.8 | 87.0 | 68.0
Zero-shot (Caption) | 58.8 | 85.6 | 76.3 | 83.1 | 63.6 | 72.7 | 40.7 | 54.8 | 44.0 | 34.9 | 60.3 | 61.6 | 50.2 | 50.7 | 38.7 | 73.8 | 53.3
\+ ICE | 75.6 | 97.1 | 93.8 | 93.0 | 78.0 | 87.7 | 38.3 | 74.0 | 59.3 | 61.3 | 74.3 | 75.7 | 67.8 | 64.0 | 54.4 | 87.5 | 68.4
Manual Prompts | 75.5 | 97.1 | 93.7 | 92.7 | 77.5 | 87.5 | 37.5 | 74.0 | 60.7 | 60.2 | 73.8 | 75.5 | 67.8 | 64.7 | 53.2 | 88.1 | 68.4
\+ ICE | 75.9 | 97.1 | 93.8 | 93.1 | 78.7 | 87.6 | 39.7 | 74.1 | 61.6 | 61.2 | 73.9 | 76.1 | 68.2 | 64.7 | 54.4 | 88.4 | 68.9
GPT Centroids | 74.8 | 97.8 | 93.3 | 92.4 | 75.8 | 87.4 | 36.4 | 73.9 | 58.8 | 63.7 | 73.2 | 75.3 | 67.3 | 63.3 | 52.6 | 86.7 | 67.5
\+ ICE | 75.2 | 97.4 | 93.0 | 92.8 | 76.3 | 87.7 | 39.1 | 74.2 | 60.2 | 64.2 | 73.7 | 75.9 | 67.4 | 63.5 | 53.5 | 87.1 | 67.9
GPT Score Mean | 74.9 | 97.6 | 93.7 | 92.4 | 76.2 | 87.3 | 36.2 | 73.9 | 58.9 | 64.9 | 73.6 | 75.5 | 67.6 | 63.5 | 52.8 | 86.8 | 67.7
\+ ICE | 75.4 | 97.4 | 93.5 | 92.8 | 77.0 | 87.6 | 39.2 | 74.2 | 60.2 | 65.5 | 73.9 | 76.1 | 67.9 | 63.7 | 53.4 | 87.2 | 68.1
Table 1: Comparison with zero-shot baselines on 15 test datasets. We always
observe that stacking our ICE method on top of baseline methods provides
consistent improvements. All methods are zero-shot and use CoCa ViT-L/14. The
caption zero-shot accuracy is reported using one caption embedding prompted by
“a photo of”. ImageNet is abbreviated INet.

#### Baselines.

We consider four existing SOTA methods as zero-shot classification baselines
for ICE: (1) a pre-trained CoCa model (Yu et al., 2022) with class embeddings
generated using the prompt “a photo of a {}”, where “{}”is replaced by the
corresponding class (2) the pre-trained model with class embeddings calculated
from the centroids of 80 intermediate embeddings generated using hand-crafted
prompts from Radford et al. (2021) (3) the pre-trained model using large
language model (LLM) generated descriptors from (Menon & Vondrick, 2022) (4) a
variation of the previous baseline, where we take the centroid of each
descriptor embedding for standard zero-shot classification.

#### Results.

In Table 1, we observe that stacking ICE achieves consistent improvements of
around 0.5% on average across cross-dataset evaluation and domain-
generalization evaluation benchmarks, with improvements of up to 3% on
datasets like FGVCAircraft and EuroSAT. We emphasize that these ZS accuracy
improvements are consistent across all datasets (Table 1) and three different
pretrained model architectures (CoCa, BLIP-2 and LLaVA in Table 3).

###  4.2 Few-Shot Classification

Although our ICE method is a zero-shot method, it can be trivially extended to
the few-shot learning setting by applying it to the few-shot finetuned model.
Accordingly, we consider SOTA methods where a pre-trained model is fine-tuned
on 16-shot ImageNet training data. Specifically, the ImageNet training dataset
contains 1000 classes with 16 images per class, for a total of 16,000 images
in the train dataset. These few-shot results are included in Table 2 of the
Appendix.

#### Baselines.

We consider two prompt learning methods, CoOp (Zhou et al., 2022b) and MaPLe
(Khattak et al., 2023), and a fine-tuning method CLIPood (Shu et al., 2023) as
our ICE baselines. We use each method for few-shot fine-tuning on the CLIP
components of the CoCa (Yu et al., 2022) architecture, and perform standard
zero-shot classification by following the CLIP framework (Radford et al.,
2021).

#### Results.

As seen in Table 2 of the Appendix, applying ICE to each baseline provides
improvements of $0.5\%$ on average across cross-dataset generalization
evaluation datasets, and smaller improvements for the domain generalization
datasets. For the methods such as CLIPood where we do not see improvements in
domain generalization on average, we find that ICE at least maintains the
average performance of the classification backbone.

###  4.3 Understanding Why ICE Provides Improvements

To better understand why ICE improves the base methods paired with it, we
analyze examples in Figure 6 where ICE correctly reclassifies a previously
incorrectly classified datapoint, where ICE preserves a previously correct
classification, where ICE fails to correctly reclassify a previously incorrect
classification, and where ICE accidentally incorrectly reclassifies a
previously correct classification.

Figure 6: A qualitative analysis on various ways ICE can affect the downstream
classification performance.

Figure 7: Top: Ablation results on varying $\xi$. Comparison between fixed
caption weight $\lambda$ and adaptive $\lambda$ using Eq. 3. Adaptive
$\lambda$ is clearly superior on Cars and ImageNet. Middle: Ablation results
on varying caption prompting and number of captions in ensemble ($\upsilon$).
$\upsilon=3$ captions is optimal. Bottom: Ablation results on varying $K$.
$K=4$ is optimal and clearly superior than bypassing Top-$K$ selection step
(denoted as max $K$ in bar plot).

In the first example of Figure 6, we observe that ICE reclassifies the
incorrectly predicted _Appenzeller Sennenhund_ class to the correct _Border
Collie_ class. One intuition for why this occurs is that Border Collies most
commonly have black and white fur, whereas Appenzeller Sennenhund dogs
typically have unique tricolor fur coats. Thus, the caption “a photo of a
black and white dog” would correspond more with the Border Collie purely based
on its bicolor fur. Additional examples of correct ICE reclassifications can
be found in Figure 1.

In the second example, we observe that ICE is able to successfully preserve an
originally correct classification. Here, since the caption agrees with the
image predicted class, ICE is able to predict the same class as before.

In the third image, ICE fails to correctly reclassify an initially incorrect
prediction. In this case, the original prediction was a _tray_ , which would
make sense since this image can technically be described as a “tray of
fruits”. Similarly, the captions describe the image as “a photo of a fruit
platter”, which can correspond more to a _plate_ or _tray_ than a
_strawberry_. Thus, the ICE predicted class still matches that of the original
class.

Finally, the fourth image is an example for how ICE can accidentally
incorrectly reclassify an initially correct prediction. Here, the correct
class is a _magnetic compass_ , as evident by the blue text written on the
instrument. While the caption “a photo of the instrument’s dial” technically
describes the image, without additional visual context, it is easy to believe
that the caption is describing a clock rather than a magnetic compass since
dials tend to be associated with clocks. Due to this ambiguity, ICE was able
to convince the initial predicted class to reclassify to the runner-up class
of _analog clock_. This example highlights the importance of why we need to
consider both the image and caption information for making the most informed
decisions.

##  5 Ablation Studies

A comprehensive ablation study on the parameters of ICE is presented in Figure
7. First, we evaluate the contribution of the adaptive $\lambda$ mechanism
proposed in Eq. 3 on the left of the figure. On ImageNet and cars, the zero-
shot accuracy of the adaptive $\lambda$ is clearly superior to the fixed
$\lambda$ for varying values of $\xi$ (in the adaptive case) and $\lambda$ (in
the fixed case). In the middle bar plot, we examine the contribution of the
Top-$K$ selection procedure outlined in Eq. 2. We compare multiple values of
$K$ (where $K=1$ is equivalent to ignoring caption embeddings, and “max $K$”
denotes score averaging over all classes without Top-$K$ selection). Values of
$K$ around $K=4$ are clearly superior to both ignoring caption embeddings and
score averaging without Top-$K$ selection. Finally, in the bar plot on the
right, we examine the contribution of ensembling multiple captions. Using
$\upsilon=3$ captions is approximately optimal. Note that the 7 individual
captions exhibit high variance in zero-shot accuracy (under the ICE
framework); this variance in performance is greatly reduced by ensembling a
small number of captions.

##  6 Limitations

While our results show consistent improvements when evaluating on several
different baseline methods across a diverse collection of datasets, we note
several limitations of our method. First, as elaborated in Section 3.4, ICE
heavily relies on the assumption that the captions can provide useful
information about the image that is not fully encoded by the image embeddings.
As seen in Section 4.3, when the captions provide unhelpful or adversarial
information and there lacks a good selection of caption scores weight
$\lambda$, ICE could decrease the base classification performance. Second,
determining a good choice for $\lambda$ is non-trivial, as it requires the
selector to have an understanding for when to trust the caption or image
scores more on a per-datapoint basis. This task is comparable with relevant
literature on failure mode prediction using confidence estimation
(Tsiligkaridis, 2020; Hendrycks & Gimpel, 2016; Zhu et al., 2023), which is
known to be challenging. Finally, generating captions can be expensive, since
each additionally generated caption requires an additional forward pass on the
base model. For image-text foundation models like CoCa (Yu et al., 2022),
which typically contain hundreds of millions of parameters, this strategy can
quickly become a time bottleneck for ICE. Thus, it is important to find a
balance between the robustness benefits reaped from the number of captions
used, and the linearly-increasing time costs for each additional caption
generated.

##  7 Conclusion

We explored the use of captioners which is understudied in the vision-language
literature for classification. We proposed a novel method for improved zero-
shot classification performance based on combining image and caption
embeddings. We showed that our method can be easily paired with existing SOTA
methods, and provide improvements of $0.5\%$ on average and up to $3\%$ across
a diverse array of datasets in cross-dataset generalization and domain
generalization. Several ablation studies are presented to study sensitivity of
parameters on performance. We performed an in-depth analysis on why our method
can help reclassify previously misclassified points, and also cover cases
where it might fail. Future work includes extending this work to better
balance the weights between image and caption scores, and considering ways to
generate more informative captions for improved downstream classification.

## Acknowledgments

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is
unlimited.

This material is based upon work supported by the Under Secretary of Defense
for Research and Engineering under Air Force Contract No. FA8702-15-D-0001.
Any opinions, findings, conclusions or recommendations expressed in this
material are those of the author(s) and do not necessarily reflect the views
of the Under Secretary of Defense for Research and Engineering.

## Impact Statement

This paper presents work whose goal is to advance the field of Machine
Learning. There are many potential societal consequences of our work, none
which we feel must be specifically highlighted here.

## References

  * Bossard et al. (2014) Bossard, L., Guillaumin, M., and Van Gool, L.  Food-101 – mining discriminative components with random forests.  In _European Conference on Computer Vision_ , 2014.
  * Bridle (1989) Bridle, J.  Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters.  In Touretzky, D. (ed.), _Advances in Neural Information Processing Systems_ , volume 2. Morgan-Kaufmann, 1989.  URL https://proceedings.neurips.cc/paper_files/paper/1989/file/0336dcbab05b9d5ad24f4333c7658a0e-Paper.pdf.
  * Cimpoi et al. (2013) Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A.  Describing textures in the wild.  _CoRR_ , abs/1311.3618, 2013.  URL http://arxiv.org/abs/1311.3618.
  * Cohen (1997) Cohen, G. H.  Align: a program to superimpose protein coordinates, accounting for insertions and deletions.  _Journal of applied crystallography_ , 30(6):1160–1161, 1997.
  * Fei-Fei et al. (2004) Fei-Fei, L., Fergus, R., and Perona, P.  Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories.  In _2004 Conference on Computer Vision and Pattern Recognition Workshop_ , pp. 178–178, 2004.  doi: 10.1109/CVPR.2004.383.
  * Helber et al. (2017) Helber, P., Bischke, B., Dengel, A., and Borth, D.  Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.  _CoRR_ , abs/1709.00029, 2017.  URL http://arxiv.org/abs/1709.00029.
  * Hendrycks & Gimpel (2016) Hendrycks, D. and Gimpel, K.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  _CoRR_ , abs/1610.02136, 2016.  URL http://arxiv.org/abs/1610.02136.
  * Hendrycks et al. (2019) Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, D.  Natural adversarial examples.  _CoRR_ , abs/1907.07174, 2019.  URL http://arxiv.org/abs/1907.07174.
  * Hendrycks et al. (2020) Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., Song, D., Steinhardt, J., and Gilmer, J.  The many faces of robustness: A critical analysis of out-of-distribution generalization.  _CoRR_ , abs/2006.16241, 2020.  URL https://arxiv.org/abs/2006.16241.
  * Khattak et al. (2023) Khattak, M. U., Rasheed, H., Maaz, M., Khan, S., and Khan, F. S.  Maple: Multi-modal prompt learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 19113–19122, 2023.
  * Krause et al. (2013) Krause, J., Stark, M., Deng, J., and Fei-Fei, L.  3d object representations for fine-grained categorization.  In _2013 IEEE International Conference on Computer Vision Workshops_ , pp. 554–561, 2013.  doi: 10.1109/ICCVW.2013.77.
  * Li et al. (2022) Li, J., Li, D., Xiong, C., and Hoi, S.  Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation.  In _International Conference on Machine Learning_ , pp. 12888–12900. PMLR, 2022.
  * Li et al. (2023) Li, J., Li, D., Savarese, S., and Hoi, S.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  _arXiv preprint arXiv:2301.12597_ , 2023.
  * Liu et al. (2023) Liu, H., Li, C., Wu, Q., and Lee, Y. J.  Visual instruction tuning.  _arXiv preprint arXiv:2304.08485_ , 2023.
  * Maji et al. (2013) Maji, S., Rahtu, E., Kannala, J., Blaschko, M. B., and Vedaldi, A.  Fine-grained visual classification of aircraft.  _CoRR_ , abs/1306.5151, 2013.  URL http://arxiv.org/abs/1306.5151.
  * Menon & Vondrick (2022) Menon, S. and Vondrick, C.  Visual classification via description from large language models, 2022.
  * Nilsback & Zisserman (2008) Nilsback, M.-E. and Zisserman, A.  Automated flower classification over a large number of classes.  In _2008 Sixth Indian Conference on Computer Vision, Graphics and Image Processing_ , pp. 722–729, 2008.  doi: 10.1109/ICVGIP.2008.47.
  * Parkhi et al. (2012) Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. V.  Cats and dogs.  In _IEEE Conference on Computer Vision and Pattern Recognition_ , 2012.
  * Radford et al. (2021) Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I.  Learning transferable visual models from natural language supervision, 2021.
  * Recht et al. (2019) Recht, B., Roelofs, R., Schmidt, L., and Shankar, V.  Do imagenet classifiers generalize to imagenet?  _CoRR_ , abs/1902.10811, 2019.  URL http://arxiv.org/abs/1902.10811.
  * Russakovsky et al. (2014) Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M. S., Berg, A. C., and Fei-Fei, L.  Imagenet large scale visual recognition challenge.  _CoRR_ , abs/1409.0575, 2014.  URL http://arxiv.org/abs/1409.0575.
  * Shu et al. (2023) Shu, Y., Guo, X., Wu, J., Wang, X., Wang, J., and Long, M.  Clipood: Generalizing clip to out-of-distributions, 2023.
  * Soomro et al. (2012) Soomro, K., Zamir, A. R., and Shah, M.  UCF101: A dataset of 101 human actions classes from videos in the wild.  _CoRR_ , abs/1212.0402, 2012.  URL http://arxiv.org/abs/1212.0402.
  * Tsiligkaridis (2020) Tsiligkaridis, T.  Failure prediction by confidence estimation of uncertainty-aware dirichlet networks.  _CoRR_ , abs/2010.09865, 2020.  URL https://arxiv.org/abs/2010.09865.
  * Vogt-Lowell et al. (2023) Vogt-Lowell, K., Lee, N., Tsiligkaridis, T., and Vaillant, M.  Robust fine-tuning of vision-language models for domain generalization.  In _IEEE High Performance Extreme Computing Conference (HPEC)_ , 2023.
  * Wang et al. (2019) Wang, H., Ge, S., Xing, E. P., and Lipton, Z. C.  Learning robust global representations by penalizing local predictive power.  _CoRR_ , abs/1905.13549, 2019.  URL http://arxiv.org/abs/1905.13549.
  * Wortsman et al. (2022) Wortsman, M., Ilharco, G., Kim, J. W., Li, M., Kornblith, S., Roelofs, R., Lopes, R. G., Hajishirzi, H., Farhadi, A., Namkoong, H., et al.  Robust fine-tuning of zero-shot models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 7959–7971, 2022.
  * Xiao et al. (2010) Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A.  Sun database: Large-scale scene recognition from abbey to zoo.  In _2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , pp. 3485–3492, 2010.  doi: 10.1109/CVPR.2010.5539970.
  * Yu et al. (2022) Yu, J., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., and Wu, Y.  Coca: Contrastive captioners are image-text foundation models, 2022.
  * Zhou et al. (2022a) Zhou, K., Yang, J., Loy, C. C., and Liu, Z.  Conditional prompt learning for vision-language models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 16816–16825, 2022a.
  * Zhou et al. (2022b) Zhou, K., Yang, J., Loy, C. C., and Liu, Z.  Learning to prompt for vision-language models.  _International Journal of Computer Vision_ , 130(9):2337–2348, jul 2022b.  doi: 10.1007/s11263-022-01653-1.  URL https://doi.org/10.1007%2Fs11263-022-01653-1.
  * Zhu et al. (2023) Zhu, F., Cheng, Z., Zhang, X.-Y., and Liu, C.-L.  Rethinking confidence calibration for failure prediction, 2023.

##  Appendix A Experiment Implementation Details

#### ICE.

For each method ICE is paired with, we simply take the image probability
distribution output by our baseline method, and apply ICE with the computed
caption scores. The hyperparameters we use to implement ICE in all experiments
are $K=5$, $\xi=0.08$, $\epsilon=1\times 10^{-12}$, $\upsilon=3$, and
$P=\\{\text{``a''},\text{``a photo of''},\text{``a photo containing''}\\}$. We
find the best empirical performance when using the centroid of the embeddings
of $3$ differently prompted captions and dynamically computing the caption
scores weight $\lambda$ using Equation 3.

#### Baselines.

We make a good-faith attempt to tune the hyperparameters of each few-shot
baseline. We use batch size 64 and SGD with momentum. Training data is sampled
in a round robin fashion to maximize class diversity within each mini-batch.
CLIPood trains the vision encoder for 750 iterations at learning rate $1\times
10^{-5}$ with adaptive margin value of 0.1. CoOp trains 3 prompt tokens
initialized with “a photo of” for 1250 iterations at learning rate $2\times
10^{-4}$ with cross-entropy loss. MaPLe trains 3 prompt tokens prepended to
each of the first 3 layers on both encoders for 750 iterations at learning
rate $1$ with cross-entropy loss.

| Source | Cross-dataset Evaluation Targets | Domain Generalization Targets
---|---|---|---
|  INet |  Caltech |  Pets |  Cars |  Flowers |  Food |  Aircraft |  SUN |  DTD |  EuroSAT |  UCF |  Average |  INet-V2 |  INet-Sketch |  INet-A |  INet-R |  Average
CLIPood | 76.6 | 97.2 | 94.3 | 92.7 | 77.5 | 87.3 | 37.4 | 74.3 | 60.3 | 59.5 | 75.2 | 75.6 | 69.5 | 64.9 | 56.6 | 88.7 | 69.9
\+ ICE | 76.7 | 97.2 | 93.9 | 93.1 | 77.8 | 87.4 | 39.6 | 74.3 | 60.8 | 61.8 | 75.7 | 76.2 | 69.2 | 64.9 | 56.7 | 88.8 | 69.9
CoOp | 76.4 | 97.4 | 93.9 | 93.2 | 77.0 | 87.6 | 39.0 | 73.4 | 59.2 | 61.2 | 74.7 | 75.7 | 69.0 | 63.3 | 55.2 | 87.7 | 68.8
\+ ICE | 76.7 | 97.2 | 93.8 | 93.5 | 77.7 | 87.6 | 40.7 | 73.6 | 60.6 | 62.0 | 75.1 | 76.2 | 69.0 | 63.4 | 55.9 | 88.1 | 69.1
MaPLe | 77.3 | 96.7 | 94.2 | 92.8 | 76.9 | 87.2 | 39.5 | 73.9 | 61.1 | 58.7 | 76.0 | 75.7 | 70.2 | 64.7 | 54.8 | 88.2 | 69.5
\+ ICE | 77.5 | 96.6 | 94.1 | 93.0 | 77.1 | 87.2 | 41.4 | 74.3 | 61.3 | 59.6 | 76.8 | 76.1 | 70.1 | 64.6 | 55.1 | 88.6 | 69.6
Table 2: Comparison with _few-shot baselines_ in the _cross-dataset evaluation_ setting and the _domain generalization_ setting. The model is fine-tuned on ImageNet with three different methods and tested on 15 total target datasets. The average accuracies are calculated separately for the two settings following prior work. In all cases, we observe that evaluating with our ICE method provides consistent improvements. All methods use CoCa ViT-L/14. |  | Cross-dataset Evaluation Targets | Domain Generalization Targets
---|---|---|---
|  INet |  Caltech |  Pets |  Cars |  Flowers |  Food |  Aircraft |  SUN |  DTD |  EuroSAT |  UCF |  Average |  INet-V2 |  INet-Sketch |  INet-A |  INet-R |  Average
CoCa Zero-shot (Image) | 75.1 | 97.6 | 93.8 | 92.7 | 77.3 | 87.5 | 36.8 | 73.6 | 57.2 | 58.5 | 73.4 | 74.8 | 67.5 | 63.5 | 53.8 | 87.0 | 68.0
CoCa Zero-shot (Caption) | 58.8 | 85.6 | 76.3 | 83.1 | 63.6 | 72.7 | 40.7 | 54.8 | 44.0 | 34.9 | 60.3 | 61.6 | 50.2 | 50.7 | 38.7 | 73.8 | 53.3
\+ ICE | 75.6 | 97.1 | 93.8 | 93.0 | 78.0 | 87.7 | 38.3 | 74.0 | 59.3 | 61.3 | 74.3 | 75.7 | 67.8 | 64.0 | 54.4 | 87.5 | 68.4
BLIP-2 Zero-shot (Image) | 73.8 | 94.6 | 93.6 | 76.9 | 79.4 | 90.9 | 32.8 | 68.0 | 52.7 | 56.2 | 74.7 | 72.0 | 68.0 | 57.9 | 68.3 | 85.5 | 69.9
BLIP-2 Zero-shot (Caption) | 44.8 | 87.3 | 29.8 | 51.3 | 49.9 | 56.2 | 7.5 | 50.0 | 46.3 | 39.4 | 61.9 | 48.0 | 42.2 | 47.0 | 42.5 | 75.6 | 51.8
\+ ICE | 74.8 | 95.5 | 93.1 | 78.6 | 79.4 | 90.4 | 32.6 | 70.3 | 54.7 | 56.3 | 75.4 | 72.6 | 68.8 | 60.0 | 69.6 | 88.0 | 71.6
LLaVA Zero-shot (Image) | 73.8 | 94.6 | 93.6 | 76.9 | 79.4 | 90.9 | 32.8 | 68.0 | 52.7 | 56.2 | 74.7 | 72.0 | 68.0 | 57.9 | 68.3 | 85.5 | 69.9
LLaVA Zero-shot (Caption) | 42.4 | 85.4 | 19.8 | 17.8 | 16.8 | 43.8 | 4.9 | 49.0 | 41.5 | 52.7 | 56.8 | 38.8 | 37.5 | 38.2 | 38.7 | 66.8 | 45.3
\+ ICE | 74.1 | 95.1 | 93.0 | 76.0 | 78.2 | 90.6 | 32.3 | 69.0 | 53.1 | 57.4 | 74.6 | 71.9 | 68.3 | 58.4 | 69.0 | 86.5 | 70.5
Table 3: Experiments using a variety of multimodal captioners, showing that
using ICE to combine the image and caption embeddings improves the overall
zero-shot accuracy. CoCa ViT-L/14 uses its own pretrained image encoder, while
BLIP-2 and LLaVA use the frozen CLIP ViT-L/14 model from (Radford et al.,
2021).

#### Caption prompts.

For all ICE results, we use a set of 3 prompt templates to generate 3 diverse
captions per image. These captions are stored in our GitHub repository under
the “captions” folder for easy reference. Caption prompts are model specific
due to the difference in pretraining.

CoCa caption prompts are dataset-agnostic: $\\{\text{``a''},\text{``a photo
of''},\text{``a photo containing''}\\}$. BLIP-2 and LLaVA are optimized for
VQA, so we found that the best classification results are achieved by
prompting with a question and answer format, using dataset-specific questions.
These prompts are listed in Tables 4 and 5

Dataset | LLaVA Caption prompt
---|---
ImageNet | “What is in this photo?”
Caltech | “What is in this photo?”
Pets | “What type of pet is in this photo?”
Cars | “What type of car is in this photo?”
Flowers | “What type of flower is in this photo?”
Food | “What type of food is in this photo”
Aircraft | “What type of aircraft is in this photo?”
SUN | “What is in this photo?”
DTD | “Describe the texture in this photo.”
EuroSAT | “What type of land use is in this satellite photo?”
UCF | “What type of action is in this photo?”
ImageNet | “Question: What is in this photo? Answer: A photo of ”
ImageNet-V2 | “What is in this photo?”
ImageNet-Sketch | “What is in this photo?”
ImageNet-A | “What is in this photo?”
ImageNet-R | “What is in this photo?”
Table 4: Prompts used to generate the first LLaVA caption. The other two captions are generated by using the phrases “be specific” or “be concise” in the prompt to elicit a diverse set of responses from the VL model. For example, the two other ImageNet prompts are: “What is in this photo? Be specific.” and “What is in this photo? Be concise.” Dataset | BLIP-2 Caption prompt
---|---
ImageNet | “Question: What is in this photo? Answer: A photo of ”
Caltech | “Question: What is in this photo? Answer: A photo of ”
Pets | “Question: What type of pet is in this photo? Answer: A photo of ”
Cars | “Question: What type of car is in this photo? Answer: A photo of ”
Flowers | “Question: What type of flower is in this photo? Answer: A photo of ”
Food | “Question: What type of food is in this photo? Answer: A photo of ”
Aircraft | “Question: What type of aircraft is in this photo? Answer: A photo of ”
SUN | “Question: What is in this photo? Answer: A photo of ”
DTD | “Question: Describe the texture in this photo. Answer: A photo of ”
EuroSAT | “Question: What type of land use is in this satellite photo? Answer: A photo of ”
UCF | “Question: What type of action is in this photo? Answer: A photo of ”
ImageNet | “Question: What is in this photo? Answer: A photo of ”
ImageNet-V2 | “Question: What is in this photo? Answer: A photo of ”
ImageNet-Sketch | “Question: What is in this photo? Answer: A photo of ”
ImageNet-A | “Question: What is in this photo? Answer: A photo of ”
ImageNet-R | “Question: What is in this photo? Answer: A photo of ”
Table 5: Prompts used to generate the first BLIP-2 caption. The other two
captions are generated by using the phrases “be specific” or “be concise” in
the prompt to elicit a diverse set of responses from the VL model. For
example, the two other ImageNet prompts are: “Question: What is in this photo?
Be specific. Answer: A photo of ” and “Question: What is in this photo? Be
concise. Answer: A photo of ”


11institutetext: University of Rijeka, Croatia.
11email<EMAIL_ADDRESS>
# Linearization for difference equations with infinite delay

Lokesh Singh 11

###### Abstract

In this article, we construct a conjugacy map for a linear difference equation
with infinite delay and corresponding nonlinear perturbation. We also prove
that the conjugacy map is one one with some additional conditions. As an
application of our result, we show that the cases of (uniform) exponential
dichotomy follow from our result.

###### keywords:

Delay Difference equations, Hartman-Grobman theorem, Linearization.

##  1 Introduction

The classical Hartman and Grobman theorem [5, 6] is a fundamental result in
the local theory of differential equations and dynamical systems. This
celebrated theorem provides a topological conjugacy (near a hyperbolic
equilibrium point $x^{*}=0$) between the dynamics of a nonlinear differential
equation $x^{\prime}=Ax+f(x)$ (on a finite-dimensonal space) and the dynamics
corresponding to the linear equation $x^{\prime}=Ax$. Later, this result was
generalized by Pugh [3] and Palis [4] to the Banach space setting. Some other
improvements of the theorem are due to Reinfelds [7], and Bates and Lu [8]. It
is well known that in general, the conjugacy in the Grobman-Hartman theorem is
only locally Hölder continuous. On the other hand, there has been a
significant amount of work concerned with formlating sufficient conditions
under which the conjugacy exhibits higher regularity properties. Some of the
early work in this direction is by Sternberg [11] and Belitskii [9, 10], while
Rodrigues et. al. [13], ElBialy [12] and Zhang and Zhang along with Lu [29,
30, 31] contributed recently.

Palmer [14] proved the first version of the Grobman-Hartman theorem for
nonautonomous differential equations by assuming that the linear part admits
an exponential dichotomy (see Subsection (3.1) for definition). Later Shi and
Xiong [17] obtained an improvement on Palmer’s result and established Hölder
continuity of conjugacies. The version of Palmer’s theorem in the case of
discrete time was first established by Aulbach and Wanner [15]. Castañeda and
Robledo [28] discussed the regularity of conjugacy map for nonautonomous
differential equations. More recently, Barreira and Valls [16] dealth with the
case when the linear part admits a nonuniform exponential dichotomy. More
recently, Dragičević, Zhang and Zhang [18, 19] discussed the higher regularity
of conjugacies in the nonautonomous setting. Some other important
contributions to nonautonomous linearization are given in [20, 21, 27].

For delay differential equations, due to several complexities, there has not
been much progress. As Sternberg mentioned in [26], in the case of delay
differential equations, the solutions form semiflows (instead of flows) and
sometimes solutions may not exist in backward time. Later, he obtained
Hartman-Grobman theorem [25] for finite delay differential equations in the
finite-dimensional setting under some restrictive conditions. Namely, he
assumed the existence of a compact global attractor on which the semiflow is
one-to -one. Benkhalti and Ezzinbi [24] obtained some improvement over
Sternberg’s result. Farkas [22], proved the Hartman-Grobman theorem for
autonomous delay differential equations admitting uniform exponential
dichotomy in finite-dimensional settings. In recent work, Barreira and Valls
[23] extended the result in continuous case of nonautonomous differential
equations with finite delay in Banach space setting. They also assumed that
the linear part admits a uniform exponential dichotomy.

In this work, our objective is to obtain a conjugacy map for nonautonomous
discrete equations with infinite delay and corresponding nonlinear delay
equations given in (2.1) and (2.6) respectively. To obtain the conjugacy map,
we assume a sufficient condition in terms of a Green type function (given in
(2.4)) associated with linear delay equation. With some additional
assumptions, we also proved that the obtained conjugacy map is one-one. As an
applications of our result, we showed that if the linear delay equation admits
(uniform) exponential dichotomy, then the corresponding Green type function
satisfies the assumptions of our Theorem 3.1 and the result follows. With
respect to existing work, our work has novelty in two senses, we are
considering infinite delay difference equation and we proved the result with
more general conditions.

This article is organized as follows. We describe our setup in Section 2. In
Section 3, we prove our main result and give an application of Theorem 3.1.

##  2 Preliminaries

Let $\mathbb{Z},\mathbb{Z}^{+}$ and $\mathbb{Z}^{-}$ denote the set of all
integers, set of nonnegative integers and set of nonpositive integers,
respectively. Let $(X,|\cdot|)$ be an arbitrary Banach space. Given a sequence
$x:\mathbb{Z}\to X$ and $m\in\mathbb{Z}$, we define $x_{m}:\mathbb{Z}^{-}\to
X$ by

$\displaystyle x_{m}(j)=x(m+j)\quad\mbox{for all }j\in\mathbb{Z}^{-}.$

Next we consider a Banach space $(\mathcal{B},\|\cdot\|_{\mathcal{B}})$ of all
sequences $\phi:\mathbb{Z}^{-}\to X$ satisfying the following assumption:

  1. (A)

There exists $J>0$, and $K,M:\mathbb{Z}^{+}\to[0,\infty)$, such that if
$x:\mathbb{Z}\to X$ with $x_{0}\in\mathcal{B}$, then for all
$n\in\mathbb{Z}^{+}$,

$\displaystyle x_{n}\in\mathcal{B}\quad\mbox{and}\quad
J|x(n)|\leq\|x_{n}\|_{\mathcal{B}}\leq K(n)\sup_{0\leq j\leq
n}|x(j)|+M(n)\|x_{0}\|_{\mathcal{B}}.$

As noted in [1], one such space satisfying the assumption (A) is Banach space
$(\mathcal{B}^{\beta},\|\cdot\|_{\mathcal{B}^{\beta}})$, defined by

$\displaystyle\mathcal{B}^{\beta}=\Big{\\{}\phi:\mathbb{Z}^{-}\to
X\Big{|}\|\phi\|_{\mathcal{B}^{\beta}}<\infty\Big{\\}},\quad\|\phi\|_{\mathcal{B}^{\beta}}:=\sup_{j\in\mathbb{Z}^{-}}[\,|\phi(j)|e^{\beta
j}],$

where $\beta$ is a real constant. Observe that the Banach Space
$(\mathcal{B}^{\beta},\|\cdot\|_{\mathcal{B}^{\beta}})$ satisfies (A) with
$J=1,M(n)=e^{-\beta n}$ and $K(n)=1$ if $\beta\geq 0$ and $K(n)=e^{-\beta n}$
if $\beta<0.$

Given a sequence of linear operators $A_{m}:\mathcal{B}\to
X,\,m\in\mathbb{Z}^{+}$, we consider a linear difference equation given by,

$\displaystyle x(m+1)=A_{m}x_{m},\qquad\mbox{for all }m\in\mathbb{Z}^{+}.$
(2.1)

Given $(n,\phi)\in\mathbb{Z}^{+}\times\mathcal{B}$, there exists a unique
sequence $x:\mathbb{Z}\to X$ such that $x_{n}=\phi$ and the sequence $x(m)$
satisfies equation (2.1) for all $m\geq n\geq 0$. The sequence $x$ is called a
solution of equation (2.1) through $(n,\phi)$ and is denoted by
$x=x(\cdot;n,\phi)$. Now we define a two parameter solution operator
$\mathcal{A}(m,n):\mathcal{B}\to\mathcal{B}$ of the linear equation (2.1) by,

$\displaystyle\mathcal{A}(m,n)\phi=x_{m}(\cdot;n,\phi)\qquad\mbox{ for all
}m\geq n\geq 0,\mbox{ and }\phi\in\mathcal{B}.$ (2.2)

Clearly, $(\mathcal{A}(m,n))_{m\geq n\geq 0}$ is a discrete evolution family
corresponding to linear equation (2.1) and it satisfies

$\displaystyle\mathcal{A}(n,n)$
$\displaystyle=Id_{\mathcal{B}}\qquad\quad\mbox{for all }n\geq 0,$
$\displaystyle\mathcal{A}(m,k)\mathcal{A}(k,n)$
$\displaystyle=\mathcal{A}(m,n)\quad\mbox{ for all }m\geq k\geq n\geq 0.$

Here $Id_{\mathcal{B}}$ denotes the identity operator on $\mathcal{B}$. Next
we consider a sequence of projection maps $(P_{n})_{n\in\mathbb{Z}^{+}}$ on
$\mathcal{B}$ such that

$\displaystyle P_{m}\mathcal{A}(m,n)=\mathcal{A}(m,n)P_{n},\quad\mbox{ for all
}m\geq n\geq 0,$ (2.3)

and

$\displaystyle\mathcal{A}(m,n)\big{|}_{\operatorname{Ker}P_{n}}:\operatorname{Ker}P_{n}\to\operatorname{Ker}P_{m}\quad\mbox{
is invertible for }m\geq n\geq 0.$

Also, for $n\geq m\geq 0$, we denote
$\mathcal{A}(m,n):=\big{(}\mathcal{A}(n,m)\big{|}_{\operatorname{Ker}P_{m}}\big{)}^{-1}:\operatorname{Ker}P_{n}\to\operatorname{Ker}P_{m}$
and $Q_{m}:=Id_{\mathcal{B}}-P_{m}$ for each $m\in\mathbb{Z}^{+}$. Now, for
$m,n\in\mathbb{Z}^{+}$, we define another operator

$\mathcal{G}(m,n):=\begin{cases}\mathcal{A}(m,n)P_{n}&\text{for $m\geq n$;}\\\
-\mathcal{A}(m,n)Q_{n}&\text{for $m<n$.}\end{cases}$ (2.4)

This operator is usually called Green operator. With respect to projection
maps $(P_{n})_{n\in\mathbb{Z}^{+}}$, we have $\mathcal{B}=E_{n}\oplus F_{n}$
for each $n\in\mathbb{Z}^{+}$. Here $E_{n}$ and $F_{n}$ are ranges of
projections $P_{n}$ and $Q_{n}$ respectively. Now, let us consider a space
$\mathcal{M}$ consisting of continuous functions,

$\displaystyle\eta:\\{(n,\phi):n\in\mathbb{Z}^{+},\phi\in
F_{n}\\}\to\mathcal{B}$

such that

$\displaystyle\|\eta\|_{\infty}:=\sup\\{\|\eta(n,\phi)\|_{\mathcal{B}}:n\in\mathbb{Z}^{+}\mbox{
and }\phi\in F_{n}\\}<\infty.$

Clearly, $(\mathcal{M},\|\cdot\|_{\infty})$ is a Banach space. We also write

$\displaystyle\eta^{n}=\eta(n,\cdot)\quad\mbox{and }\quad
h^{n}=Id_{F_{n}}+\eta^{n}.$

Here $Id_{F_{n}}$ is identity map on subspace $F_{n}$. Furthermore, for each
$m\in\mathbb{Z}^{+}$, let $f_{m}\colon\mathcal{B}\to X$, be a sequence of maps
such that $f_{m}(0)=0$ for all $m\in\mathbb{Z}^{+}$ and there exist numbers
$c_{m}>0$ satisfying

$|f_{m}(\phi)-f_{m}(\psi)|\leq c_{m}\min{(1,\|\phi-\psi\|_{\mathcal{B}})},$
(2.5)

for every $m\in\mathbb{Z}^{+}$ and $\phi,\psi\in\mathcal{B}$. Finally, we
consider a _semilinear difference equation_ given by

$x(m+1)=A_{m}x_{m}+f_{m}(x_{m})\qquad\text{ for all }m\in\mathbb{Z}^{+}.$
(2.6)

Given $(n,\phi)\in\mathbb{Z}^{+}\times\mathcal{B}$, there exists a unique
sequence $x:\mathbb{Z}\to X$ such that $x_{n}=\phi$ and $x$ satisfies the
semilinear difference equation (2.6). We write the solution of equation (2.6)
in terms of operator $\mathcal{R}(m,n)$ given by,

$\displaystyle x_{m}=\mathcal{R}(m,n)x_{n}\qquad\mbox{ for all }m\geq n\geq
0.$ (2.7)

Now we recall [1] and [2] to give the variation of constants formula for a
difference equation given by

$\displaystyle x(m+1)=A_{m}x_{m}+p_{m},$ (2.8)

where $(p_{m})_{m\in\mathbb{Z}^{+}}$ is a sequence in $X$. Define
$\Gamma:\mathbb{Z}^{-}\to\mathcal{L}(X)$ by

$\displaystyle\Gamma(j)=\begin{cases}Id_{X}&\text{for $j=0$;}\\\
0_{X}&\text{for $j<0$.}\end{cases}$

The symbol $\mathcal{L}(X)$ denotes the space of bounded linear operators in
X. For $v\in X$, define $\Gamma v:\mathbb{Z}^{-}\to X$ by

$\displaystyle(\Gamma v)(j):=\Gamma(j)v=\begin{cases}v&\text{for $j=0$;}\\\
0&\text{for $j<0$.}\end{cases}$

If $x:\mathbb{Z}\to X$ is defined by $x(j)=0$ for $j\leq 0$ and $x(j)=v$ for
$j>0$, then $x_{0}=0$ and $x_{1}=\Gamma v$. Since $x_{0}=0\in\mathcal{B}$, by
assumption (A), we have that $\Gamma v=x_{1}\in\mathcal{B}$ and

$\displaystyle\|\Gamma v\|_{\mathcal{B}}\leq K(1)|v|.$ (2.9)

######  Theorem 2.1.

[1] Let $\phi\in\mathcal{B}$. A sequence $x:\mathbb{Z}\to X$ is a solution of
$(\ref{diff. equ})$ with initial value $x_{0}=\phi$ if and only if for $m\geq
0$, the segment $x_{m}$ satisfy the following relation in $\mathcal{B}$,

$\displaystyle
x_{m}=\mathcal{A}(m,0)\phi+\sum_{k=0}^{m-1}\mathcal{A}(m,k+1)(\Gamma
p_{k}),\quad m\geq 0.$ (2.10)

As a corollary of above result, we have that $x:\mathbb{Z}\to X$ is a solution
of (2.6) if and only if $\phi=x_{0}\in\mathcal{B}$ and

$\displaystyle
x_{m}=\mathcal{A}(m,0)\phi+\sum_{k=0}^{m-1}\mathcal{A}(m,k+1)(\Gamma
f_{k}(x_{k})),\quad m\geq 0.$

Let $x=x(\cdot;0,\phi)$ be a solution of equation (2.6) for
$\phi\in\mathcal{B}$, then using constants of variation formula from Theorem
2.1, for $m\geq n\geq 0$, we have

$\displaystyle x_{m}$
$\displaystyle=\mathcal{A}(m,0)\phi+\sum_{k=0}^{m-1}\mathcal{A}(m,k+1)(\Gamma
f_{k}(x_{k}))$
$\displaystyle=\mathcal{A}(m,0)\phi+\sum_{k=0}^{n-1}\mathcal{A}(m,k+1)(\Gamma
f_{k}(x_{k}))+\sum_{k=n}^{m-1}\mathcal{A}(m,k+1)(\Gamma f_{k}(x_{k}))$
$\displaystyle=\mathcal{A}(m,n)\big{\\{}\mathcal{A}(n,0)\phi+\sum_{k=0}^{n-1}\mathcal{A}(n,k+1)(\Gamma
f_{k}(x_{k}))\big{\\}}$ $\displaystyle\hskip
170.71652pt+\sum_{k=n}^{m-1}\mathcal{A}(m,k+1)(\Gamma f_{k}(x_{k}))$
$\displaystyle=\mathcal{A}(m,n)x_{n}+\sum_{k=n}^{m-1}\mathcal{A}(m,k+1)(\Gamma
f_{k}(x_{k})),$

for all $m\geq n\geq 0$. Therefore, we obtain that $x:\mathbb{Z}\to X$ is a
solution of (2.6) if and only if $x_{0}\in\mathcal{B}$ and for $m\geq n\geq 0$

$\displaystyle
x_{m}=\mathcal{A}(m,n)x_{n}+\sum_{k=n}^{m-1}\mathcal{A}(m,k+1)(\Gamma
f_{k}(x_{k})).$ (2.11)

##  3 Main Result

The following theorem is our main result.

######  Theorem 3.1.

Assume that the semilinear equation $(\ref{dis semilin eq})$ admits

$q:=\sup_{m\in\mathbb{Z}^{+}}\bigg{(}\sum_{n\in\mathbb{Z}^{+}}c_{n}\|\mathcal{G}(m,n+1)\|\bigg{)},\quad\mbox{
with }\,K(1)q<1.$ (3.1)

Then there exists a function $\eta\in\mathcal{M}$, such that for every $m\geq
n\geq 0$

$\displaystyle h^{m}\circ\mathcal{A}(m,n)=\mathcal{R}(m,n)\circ
h^{n},\quad\mbox{ on }F_{n}.$ (3.2)

Moreover, each map $h^{n}$ is one-to-one provided that the condition $(\ref{cc
disc})$ holds with a constant sequence of Lipschitz constants
$(c_{n})_{n\in\mathbb{Z}^{+}}$.

First we prove a lemma which will be used to establish that $h^{n}$ is one-one
for $n\in\mathbb{Z}^{+}$. Set

$\displaystyle a_{m,n}:=\|\mathcal{G}(m,n)\|.$ (3.3)

######  Lemma 3.2.

Assume that the sequence of Lipschitz constants $(c_{n})_{n\in\mathbb{Z}^{+}}$
in $(\ref{lipn disc})$ is constant and that $(\ref{cc disc})$ holds. Then, for
each fixed $m\in\mathbb{Z}^{+}$, we have that

$\displaystyle\lim_{n\to\infty}a_{m,n}=0.$

######  Proof 3.3.

Let $C>0$ be such that $c_{n}=C$ for all $n\in\mathbb{Z}^{+}$. By (3.1), it
follows that

$\displaystyle\sup_{m\in\mathbb{Z}^{+}}\bigg{(}\sum_{n\in\mathbb{Z}^{+}}c_{n}\|\mathcal{G}(m,n+1)\|\bigg{)}=C\sup_{m\in\mathbb{Z}^{+}}\Big{(}\sum_{n\in\mathbb{Z}^{+}}a_{m,n+1}\Big{)}<\infty.$

Therefore, for each fixed $m\in\mathbb{Z}^{+}$,

$\displaystyle\sum_{n\in\mathbb{Z}^{+}}a_{m,n+1}<\infty.$

Hence, $\lim_{n\to\infty}a_{m,n+1}=0$ for each fixed $m\in\mathbb{Z}^{+}$.
Equivalently,

$\displaystyle\lim_{n\to\infty}a_{m,n}=0\qquad\mbox{ for each fixed
}m\in\mathbb{Z}^{+}.$ (3.4)

Now we give our proof of Theorem 3.1.

######  Proof 3.4.

Let us consider a map $F:\mathcal{M}\to\mathcal{M}$ given by

$\displaystyle
F(\eta)(n,\phi)=\sum_{m\in\mathbb{Z}^{+}}\mathcal{G}(n,m+1)\Gamma
f_{m}\big{(}\mathcal{A}(m,n)\phi+\eta^{m}(\mathcal{A}(m,n)\phi)\big{)},$ (3.5)

where $\eta\in\mathcal{M}$ and $(n,\phi)\in\mathbb{Z}^{+}\times F_{n}$.
Furthermore, since $f_{m}(0)=0$ for each $m\in\mathbb{Z}^{+}$ and using (2.5),
we have,

$\displaystyle|f_{m}(\phi)|\leq c_{m}\min{(1,\|\phi\|_{\mathcal{B}})}\leq
c_{m},\quad\mbox{ for all }\phi\in\mathcal{B}.$

Therefore, using above estimate,

$\displaystyle\|F(\eta)\|_{\infty}\leq\sup_{n\in\mathbb{Z}^{+}}\Big{(}K(1)\sum_{m\in\mathbb{Z}^{+}}c_{m}\|\mathcal{G}(n,m+1)\|\Big{)}=K(1)q<\infty.$

Therefore, the operator $F$ is well defined. We now claim that $F$ is a
contraction map. For each $\eta,\xi\in\mathcal{M}$ we have

$\displaystyle|F(\eta)(n,\phi)-F(\xi)(n,\phi)|$ $\displaystyle\leq
K(1)\sum_{m\in\mathbb{Z}^{+}}\|\mathcal{G}(n,m+1)\||f_{m}\big{(}\mathcal{A}(m,n)\phi+\eta^{m}(\mathcal{A}(m,n)\phi)\big{)}-$
$\displaystyle\hskip
142.26378ptf_{m}\big{(}\mathcal{A}(m,n)\phi+\xi^{m}(\mathcal{A}(m,n)\phi)\big{)}|$
$\displaystyle\leq
K(1)\sum_{m\in\mathbb{Z}^{+}}\|\mathcal{G}(n,m+1)\|c_{m}|\eta^{m}(\mathcal{A}(m,n)\phi)-\xi^{m}(\mathcal{A}(m,n)\phi)|$
$\displaystyle\leq
K(1)\sum_{m\in\mathbb{Z}^{+}}c_{m}\|\mathcal{G}(n,m+1)\|\|\eta-\xi\|_{\infty}.$

Therefore,

$\displaystyle\|F(\eta)-F(\xi)\|_{\infty}\leq K(1)q\,\|\eta-\xi\|_{\infty}.$

As $K(1)q<1$, the operator $F$ is a contraction map. Therefore it has a unique
fixed point function, say $\eta$ i.e. $F(\eta)=\eta$. Now, for $\phi\in
F_{n}$, we have

$\displaystyle\eta^{n}(\phi)$
$\displaystyle=\sum_{m\in\mathbb{Z}^{+}}\mathcal{G}(n,m+1)\Gamma
f_{m}\big{(}\mathcal{A}(m,n)\phi+\eta^{m}(\mathcal{A}(m,n)\phi)\big{)}$
$\displaystyle=\sum_{m\in\mathbb{Z}^{+}}\mathcal{G}(n,m+1)\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$ (3.6)

Note that,

$\displaystyle Q_{n}\eta^{n}(\phi)=-\sum_{m\geq
n}\mathcal{A}(n,m+1)Q_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$ (3.7)

Take $0\leq p\leq n$. By (3.7), for $F_{p}\ni\psi=\mathcal{A}(p,n)\phi$ we
have that

$\displaystyle Q_{p}\eta^{p}(\psi)=-\sum_{m\geq
p}\mathcal{A}(p,m+1)Q_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,p)\psi)\big{)}.$ (3.8)

Also from (3.7),

$\displaystyle\mathcal{A}(p,n)Q_{n}\eta^{n}(\phi)=-\sum_{m\geq
n}\mathcal{A}(p,m+1)Q_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$

Using (3.8) and the above relation, we obtain that

$\displaystyle\mathcal{A}(p,n)Q_{n}\eta^{n}(\phi)=Q_{p}\eta^{p}$
$\displaystyle(\mathcal{A}(p,n)\phi)$
$\displaystyle+\sum_{m=p}^{n-1}\mathcal{A}(p,m+1)Q_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$

Equivalently,

$\displaystyle Q_{n}\eta^{n}(\phi)=\mathcal{A}(n,p)Q_{p}\eta^{p}$
$\displaystyle(\mathcal{A}(p,n)\phi)$
$\displaystyle+\sum_{m=p}^{n-1}\mathcal{A}(n,m+1)Q_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$ (3.9)

Similarly, from (3.4), it follows that

$\displaystyle P_{n}\eta^{n}(\phi)=\sum_{m<n}\mathcal{A}(n,m+1)P_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$ (3.10)

For $F_{p}\ni\psi=\mathcal{A}(p,n)\phi$, we have that

$\displaystyle P_{p}\eta^{p}(\psi)=\sum_{m<p}\mathcal{A}(p,m+1)P_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,p)\psi)\big{)}.$ (3.11)

Hence,

$\displaystyle P_{n}\eta^{n}(\phi)=\mathcal{A}(n,p)P_{p}$
$\displaystyle\eta^{p}(\mathcal{A}(p,n)\phi)$
$\displaystyle+\sum_{m=p}^{n-1}\mathcal{A}(n,m+1)P_{m+1}\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$ (3.12)

By adding (3.4) and (3.4), we obtain that

$\displaystyle\eta^{n}(\phi)=\mathcal{A}(n,p)\eta^{p}(\mathcal{A}(p,n)\phi)+\sum_{m=p}^{n-1}\mathcal{A}(n,m+1)\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$

Since $h^{n}=Id_{F_{n}}+\eta^{n}$, we conclude that

$\displaystyle h^{n}(\phi)=\mathcal{A}(n,p)\big{(}h^{p}($
$\displaystyle\mathcal{A}(p,n)\phi)\big{)}+\sum_{m=p}^{n-1}\mathcal{A}(n,m+1)\Gamma
f_{m}\big{(}h^{m}(\mathcal{A}(m,n)\phi)\big{)}.$

Moreover,

$\displaystyle
h^{n}(\mathcal{A}(n,p)\psi)=\mathcal{A}(n,p)h^{p}(\psi)+\sum_{m=p}^{n-1}\mathcal{A}(n,m+1)\Gamma
f_{m}(h^{m}(\mathcal{A}(m,p)\psi))$ (3.13)

On the other hand, by the variation of constants formula in (2.11) and the
definition of $\mathcal{R}(m,n)$ in (2.7), for all $n\geq p\geq 0$, we have,

$\displaystyle\mathcal{R}(n,p)h^{p}(\psi)=\mathcal{A}(n,p)h^{p}(\psi)+\sum_{m=p}^{n-1}\mathcal{A}(n,m+1)\Gamma
f_{m}\big{(}\mathcal{R}(m,p)h^{p}(\psi)\big{)}.$

Comparing above relation with (3.13), and using strong induction on $n$, we
obtain,

$\displaystyle\mathcal{R}(n,p)h^{p}(\psi)=h^{n}(\mathcal{A}(n,p)\psi),\quad\mbox{
for all }n\geq p\geq 0\mbox{ and }\psi\in F_{p}.$ (3.14)

Hence, we proved that (3.2) holds.

Now we show that the map $h^{p}$ is one-to-one for $p\in\mathbb{Z}^{+}$.
Observe that

$\displaystyle\|\mathcal{G}(p,n)\|=\|\mathcal{A}(p,n)Q_{n}\|=a_{p,n}\quad\mbox{
for all }n>p\geq 0.$

Now let $n>p\geq 0$ and $\phi_{1},\phi_{2}\in F_{p}$ such that

$\displaystyle h^{p}(\phi_{1})=h^{p}(\phi_{2}).$

Using above relation and (3.14), we have,

$\displaystyle
h^{n}(\mathcal{A}(n,p)\phi_{1})=h^{n}(\mathcal{A}(n,p)\phi_{2}).$

Thus,

$\displaystyle\mathcal{A}(n,p)(\phi_{1}-\phi_{2})=-\big{(}\eta^{n}(\mathcal{A}(n,p)\phi_{1})-\eta^{n}(\mathcal{A}(n,p)\phi_{2})\big{)}.$
(3.15)

Note that, as $\phi_{1},\phi_{2}\in F_{p}$,

$\displaystyle\phi_{1}-\phi_{2}=\mathcal{A}(p,n)\mathcal{A}(n,p)Q_{p}\big{(}\phi_{1}-\phi_{2}\big{)}=\mathcal{A}(p,n)Q_{n}\mathcal{A}(n,p)(\phi_{1}-\phi_{2}).$

Also, $\mathcal{A}(p,n)Q_{n}$ is a nonzero map in $\mathcal{B},$ as
$\mathcal{A}(p,n)\big{|}_{F_{n}}$ is invertible. Therefore, we have,

$\displaystyle\|\mathcal{A}(n,p)(\phi_{1}-\phi_{2})\|_{\mathcal{B}}$
$\displaystyle\geq\frac{\|\phi_{1}-\phi_{2}\|_{\mathcal{B}}}{\|\mathcal{A}(p,n)Q_{n}\|}$
$\displaystyle=\frac{\|\phi_{1}-\phi_{2}\|_{\mathcal{B}}}{a_{p,n}}.$

If $\phi_{1}\neq\phi_{2}$, then it follows from (3.4) that the function
$n\to\mathcal{A}(n,p)(\phi_{1}-\phi_{2})$ is unbounded. However, the right
hand side of (3.15), is bounded by $2\|\eta\|_{\infty}$, which leads to a
contradiction. Therefore $\phi_{1}=\phi_{2}$ and hence $h^{p}$ is one-one for
all $p\in\mathbb{Z}^{+}$.

This completes the proof of our Theorem 3.1.

###  3.1 Uniform Exponential Dichotomy Case:

Let us assume that (2.1) admits uniform exponential dichotomy, i.e. for each
$n\in\mathbb{Z}^{+}$, there exists projection $P_{n}$ such that:

  1. 1.

$\mathcal{A}(m,n)P_{n}=P_{m}\mathcal{A}(m,n)$ for all $m\geq n\geq 0$;

  2. 2.

for $m\geq n$,
$\mathcal{A}(m,n)\big{|}_{\operatorname{Ker}P_{n}}:\operatorname{Ker}P_{n}\to\operatorname{Ker}P_{m}$
is an invertible map and we denote the inverse map
$\big{(}\mathcal{A}(m,n)\big{|}_{\operatorname{Ker}P_{n}}\big{)}^{-1}$ by
$\mathcal{A}(n,m)$;

  3. 3.

there exists $D,\lambda>0$, such that

$\displaystyle\|\mathcal{A}(m,n)P_{n}\|\leq De^{-\lambda(m-n)}\qquad\mbox{ for
all }m\geq n;$

and

$\displaystyle\|\mathcal{A}(m,n)\big{(}Id_{\mathcal{B}}-P_{n}\big{)}\|\leq
De^{-\lambda(n-m)}\qquad\mbox{ for all }m<n.$

In particular, we have that,

$\displaystyle\|\mathcal{G}(m,n)\|\leq De^{-\lambda|m-n|}\qquad\mbox{ for all
}m,n\in\mathbb{Z}^{+}.$

By comparing the above expression with (3.3), we have that $a_{m,n}\leq
De^{-\lambda|m-n|}$. Finally, let $C>0$ be a constant such that $c_{m}=C$ for
all $m\in\mathbb{Z^{+}}$. Using above estimates, we have

$\displaystyle
C\sup_{m\in\mathbb{Z}^{+}}\Big{(}\sum_{n\in\mathbb{Z}^{+}}a_{m,n+1}\Big{)}$
$\displaystyle\leq
C\sup_{m\in\mathbb{Z}^{+}}\Big{(}\sum_{n\in\mathbb{Z}^{+}}De^{-\lambda|m-n-1|}\Big{)}$
$\displaystyle=C\,D\sup_{m\in\mathbb{Z}^{+}}\Big{(}\frac{e^{-\lambda}(1-e^{-\lambda(m-1)})}{1-e^{-\lambda}}+\frac{1}{1-e^{-\lambda}}\Big{)}<\infty.$

Using the constant $C$, we can make sure that equation (3.1) is satisfied.
Therefore, if the linear delay difference equation (2.1) satisfies uniform
exponential dichotomy then our Theorem 3.1 is applicable.

#### Acknowledgements

Author would like to thanks Prof. D. Dragičević for his valuable suggestions
throughout the process of solving and writing this article. The Author is
supported by Croatian Science Foundation under the project IP-2019-04-1239.

## References

  * [1] Murakami, S.: Representation of solutions of linear functional difference equations in phase space. Nonlinear Anal. 30, pp. 1153-1164 (1997).
  * [2] Dragičević, D., and Pituk, M.: Shadowing for nonautonomous difference equations with infinite delay. Applied Mathematics Letters, 120, 107284 (2021).
  * [3] Pugh, C.: On a Theorem of P. Hartman. American Journal of Mathematics, 91(2), pp. 363–367 (1969).
  * [4] Palis J.,: On the local structure of hyperbolic points in Banach spaces. An. Acad. Brasil. Cienc. 40, pp. 263–266 (1968).
  * [5] Hartman, P.: On the local linearization of differential equations. Proc. Amer. Math. Soc., 14, pp. 568-573 (1963).
  * [6] Grobman, D.: Homeomorphism of systems of differential equations. Dokl. Akad. Nauk SSSR, 128, pp. 880-881 (1959).
  * [7] Reinfelds, A.: A generalized theorem of Grobman and Hartman. Latv. Mat. Ezhegodnik, 29, pp. 84-88 (1985).
  * [8] Bates, W., Lu, K.: A Hartman-Grobman theorem for the Cahn-Hilliard and phase-field equations. J. Dynam. Differential Equations, 6(1), pp. 101-145 (1994).
  * [9] Belitskii, G.R.: Functional equations and the conjugacy of difeomorphism of fnite smoothness class. Funct. Anal. Appl. 7, pp. 268–277 (1973).
  * [10] Belitskii, G.R.: Equivalence and normal forms of germs of smooth mappings. Russian Math. Surv. 33, pp. 107–177 (1978).
  * [11] Sternbger, S.: Local contractions and a theorem of Poincare. Am. J. Math. 79, pp. 809–824 (1957).
  * [12] ElBialy, M.S.: Smooth conjugacy and linearization near resonant fxed points in Hilbert spaces. Houston J. Math. 40, pp. 467–509 (2014).
  * [13] Rodrigues, H.M., Solà-Morales, J.: Invertible contractions and asymptotically stable ODEs that are not C1-linearizable. J. Dyn. Difer. Equ. 18, pp. 961–974 (2006).
  * [14] Palmer, K.: A generalization of Hartmans linearization theorem. J. Math. Anal. Appl. 41, pp. 753–758 (1973).
  * [15] Aulbach, B., Wanner, T.: Topological simplifcation of nonautonomous diference equations. J. Differ. Equ. Appl. 12, pp. 283–296 (2006).
  * [16] Barreira L., Valls C.: A Grobman–Hartman theorem for nonuniformly hyperbolic dynamics. J. diff. equ. 228, pp. 285–310 (2006).
  * [17] Shi, J.L., Xiong, K.Q.: On Hartmans linearization theorem and Palmers linearization theorem. J. Math. Anal. Appl. 192, pp. 813–832 (1995).
  * [18] Dragičević, D., Zhang, W., Zhang, W.: Smooth linearization of nonautonomous diference equations with a nonuniform dichotomy. Math. Z. 292, pp. 1175–1193 (2019).
  * [19] Dragičević, D., Zhang, W., Zhang, W.: Smooth linearization of nonautonomous diferential equations with a nonuniform dichotomy. Proc. Lond. Math. Soc. 121, pp. 32–50 (2020).
  * [20] Backes, L., Dragičević, D.: A generalized Grobman–Hartman theorem for nonautonomous dynamics. Collect. Math. (2021).
  * [21] Barreira, L., Dragičević, D., Valls C.:Existence of conjugacies and stable manifold theorem via suspensions. Elec. J. of Diff. Equa. Vol. 2017, No. 172, pp. 1–11 (2017).
  * [22] Farkas G.: A Hartman-Grobman result for retarded functional differential equations with an application to the numerics around hyperbolic equilibria, Z. Angew. Math. Phys. 52, pp. 421-432 (2001).
  * [23] Barreira, L., Valls, C.: Perturbations of delay equations. J. Differ. Equ. 269,pp. 7015-7041 (2020).
  * [24] Benkhalti, R., Ezzinbi, K.: A Hartman and Grobman theorem for some partial functional differential equations, Int. J. of Bif. and Cha., Vol. 10, No. 5, pp. 1165–1169(2000).
  * [25] Sternberg, N.: A Hartman-Grobman Theorem for a Class of Retarded Functional Differential Equations, J. Math. Anal. Appl. 176, pp. 156-165 (1993).
  * [26] Sternberg, N.: A Hartman-Grobman theorem for maps, in: Ordinary and Delay Differential Equations, Edinburg, TX, 1991, in: Pitman Res. Notes Math. Ser., vol. 272, Longman Sci. Tech., Harlow, pp. 223–227 (1992).
  * [27] Dragičević, D.: Global smooth linearization of nonautonomous contractions on Banach spaces, Electron. J. Qual. Theory Differ. Equ., Paper No. 12, pp. 1-19 (2022).
  * [28] A. Castañeda, G. Robledo,: Differentiability of Palmer’s linearization theorem and converse result for density function, J. Differential Equations 259, pp. 4634–4650 (2015).
  * [29] W. M. Zhang, W. N. Zhang,: C1 linearization for planar contractions, J. Funct. Anal. 260, No. 7, 2043–2063 (2011).
  * [30] W. M. Zhang, W. N. Zhang, Sharpness for C1 linearization of planar hyperbolic diffeomorphisms, J. Differential Equations 257, pp. 4470–4502 (2014).
  * [31] W. M. Zhang, K. Lu, W. N. Zhang,: Differentiability of the conjugacy in the Hartman-Grobman Theorem, Trans. Amer. Math. Soc. 369, 4995-5030 (2017).