In this work, we approach the minimization of a continuously differentiable convex function under linear equality constraints by a second-order dynamical system with an asymptotically vanishing damping term. The system under consideration is a time rescaled version of another system previously found in the literature. We show fast convergence of the primal-dual gap, the feasibility measure, and the objective function value along the generated trajectories. These convergence rates now depend on the rescaling parameter, and thus can be improved by choosing said parameter appropriately. When the objective function has a Lipschitz continuous gradient, we show that the primal-dual trajectory asymptotically converges weakly to a primal-dual optimal solution to the underlying minimization problem. We also exhibit improved rates of convergence of the gradient along the primal trajectories and of the adjoint of the corresponding linear operator along the dual trajectories. Even in the unconstrained case, some trajectory convergence result seems to be new. We illustrate the theoretical outcomes through numerical experiments.

Key Words. Augmented Lagrangian method, primal-dual dynamical system, damped inertial dynamics, Nesterov's accelerated gradient method, Lyapunov analysis, time rescaling, convergence rate, trajectory convergence

AMS subject classification. 37N40, 46N10, 65K10, 90C25

§ INTRODUCTION

§.§ Problem statement and motivation

In this paper we will consider the optimization problem

\begin{equation}
%	\tag{$\mathcal{P}$}
\label{intro:pb}
\begin{array}{rl}
\min & f \left( x \right), \\
\textrm{subject to} 	& Ax = b
\end{array}
\end{equation}

\begin{equation}
%	\tag{$\mathcal{H}_{\mathcal{P}}$}
\label{intro:hypo}
\begin{cases}
\sX, \sY \textrm{ are real Hilbert spaces}; \\
f \colon \sX \to \sR \textrm{ is a continuously differentiable convex function}; \\
A \colon \sX \to \sY \textrm{ is a continuous linear operator and } b \in \sY; \\
\textrm{the set } \mathbb{S} \textrm{ of primal-dual optimal solutions of } \eqref{intro:pb} \textrm{ is assumed to be  nonempty}.
\end{cases}
\end{equation}

This model formulation underlies many important applications in various areas, such as image recovery [50], machine learning [41, 57], the energy dispatch of power grids [73, 74], distributed optimization [58, 76] and network optimization [70, 75].

In recent years, there has been a flurry of research on the relationship between continuous time dynamical systems and the numerical algorithms that arise from their discretizations.
For the unconstrained optimization problem, it has been known that inertial systems with damped velocities enjoy good convergence properties.
For a convex, smooth function $f : \mathcal{X} \to \mathbb{R}$, Polyak is the first to consider the heavy ball with friction (HBF) dynamics [66, 65]

\begin{equation} \label{eq:heavy ball with friction}
\tag{HBF}
\ddot{x}(t) + \gamma \dot{x}(t) + \nabla f(x(t)) = 0 .
\end{equation}

Alvarez and Attouch continue the line of this study, focusing on inertial dynamics with a fixed viscous damping coefficient [2, 3, 19].
Later on, Cabot, Engler, and Gadat [43, 44] consider the system that replaces $\gamma$ with a time dependent damping coefficient $\gamma \left( t \right)$.
In [72], Su, Boyd, and Candès showed that it turns out one can achieve fast convergence rates by introducing a time dependent damping coefficient which vanishes in a controlled manner, neither too fast nor too slowly, as $t$ goes to infinity

\begin{equation} \label{eq:AVD}
\tag{AVD}
\ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \nabla f(x(t)) = 0.
\end{equation}

For $\alpha \geq 3$, the authors showed that a solution $x \colon \left[ t_{0}, +\infty \right) \to \mathcal{X}$ to (<ref>) satisfies $f(x(t)) - f(x_{*}) = \mathcal{O}\left(\frac{1}{t^{2}}\right)$ as $t \to +\infty$.
In fact, the choice $\alpha = 3$ provides a continuous limit counterpart to Nesterov's celebrated accelerated gradient algorithm [61, 63, 30].
Weak convergence of the trajectories to minimizers of $f$ when $\alpha > 3$ has been shown by Attouch, Chbani, Peypouquet, and Redont in [22] and May in [59], together with the improved rates of convergence $f(x(t)) - f(x_{*}) = o\left( \frac{1}{t^{2}}\right)$ as $t\to +\infty$.
In the meantime, similar results for the discrete counterpart were also reported by Chambolle and Dossal in [45], and by Attouch and Peypouquet in [20] .

In [15], Attouch, Chbani, and Riahi proposed an inertial proximal type algorithm, which results from a discretization of the time rescaled (<ref>) system

\[
\ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \delta(t) \nabla f(x(t)) = 0,
\]

where $\delta \colon \left[ t_{0} , + \infty \right) \to \sR_{+}$ is the time scaling function satisfies a certain growth condition and that also enter into convergence rate statement $f \left( x \left( t \right) \right) - f \left( x_{*} \right) = \bO \left( \frac{1}{t^{2} \delta \left( t \right)} \right)$ as $t \to + \infty$.

The resulting algorithm obtained by the authors is considerably simpler than the founding proximal point algorithm proposed by Güler in [51], while providing comparable convergence rates for the functional values.

In order to approach constrained optimization problems, Augmented Lagrangian Method (ALM) [68] (for linearly constrained problems) and Alternating Direction Method of Multipliers (ADMM) [49, 41] (for problems with separable objectives and block variables linearly coupled in the constraints) and some of their variants have been shown to enjoy substantial success.

Continuous-time approaches for structured convex minimization problems formulated in the spirit of the full splitting paradigm have been recently addressed in [40] and, closely connected to our approach, in [75, 52, 11, 34], to which we will have a closer look in Subsection <ref>.
The temporal discretization resulting from these dynamics gives rise to the numerical algorithm with fast convergence rates [54, 53] and with a convergence guarantee for the generated iterate [33], without additional assumptions such as strong convexity.

In this paper, we will investigate a second-order dynamical system with asymptotic vanishing damping and time rescaling term, which is associated with the optimization problem (<ref>) and formulated in terms of its augmented Lagrangian.
The case when the time rescaling term does not appear has been established in [34].
We show that by introducing this time rescaling function, we are able to derive faster convergence rates for the primal-dual gap, the feasibility measure, and the objective function value along generated trajectories while still maintaining the asymptotic behaviour of the trajectories towards a primal-dual optimal solution.
On the other hand, this work can also be viewed as an extension of the time rescaling technique derived in [15, 18] for the constrained case.
To our knowledge, the trajectory convergence for dynamics with time scaling seems to be new, even in the unconstrained case.

§.§ Notations and a preliminary result

For both Hilbert spaces $\sX$ and $\sY$, the Euclidean inner product and the associated norm will be denoted by $\left\langle \cdot , \cdot \right\rangle$ and $\left\lVert \cdot \right\rVert$, respectively. The Cartesian product $\sX \times \sY$ will be endowed with the inner product and the associated norm defined for $\left( x , \lambda \right) , \left( z , \mu \right) \in \sX \times \sY$ as

\begin{equation*}
\left\langle \left( x , \lambda \right) , \left( z , \mu \right) \right\rangle
= \left\langle x , z \right\rangle + \left\langle \lambda , \mu \right\rangle
\qquad \textrm{ and } \qquad
\left\lVert \left( x , \lambda \right) \right\rVert
= \sqrt{\left\lVert x \right\rVert ^{2} + \left\lVert \lambda \right\rVert ^{2}},
\end{equation*}

Let $f \colon \sX \to \sR$ be a continuously differentiable convex function such that $\nabla f$ is $\ell-$Lipschitz continuous.
For every $x, y \in \sX$ it holds (see <cit.>)

\begin{equation}
\label{pre:f-bound}
0 \leq \dfrac{1}{2 \ell} \left\lVert \nabla f \left( x \right) - \nabla f \left( y \right) \right\rVert ^{2} \leq f \left( x \right) - f \left( y \right) - \left\langle \nabla f \left( y \right) , x - y \right\rangle \leq \dfrac{\ell}{2} \left\lVert x - y \right\rVert ^{2} .
\end{equation}

§ THE PRIMAL-DUAL DYNAMICAL APPROACH

§.§ Augmented Lagrangian formulation

Consider the saddle point problem

\begin{equation}
\label{intro:sd}
\min_{x \in \sX} \max_{\lambda \in \sY} \Lag \left( x , \lambda \right)
\end{equation}

associated to problem (<ref>), where $\Lag \colon \sX \times \sY \to \sR$ denotes the Lagrangian function

\begin{equation*}
\Lag \left( x , \lambda \right) := f \left( x \right) + \left\langle \lambda , Ax - b \right\rangle .
\end{equation*}

Under the assumptions (<ref>), $\Lag$ is convex with respect to $x \in \sX$ and affine with respect to $\lambda \in \sY$. A pair $\left( x_{*} , \lambda_{*} \right) \in \sX \times \sY$ is said to be a saddle point of the Lagrangian function $\Lag$ if for every $\left( x , \lambda \right) \in \sX \times \sY$

\begin{equation}
\label{intro:sadde-point}
\Lag \left( x_{*} , \lambda \right) \leq \Lag \left( x_{*} , \lambda_{*} \right) \leq \Lag \left( x , \lambda_{*} \right).
\end{equation}

If $\left( x_{*} , \lambda_{*} \right) \in \sX \times \sY$ is a saddle point of $\Lag$ then $x_{*} \in \sX$ is an optimal solution of (<ref>), and $\lambda_{*} \in \sY$ is an optimal solution of its Lagrange dual problem. If $x_{*} \in \sX$ is an optimal solution of (<ref>) and a suitable constraint qualification is fulfilled, then there exists an optimal solution $\lambda_{*} \in \sY$ of the Lagrange dual problem such that $\left( x_{*} , \lambda_{*} \right) \in \sX \times \sY$ is a saddle point of $\Lag$. For details and insights into the topic of constraint qualifications for convex duality we refer to [27, 31].

The set of saddle points of $\Lag$, called also primal-dual optimal solutions of (<ref>), will be denoted by $\sol$ and, as stated in the assumptions, it will be assumed to be nonempty.  The set of feasible points of (<ref>) will be denoted by $\Fea := \left\lbrace x \in \sX \colon Ax = b \right\rbrace$ and the optimal objective value of (<ref>) by $f_{*}$.

The system of primal-dual optimality conditions for (<ref>) reads

\begin{equation}
\label{intro:opt-Lag}
\left( x_{*} , \lambda_{*} \right) \in \sol
\Leftrightarrow \begin{cases}
\nabla_{x} \Lag \left( x_{*} , \lambda_{*} \right) 			& = 0 \\
\nabla_{\lambda} \Lag \left( x_{*} , \lambda_{*} \right) 	& = 0
\end{cases} \Leftrightarrow \begin{cases}
\nabla f \left( x_{*} \right) + A^{*} \lambda_{*} 	& = 0 \\
Ax_{*} - b 													& = 0
\end{cases},
\end{equation}

where $A^{*} : \sY \rightarrow \sX$ denotes the adjoint operator of $A$.

For $\beta \geq 0$, we consider also the augmented Lagrangian $\Lb \colon \sX \times \sY \to \sR$ associated with (<ref>)

\begin{equation}
\label{intro:aug-Lag}
\Lb \left( x , \lambda \right) := \Lag \left( x , \lambda \right) + \dfrac{\beta}{2} \left\lVert Ax - b \right\rVert ^{2} = f \left( x \right) + \left\langle \lambda , Ax - b \right\rangle + \dfrac{\beta}{2} \left\lVert Ax - b \right\rVert ^{2} .
\end{equation}

For every $(x, \lambda) \in \Fea \times \sY$ it holds

\begin{equation}
\label{intro:Fea:eq}
f \left( x \right) = \Lb \left( x , \lambda \right) = \Lag \left( x , \lambda \right) .
\end{equation}

If $\left( x_{*} , \lambda_{*} \right) \in \sol$, then we have for every $\left( x , \lambda \right) \in \sX \times \sY$

\begin{equation*}
\Lag \left( x_{*} , \lambda \right) = \Lb \left( x_{*} , \lambda \right)
\leq \Lag \left( x_{*} , \lambda_{*} \right) = \Lb \left( x_{*} , \lambda_{*} \right)
\leq \Lag \left( x , \lambda_{*} \right) \leq \Lb \left( x , \lambda_{*} \right).
\end{equation*}

In addition, from (<ref>) we have

\begin{equation*} \label{intro:opt-Lb}
\begin{split}
\left( x_{*} , \lambda_{*} \right) \in \sol
\begin{cases}
\nabla f \left( x_{*} \right) + A^{*} \lambda_{*} 	& = 0 \\
Ax_{*} - b 													& = 0
\end{cases}
\:\Leftrightarrow\:
\begin{cases}
\nabla f \left( x_{*} \right) + A^{*} \lambda_{*} + \beta A^{*}(Ax_{*} - b) &= 0 \\
Ax_{*} - b &= 0
\end{cases} \\
\begin{cases}
\nabla_{x} \mathcal{L}_{\beta}(x_{*}, \lambda_{*}) &= 0 \\
\nabla_{\lambda} \mathcal{L}_{\beta}(x_{*}, \lambda_{*}) &= 0.
\end{cases}
\end{split}
\end{equation*}

In other words, for any $\beta \geq 0$ the sets of saddle points of $\mathcal{L}$ and $\mathcal{L}_{\beta}$ are identical.

§.§ The primal-dual asymptotic vanishing damping dynamical system with time rescaling

The dynamical system which we associate to (<ref>) and investigate in this paper reads

\begin{equation}
%		\tag{$\mathrm{TRIALS}$}
\label{ds:timerescaled}
\begin{dcases}
\ddot{x} \left( t \right) + \dfrac{\alpha}{t} \dot{x} \left( t \right) + \delta \left( t \right) \nabla_{x} \Lb \Bigl( x \left( t \right) , \lambda \left( t \right) + \theta t \dot{\lambda} \left( t \right) \Bigr)  		& = 0 \\
\ddot{\lambda} \left( t \right) + \dfrac{\alpha}{t} \dot{\lambda} \left( t \right) - \delta \left( t \right) \nabla_{\lambda} \Lb \Bigl( x \left( t \right) + \theta t \dot{x} \left( t \right) , \lambda \left( t \right) \Bigr) 	& = 0 \\
\Bigl( x \left( t_{0} \right) , \lambda \left( t_{0} \right) \Bigr) 			= \Bigl( x_{0} , \lambda_{0} \Bigr) \textrm{ and }
\Bigl( \dot{x} \left( t_{0} \right) , \dot{\lambda} \left( t_{0} \right) \Bigr) = \Bigl( \dot{x}_{0} , \dot{\lambda}_{0} \Bigr)
\end{dcases},
\end{equation}

where $t_0 >0$, $\alpha > 0$, $\theta > 0$, $\delta \colon \left[ t_{0}, +\infty \right) \to \mathbb{R}$ is a nonnegative continuously differentiable function and $\left( x_{0} , \lambda_{0} \right), \bigl( \dot{x}_{0} , \dot{\lambda}_{0} \bigr) \in \sX \times \sY$ are the initial conditions. Replacing the expressions of the partial gradients of $\Lb$ into the system leads to the following formulation for (<ref>):

\begin{equation*}
\begin{dcases}
\ddot{x} \left( t \right) + \dfrac{\alpha}{t} \dot{x} \left( t \right) + \delta \left( t \right) \nabla f \left( x \left( t \right) \right) + \delta \left( t \right) A^{*} \left( \lambda \left( t \right) + \theta t \dot{\lambda} \left( t \right) \right) + \delta \left( t \right) \beta A^{*} \Bigl( Ax \left( t \right) - b \Bigr)		& = 0 \\
\ddot{\lambda} \left( t \right) + \dfrac{\alpha}{t} \dot{\lambda} \left( t \right) - \delta \left( t \right) \Bigl( A \bigl( x \left( t \right) + \theta t \dot{x} \left( t \right) \bigr) - b \Bigr) 	& = 0 \\
\Bigl( x \left( t_{0} \right) , \lambda \left( t_{0} \right) \Bigr) 			= \Bigl( x_{0} , \lambda_{0} \Bigr) \textrm{ and }
\Bigl( \dot{x} \left( t_{0} \right) , \dot{\lambda} \left( t_{0} \right) \Bigr) = \Bigl( \dot{x}_{0} , \dot{\lambda}_{0} \Bigr).
\end{dcases}
\end{equation*}

The case (<ref>) in which there is no time rescaling, i.e., when $\delta(t) \equiv 1$, was studied by Zeng, Lei, and Chen in [75], and by Boţ and Nguyen in [34].
The system with more general damping, extrapolation and time rescaling coefficients was addressed by He, Hu, and Fang in [52, 55] and by Attouch, Chbani, Fadili, and Riahi in [11].

It is well known that the viscous damping term $\frac{\alpha}{t}$ has a vital role in achieving fast convergence in unconstrained minimization [12, 14, 59].
The role of the extrapolation $\theta t$ is to induce more flexibility in the dynamical system and in the associated discrete schemes, as it has been recently noticed in [11, 16, 52, 75].
The time scaling function $\delta \left( \cdot \right)$ has the role to further improve the rates of convergence of the objective function value along the trajectory, as it was noticed in the context of uncostrained minimization problems in [10, 15, 17] and of linearly constrained minimization problems in [11, 55].

Finally, we mention that extending the results in this paper to the multi-block case is possible. For further details, we refer the readers to <cit.>.

§.§ Associated monotone inclusion problem

The optimality system (<ref>) can be equivalently written  as

\begin{equation}\label{moninclusion}
\TL \left( x_{*} , \lambda_{*} \right) = 0 ,
\end{equation}

\begin{equation*}
\TL \colon \sX \times \sY \to \sX \times \sY, \quad \TL \left( x , \lambda \right)
= \begin{pmatrix}
\nabla_{x} \Lag \left( x , \lambda \right) \\ - \nabla_{\lambda} \Lag \left( x , \lambda \right)
\end{pmatrix}
= \begin{pmatrix}
\nabla f \left( x \right) + A^{*} \lambda \\ b-Ax
\end{pmatrix},
\end{equation*}

is the maximally monotone operator associated with the convex-concave function $\Lag$. Indeed, it is immediate to verify that $\TL$ is monotone. Since it is also continuous, it is maximally monotone (see, for instance, <cit.>). Therefore $\sol$ can be interpreted as the set of zeros of the maximally monotone operator $\TL$, which means that it is a closed convex subset of $\sX \times \sY$ (see, for instance, <cit.>).

Even in the non-rescaling case, applying the fast continuous-time approaches recently proposed in [21, 5] to the solving of (<ref>) would require the use of the Moreau-Yosida approximation of the operator $\TL$, for which in general no closed formula is available. The resulting dynamical system would therefore not be formulated in the spirit of the full splitting algorithm, which is undesirable from the point of view of numerical computations.
We mention the work [32], which is related to this time rescaling approach.

§ FASTER CONVERGENCE RATES VIA TIME RESCALING

In this section we will derive fast convergence rates for the primal-dual gap, the feasibility measure, and the objective function value along the trajectories generated by the dynamical system (<ref>).
We will make the following assumptions on the parameters $\alpha$, $\theta$, $\beta$ and the function $\delta$ throughout this section.

In (<ref>), assume that $\delta : \left[t_{0}, +\infty\right) \to (0, + \infty)$ is continuously differentiable. Moreover, suppose that the parameters $\alpha, \beta$, $\theta$ and the function $\delta$ satisfy

\begin{equation}
\label{cond:gap}
\alpha \geq 3 , \quad \beta \geq 0 , \quad \dfrac{1}{2} \geq \theta \geq \dfrac{1}{\alpha - 1} \quad \textrm{ and } \quad
\sup_{t\geq t_{0}}\frac{t \dot{\delta}(t)}{\delta(t)} \leq \frac{1 - 2\theta}{\theta} .
\end{equation}

Besides the first three conditions that are known previously in [34], it is worth pointing out that we can deduce from the last one the following inequality for every $t \geq t_{0}$:

\begin{equation}
\label{cond:alpha}
\dfrac{t \dot{\delta} \left( t \right)}{\delta \left( t \right)} \leq \dfrac{1 - 2 \theta}{\theta} = \dfrac{1}{\theta} - 2 \leq \alpha - 3 .
\end{equation}

This gives a connection to the condition which appears in [15]. A few more comments regarding the function $\delta$ will come later, after the convergence rates statements.

§.§ The energy function

Let $\left( x , \lambda \right) \colon \left[ t_{0} , + \infty \right) \to \sX \times \sY$ be a solution of (<ref>). Let
$(x_{*}, \lambda_{*}) \in \mathbb{S}$ be fixed,  we define the energy function $\E \colon \left[ t_{0} , + \infty \right) \to \sR$

\begin{equation}
\label{ener:E}
\E \left( t \right)
:= \theta^{2} t^{2} \delta(t) \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) + \dfrac{1}{2} \left\lVert v \left( t \right) \right\rVert ^{2} + \dfrac{\xi}{2} \left\lVert \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) \right\rVert ^{2} ,
\end{equation}

\begin{align}
v \left( t \right) & := \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) + \theta t \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) , \label{ener:v} \\
\xi  & := \alpha \theta - \theta - 1 \geq 0 . \label{ener:xi}
\end{align}

Notice that due to (<ref>), we have

\begin{equation}
\label{ener:pos}
\E \left( t \right) \geq 0 \qquad \forall t \geq t_{0} .
\end{equation}

In addition, according to (<ref>) and (<ref>), we have for every $t \geq t_0$

\begin{align}
\Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right)
& = \Lag \left( x \left( t \right) , \lambda_{*} \right) - \Lag \left( x_{*} , \lambda \left( t \right) \right) + \dfrac{\beta}{2} \left\lVert Ax \left( t \right) - b \right\rVert ^{2} \label{ener:PD-gap:Lag} \\
& = \Lag \left( x \left( t \right) , \lambda_{*} \right) - f_{*} + \dfrac{\beta}{2} \left\lVert Ax \left( t \right) - b \right\rVert ^{2} \nonumber \\
& = f \left( x \left( t \right) \right) - f_{*} + \left\langle \lambda_{*} , Ax \left( t \right) - b \right\rangle + \dfrac{\beta}{2} \left\lVert Ax \left( t \right) - b \right\rVert ^{2} \geq 0 , \label{ener:PD-gap}
\end{align}

where $f_{*}$ denotes the optimal objective  value of (<ref>).

Assumption <ref> implies the nonnegativity of following quantity, which will appear many times in our analysis:

\begin{equation}\label{eq:definition of sigma}
\sigma \colon \left[ t_{0} , + \infty \right) \to \sR_{+} , \quad \sigma(t) := \frac{1 - 2\theta}{\theta}\delta(t) - t \dot{\delta}(t) .
\end{equation}

Let $\left( x , \lambda \right) \colon \left[ t_{0} , + \infty \right) \to \sX \times \sY$ be a solution of (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. For every $t \geq t_{0}$ it holds

\begin{align*}
\dfrac{d}{dt} \E \left( t \right)
\leq &- \theta^{2} t \sigma(t) \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) - \dfrac{1}{2} \beta \theta t \delta \left( t \right) \left\lVert Ax \left( t \right) - b \right\rVert ^{2} \nonumber \\
&- \xi \theta t \left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert ^{2} .
\end{align*}

Let $t \geq t_{0}$ be fixed.
Since $x_{*} \in \Fea$, we have

\begin{align*}
\nabla_{x} \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) & = \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) = \nabla f \left( x \left( t \right) \right) + A^{*} \lambda_{*} + \beta A^{*} \left( Ax \left( t \right) - b \right) , \nonumber \\
\nabla_{\lambda} \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) & = - \nabla_{\lambda} \Lb \left( x_{*} , \lambda \left( t \right) \right) = 0 .
\end{align*}

Under these expressions, the system (<ref>) can be equivalently written as

\begin{align*}
\left(\ddot{x} \left( t \right) , \ddot{\lambda} \left( t \right)\right)
=& - \dfrac{\alpha}{t} \left(\dot{x} \left( t \right) , \dot{\lambda} \left( t \right)\right)
- \delta \left( t \right) \bigl( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \bigr) \nonumber \\
&- \delta(t)\left(A^{*} \left( \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) \right) ,
- \Bigl( A \left( x \left( t \right) + \theta t \dot{x} \left( t \right) \right) - b \Bigr)\right) ,
\end{align*}

which leads to

\begin{align*}
\dot{v} \left( t \right)
=& \: (1 + \theta)\left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) + \theta t \left( \ddot{x} \left( t \right) , \ddot{\lambda} \left( t \right) \right) \nonumber \\
=& \: - \xi \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) - \theta t \delta \left( t \right) \left( t \right) \bigl( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \bigr) \nonumber \\
&\:- \theta t \delta \left( t \right) \begin{pmatrix}
A^{*} \left( \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) \right) ,
- \Bigl( A \left( x \left( t \right) + \theta t \dot{x} \left( t \right) \right) - b \Bigr)
\end{pmatrix}.
\end{align*}

We get from the distributive property of inner product

\begin{align*}
& \left\langle v \left( t \right) , \dot{v} \left( t \right) \right\rangle \nonumber \\
= 	& - \xi \left\langle \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) , \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rangle
- \xi \theta t \left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert ^{2} \nonumber \\
& - \theta t \delta \left( t \right) \left\langle \left( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \right) , \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) \right\rangle \nonumber \\
& - \theta^{2} t^{2} \delta \left( t \right) \left\langle \left( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \right) , \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rangle \nonumber \\
& - \theta t \delta \left( t \right) \left\langle \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) , Ax \left( t \right) - Ax_{*} \right\rangle
- \theta^{2} t^{2} \delta \left( t \right) \left\langle \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) , A \dot{x} \left( t \right) \right\rangle \nonumber \\
& + \theta t \delta \left( t \right) \left\langle A \left( x \left( t \right) + \theta t \dot{x} \left( t \right) \right) - b , \lambda \left( t \right) - \lambda_{*} \right\rangle
+ \theta^{2} t^{2} \delta \left( t \right) \left\langle A \left( x \left( t \right) + \theta t \dot{x} \left( t \right) \right) - b , \dot{\lambda} \left( t \right) \right\rangle .
\end{align*}

Since $x_{*} \in \Fea$, the last four terms in the above identity vanish. Indeed,

\begin{align*}
& - \left\langle \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) , Ax \left( t \right) - Ax_{*} \right\rangle
- \theta t \left\langle \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) , A \dot{x} \left( t \right) \right\rangle \nonumber \\
&  + \left\langle A \left( x \left( t \right) + \theta t \dot{x} \left( t \right) \right) - b , \lambda \left( t \right) - \lambda_{*} \right\rangle
+ \theta t \left\langle A \left( x \left( t \right) + \theta t \dot{x} \left( t \right) \right) - b , \dot{\lambda} \left( t \right) \right\rangle \nonumber \\
= \ 	& - \left\langle \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) , Ax \left( t \right) - b \right\rangle
- \theta t \left\langle \lambda \left( t \right) - \lambda_{*} + \theta t \dot{\lambda} \left( t \right) , A \dot{x} \left( t \right) \right\rangle \nonumber \\
& + \!\left\langle Ax \left( t \right) - b , \lambda \left( t \right) - \lambda_{*} \right\rangle
+ \theta t \left\langle A \dot{x} \left( t \right) , \lambda \left( t \right) - \lambda_{*} \right\rangle \nonumber \\
& + \theta t \left\langle Ax \left( t \right) - b , \dot{\lambda} \left( t \right) \right\rangle
+ \theta^{2} t^{2} \left\langle A \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right\rangle \\
=  \ & \ 0 .
\end{align*}

Therefore, differentiating $\mathcal{E}$ with respect to $t$ gives

\begin{align}
\dfrac{d}{dt} \E \left( t \right)
=& \: \theta^{2} t \left( 2 \delta \left( t \right) + t \dot{\delta} \left( t \right) \right) \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) \nonumber \\
& + \theta^{2} t^{2} \delta(t) \left\langle \left( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \right) , \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rangle \nonumber \\
&+ \left\langle v \left( t \right) , \dot{v} \left( t \right) \right\rangle
+ \xi \left\langle \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) , \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rangle \nonumber \\
= & \: \theta^{2} t \left( 2 \delta \left( t \right) + t \dot{\delta} \left( t \right) \right) \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) - \xi \theta t \left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert ^{2} \nonumber \\
& - \theta t \delta(t) \left\langle \left( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \right) , \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) \right\rangle . \label{dec:pre}
\end{align}

Furthermore, the convexity of $f$ and the fact that $x_{*} \in \Fea$ guarantee

\begin{align}
& - \left\langle \left( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \right) , \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) \right\rangle \nonumber \\
%= \left\langle \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , x_{*} - x \left( t \right) \right\rangle
= \ 	& \left\langle \nabla f \left( x \left( t \right) \right) , x_{*} - x \left( t \right) \right\rangle
+ \left\langle A^{*} \lambda_{*} , x_{*} - x \left( t \right) \right\rangle
+ \beta \left\langle A^{*} \left( Ax \left( t \right) - b \right) , x_{*} - x \left( t \right) \right\rangle \nonumber \\
\leq \ 	& - \left( f \left( x \left( t \right) \right) - f \left( x_{*} \right) \right) - \left\langle \lambda_{*} , Ax \left( t \right) - b \right\rangle - \beta \left\lVert Ax \left( t \right) - b \right\rVert ^{2} \label{dec:conv} \\
= \ 	& - \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) - \dfrac{\beta}{2} \left\lVert Ax \left( t \right) - b \right\rVert ^{2} \nonumber,
\end{align}

where we recall that the second equality comes from (<ref>). By multiplying this inequality by $\theta t \delta(t)$ and combining it with (<ref>), the coefficient attached to the primal-dual gap becomes

\[
\theta^{2} t \left( 2\delta(t) + t \dot{\delta}(t)\right) - \theta t \delta(t) = - \theta^{2} t \left( \frac{1 - 2\theta}{\theta} \delta(t) - t\dot{\delta}(t)\right) = - \theta^{2} t \sigma(t),
\]

which finally gives the desired statement.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ be a solution of (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. The following statements are true:

* it holds

∫_t_0^+∞ t σ(t)[ℒ( x(t), λ_* ) - ℒ(x_*, λ(t))]dt ≤        (t_0)     < +∞,

β∫_t_0^+∞ t δ(t) ‖Ax ( t ) - b ‖^2dt ≤     2(t_0)/θ     < +∞,

ξ∫_t_0^+∞ t ‖( ẋ ( t ) , λ̇ ( t ) ) ‖^2 ≤     (t_0)/θ      < +∞;

* if, in addition, $\alpha > 3$ and $\frac{1}{2} \geq \theta > \frac{1}{\alpha - 1}$, then the trajectory $(x(t), \lambda(t))_{t\geq t_{0}}$ is bounded and the convergence rate of its velocity is

\[
\left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert = \mathcal{O}\left(\frac{1}{t}\right) \quad \text{as} \quad t\to +\infty.
\]

(i) Recall that Assumption <ref> implies $\sigma(t) \geq 0$ for all $t\geq t_{0}$ and $\xi \geq 0$. Moreover, $(x_{*}, \lambda_{*}) \in \mathbb{S}$ yields $x_{*} \in \mathbb{F}$. Therefore, we can apply  observation (<ref>) and Lemma <ref> to obtain, for every $t\geq t_{0}$,

\begin{align}
\frac{d}{dt}\E(t) \leq& \: -\theta^{2} t \sigma(t) \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) \\
& - \frac{1}{2}\beta \theta t \delta(t) \left\lVert Ax \left( t \right) - b \right\rVert^{2} - \xi \theta t \left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert^{2}
\leq 0. \label{eq:E not increasing}
\end{align}

This means that $\E$ is nonincreasing on $\left[ t_{0}, +\infty \right)$. For every $t\geq t_{0}$, by integrating (<ref>) from $t_{0}$ to $t$, we obtain

\begin{align*}
&\theta^{2}\int_{t_{0}}^{t} t\sigma(t)\Bigl(\mathcal{L}(x(s), \lambda_{*}) - \mathcal{L}(x_{*}, \lambda(s))\Bigr) ds \\
& + \frac{\beta \theta}{2}\int_{t_{0}}^{t}s \delta(s)\|Ax(s) - b\|^{2}ds + \xi \theta \int_{t_{0}}^{t}s\left\|\left(\dot{x}(s), \dot{\lambda}(s)\right)\right\|^{2} ds \\
\leq \:\: &\E(t_{0}) - \E(t) \leq \E(t_{0}),
\end{align*}

where the last inequality follows from (<ref>). Since all quantities inside the integrals are nonnegative, we obtain (<ref>)-(<ref>) by letting $t \to +\infty$.

(ii) Inequality (<ref>) tells us that $\E$ is nonincreasing on $\left[ t_{0}, +\infty \right)$. Hence, for every $t\geq t_{0}$ it holds

\begin{equation}
\label{eq:energy is bounded}
\begin{split}
&\theta^{2} t^{2} \delta(t) \Bigl(\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\Bigr) + \left\|v(t)\right\|^{2} + \frac{\xi}{2}\left\|\bigl(x(t), \lambda(t)\bigr) - (x_{*}, \lambda_{*})\right\|^{2}
\leq \E(t_{0}).
\end{split}
\end{equation}
Assuming $\alpha > 3$ and $\frac{1}{2} \geq \theta > \frac{1}{\alpha - 1}$, we immediately see $\xi > 0$. From (<ref>) we obtain, for every $t\geq t_{0}$,

\begin{equation}
\label{eq:trajectory bounded}
\left\|\bigl(x(t), \lambda(t)\bigr) - (x_{*}, \lambda_{*})\right\|^{2} \leq \frac{2\E(t_{0})}{\xi},
\end{equation}

which implies the boundedness of the trajectory. On the other hand, the same inequality gives for all $t\geq t_{0}$

\begin{equation}
\left\|v(t)\right\| = \left\|\bigl(x(t), \lambda(t)\bigr) - (x_{*}, \lambda_{*}) + \theta t \left(\dot{x}(t), \dot{\lambda}(t)\right)\right\| \leq \sqrt{2\E(t_{0})}.
\end{equation}

Using the triangle inequality and (<ref>), we obtain for all $t\geq t_{0}$

\begin{align}
t\left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert &\leq \frac{1}{\theta} \left(\left\|\bigl(x(t), \lambda(t)\bigr) - (x_{*}, \lambda_{*})\right\| + \left\|v\right\|\right) \nonumber\\
&\leq \frac{1}{\theta}\left(\sqrt{\frac{2\E(t_{0})}{\xi}} + \sqrt{2\E(t_{0})}\right) = \frac{1}{\theta}\left(\frac{1}{\sqrt{\xi}} + 1\right)\sqrt{2\E(t_{0})}, \label{eq:rate 1/t for velocity}
\end{align}

which gives the desired convergence rate.

§.§ Fast convergence rates for the primal-dual gap, the feasibility measure and the objective function value

The following are the main convergence rates results of the paper.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X}\times \mathcal{Y}$ be a solution of (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. The following statements are true:

* for every $t\geq t_{0}$ it holds

\begin{equation}
\label{eq:rate primal-dual gap}
0 \leq \mathcal{L}\left( x(t), \lambda_{*} \right) - \mathcal{L}(x_{*}, \lambda(t)) \leq \frac{\E(t_{0})}{\theta^{2} t^{2} \delta(t)};
\end{equation}

* for every $t\geq t_{0}$ it holds

\begin{equation}\label{eq:feasibility condition}
\left\lVert Ax \left( t \right) - b \right\rVert \leq \frac{2 C_{1}}{t^{2} \delta(t)},
\end{equation}

\[
C_{1} := \sup_{t\geq t_{0}} t \left\lVert \dot{\lambda} \left( t \right) \right\rVert + (\alpha - 1) \sup_{t\geq t_{0}} \left\lVert \lambda \left( t \right) - \lambda_{*} \right\rVert + t_{0}^{2} \delta(t_{0}) \left\lVert Ax(t_{0}) - b \right\rVert + t_{0}\left\lVert \dot{\lambda} \left( t_{0} \right) \right\rVert;
\]

* for every $t\geq t_{0}$ it holds

\begin{equation}\label{eq:functional values}
\left\lvert f \left( x \left( t \right) \right) - f_{*} \right\rvert \leq \left(\frac{\E(t_{0})}{\theta^{2}} + 2 C_{1} \left\lVert \lambda_{*} \right\rVert\right) \frac{1}{t^{2} \delta(t)}.
\end{equation}

(i) We have already established that $\E$ is nonincreasing on $\left[ t_{0}, +\infty \right)$. Therefore, from the expression for $\E$ and relation (<ref>) we deduce

\begin{equation}\label{eq:aux primal dual gap}
\theta^{2} t^{2} \delta(t) \bigl[\mathcal{L}\left( x(t), \lambda_{*} \right) - \mathcal{L}(x_{*}, \lambda(t)) \bigr] \leq \E(t_{0}) \quad \forall t\geq t_{0},
\end{equation}

and the first claim follows.

(ii) From the second line of (<ref>), for every $t\geq t_{0}$ we have

\begin{equation}\label{eq:alt proof rates 1}
t \ddot{\lambda}(t) + \alpha \dot{\lambda}(t) = t\delta(t) \Bigl(A\bigl(x(t) + \theta t \dot{x}(t)\bigr) - b\Bigr) = t\delta(t)\bigl( Ax(t) - b\bigr) + \theta t^{2} \delta(t) A\dot{x}(t).
\end{equation}

Fix $t\geq t_{0}$. On the one hand, integration by parts yields

\begin{equation}\label{eq:alt proof rates 2}
\begin{split}
\int_{t_{0}}^{t}\bigl( s \ddot{\lambda}(s) + \alpha \dot{\lambda}(s)\bigr)ds &= t \dot{\lambda}(t) - t_{0} \dot{\lambda}(t_{0}) - \int_{t_{0}}^{t}\dot{\lambda}(s)ds + \alpha \int_{t_{0}}^{t} \dot{\lambda}(s)ds \\
&= t \dot{\lambda}(t) - t_{0} \dot{\lambda}(t_{0}) + (\alpha - 1)(\lambda(t) - \lambda(t_{0})).
\end{split}
\end{equation}

On the other hand, again integrating by parts leads to

\begin{equation}\label{eq:alr proof rates 3}
\begin{split}
\int_{t_{0}}^{t}s^{2} \delta(s) A\dot{x}(s) ds =& \: t^{2} \delta(t) (Ax(t) - b) - t_{0}^{2} \delta(t_{0}) (Ax(t_{0}) - b) \\
&\: - \int_{t_{0}}^{t}\bigl( 2 s \delta(s) + s^{2} \dot{\delta}(s)\bigr) (Ax(s) - b)ds.
\end{split}
\end{equation}

Now, integrating (<ref>) from $t_{0}$ to $t$ and using (<ref>) and (<ref>) gives us

\begin{align}
&t \dot{\lambda}(t) - t_{0} \dot{\lambda}(t_{0}) + (\alpha - 1)(\lambda(t) - \lambda(t_{0})) \nonumber\\
= \: &\int_{t_{0}}^{t}s \delta(s) (Ax(s) - b) ds + \theta \int_{t_{0}}^{t} s^{2} \delta(s) A\dot{x}(s) ds \nonumber\\
= \: & t^{2} \delta(t) (Ax(t) - b) - t_{0}^{2} \delta(t_{0})(Ax(t_{0}) - b) + \int_{t_{0}}^{t} s \bigl[(1 - 2\theta)\delta(s) - \theta s \dot{\delta}(s)\bigr] (Ax(s) - b) ds \nonumber\\
= \: & t^{2} \delta(t) (Ax(t) - b) - t_{0}^{2} \delta(t_{0})(Ax(t_{0}) - b) + \int_{t_{0}}^{t}\frac{(1 - 2\theta)\delta(s) - \theta s \dot{\delta}(s)}{s \delta(s)} s^{2} \delta(s) (Ax(s) - b) ds.
\end{align}

It follows that, for every $t \geq t_{0}$, we have

\begin{equation}
\left\| t^{2} \delta(t) (Ax(t) - b) + \int_{t_{0}}^{t}\frac{(1 - 2\theta)\delta(s) - \theta s \dot{\delta}(s)}{s \delta(s)} s^{2} \delta(s) (Ax(s) - b) ds \right\| \leq C_{1},
\end{equation}

\[
C_{1} = \sup_{t\geq t_{0}} t\left\lVert \dot{\lambda} \left( t \right) \right\rVert + (\alpha - 1) \sup_{t\geq t_{0}}\| \lambda(t) - \lambda \left( t_{0} \right) \| + t_{0}^{2} \delta(t_{0}) \left\lVert Ax(t_{0}) - b \right\rVert + t_{0}\left\lVert \dot{\lambda} \left( t_{0} \right) \right\rVert < + \infty,
\]

and this quantity is finite in light of (<ref>) and (<ref>). Now, we set

\[
g(t) := t^{2} \delta(t) \left\lVert Ax \left( t \right) - b \right\rVert, \qquad a(t) := \frac{(1 - 2\theta)\delta(t) - \theta t \dot{\delta}(t)}{t \delta(t)} \qquad \forall t\geq t_{0}
\]

and we apply Lemma <ref> to deduce that

\begin{equation}\label{eq:aux feasibility condition}
t^{2} \delta(t) \left\lVert Ax \left( t \right) - b \right\rVert \leq 2C_{1} \quad \forall t\geq t_{0}.
\end{equation}

(iii) For a fixed $t\geq t_{0}$, we have

\[
\mathcal{L}\left( x(t), \lambda_{*} \right) - \mathcal{L}(x_{*}, \lambda(t)) = f(x(t)) - f(x_{*}) + \langle \lambda_{*}, Ax(t) - b\rangle.
\]

Therefore, from using (<ref>) and (<ref>) we obtain, for every $t\geq t_{0}$,

\begin{align*}
\left\lvert f \left( x \left( t \right) \right) - f_{*} \right\rvert &\leq \mathcal{L}\left( x(t), \lambda_{*} \right) - \mathcal{L}(x_{*}, \lambda(t)) + \left\lVert \lambda_{*} \right\rVert \left\lVert Ax \left( t \right) - b \right\rVert \\
&\leq \left(\frac{\E(t_{0})}{\theta^{2}} + 2C_{1} \left\lVert \lambda_{*} \right\rVert\right)\frac{1}{t^{2} \delta(t)},
\end{align*}

which leads to the last statement.

Some comments regarding the previous proof and results are in order.

The proof we provided here is significantly shorter than the one derived in [34] thanks to Lemma <ref>. This Lemma is inspired by the one used in [54] for showing the fast convergence to zero of the feasibility measure, although the authors study a different dynamical system.
On the other hand, when $\delta \left( t \right) \equiv 1$, the results in [34] is more robust than the one we obtain here, as it gives the $\bO \left( \frac{1}{t^{2}} \right)$ rates for the sum of primal-dual gap and feasibility measure, rather than each one individually.

Here are some remarks comparing our rates of convergence to those in [11, 55].

* Primal-dual gap: According to (<ref>), the following rate of convergence for the primal-dual is exhibited:

\[
\mathcal{L}\bigl(x(t), \lambda_{*}\bigr) - \mathcal{L}\bigl(x_{*}, \lambda(t)\bigr) = \mathcal{O}\left(\frac{1}{t^{2} \delta(t)}\right) \quad \text{as} \quad t\to +\infty,
\]

which coincides with the findings of [11, 55].

* Feasibility measure: According to (<ref>), we have

\[
\left\lVert Ax \left( t \right) - b \right\rVert = \mathcal{O}\left(\frac{1}{t^{2} \delta(t)}\right) \quad \text{as} \quad t\to +\infty,
\]

which improves the rate $\mathcal{O}\left( \frac{1}{t \sqrt{\delta(t)}}\right)$ reported in [11, 55].

* Functional values: Relation (<ref>) tells us that

\[
\left\lvert f \left( x \left( t \right) \right) - f_{*} \right\rvert = \mathcal{O}\left(\frac{1}{t^{2} \delta(t)}\right) \quad \text{as} \quad t\to +\infty.
\]

In [11], only the upper bound presents this order of convergence. The lower bound obtained is of order $\mathcal{O}\left(\frac{1}{t \sqrt{\delta(t)}}\right)$ as $t \to +\infty$. In [55], there are no comments on the rate attained by the functional values in the case of a general time rescaling parameter.

To further illustrate, notice that

\[
\delta(t) := \delta_{0} t^{n} \qquad \forall t\geq t_{0}
\]

fulfills Assumption <ref> provided that $\delta_{0} > 0, 0 \leq n \leq \frac{1 - 2\theta}{\theta} = \frac{1}{\theta} - 2$.
Therefore, all the statements derived are above are of order $\bO \left( \frac{1}{t^{1/\theta}} \right)$.
If we desire to obtain fast convergence rates, we must take $\theta$ small, which in the light of Assumption <ref> can be achieved by choosing large enough $\alpha$.
Such behavior can be seen in the unconstrained case [15] and other settings [55, 32].

§ WEAK CONVERGENCE OF THE TRAJECTORY TO A PRIMAL-DUAL SOLUTION

In this section we will show that the solutions to (<ref>) weakly converge to an element of $\mathbb{S}$.
The fact that $\delta \left( t \right)$ enters the convergence rate statement suggests that one can benefit from this time rescaling function when it is at least nondecreasing on $\left[ t_{0}, +\infty \right)$. We are, in fact, going to need this condition when showing trajectory convergence.

In (<ref>), assume that $\nabla f$ is $\ell$-Lipschitz continuous for some $\ell > 0$ and that $\delta : [t_{0}, +\infty) \to (0, +\infty)$ is continuously differentiable and nondecreasing. Moreover, suppose that the parameters $\alpha, \beta$, $\theta$ and the function $\delta$ satisfy

\begin{equation*}
\alpha > 3 , \quad \beta \geq 0 , \quad \dfrac{1}{2} > \theta > \dfrac{1}{\alpha - 1}, \quad \sup_{t\geq t_{0}}\frac{t \dot{\delta}(t)}{\delta(t)} < \frac{1 - 2\theta}{\theta} .
\end{equation*}

Assumption <ref> entails the existence of $C_{2} > 0$ such that

\begin{equation}\label{eq: assumption 2 fact 1}
\frac{t \dot{\delta}(t)}{\delta(t)} + C_{2} \leq \frac{1 - 2\theta}{\theta} \quad \forall t \geq t_{0}.
\end{equation}

and therefore it follows further from the nondecreasing property of $\delta$  that

\begin{equation}\label{eq: assumption 2 fact 2}
0 < C_{2} \delta(t_{0}) \leq C_{2} \delta(t) \leq (1 - 2\theta)\delta(t) - \theta t \dot{\delta}(t) \quad \forall t\geq t_{0} .
\end{equation}

Moreover, from (<ref>), for every $t\geq t_{0}$, we have

\[
0 < C_{2} \leq \frac{1 - 2\theta}{\theta} - \frac{t \dot{\delta}(t)}{\delta(t)} = \frac{\sigma(t)}{\delta(t)},
\]

which gives

\begin{equation} \label{eq:inequality for delta and sigma}
\delta(t) \leq \frac{\sigma(t)}{C_{2}} \quad\forall t\geq t_{0}.
\end{equation}

We also mention that in the setting of Assumption <ref>, the dynamical system (<ref>) has a unique global twice continuously differentiable solution.
The proof follows the same argument as in <cit.>, which relies on the fact that (<ref>) can be rewritten as a first-order dynamical system.
More precisely, $\left( x, \lambda \right) \colon \left[ t_{0}, + \infty \right) \to \mathcal{X} \times \mathcal{Y}$ is a solution to (<ref>) if and only if $(x, \lambda, y, \nu) \colon \left[ t_{0}, + \infty \right) \to \mathcal{X} \times \mathcal{Y} \times \mathcal{X} \times \mathcal{Y}$ is a solution to

\begin{equation*}
\begin{dcases}
\bigl(\dot{x}(t), \dot{\lambda}(t), \dot{y}(t), \dot{\nu}(t)\bigr) &= F \left( t, x(t), \lambda(t), y(t), \nu(t) \right) \\
\left( x(t_{0}), \lambda(t_{0}), y(t_{0}), \nu(t_{0}) \right) &= \left( x_{0}, \lambda_{0}, \dot{x}_{0}, \dot{\lambda}_{0} \right),
\end{dcases}
\end{equation*}

where $F \colon \left[ t_{0}, + \infty \right) \times \mathcal{X} \times \mathcal{Y} \times \mathcal{X} \times \mathcal{Y} \to \mathcal{X} \times \mathcal{Y} \times \mathcal{X} \times \mathcal{Y}$ is given by

\begin{align*}
&F(t, z, \mu, w, \eta) \\
&:= \left( w, \eta, -\frac{\alpha}{t}w - \delta(t)\left[ \nabla f(z) + A^{*}\bigl( \mu + \theta t \eta\bigr) + \beta A^{*}\bigl( Az - b\bigr)\right], -\frac{\alpha}{t}\eta + \delta(t)\bigl[ A\bigl( z + \theta t w\bigr) - b\bigr]\right).
\end{align*}

We omit the details proof and only state the result in the following theorem.

For every choice of initial conditions

\[
x(t_{0}) = x_{0}, \quad \lambda(t_{0}) = \lambda_{0}, \quad \dot{x}(t_{0}) = \dot{x}_{0} \quad \text{and} \quad \dot{\lambda}(t_{0}) = \dot{\lambda}_{0}
\]

the system (<ref>) has a unique global twice continuously differentiable solution $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$.

The additional Lipschitz continuity condition of $\nabla f$ and the fact that $\delta$ is nondecreasing give rise to the following two essential integrability statements.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ be a solution of (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. Then it holds

\begin{equation}\label{eq:integral for gradient}
\int_{t_{0}}^{+\infty}t\delta(t) \left\|\nabla f(x(t)) - \nabla f(x_{*})\right\|^{2} dt < +\infty
\end{equation}

\begin{equation}\label{eq:integral for feasibility condition}
\int_{t_{0}}^{+\infty} t\delta(t) \left\lVert Ax \left( t \right) - b \right\rVert^{2} dt < +\infty.
\end{equation}

Thanks to $\nabla f$ being $\ell$-Lipschitz continuous, we can use (<ref>) to refine relation (<ref>) in the proof of Lemma <ref>

\begin{align*}
&- \left\langle \left( \nabla_{x} \Lb \left( x \left( t \right) , \lambda_{*} \right) , 0 \right) , \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) - (x_{*}, \lambda_{*}) \right\rangle \\
= \     & \langle \nabla f(x(t)), x_{*} - x(t)\rangle + \left\langle A^{*}\lambda_{*}, x_{*} - x(t)\right\rangle + \beta\left\langle A^{*}(Ax(t) - b), x_{*} - x(t)\right\rangle \\
\leq \  & - (f(x(t)) - f(x_{*})) - \frac{1}{2\ell}\left\| \nabla f(x(t)) - \nabla f(x_{*})\right\|^{2} - \langle \lambda_{*}, Ax(t) - b\rangle - \beta \left\lVert Ax \left( t \right) - b \right\rVert^{2} \\
= \     & - \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) - \frac{1}{2\ell} \left\| \nabla f(x(t)) - \nabla f(x_{*})\right\|^{2} - \frac{\beta}{2}\left\lVert Ax \left( t \right) - b \right\rVert^{2}.
\end{align*}

Consequently, combining this inequality with (<ref>) yields, for every $t\geq t_{0}$

\begin{align*}
\frac{d}{dt}\E(t) \leq& - \theta^{2} t \sigma(t)  \left( \Lb \left( x \left( t \right) , \lambda_{*} \right) - \Lb \left( x_{*} , \lambda \left( t \right) \right) \right) - \xi \theta t \left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert^{2} \\
&- \frac{\theta t\delta(t)}{2\ell}\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} - \frac{\theta \beta t \delta(t)}{2}\left\lVert Ax \left( t \right) - b \right\rVert^{2} \\
\leq& - \frac{\theta t \delta(t)}{2\ell} \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2}.
\end{align*}

Integration of this inequality produces (<ref>).

The finiteness of the second integral is only entailed by (<ref>) when $\beta > 0$. For the general case $\beta \geq 0$, we use (<ref>) and the fact that $\delta$ is nondecreasing on $\left[ t_{0}, +\infty \right)$ to obtain

\[
\int_{t_{0}}^{+ \infty} t\delta(t) \left\lVert Ax \left( t \right) - b \right\rVert^{2} dt \leq 4 C_{1}^{2} \int_{t_{0}}^{+ \infty} \frac{1}{t^{3} \delta(t)} dt \leq \frac{4 C_{1}^{2}}{\delta(t_{0})} \int_{t_{0}}^{+ \infty} \dfrac{1}{t^{3}} dt < + \infty,
\]

and the proof is complete.

Now, for a given primal-dual solution $(x_{*}, \lambda_{*}) \in \mathbb{S}$, we define the following mappings on $\left[ t_{0}, +\infty \right)$

\begin{align}
W(t) &:= \delta(t) \bigl[\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\bigr] + \frac{1}{2}\left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert^{2} \geq 0, \label{eq:formula for W}\\
\varphi(t) &:= \frac{1}{2}\left\|\bigl(x(t), \lambda(t)\bigr) - (x_{*}, \lambda_{*})\right\|^{2} \geq 0.
\end{align}

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ a solution of (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. The following inequality holds for every $t\geq t_{0}$:

\begin{equation}
\ddot{\varphi}(t) + \frac{\alpha}{t} \dot{\varphi}(t) + \theta t \dot{W}(t) + \frac{\delta(t)}{2\ell}\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} + \frac{\beta \delta(t)}{2}\left\lVert Ax \left( t \right) - b \right\rVert^{2} \leq 0.
\end{equation}

Fix $t\geq t_{0}$. Differentiating $W$ with respect to time yields

\begin{align}
\dot{W}(t) = \: &\dot{\delta}(t)\Bigl[\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\Bigr] \nonumber\\
&+ \delta(t) \Bigl[\left\langle \nabla_{x}\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right), \dot{x}(t)\right\rangle - \left\langle \nabla_{\lambda}\mathcal{L}_{\beta}(x_{*}, \lambda(t)), \dot{\lambda}(t)\right\rangle\Bigr] \nonumber\\
&+ \langle \ddot{x}(t), \dot{x}(t)\rangle + \langle \ddot{\lambda}(t), \dot{\lambda}(t)\rangle. \nonumber
\end{align}

Recall the formulas for the gradients of $\mathcal{L}$:

\begin{align*}
\nabla_{x}\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) &= \nabla f(x(t)) + A^{*}\lambda_{*} + \beta A^{*}(Ax(t) - b), \\
\nabla_{\lambda}\mathcal{L}_{\beta}(x_{*}, \lambda(t)) &= Ax_{*} - b = 0,
\end{align*}

since $x_{*} \in \mathbb{F}$.
Plugging this into the expression for $\dot{W}(t)$ gives us

\begin{align*}
\dot{W}(t) = \: &\dot{\delta}(t)\Bigl[\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\Bigr] \\
&+ \delta(t) \left\langle \nabla_{x}\mathcal{L}_{\beta}(x(t), \lambda(t) + \theta t \dot{\lambda}(t)), \dot{x}(t)\right\rangle + \langle \ddot{x}(t), \dot{x}(t)\rangle \\
&- \delta(t) \left\langle \lambda(t) - \lambda_{*} + \theta t \dot{\lambda}(t), A\dot{x}(t)\right\rangle \\
&- \delta(t) \left\langle \nabla_{\lambda}\mathcal{L}_{\beta}(x(t) + \theta t \dot{x}(t), \lambda(t)), \dot{\lambda(t)}\right\rangle + \langle \ddot{\lambda}(t), \dot{\lambda}(t)\rangle \\
&+ \delta(t) \left\langle Ax(t) - b + \theta t A\dot{x}(t), \dot{\lambda}(t)\right\rangle.
\end{align*}

By regrouping and using (<ref>), we arrive at

\begin{equation}
\label{eq: form of W prime}
\begin{split}
\dot{W}(t) = \: &\dot{\delta}(t)\Bigl[\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\Bigr] \\
&- \frac{\alpha}{t}\left\lVert \dot{x} \left( t \right) \right\rVert^{2} - \frac{\alpha}{t}\|\dot{\lambda}(t)\|^{2} - \delta(t) \left\langle \lambda(t) - \lambda_{*}, A\dot{x}(t)\right\rangle + \delta(t) \bigl\langle Ax(t) - b, \dot{\lambda}(t)\bigr\rangle.
\end{split}
\end{equation}

On the other hand, by the chain rule, we have

\begin{align*}
\dot{\varphi}(t) &= \langle x(t) - x_{*}, \dot{x}(t)\rangle + \bigl\langle \lambda(t) - \lambda_{*}, \dot{\lambda}(t)\bigr\rangle, \\
\ddot{\varphi}(t) &= \langle x(t) - x_{*}, \ddot{x}(t)\rangle + \left\lVert \dot{x} \left( t \right) \right\rVert^{2} + \bigl\langle \lambda(t) - \lambda_{*}, \ddot{\lambda}(t)\bigr\rangle + \bigl\|\dot{\lambda}(t)\bigr\|^{2}.
\end{align*}

By combining these relations, (<ref>) and the fact that $x_{*} \in \mathbb{F}$, we get

\begin{align}
\ddot{\varphi}(t) + \frac{\alpha}{t}\dot{\varphi}(t) = \: &\left\langle x(t) - x_{*}, \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t)\right\rangle + \left\langle \lambda(t) - \lambda_{*}, \ddot{\lambda}(t) + \frac{\alpha}{t}\dot{\lambda}(t)\right\rangle
+ \left\lVert \dot{x} \left( t \right) \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2} \nonumber\\
= \: &- \bigl\langle x(t) - x_{*}, \delta(t) \nabla_{x}\mathcal{L}_{\beta}(x(t), \lambda(t) + \theta t \dot{\lambda}(t))\bigr\rangle \nonumber\\
&+ \bigl\langle \lambda(t) - \lambda_{*}, \delta(t) \nabla_{\lambda}\mathcal{L}_{\beta}(x(t) + \theta t \dot{\lambda}(t), \lambda(t))\bigr\rangle + \left\lVert \dot{x} \left( t \right) \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2} \nonumber\\
= \: &-\bigl\langle x(t) - x_{*}, \delta(t) \nabla_{x} \mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right)\bigr\rangle - \bigl\langle Ax(t) - b, \delta(t) \bigl(\lambda(t) - \lambda_{*} + \theta t \dot{\lambda}(t)\bigr)\bigr\rangle \nonumber\\
&+ \bigl\langle \lambda(t) - \lambda_{*}, \delta(t) \bigl(Ax(t) - b + \theta t A\dot{x}(t)\bigr)\bigr\rangle + \left\lVert \dot{x} \left( t \right) \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2} \nonumber \\
= \: &-\delta(t) \bigl\langle x(t) - x_{*}, \nabla_{x} \mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right)\bigr\rangle \nonumber\\
&-\theta t \delta(t) \bigl\langle Ax(t) - b, \dot{\lambda}(t)\bigr\rangle + \theta t \delta(t) \bigl\langle \lambda(t) - \lambda_{*}, A \dot{x}(t)\bigr\rangle \nonumber\\
&+ \left\lVert \dot{x} \left( t \right) \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2} \label{eq: form of phi}
\end{align}

The Lipschitz continuity of $\nabla f$ entails

\begin{align*}
&-\left\langle x(t) - x_{*}, \nabla_{x} \mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) \right\rangle \\
&= -\langle x(t) - x_{*}, \nabla f(x(t))\rangle - \bigl\langle x(t) - x_{*}, A^{*}\lambda_{*}\bigr\rangle - \beta \left\lVert Ax \left( t \right) - b \right\rVert^{2} \\
&\leq -(f(x(t)) - f(x_{*})) - \frac{1}{2\ell} \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} - \langle \lambda_{*}, Ax(t) - b\rangle - \beta\left\lVert Ax \left( t \right) - b \right\rVert^{2} \\
&= -\Bigl( \mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\Bigr) - \frac{1}{2\ell}\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} - \frac{\beta}{2}\left\lVert Ax \left( t \right) - b \right\rVert^{2}.
\end{align*}

This, together with (<ref>), implies

\begin{align}
\ddot{\varphi}(t) + \frac{\alpha}{t}\dot{\varphi}(t) \leq \: &- \delta(t)\Bigl( \mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\Bigr) - \theta t \delta(t) \bigl\langle Ax(t) - b, \dot{\lambda}(t)\bigr\rangle \nonumber\\
&+ \theta t \delta(t) \bigl\langle \lambda(t) - \lambda_{*}, A\dot{x}(t)\bigr\rangle + \left\lVert \dot{x} \left( t \right) \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2} \nonumber\\
&- \frac{\delta(t)}{2\ell}\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} - \frac{\beta \delta(t)}{2} \left\lVert Ax \left( t \right) - b \right\rVert^{2}. \label{eq: inequality for phi}
\end{align}

Multiplying (<ref>) by $\theta t > 0$ and then adding the result to (<ref>) yields

\begin{align*}
\ddot{\varphi}(t) + \frac{\alpha}{t}\dot{\varphi}(t) + \theta t \dot{W}(t) = \: &-\bigl(\delta(t) - \theta t \dot{\delta}(t)\bigr) \Bigl(\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\Bigr) \\
&-\frac{\delta(t)}{2\ell}\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} - \frac{\beta \delta(t)}{2}\left\lVert Ax \left( t \right) - b \right\rVert^{2} \\
&+(1 - \theta \alpha) \left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert^{2} \\
&\leq -\frac{\delta(t)}{2\ell}\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} - \frac{\beta \delta(t)}{2}\left\lVert Ax \left( t \right) - b \right\rVert^{2},
\end{align*}

where the last inequality follows from Assumption <ref>:

\begin{align*}
1 - \theta \alpha & \leq -\theta < 0 , \\
- \delta(t) + \theta t \dot{\delta}(t) &\leq (2\theta - 1)\delta(t) + \theta t \dot{\delta}(t) \leq 0.
\end{align*}

The desired result then follows after some rearranging.

The following lemma ensures that the first condition of Opial's Lemma is met.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ be a solution to (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. Then the positive part $[\dot{\varphi}]_{+}$ of $\dot{\varphi}$ belongs to $\sL^{1}\left[ t_{0}, +\infty \right)$ and the limit $\lim_{t \to +\infty} \varphi(t)$ exists.

For any $t\geq t_{0}$, we multiply (<ref>) by $t$ and drop the last two norm squared terms to obtain

\[
t \ddot{\varphi}(t) + \alpha \dot{\varphi}(t) + \theta t^{2} \dot{W}(t) \leq 0.
\]

Recall from (<ref>) that for every $t\geq t_{0}$ we have

\begin{equation}\label{eq:formula for tW(t)}
tW(t) = t\delta(t) \bigl[\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\bigr] + \frac{t}{2}\left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert^{2}.
\end{equation}

On the one hand, according to (<ref>), the second summand of the previous expression belongs to $\sL^{1}\left[ t_{0}, +\infty \right)$. On the other hand, using (<ref>) and (<ref>), we assert that

\begin{equation*}
\int_{t_{0}}^{+\infty} t \delta(t) \bigl[\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\bigr] dt
\leq \frac{1}{C_{2}} \int_{t_{0}}^{+\infty} t \sigma(t) \bigl[\mathcal{L}_{\beta}\left( x(t), \lambda_{*} \right) - \mathcal{L}_{\beta}(x_{*}, \lambda(t))\bigr] dt
< +\infty.
\end{equation*}

Hence, the first summand of (<ref>) also belongs to $ \sL^{1}\left[ t_{0}, +\infty \right)$, which implies that the mapping $t \mapsto tW(t)$ belongs to $\sL^{1}\left[ t_{0}, +\infty \right)$ as well. For achieving the desired conclusion, we make use of Lemma <ref> with $\phi := \varphi$ and $w := \theta W$.

The following results guarantee that the second condition of Opial's Lemma is also met.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ be a solution to (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. The following inequality holds for every $t\geq t_{0}$

\begin{align*}
&\frac{\alpha}{t \delta(t)} \frac{d}{dt}\left\lVert \left( \dot{x} \left( t \right) , \dot{\lambda} \left( t \right) \right) \right\rVert^{2} + 2\left\langle \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\right\rangle \\
&+\theta \frac{d}{dt}\Bigl(t \delta(t) \left\| A^{*}(\lambda(t) - \lambda_{*})\right\|^{2}\Bigr) + \bigl((1 - \theta)\delta(t) - \theta t \dot{\delta}(t)\bigr) \left\|A^{*}(\lambda(t) - \lambda_{*})\right\|^{2} \\
\leq \:\: &\delta(t) \Bigl[ 2\| \nabla f(x(t)) - \nabla f(x_{*})\|^{2} + \left(2\beta^{2} \|A\|^{2} + 1\right)\left\lVert Ax \left( t \right) - b \right\rVert^{2}\Bigr].
\end{align*}

Let $t \geq t_{0}$ be fixed. From (<ref>) and the fact that $A^{*}\lambda_{*} = -\nabla f(x_{*})$, we have

\begin{align}
& \delta^{2} \left( t \right) \bigl\| \nabla f(x(t)) - \nabla f(x_{*}) + \beta A^{*}(Ax(t) - b)\bigr\|^{2} \nonumber\\
= \     & \left\|\ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) + \delta(t) A^{*}\bigl(\lambda(t) - \lambda_{*} + \theta t \dot{\lambda}(t)\bigr)\right\|^{2} \nonumber\\
= \     & \left\| \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t)\right\|^{2} + \delta^{2}(t) \bigl\| A^{*}\bigl( \lambda(t) - \lambda_{*} + \theta t \dot{\lambda}(t)\bigr)\bigr\|^{2} \nonumber\\
& \quad + 2 \delta(t) \left\langle \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t), A^{*}\bigl(\lambda(t) - \lambda_{*}\bigr)\right\rangle + 2\theta t \delta(t) \bigl\langle \ddot{x}(t), A^{*} \dot{\lambda}(t)\bigr\rangle + 2\alpha \theta t \delta(t) \bigl\langle \dot{x}(t), A^{*} \dot{\lambda}(t)\bigr\rangle. \label{eq:technical inequality 1}
\end{align}

Again using (<ref>) yields

\begin{align}
\delta^{2} \left( t \right) \left\lVert Ax \left( \left( t \right) \right) - b \right\rVert ^{2} = \: &\left\| \ddot{\lambda}(t) + \frac{\alpha}{t}\dot{\lambda}(t) - \theta t \delta(t) A\dot{x}(t)\right\|^{2} \nonumber\\
= \: &\left\| \ddot{\lambda}(t) + \frac{\alpha}{t}\dot{\lambda}(t)\right\|^{2} + \theta^{2} t^{2} \delta^{2} \left( t \right) \left\lVert A \dot{x} \left( t \right) \right\rVert^{2} \nonumber\\
& -2\theta t \delta(t) \bigl\langle \ddot{\lambda}(t), A\dot{x}(t)\bigr\rangle - 2\alpha\theta \delta(t) \bigl\langle \dot{\lambda}(t), A\dot{x}(t)\bigr\rangle \label{eq:technical inequality 2}
\end{align}

Adding (<ref>) and (<ref>) together produces

\begin{align}
& \delta^{2} \left( t \right) \bigl\| \nabla f(x(t)) - \nabla f(x_{*}) + \beta A^{*}(Ax(t) - b)\bigr\|^{2} + \delta^{2} \left( t \right) \left\lVert Ax \left( \left( t \right) \right) - b \right\rVert ^{2} \nonumber\\
&= \left\|\bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr) + \frac{\alpha}{t}\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\right\|^{2} + \delta^{2}(t)\bigl\| A^{*}\bigl( \lambda(t) - \lambda_{*} + \theta t \dot{\lambda}(t)\bigr)\bigr\|^{2} + \theta^{2} t^{2} \delta^{2}(t) \left\lVert A \dot{x} \left( t \right) \right\rVert^{2} \nonumber\\
&\quad + 2 \theta t \delta(t) \bigl\langle \ddot{x}(t), A^{*}\dot{\lambda}(t)\bigr\rangle - 2 \theta t \delta(t) \bigl\langle \ddot{\lambda}(t), A\dot{x}(t)\bigr\rangle + 2 \delta(t) \left\langle \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\right\rangle. \label{eq:technical inequality 3}
\end{align}

On the one hand, we have

\begin{align}
&\theta^{2} t^{2} \delta^{2}(t) \left\lVert A \dot{x} \left( t \right) \right\rVert^{2} + 2\theta t \delta(t) \bigl\langle \ddot{x}(t), A^{*}\dot{\lambda}(t)\bigr\rangle - 2 \theta t \delta(t) \bigl\langle \ddot{\lambda}(t), A\dot{x}(t)\bigr\rangle \nonumber\\
= \     & \theta^{2} t^{2} \delta^{2}(t) \bigl\| \bigl(A^{*}\dot{\lambda}(t), -A\dot{x}(t)\bigr)\bigr\|^{2} - \theta^{2} t^{2} \delta^{2}(t) \bigl\|A^{*} \dot{\lambda}(t)\bigr\|^{2} + 2 \theta t \delta(t) \bigl\langle \bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr), \bigl( A^{*}\dot{\lambda}(t), - A\dot{x}(t)\bigr)\bigr\rangle \nonumber\\
= \     & - \bigl\|\bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr)\bigr\|^{2} + \bigl\|\bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr) + \theta t \delta(t) A^{*}\bigl( A^{*}\dot{\lambda}(t), -A\dot{x}(t)\bigr)\bigr\|^{2} - \theta^{2} t^{2} \delta^{2}(t) \bigl\|A^{*}\dot{\lambda}(t)\bigr\|^{2} \nonumber\\
\geq \  & - \bigl\| \bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr)\bigr\|^{2} - \theta^{2} t^{2} \delta^{2}(t) \bigl\|A^{*}\dot{\lambda}(t)\bigr\|^{2}. \label{eq:technical inequality 4}
\end{align}

On the other hand, it holds

\begin{align}
&\left\| \bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr) + \frac{\alpha}{t}\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\right\|^{2} - \bigl\|\bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr)\bigr\|^{2} \nonumber\\
= \     & \frac{\alpha^{2}}{t^{2}} \bigl\|\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\bigr\|^{2} + 2\frac{\alpha}{t}\bigl\langle \bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr), \bigl( \dot{x}(t), \dot{\lambda}(t)\bigr)\bigr\rangle \geq \frac{\alpha}{t} \frac{d}{dt}\bigl\| \bigl( \dot{x}(t), \dot{\lambda}(t)\bigr)\bigr\|^{2}. \label{eq:technical inequality 5}
\end{align}

\begin{align}
&\delta(t)\bigl\| A^{*}\bigl( \lambda(t) - \lambda_{*} + \theta t \dot{\lambda}(t)\bigr)\bigr\|^{2} - \theta^{2} t^{2} \delta(t) \bigl\|A^{*} \dot{\lambda}(t)\bigr\|^{2} \nonumber\\
= \     & \delta(t)\bigl\| A^{*}( \lambda(t) - \lambda_{*})\bigr\|^{2} + 2\theta t \delta(t) \bigl\langle A A^{*}( \lambda(t) - \lambda_{*}), \dot{\lambda(t)}\bigr\rangle \nonumber\\
= \     & \bigl((1 - \theta)\delta(t) - \theta t \dot{\delta}(t)\bigr) \bigl\| A^{*} \bigl( \lambda(t) - \lambda_{*}\bigr)\bigr\|^{2} + \theta \delta(t) \bigl\| A^{*}( \lambda(t) - \lambda_{*})\bigr\|^{2} \nonumber\\
&\quad + \theta t \dot{\delta}(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} + \theta \frac{d}{dt} \bigl\| A^{*}(\lambda(t) - \lambda_{*})\|^{2} \nonumber\\
= \     & \bigl((1 - \theta)\delta(t) - \theta t \dot{\delta}(t)\bigr) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} + \theta \frac{d}{dt}\Bigl(t \delta(t) \bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2}\Bigr). \label{eq:technical inequality 6}
\end{align}

Now, using (<ref>), (<ref>) and (<ref>) in (<ref>) yields

\begin{align*}
& \delta^{2} \left( t \right) \left( t \right) \bigl\| \nabla f(x(t)) - \nabla f(x_{*}) + \beta A^{*}(Ax(t) - b)\bigr\|^{2} + \delta^{2} \left( t \right) \left\lVert Ax \left( \left( t \right) \right) - b \right\rVert ^{2} \\
\geq \  & \left\|\bigl(\ddot{x}(t), \ddot{\lambda}(t)\bigr) + \frac{\alpha}{t}\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\right\|^{2} + \delta^{2}(t)\bigl\| A^{*}\bigl( \lambda(t) - \lambda_{*} + \theta t \dot{\lambda}(t)\bigr)\bigr\|^{2} \\
&\quad - \bigl\| \bigl( \ddot{x}(t), \ddot{\lambda}(t)\bigr)\bigr\|^{2} - \theta^{2} t^{2} \delta^{2}(t) \bigl\|A^{*}\dot{\lambda}(t)\bigr\|^{2} + 2 \delta(t) \left\langle \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\right\rangle \\
\geq \  & \frac{\alpha}{t}\frac{d}{dt}\bigl\|\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\bigr\|^{2} + 2\delta(t) \left\langle \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\right\rangle \\
&\quad +\delta(t) \left[ \bigl((1 - \theta)\delta(t) - \theta t \dot{\delta}(t)\bigr)\bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2} + \theta \frac{d}{dt} \Bigl( t \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2}\Bigr)\right]
\end{align*}

Finally, since

\begin{align*}
&\bigl\| \nabla f(x(t)) - \nabla f(x_{*}) + \beta A^{*}(Ax(t) - b)\bigr\|^{2} + \left\lVert Ax \left( \left( t \right) \right) - b \right\rVert ^{2} \\
\leq \  & 2\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} + (2\beta \|A\|^{2} + 1) \left\lVert Ax \left( t \right) - b \right\rVert^{2},
\end{align*}

the conclusion follows after dividing the inequality by $\delta(t)$.

The following proposition provides us with the main integrability result that will be used for verifying the second condition of Opial's Lemma.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ be a solution to (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. Then it holds

\begin{equation}\label{eq:integral for dual variable}
\int_{t_{0}}^{+\infty} t \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} dt < +\infty.
\end{equation}

Recall that from Lemma <ref>, for every $t\geq t_{0}$, we have

\[
\ddot{\varphi}(t) + \frac{\alpha}{t}\dot{\varphi}(t) + \theta t \dot{W}(t) \leq - \frac{\delta(t)}{2\ell}\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} - \frac{\beta \delta(t)}{2}\left\lVert Ax \left( t \right) - b \right\rVert^{2}.
\]

Now, to this inequality we add the one produced by Lemma <ref>. For every $t\geq t_{0}$, it holds

\begin{align}
&\ddot{\varphi}(t) + \frac{\alpha}{t}\dot{\varphi}(t) + \theta t \dot{W}(t) + \frac{\alpha}{t\delta(t)}\bigl\|\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\bigr\|^{2} \nonumber\\
&+ \theta \frac{d}{dt}\Bigl(t\delta(t) \bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2}\Bigr) + 2\left\langle \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\right\rangle \nonumber\\
\leq \: &-\bigl( (1 - \theta)\delta(t) - \theta t \dot{\delta}(t)\bigr)\bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2} + \left(2 - \frac{1}{2\ell}\right) \delta(t) \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} \nonumber\\
&+\left(2\beta^{2} \|A\|^{2} + 1 - \frac{\beta}{2}\right) \delta(t) \left\lVert Ax \left( t \right) - b \right\rVert^{2} \nonumber\\
\leq \: &-\bigl( (1 - \theta)\delta(t) - \theta t \dot{\delta}(t)\bigr)\bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2} + C_{3} \delta(t) \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} \nonumber\\
&+C_{4} \delta(t) \left\lVert Ax \left( t \right) - b \right\rVert^{2}, \label{eq: inequality we will have to integrate}
\end{align}

\[
C_{3} := \left[2 - \frac{1}{2\ell}\right]_{+} \geq 0 \quad \text{and} \quad C_{4} := \left[2\beta^{2}\|A\|^{2} + 1 - \frac{\beta}{2}\right]_{+} \geq 0.
\]

Mutiplying (<ref>) by $t^{\alpha}$ and integrating from $t_{0}$ to $t$, we obtain

\begin{align}
&I_{1}(t) + \theta I_{2}(t) + \alpha I_{3}(t) + \theta I_{4}(t) + 2 I_{5}(t) \nonumber\\
&\leq \int_{t_{0}}^{t}s^{\alpha} \bigl((1 - \theta)\delta(s) - \theta s \dot{\delta}(s)\bigr) \bigl\|A^{*}(\lambda(s) - \lambda_{*})\bigr\|^{2} ds + C_{3} \int_{t_{0}}^{t} s^{\alpha} \delta(s) \|\nabla f(x(s)) - \nabla f(x_{*})\|^{2} ds \nonumber\\
&\quad + C_{4}\int_{t_{0}}^{t}s^{\alpha} \delta(s) \|Ax(s) - b\|^{2} ds, \label{eq: inequality for I_1, I_2, I_3, I_4, and I_5}
\end{align}

\begin{align*}
I_{1}(t) &:= \int_{t_{0}}^{t}\bigl(s^{\alpha} \ddot{\varphi}(s) + \alpha s^{\alpha - 1}\dot{\varphi}(s)\bigr) ds, \\
I_{2}(t) &:= \int_{t_{0}}^{t} s^{\alpha + 1}\dot{W}(s) ds, \\
I_{3}(t) &:= \int_{t_{0}}^{t} \frac{s^{\alpha - 1}}{\delta(s)} \frac{d}{ds} \bigl\|\bigl(\dot{x}(s), \dot{\lambda}(s)\bigr)\bigr\|^{2} ds, \\
I_{4}(t) &:= \int_{t_{0}}^{t} s^{\alpha} \frac{d}{ds}\Bigl(s\delta(s) \bigl\|A^{*}(\lambda(s) - \lambda_{*})\bigr\|^{2}\Bigr) ds, \\
I_{5}(t) &:= \int_{t_{0}}^{t}\bigl\langle s^{\alpha} \ddot{x}(s) + \alpha s^{\alpha - 1}\dot{x}(s), A^{*}(\lambda(s) - \lambda_{*})\bigr\rangle ds.
\end{align*}

We will furnish five different inequalities from computing each of these integrals separately. Let $t\geq t_{0}$ be fixed.

$\bullet$ The integral $I_{1}(t)$. By the chain rule, for $s \geq t_{0}$ it holds

\[
s^{\alpha}\ddot{\varphi}(s) + \alpha s^{\alpha - 1}\dot{\varphi}(s) = \frac{d}{ds}\bigl( s^{\alpha} \dot{\varphi}(s)\bigr),
\]

which leads to

\begin{equation}\label{eq: inequality for I_1}
0 = I_{1}(t) - t^{\alpha}\dot{\varphi}(t) + t_{0}^{\alpha}\dot{\varphi}(t_{0}) \leq I_{1}(t) - t^{\alpha}\dot{\varphi}(t) + |t_{0}^{\alpha}\dot{\varphi}(t_{0})|.
\end{equation}

$\bullet$ The integral $I_{2}(t)$. Integration by parts gives

\[
I_{2}(t) = t^{\alpha + 1}W(t) - t_{0}^{\alpha + 1}W(t_{0}) - (\alpha + 1)\int_{t_{0}}^{t}s^{\alpha}W(s)ds,
\]

which implies

\begin{equation}\label{eq: inequality for I_2}
0 \leq t^{\alpha + 1}W(t) = I_{2}(t) + t_{0}^{\alpha + 1}W(t_{0}) + (\alpha + 1)\int_{t_{0}}^{t}s^{\alpha}W(s)ds.
\end{equation}

$\bullet$ The integral $I_{3}(t)$. Again by integrating by parts, we get

\begin{align*}
I_{3}(t) = \: &\frac{t^{\alpha - 1}}{\delta(t)} \bigl\|\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\bigr\|^{2} - \frac{t_{0}^{\alpha - 1}}{\delta(t_{0})} \bigl\|\bigl(\dot{x}(t_{0}), \dot{\lambda}(t_{0})\bigr)\bigr\|^{2} \\
&- \int_{t_{0}}^{t} \left[ \frac{(\alpha - 1) s^{\alpha - 2} \delta(s) - s^{\alpha - 1}\dot{\delta}(s)}{\delta^{2}(s)}\right] \bigl\|\bigl(\dot{x}(s), \dot{\lambda}(s)\bigr)\bigr\|^{2} ds.
\end{align*}

For $s\geq t_{0}$, according to Assumption <ref> we have $\dot{\delta}(s) \geq 0$, hence $\delta$ is monotonically increasing and therefore

\[
\frac{(\alpha - 1) s^{\alpha - 2} \delta(s) - s^{\alpha - 1} \delta(s)}{\delta^{2}(s)} \leq \frac{(\alpha - 1)s^{\alpha - 2} \delta(s)}{\delta^{2}(s)} \leq \frac{(\alpha - 1)s^{\alpha}}{t_{0}^{2} \delta(t_{0})}.
\]

It follows that

\begin{align}
0 &\leq \frac{t^{\alpha - 1}}{\delta(t)}\bigl\|\bigl(\dot{x}(t), \dot{\lambda}(t)\bigr)\bigr\|^{2} \nonumber\\
&= I_{3}(t) + \frac{t_{0}^{\alpha - 1}}{\delta(t_{0})}\bigl\|\bigl(\dot{x}(t_{0}), \dot{\lambda}(t_{0})\bigr)\bigr\|^{2} + \int_{t_{0}}^{t} \left[ \frac{(\alpha - 1) s^{\alpha - 2} \delta(s) - s^{\alpha - 1}\dot{\delta}(s)}{\delta^{2}(s)}\right] \bigl\|\bigl(\dot{x}(s), \dot{\lambda}(s)\bigr)\bigr\|^{2} ds \nonumber\\
&\leq I_{3}(t) + \frac{t_{0}^{\alpha - 1}}{\delta(t_{0})}\bigl\|\bigl(\dot{x}(t_{0}), \dot{\lambda}(t_{0})\bigr)\bigr\|^{2} + \frac{\alpha - 1}{t_{0}^{2} \delta(t_{0})}\int_{t_{0}}^{t} s^{\alpha} \bigl\| \bigl( \dot{x}(s), \dot{\lambda}(s)\bigr)\bigr\|^{2} ds. \label{eq: inequality for I_3}
\end{align}

$\bullet$ The integral $I_{4}(t)$. Yet again, integration by parts produces

\begin{equation*}
I_{4}(t) = t^{\alpha + 1} \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} - t_{0}^{\alpha + 1} \delta(t_{0}) \bigl\| A^{*}(\lambda(t_{0}) - \lambda_{*})\bigr\|^{2} - \alpha \int_{t_{0}}^{t}s^{\alpha} \delta(s) \left\lVert A^{*}(\lambda(s) - \lambda_{*}) \right\rVert^{2} ds,
\end{equation*}

and from here

\begin{equation}\label{eq: inequality for I_4}
t^{\alpha + 1} \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} = I_{4}(t) + t_{0}^{\alpha + 1} \delta(t_{0}) \bigl\| A^{*}(\lambda(t_{0}) - \lambda_{*})\bigr\|^{2} + \alpha \int_{t_{0}}^{t}s^{\alpha} \delta(s) \left\lVert A^{*}(\lambda(s) - \lambda_{*}) \right\rVert^{2} ds.
\end{equation}

$\bullet$ The integral $I_{5}(t)$. Integration by parts entails

\begin{align*}
I_{5}(t) &= \int_{t_{0}}^{t} \left\langle \frac{d}{ds}\bigl( s^{\alpha} \dot{x}(s)\bigr), A^{*}(\lambda(s) - \lambda_{*})\right\rangle ds \\
&= t^{\alpha} \bigl\langle \dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\bigr\rangle - t_{0}^{\alpha} \bigl\langle \dot{x}(t_{0}), A^{*}(\lambda(t_{0}) - \lambda_{*})\bigr\rangle - \int_{t_{0}}^{t} s^{\alpha} \bigl\langle \dot{x}(s), A^{*}\dot{\lambda}(s)\bigr\rangle ds.
\end{align*}

By the Cauchy-Schwarz inequality, we deduce that

\[
\int_{t_{0}}^{t} s^{\alpha} \bigl\langle \dot{x}(s), A^{*}\dot{\lambda}(s)\bigr\rangle ds \leq \left\lVert A \right\rVert \int_{t_{0}}^{t} s^{\alpha} \left\lVert \dot{x}(s) \right\rVert \left\lVert \dot{\lambda}(s) \right\rVert ds \leq \dfrac{\left\lVert A \right\rVert}{2} \int_{t_{0}}^{t} s^{\alpha}\Bigl( \bigl\|\dot{x}(s)\bigr\|^{2} + \bigl\| \dot{\lambda}(s)\bigr\|^{2}\Bigr) ds,
\]

and thus

\begin{equation}\label{eq: inequality for I_5}
\begin{split}
0 \leq \: &I_{5}(t) - t^{\alpha} \bigl\langle \dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\bigr\rangle +  \bigl|t_{0}^{\alpha} \bigl\langle \dot{x}(t_{0}), A^{*}(\lambda(t_{0}) - \lambda_{*})\bigr\rangle\bigr| \\
&+ \dfrac{\left\lVert A \right\rVert}{2} \int_{t_{0}}^{t} s^{\alpha} \bigl\| \bigl( \dot{x}(s), \dot{\lambda}(s)\bigr)\bigr\|^{2} ds.
\end{split}
\end{equation}

Now, to the equality (<ref>), we add the inequalities (<ref>), (<ref>), (<ref>) and (<ref>) and then proceed to employ (<ref>):

\begin{align}
&\theta t^{\alpha + 1} \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} \nonumber\\
\leq \: &I_{1}(t) + \theta I_{2}(t) + \alpha I_{3}(t) + \theta I_{4}(t) + 2I_{5} - t^{\alpha} \dot{\varphi}(t) \nonumber\\
&+\int_{t_{0}}^{t}s^{\alpha} \left[ \theta (\alpha + 1) W(s) + \left( \frac{\alpha(\alpha - 1)}{t_{0}^{2} \delta(t_{0})} + \left\lVert A \right\rVert\right)\right] \bigl\|\bigl(\dot{x}(s), \dot{\lambda}(s)\bigr)\bigr\|^{2} ds \nonumber\\
&+ \theta \alpha \int_{t_{0}}^{t} s^{\alpha} \delta(s) \left\lVert A^{*}(\lambda(s) - \lambda_{*}) \right\rVert^{2} ds - 2 t^{\alpha} \bigl\langle \dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\bigr\rangle + C_{5} \nonumber\\
\leq \: & -t^{\alpha} \dot{\varphi}(t) + \int_{t_{0}}^{t} s^{\alpha} V(s) ds + \int_{t_{0}}^{t} s^{\alpha} \Bigl[ \bigl(\theta(\alpha + 1) - 1\bigr)\delta(t) + \theta s \dot{\delta}(s)\Bigr] \left\lVert A^{*}(\lambda(s) - \lambda_{*}) \right\rVert^{2} ds \nonumber\\
&-2 t^{\alpha} \bigl\langle \dot{x}(t), A^{*}(\lambda(t) - \lambda_{*})\bigr\rangle + C_{5}, \label{eq: we divide this one by t^alpha}
\end{align}

where, for $s\geq t_{0}$,

\begin{align*}
V(s) := \: &\theta (\alpha + 1) W(s) + \left( \frac{\alpha(\alpha - 1)}{t_{0}^{2} \delta(_{0})} + \left\lVert A \right\rVert\right) \bigl\| \bigl( \dot{x}(s), \dot{\lambda}(s)\bigr)\bigr\|^{2} \\
&+ C_{3} \delta(s) \|\nabla f(x(s)) - \nabla f(x_{*})\|^{2} + C_{4} \delta(s) \|Ax(s) - b\|^{2},
\end{align*}

and the constant $C_{5}$ is given by

\begin{align*}
C_{5} := \: &t_{0}^{\alpha} |\dot{\varphi}(t_{0})| + \theta t_{0}^{\alpha + 1}W(t_{0}) + \alpha \frac{t_{0}^{\alpha - 1}}{\delta(t_{0})}\bigl\|\bigl(\dot{x}(t_{0}, \dot{\lambda}(t_{0}))\bigr)\bigr\|^{2} \\
&+\theta t_{0}^{\alpha + 1} \delta(t_{0}) \bigl\| A^{*}(\lambda(t_{0}) - \lambda_{*})\bigr\|^{2} + 2 t_{0}^{\alpha} \bigl| \bigl\langle \dot{x}(t_{0}), A^{*}(\lambda(t_{0}) - \lambda_{*})\bigr\rangle\bigr| \geq 0.
\end{align*}

Now we divide (<ref>) by $t^{\alpha}$, thus obtaining

\begin{align*}
\theta t \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} \leq \: &- \dot{\varphi}(t) + \frac{1}{t^{\alpha}}\int_{t_{0}}^{t} s^{\alpha} V(s) ds \\
&+ \frac{1}{t^{\alpha}}\int_{t_{0}}^{t} s^{\alpha} \Bigl[ \bigl(\theta(\alpha + 1) - 1\bigr)\delta(s) + \theta s \dot{\delta}(s)\Bigr] \left\lVert A^{*}(\lambda(s) - \lambda_{*}) \right\rVert^{2} ds \\
&-2 \bigl\langle \dot{x}(t) , A^{*}(\lambda(t) - \lambda_{*})\bigr\rangle + \frac{C_{5}}{t^{\alpha}}.
\end{align*}

Now, we integrate this inequality from $t_{0}$ to $r$. We get

\begin{align}
&\theta \int_{t_{0}}^{r} t \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} dt \nonumber\\
\leq \  & \varphi(t_{0}) - \varphi(r)  + \int_{t_{0}}^{r}\frac{1}{t^{\alpha}} \left( \int_{t_{0}}^{t} s^{\alpha} V(s) ds\right) dt \nonumber\\
&\quad + \int_{t_{0}}^{r} \frac{1}{t^{\alpha}} \left(\int_{t_{0}}^{t} s^{\alpha} \Bigl[ \bigl(\theta(\alpha + 1) - 1\bigr)\delta(s) + \theta s \dot{\delta}(s)\Bigr] \left\lVert A^{*}(\lambda(s) - \lambda_{*}) \right\rVert^{2} ds\right) dt \nonumber\\
&\quad - 2 \int_{t_{0}}^{r} \bigl\langle A\dot{x}(t), \lambda(t) - \lambda_{*}\bigr\rangle dt + C_{5} \int_{t_{0}}^{r} t^{\alpha} dt. \label{eq: double integrals}
\end{align}

We now recall some important facts. First of all, we have

\begin{equation}\label{eq: fact 1}
\int_{t_{0}}^{r}\frac{1}{t^{\alpha}} dt \leq \frac{1}{(\alpha - 1)t_{0}^{\alpha - 1}}.
\end{equation}

In addition, according to Lemma <ref>, it holds

\begin{equation}\label{eq: fact 2}
\int_{t_{0}}^{r} \frac{1}{t^{\alpha}} \left(\int_{t_{0}}^{t} s^{\alpha} V(s) ds\right)dt \leq \frac{1}{\alpha - 1}\int_{t_{0}}^{r}t V(t) dt,
\end{equation}

\begin{align}\label{eq: fact 3}
& \int_{t_{0}}^{r} \frac{1}{t^{\alpha}} \left(\int_{t_{0}}^{t} s^{\alpha} \Bigl[ \bigl(\theta(\alpha + 1) - 1\bigr)\delta(s) + \theta s \dot{\delta}(s)\Bigr] \left\lVert A^{*}(\lambda(s) - \lambda_{*}) \right\rVert^{2} ds\right) dt \nonumber \\
\leq \ 	& \frac{1}{\alpha - 1}\int_{t_{0}}^{r} \Bigl[ \bigl(\theta(\alpha + 1) - 1\bigr)\delta(t) + \theta t \dot{\delta}(t)\Bigr] \left\lVert A^{*}(\lambda(t) - \lambda_{*}) \right\rVert^{2} dt,
\end{align}

Finally, integrating by parts leads to

\begin{align}
&- \int_{t_{0}}^{r} \bigl\langle A \dot{x}(t), \lambda(t) - \lambda_{*}\bigr\rangle dt \nonumber\\
= \     & - \bigl\langle Ax(r) - b, \lambda(r) - \lambda_{*}\bigr\rangle + \bigl\langle Ax(t_{0}) - b, \lambda(t_{0}) - \lambda_{*}\bigr\rangle + \int_{t_{0}}^{r} \bigl\langle Ax(t) - b, \dot{\lambda}(t)\bigr\rangle dt \nonumber\\
\leq \  & \|Ax(r) - b\|\| \lambda(r) - \lambda_{*}\| + \left\lVert Ax(t_{0}) - b \right\rVert \|\lambda(t_{0}) - \lambda_{*}\| + \int_{t_{0}}^{r} \bigl\langle Ax(t) - b, \dot{\lambda}(t)\bigr\rangle dt \nonumber\\
\leq \  & \sup_{t\geq t_{0}} \{\left\lVert Ax \left( t \right) - b \right\rVert\left\lVert \lambda \left( t \right) - \lambda_{*} \right\rVert\} + \left\lVert Ax(t_{0}) - b \right\rVert\| \lambda(t_{0} - \lambda_{*})\| \nonumber\\
&\quad + \frac{1}{2}\int_{t_{0}}^{r} \bigl( \left\lVert Ax \left( t \right) - b \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2}\bigr) dt. \label{eq: fact 4}
\end{align}

The supremum term is finite due to the boundedness of the trajectory. Now, by using the nonnegativity of $\varphi$ and the facts (<ref>), (<ref>), (<ref>) and (<ref>) on (<ref>), we come to

\begin{align}
&\dfrac{\theta}{\alpha - 1} \int_{t_{0}}^{r}t \sigma(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} dt \nonumber\\
= \ 	& \int_{t_{0}}^{r} \left[ \theta \delta(t) - \frac{\bigl(\theta(\alpha + 1) - 1\bigr)\delta(t) + \theta t \dot{\delta}(t)}{\alpha - 1}\right] t \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} dt \nonumber\\
\leq \ 	& \frac{1}{\alpha - 1}\int_{t_{0}}^{r} t V(t) dt + \int_{t_{0}}^{r} t \left( \left\lVert Ax \left( t \right) - b \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2} \right) dt + C_{6}, \label{eq: integral for dual trajectory}
\end{align}

\begin{align*}
C_{6} := \varphi(t_{0}) + 2 \sup_{t\geq t_{0}} \left\lbrace \left\lVert Ax \left( t \right) - b \right\rVert\left\lVert \lambda \left( t \right) - \lambda_{*} \right\rVert \right\rbrace + 2 \left\lVert Ax(t_{0}) - b \right\rVert\| \lambda(t_{0} - \lambda_{*})\| + \frac{C_{5}}{(\alpha - 1)t_{0}^{\alpha - 1}}.
\end{align*}

According to (<ref>) and (<ref>) in Theorem <ref>, as well as Lemma <ref>, we know that the mappings $t \mapsto t V(t)$ and $t \mapsto t \left( \left\lVert Ax \left( t \right) - b \right\rVert^{2} + \left\lVert \dot{\lambda} \left( t \right) \right\rVert \right)$ belong to $\sL^{1}\left[ t_{0}, +\infty \right)$. Therefore, by taking the limit as $r \to +\infty$ in (<ref>) we obtain

\[
\int_{t_{0}}^{+\infty} t \sigma(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} dt < +\infty.
\]

Again, from (<ref>) we conclude that

\[
\int_{t_{0}}^{+\infty} t \delta(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} dt \leq \frac{1}{C_{2}} \int_{t_{0}}^{+\infty} t \sigma(t) \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert^{2} dt < +\infty,
\]

which completes the proof.

The following result is the final step towards the second condition of Opial's Lemma.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ be a solution to (<ref>) and $(x_{*}, \lambda_{*}) \in \mathbb{S}$. Then it holds

\begin{equation} \label{eq:rates for gradient and dual variable}
\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert = o \left(\frac{1}{\sqrt{t} \sqrt[4]{\delta(t)}}\right) \textrm{ and } \left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert = o \left(\frac{1}{\sqrt{t} \sqrt[4]{\delta(t)}}\right) \textrm{ as } t \to +\infty.
\end{equation}

\[
\left\lVert \nabla_{x} \Lag \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) \right\rVert  = \left\lVert \nabla f \left( x \left( t \right) \right) + A^{*} \lambda \left( t \right) \right\rVert = o\left(\frac{1}{\sqrt{t} \sqrt[4]{\delta(t)}}\right) \quad \textrm{as} \quad t\to +\infty,
\]

while, as seen earlier,

\[
\left\lVert \nabla_{\lambda} \Lag \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) \right\rVert = \left\lVert Ax \left( t \right) - b \right\rVert =  \mathcal{O}\left(\frac{1}{t^{2} \delta(t)}\right) \quad \textrm{as} \quad t\to +\infty.
\]

We first show the gradient rate. For $t\geq t_{0}$, it holds

\begin{align}
\frac{d}{dt}\Bigl(t \sqrt{\delta(t)} \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2}\Bigr)
&= \left( \sqrt{\delta(t)} + \frac{t\dot{\delta}(t)}{2\sqrt{\delta(t)}}\right) \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} \nonumber\\
& \quad + 2 t \sqrt{\delta(t)} \left\langle \nabla f(x(t)) - \nabla f(x_{*}), \frac{d}{dt}\nabla f(x(t))\right\rangle. \label{eq: improved rate 0}
\end{align}

On the one hand, by Assumption <ref>, we can write

\begin{equation}\label{eq: improved rate 1}
\left( \sqrt{\delta(t)} + \frac{t\dot{\delta}(t)}{2\sqrt{\delta(t)}}\right) = \left( \sqrt{\delta(t)} + \frac{\sqrt{\delta(t)}}{2} \cdot \frac{t\dot{\delta}(t)}{\delta(t)}\right) \leq \left(1 + \frac{1 - 2\theta}{2\theta}\right) \sqrt{\delta(t)} = \frac{1}{2\theta} \sqrt{\delta(t)}.
\end{equation}

Since $\delta$ is nondecreasing, for $t \geq t_{0}$ we have $\sqrt{\delta(t)} \geq \sqrt{\delta(t_{0})} > 0$. Set $t_{1} := \max\left\{t_{0}, \frac{1}{\sqrt{t_{0}}}\right\}$. Therefore, for $t \geq t_{1}$ it holds

\[
\frac{1}{\sqrt{\delta(t)}} \leq \frac{1}{\sqrt{\delta(t_{0})}} = t_{1} \leq t
\]

and thus

\begin{equation}\label{eq: improved rate 2}
\sqrt{\delta(t)} \leq t \delta(t).
\end{equation}

On the other hand, for every $t \geq t_{1}$ we deduce

\begin{align}
2 t \sqrt{\delta(t)} \left\langle \nabla f(x(t)) - \nabla f(x_{*}), \frac{d}{dt}\nabla f(x(t))\right\rangle
&= 2 t \left\langle \sqrt{\delta(t)} \bigl[\nabla f(x(t)) - \nabla f(x_{*})\bigr], \frac{d}{dt}\nabla f(x(t))\right\rangle \nonumber\\
&\leq t\delta(t) \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} + t\left\|\frac{d}{dt}\nabla f(x(t))\right\|^{2} \nonumber\\
&\leq t\delta(t) \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} + \ell^{2} t \bigl\| \dot{x}(t)\bigr\|^{2} \label{eq: improved rate 3},
\end{align}

where the last inequality is a consequence of the $\ell$-Lipschitz continuity of $\nabla f$.

By combining (<ref>), (<ref>) and (<ref>), from (<ref>) we assert that for every $t\geq t_{1}$

\begin{equation*}
\frac{d}{dt}\Bigl(t \sqrt{\delta(t)} \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2}\Bigr) \leq \left( 1 + \frac{1}{2\theta} \right) t \delta(t) \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} + \ell^{2} t \bigl\| \dot{x}(t)\bigr\|^{2} .
\end{equation*}

According to (<ref>) and (<ref>), the right hand side of the previous inequality belongs to $\sL^{1} [t_{1}, +\infty)$. Since $\delta$ is nondecreasing, for every $t \geq t_{1}$ we have

\[
\sqrt{\delta(t)} = \sqrt{\delta(t)} \cdot \frac{\sqrt{\delta(t)}}{\sqrt{\delta(t)}} \leq \frac{\delta(t)}{\sqrt{\delta(t_{0})}},
\]

\begin{equation}\label{eq:inequality for applying lemma a3}
\int_{t_{1}}^{+\infty} t \sqrt{\delta(t)} \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} dt \leq \frac{1}{\sqrt{\delta(t_{0})}} \int_{t_{1}}^{+\infty} t \delta(t) \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} dt < +\infty,
\end{equation}

i.e., the function being differentiated also belongs to $\sL^{1} [t_{1}, +\infty)$. Therefore, Lemma <ref> gives us

\[
t \sqrt{\delta(t)} \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert^{2} \to 0 \quad \text{as} \quad t\to +\infty.
\]

Proceeding in the exact same way, for every $t\geq t_{1}$ we have

\begin{align*}
&\frac{d}{dt} \Bigl( t \sqrt{\delta(t)} \bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2}\Bigr) \\
= \     & \left( \sqrt{\delta(t)} + \frac{t \dot{\delta}(t)}{2\sqrt{\delta(t)}}\right) \bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2} + 2 t \sqrt{\delta(t)} \bigl\langle A A^{*}(\lambda(t) - \lambda_{*}), \dot{\lambda}(t)\bigr\rangle \\
\leq \  & \left( \frac{1}{2\theta} + \|A\|^{2} \right) t \delta(t) \bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2} + t \left\lVert \dot{\lambda} \left( t \right) \right\rVert^{2}.
\end{align*}

According to (<ref>) and (<ref>), the right hand side of the previous inequality belongs to $\sL^{1}[t_{1}, +\infty)$. Arguing as in (<ref>), we deduce that the function being differentiated also belongs to $\sL^{1}[t_{1}, +\infty)$. Again applying Lemma <ref>, we come to

\[
t \sqrt{\delta(t)} \bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\|^{2} \to 0 \quad \text{as} \quad t\to +\infty.
\]

Finally, recalling that $A^{*}\lambda_{*} = -\nabla f(x_{*})$, we deduce from the triangle inequality that

\begin{align*}
\left\lVert \nabla_{x} \Lag \bigl( x \left( t \right) , \lambda \left( t \right) \bigr) \right\rVert  &= \left\lVert \nabla f \left( x \left( t \right) \right) + A^{*} \lambda \left( t \right) \right\rVert \\
&\leq \left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert + \bigl\|A^{*}(\lambda(t) - \lambda_{*})\bigr\| \nonumber \\
& = o\left(\frac{1}{\sqrt{t} \sqrt[4]{\delta(t)}}\right) \quad \textrm{as} \quad t\to +\infty,
\end{align*}

and the third claim follows.

The previous theorem also has its own interest. It tells us that the time rescaling parameter also plays a role in accelerating the rates of convergence for $\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert$ and $\|A^{*}(\lambda(t) - \lambda_{*})\|$ as $t\to +\infty$. Moreover, we deduce from (<ref>) that the mapping $(x, \lambda) \mapsto (\nabla f(x), A^{*}\lambda)$ is constant along $\mathbb{S}$, as reported in <cit.>.

We now come to the final step and show weak convergence of the trajectories of (<ref>) to elements of $\mathbb{S}$.

Let $(x, \lambda) : \left[ t_{0}, +\infty \right) \to \mathcal{X} \times \mathcal{Y}$ be a solution to (<ref>) and $\left( x_{*} , \lambda_{*} \right) \in \sol$. Then $\bigl(x(t), \lambda(t)\bigr)$ converges weakly to a primal-dual solution of (<ref>) as $t\to +\infty$.

For proving this theorem, we make use of Opial's Lemma (see Lemma <ref>). Lemma <ref> tells us that $\lim_{t\to +\infty}\left\|\bigl(x(t), \lambda(t)\bigr) - (x_{*}, \lambda_{*})\right\|$ exists for every $(x_{*}, \lambda_{*}) \in \mathbb{S}$, which proves condition (i) of Opial's Lemma.

In order to show condition (ii), we recall the operator $\mathcal{T}_{\mathcal{L}}$ defined in (<ref>) by

\[
\TL \left( x , \lambda \right)
\begin{pmatrix}
\nabla f \left( x \right) + A^{*} \lambda \\ b-Ax
\end{pmatrix}
\quad
\forall (x, \lambda) \in \mathcal{X} \times \mathcal{Y}.
\]

Fix $(x_{*}, \lambda_{*}) \in \mathbb{S}$ (in other words, $\mathcal{T}_{\mathcal{L}}(x_{*}, \lambda_{*}) = 0$) and take $(\tilde{x}, \tilde{\lambda})$ any weak sequential cluster point of $\bigl( x(t), \lambda(t)\bigr)$ as $t\to +\infty$, which means there exists a strictly increasing sequence $(t_{n})_{n\in \mathbb{N}} \subseteq [t_{0}, +\infty)$ such that

\[
\bigl( x(t_{n}), \lambda(t_{n})\bigr) \rightharpoonup (\tilde{x}, \tilde{\lambda}) \quad \text{as} \quad n \to +\infty.
\]

Given that $\delta$ is nondecreasing on $\left[ t_{0}, +\infty \right)$, from Theorem <ref> and (<ref>) we deduce that

\begin{align*}
\nabla f(x(t_{n})) - \nabla f(x_{*}) &\to 0, \\
A^{*}\lambda(x(t_{n})) - A^{*}\lambda_{*} &\to 0, \\
Ax(t_{n}) - b &\to 0
\end{align*}

as $n\to +\infty$. Since $A^{*}\lambda_{*} = -\nabla f(x_{*})$, the previous three statements imply

\[
\mathcal{T}_{\mathcal{L}}(x(t_{n}), \lambda(t_{n})) \to 0 \quad \text{as} \quad n \to +\infty.
\]

Since we already had $(x(t_{n}), \lambda(t_{n})) \rightharpoonup (\tilde{x}, \tilde{\lambda})$ as $n\to +\infty$, and the graph of $\mathcal{T}_{\mathcal{L}}$ is sequentially closed in $(\mathcal{X} \times \mathcal{Y})^{\text{weak}} \times (\mathcal{X} \times \mathcal{Y})^{\text{strong}}$ (see <cit.>), we finally come to

\[
\mathcal{T}_{\mathcal{L}}(\tilde{x}, \tilde{\lambda}) = 0,
\]

which is to say, $(\tilde{x}, \tilde{\lambda}) \in \mathbb{S}$. The proof is thus concluded.

In case $A := 0$ and $b := 0$, the optimization problem (<ref>) reduces to the unconstrained optimization problem

\begin{equation}
\label{intro:pb-un}
\min\limits_{x \in \sX} f \left( x \right) .
\end{equation}

Indeed, the system of optimality conditions (<ref>) read in this case

\begin{equation*}
\left( x_{*} , \lambda_{*} \right) \in \sol
\Leftrightarrow \nabla f \left( x_{*} \right) = 0 \textrm{ and } \lambda_{*} \in \sY,
\end{equation*}

in particular, $x_{*} \in \sX$ is an optimal solution of (<ref>) if and only if $\nabla f \left( x_{*} \right) = 0$.
The system (<ref>) becomes

\begin{equation*}
\begin{dcases}
\ddot{x} \left( t \right) + \dfrac{\alpha}{t} \dot{x} \left( t \right) + \delta \left( t \right) \nabla f \left( x \left( t \right) \right)    		& = 0 \\
\ddot{\lambda} \left( t \right) + \dfrac{\alpha}{t} \dot{\lambda} \left( t \right)	& = 0 \\
\Bigl( x \left( t_{0} \right) , \lambda \left( t_{0} \right) \Bigr) 			= \Bigl( x_{0} , \lambda_{0} \Bigr) \textrm{ and }
\Bigl( \dot{x} \left( t_{0} \right) , \dot{\lambda} \left( t_{0} \right) \Bigr) = \Bigl( \dot{x}_{0} , \dot{\lambda}_{0} \Bigr)
\end{dcases}.
\end{equation*}

The dynamical system in $x$ is reads

\begin{equation*}
\begin{dcases}
\ddot{x} \left( t \right) + \dfrac{\alpha}{t} \dot{x} \left( t \right) + \nabla f \left( x \left( t \right) \right)   = 0 \\
x(t_0) = x_0 \ \textrm{ and } \dot{x}(t_0) = \dot{x}_0
\end{dcases},
\end{equation*}

for $\alpha \geq 3$, and is nothing else than Nesterov's accelerated gradient system. The trajectory generated by the system in $\lambda$ is $\lambda(t) = \frac{\dot{\lambda}_0 t_0^\alpha}{1-\alpha} t^{1-\alpha} + \lambda_0 - \frac{\dot{\lambda}_0 t_0}{1-\alpha}$ for every $t \geq t_{0}$. The parameters $\theta$ and $\beta$ plays no role in the system. Therefore, the condition on $\delta \left( t \right)$ now becomes

\begin{equation*}
\sup_{t\geq t_{0}}\frac{t \dot{\delta}(t)}{\delta(t)} \leq \alpha - 3 ,
\end{equation*}

which is exactly the one imposed in [15], see also [18].

If $\alpha \geq 3$, then Theorem <ref> (iii) says that $f(x(t))$ converges to $f_*$ with a rate of convergence of $\bO \left( \frac{1}{t^{2} \delta \left( t \right)} \right)$ as $t \to + \infty$, which is the rate derived in [15, 18].

If $\alpha > 3$, then Theorem <ref> gives that the trajectory $x(t)$ converges weakly to an optimal solution of (<ref>), as $t \to + \infty$. Notice that no trajectory convergence has been reported in [15, 18].

Finally, we mention that the convergence of the trajectory in the critical case $\alpha = 3$ is still an open question, even for non time rescaling case ([12, 72]), as it is the convergence of the iterates of the original Nesterov's acceleration algorithm ([30, 61]).

§ NUMERICAL EXPERIMENTS

We will illustate the theoretical results by two numerical examples, with $\mathcal{X} = \mathbb{R}^{4}$ and $\mathcal{Y} = \mathbb{R}^{2}$. We will address two minimization problems with linear constraints; one with a strongly convex objective function and another with a convex objective function which is not strongly convex. In both cases, the linear constraints are dictated by

\[
A =
\begin{bmatrix}
1 & -1 & -1 & 0 \\
0 & 1 & 0 & -1
\end{bmatrix}
\qquad
\text{and}
\qquad
b =
\begin{bmatrix}
0 \\
\end{bmatrix}.
\]

Consider the minimization problem

\begin{equation*}
\begin{array}{rl}
\min & f(x_{1}, x_{2}, x_{3}, x_{4}) := (x_{1} - 1)^{2} + (x_{2} - 1)^{2} + x_{3}^{2} + x_{4}^{2} \\
\textrm{subject to} 	& x_{1} - x_{2} - x_{3} = 0 \\
& x_{2} - x_{4} = 0.
\end{array}
\end{equation*}

The optimality conditions can be calculated and lead to the following primal-dual solution pair

\[
x_{*} =
\begin{bmatrix}
0.8 \\
0.6 \\
0.2 \\
\end{bmatrix}
\qquad
\text{and}
\qquad
\lambda_{*} =
\begin{bmatrix}
0.4 \\
\end{bmatrix}.
\]

Consider the minimization problem

\begin{equation*}
\begin{array}{rl}
\min & f(x_{1}, x_{2}, x_{3}, x_{4}) := \log\left( 1 + e^{-x_{1} - x_{2}}\right) + x_{3}^{2} + x_{4}^{2} \\
\textrm{subject to} 	& x_{1} - x_{2} - x_{3} = 0 \\
& x_{2} - x_{4} = 0.
\end{array}
\end{equation*}

This problem is similar to the regularized logistic regression frequently used in machine learning.
We cannot explicitly calculate the optimality conditions as in the previous case; instead, we use the last solution in the numerical experiment as the approximate solution.

To comply with Assumption <ref>, we choose $t_{0} > 0$, $\alpha = 8$, $\beta = 10$, $\theta = \frac{1}{6}$, and we test four different choices for the rescaling parameter: $\delta(t) = 1$ (i.e., the (PD-AVD) dynamics in [75, 34]), $\delta(t) = t$, $\delta(t) = t^{2}$ and $\delta(t) = t^{3}$. In both examples, the initial conditions are

\[
x(t_{0}) =
\begin{bmatrix}
0.5 \\
0.5 \\
0.5 \\
\end{bmatrix},
\quad
\lambda(t_{0}) =
\begin{bmatrix}
0.2 \\
\end{bmatrix},
\quad
\dot{x}(t_{0}) =
\begin{bmatrix}
0.5 \\
0.5 \\
0.5 \\
\end{bmatrix}
\quad
\text{and}
\quad
\dot{\lambda}(t_{0}) =
\begin{bmatrix}
0.5 \\
\end{bmatrix}.
\]

For each choice of $\delta$, we plot, using a logarithmic scale, the primal-dual gap $\mathcal{L}\bigl(x(t), \lambda_{*}\bigr) - \mathcal{L}\bigl(x_{*}, \lambda(t)\bigr)$, the feasibility measure $\left\lVert A x \left( t \right) - b \right\rVert$ and  the functional values $\left\lvert f \left( x \left( t \right) \right) - f_{*} \right\rvert$, to highlight the theoretical result in Theorem <ref>.
We also illustrate the findings from Theorem <ref>, namely, we plot the quantities $\left\lVert \nabla f \left( x \left( t \right) \right) - \nabla f \left( x_{*} \right) \right\rVert$ and $\left\lVert A^{*} \left( \lambda \left( t \right) - \lambda_{*} \right) \right\rVert$, as well as the velocity $\Vert ( \dot{x} (t) , \dot{\lambda} (t) ) \Vert$.

Figure <ref> and <ref> display these plots for Example <ref> and <ref>, respectively.
As predicted by the theory, choosing faster-growing time rescaling parameters yields better convergence rates. This is not the case for the velocities.

The function $\delta \left( t \right)$ influences convergence behaviour in Example <ref>

The function $\delta \left( t \right)$ influences convergence behaviour in Example <ref>

§ APPENDIX

Here we collect the auxiliary results that are required to carry out many steps in out analysis.

A proof for the following lemma in the finite-dimensional case can be found in <cit.>. The proof for the infinite-dimensional case is short and virtually identical, so we include it here for the sake of completeness..

Assume that $t_{0} > 0$, $g : \left[ t_{0}, +\infty \right) \to \mathcal{Y}$ is a continuous differentiable function, $a : \left[ t_{0}, +\infty \right) \to [0, +\infty)$ is a continuous function, and $C \geq 0$. If, in the sense of Bochner integrability, we have

\begin{equation}\label{eq:boundedness hypothesis}
\left\| g(t) + \int_{t_{0}}^{t} a(s) g(s) ds\right\| \leq C \quad \forall t\geq t_{0}
\end{equation}

\[
\sup_{t\geq t_{0}}\|g(t)\| \leq 2C < + \infty.
\]

Define, for every $t\geq t_{0}$,

\[
G(t) := e^{\int_{t_{0}}^{t} a(s) ds} \int_{t_{0}}^{t} a(s) g(s) ds.
\]

Fix $t\geq t_{0}$. The time derivative of $G$ reads

\[
\dot{G}(t) = a(t) e^{\int_{t_{0}}^{t} a(s) ds} \int_{t_{0}}^{t} a(s) g(s) ds + e^{\int_{t_{0}}^{t} a(s) ds} a(t) g(t)
= a(t) e^{\int_{t_{0}}^{t} a(s) ds} \left[ g(t) + \int_{t_{0}}^{t} a(s) g(s) ds\right],
\]

so by using (<ref>) and the previous equality we arrive at

\begin{equation} \label{eq:derivative of G}
\bigl\| \dot{G}(t)\bigr\| \leq C a(t) e^{\int_{t_{0}}^{t} a(s) ds} = C \frac{d}{dt}\left( e^{\int_{t_{0}}^{t} a(s) ds}\right).
\end{equation}

Since $G(t_{0}) = 0$, we have

\[
G(t) = G(t) - G(t_{0}) = \int_{t_{0}}^{t} \dot{G}(s) ds,
\]

so by employing (<ref>) and the previous equality we obtain, for every $t\geq t_{0}$,

\begin{align*}
e^{\int_{t_{0}}^{t} a(s) ds} \left\| \int_{t_{0}}^{t} a(s) g(s) ds\right\| &= \|G(t)\| \leq \int_{t_{0}}^{t}\bigl\| \dot{G}(s)\bigr\| ds \leq C \int_{t_{0}}^{t} \frac{d}{ds}\left(e^{\int_{t_{0}}^{s} a(\tau) d\tau}\right) ds \\
&\leq C \left[ e^{\int_{t_{0}}^{t} a(s) ds} - 1\right]
\leq C e^{\int_{t_{0}}^{t} a(s) ds}.
\end{align*}

Dividing both sides of the previous inequality by $e^{\int_{t_{0}}^{t} a(s) ds}$ gives us

\begin{equation}\label{eq:boundedness hypothesis 2}
\left\| \int_{t_{0}}^{t} a(s) g(s) ds\right\| \leq C \quad \forall t\geq t_{0}.
\end{equation}

Now, by putting (<ref>) and (<ref>) we finally come to

\[
\|g(t)\| \leq \left\| g(t) + \int_{t_{0}}^{t} a(s) g(s) ds\right\| + \left\| \int_{t_{0}}^{t} a(s) g(s) ds\right\| \leq 2C \quad \forall t\geq t_{0},
\]

which leads to the announced statement.

The proofs for the following results can be found in <cit.> and <cit.>, respectively.

Let $0 < t_{0} \leq r \leq +\infty$ and $h :\left[ t_{0}, +\infty \right) \to [0, +\infty)$ be a continuous function. For every $\alpha > 1$ it holds

\[
\int_{t_{0}}^{r} \frac{1}{t^{\alpha}} \left[ \int_{t_{0}}^{t}s^{\alpha - 1} h(s)\right]dt \leq \frac{1}{\alpha - 1} \int_{t_{0}}^{r} h(t) dt.
\]

If $r = +\infty$, then equality holds.

Let $t_{0} > 0$, $1 \leq p < +\infty$ and $1\leq q \leq +\infty$. Suppose that $F \in \sL^{p}\left[ t_{0}, +\infty \right)$ is a locally absolutely continuous nonnegative function, $G \in \sL^{q}\left[ t_{0}, +\infty \right)$ and

\[
\dot{F}(t) \leq G(t) \qquad \forall t\geq t_{0}.
\]

Then, $\lim_{t\to +\infty} F(t) = 0$.

The following lemma is a slight variation of results already present in the literature. See, for example, <cit.>.

Let $t_{0} > 0$, $\alpha > 1$, and let $\phi : [t_{0}, +\infty) \to \mathbb{R}$ be a twice continuously differentiable function bounded from below. Furthermore, assume $w : [t_{0}, +\infty) \to [0, +\infty)$ to be a continuously differentiable function such that $t\mapsto t w(t)$ belongs to $\mathbb{L}^{1}[t_{0}, +\infty)$ and

\[
t\ddot{\phi}(t) + \alpha \dot{\phi}(t) + t^{2}\dot{w}(t) \leq 0 \qquad \forall t\geq t_{0}.
\]

Then, the positive part $[\dot{\phi}]_{+}$ of $\dot{\phi}$ belongs to $\mathbb{L}^{1}[t_{0}, +\infty)$ and the limit $\lim_{t\to +\infty} \phi(t)$ is a real number.

Fix $t\geq t_{0}$. Adding $(\alpha + 1)t w(t)$ to both sides of the previous inequality and then multiplying it by $t^{\alpha - 1}$ yields

\[
\frac{d}{dt}\bigl(t^{\alpha} \dot{\phi}(t)\bigr) + \frac{d}{dt}\bigl(t^{\alpha + 1}w(t)\bigr) \leq (\alpha + 1)t^{\alpha}w(t).
\]

Since the previous inequality holds for any $t\geq t_{0}$, we can integrate it from $t_{0}$ to $t\geq t_{0}$ to get

\[
t^{\alpha} \dot{\phi}(t) - t_{0}^{\alpha} \dot{\phi}(t_{0}) + t^{\alpha + 1}w(t) - t_{0}^{\alpha + 1}w(t_{0}) \leq (\alpha + 1) \int_{t_{0}}^{t} s^{\alpha}w(s) ds.
\]

After dropping the nonnegative term $t^{\alpha + 1}w(t)$ and dividing by $t^{\alpha}$ we arrive at

\[
\dot{\phi}(t) \leq \frac{\widetilde{C}}{t^{\alpha}} + \frac{\alpha + 1}{t^{\alpha}} \int_{t_{0}}^{t}s^{\alpha} w(s) ds \qquad \forall t\geq t_{0},
\]

\[
\widetilde{C} := t_{0}^{\alpha} \bigl| \dot{\phi}(t_{0})\bigr| + t_{0}^{\alpha + 1}w(t_{0}),
\]

which further leads to

\[
[\dot{\phi}(t)]_{+} \leq \frac{\widetilde{C}}{t^{\alpha}} + \frac{\alpha + 1}{t^{\alpha}} \int_{t_{0}}^{t}s^{\alpha}w(s) ds \qquad \forall t\geq t_{0}.
\]

Now, we integrate this inequality from $t_{0}$ to $r\geq t_{0}$ and we apply Lemma <ref> with $h : [t_{0}, +\infty) \to [0, +\infty)$ given by $h(s) := sw(s)$ to obtain

\begin{align*}
\int_{t_{0}}^{r}[\dot{\phi}(t)]_{+}dt &\leq \widetilde{C}\int_{t_{0}}^{r} \frac{1}{t^{\alpha}}dt + (\alpha + 1) \int_{t_{0}}^{r}\frac{1}{t^{\alpha}} \left[ \int_{t_{0}}^{t} s^{\alpha - 1}\cdot sw(s)ds\right]dt \\
&\leq \frac{\widetilde{C}}{1 - \alpha}\left( \frac{1}{t_{0}^{\alpha - 1}} - \frac{1}{r^{\alpha - 1}}\right) + \frac{\alpha + 1}{1 - \alpha} \int_{t_{0}}^{r}tw(t) dt.
\end{align*}

By hypothesis, as $r\to +\infty$, the right hand side of the previous inequality is finite. In other words,

\[
\int_{t_{0}}^{+\infty}[\dot{\phi}(t)]_{+}dt < +\infty.
\]

The previous statement, together with the fact that we assumed that $\phi$ was bounded from below, allow us to deduce that the function $\psi : [t_{0}, +\infty) \to \mathbb{R}$ given by

\[
\psi(t) := \phi(t) - \int_{t_{0}}^{t}[\dot{\phi}(s)]_{+}ds
\]

is also bounded from below. An easy computation shows that $\dot{\psi}$ is nonpositive on $[t_{0}, +\infty)$, thus $\psi$ is nonincreasing on $[t_{0}, +\infty)$. These facts imply that $\lim_{t\to +\infty}\psi(t)$ is a real number. Finally, we conclude that

\[
\lim_{t\to +\infty} \phi(t) = \lim_{t\to +\infty}\psi(t) + \int_{t_{0}}^{+\infty}[\dot{\phi}(s)]_{+}ds \: \in\: \mathbb{R}.
\]

The proof for Opial's Lemma can be found in [64].

Let $\mathcal{H}$ be a real Hilbert space, $S \subseteq \mathcal{H}$ a nonempty set, $t_{0} > 0$ and $z : \left[ t_{0}, +\infty \right) \to \mathcal{H}$ a mapping that satisfies

* for every $z_{*} \in S$, $\lim_{t\to +\infty} \|z(t) - z_{*}\|$ exists;

* every weak sequential cluster point of the trajectory $z(t)$ as $t \to +\infty$ belongs to $S$.

Then, $z(t)$ converges weakly to an element of $S$ as $t \to +\infty$.

[1]
B. Abbas, H. Attouch, B.F. Svaiter.
Newton-like dynamics and forward–backward methods for structured monotone inclusions in Hilbert spaces.
Journal of Optimization Theory and Applications 161(2), 331–360 (2014)

[2]
F. Alvarez.
On the minimizing property of a second order dissipative system in Hilbert spaces.
SIAM Journal on Control and Optimization 38(4), 1102–1119 (2000)

[3]
F. Alvarez, H. Attouch, J. Bolte, P. Redont.
A second-order gradient-like dissipative dynamical system with Hessian-driven damping: Application to optimization and mechanics.
Journal de Mathématiques Pures et Appliquées 81(8), 747–779 (2002)

[5]
H. Attouch.
Fast inertial proximal ADMM algorithms for convex structured optimization with linear constraint.
Minimax Theory and its Applications 6(1), 1–24 (2021)

[10]
H. Attouch, Z. Chbani, J. Fadili, H. Riahi.
First-order optimization algorithms via inertial systems with Hessian driven damping.
Mathematical Programming 193, 113–155 (2022)

[11]
H. Attouch, Z. Chbani, J. Fadili, H. Riahi.
Fast convergence of dynamical ADMM via time scaling of damped inertial dynamics.
Journal of Optimization Theory and Applications 193, 704–736 (2022)

[12]
H. Attouch, Z. Chbani, J. Peypouquet, P. Redont.
Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity.
Mathematical Programming 168 (1), 123–175 (2018)

[13]
H. Attouch, Z. Chbani, H. Riahi.
Combining fast inertial dynamics for convex optimization with
Tikhonov regularization.
Journal of Mathematical Analysis and Applications 457, 1065–1094 (2018)

[14]
H. Attouch, Z. Chbani, H. Riahi.
Rate of convergence of the Nesterov accelerated gradient method in the subcritical case $\alpha \leq 3$.
ESAIM: Control, Optimisation and Calculus of Variations 25, 2 (2019)

[15]
H. Attouch, Z. Chbani, H. Riahi.
Fast proximal methods via time scaling of damped inertial dynamics.
SIAM Journal on Optimization 29(3), 2227–2256 (2019)

[16]
H. Attouch, Z. Chbani, H. Riahi.
Fast convex optimization via a third-order in time evolution equation.
Optimization 71(5), 1275–1304 (2022)

[17]
H. Attouch, Z. Chbani, H. Riahi.
Fast convex optimization via time scaling of damped inertial gradient dynamics.
Pure and Applied Functional Analysis.

[18]
H. Attouch, Z. Chbani, H. Riahi.
Fast Convex Optimization via Time Scaling of Damped Inertial Gradient Dynamics.

[19]
H. Attouch, X. Goudou, P. Redont.
The heavy ball with friction method. I. The continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system.
Communications in Contemporary Mathematics 2(1), 1–34 (2000)

[20]
H. Attouch, J. Peypouquet.
The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than $1/k^2$.
SIAM Journal on Optimization 26(3), 1824–1834 (2016)

[21]
H. Attouch, J. Peypouquet.
Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators.
Mathematical Programming 174 (1-2), 391-432 (2019)

[22]
H. Attouch, Z. Chbani, J. Peypouquet, P. Redont
“Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
Mathematical Programming 168, 123–175 (2018)

[27]
H.H. Bauschke, P.L. Combettes.
Convex Analysis and Monotone Operator Theory in Hilbert Spaces.
CMS Books in Mathematics, Springer, New York (2017)

[30]
A. Beck and M. Teboulle.
A fast iterative shrinkage-thresholding algorithm for linear inverse problems.
SIAM Journal on Imaging Sciences 2(1), 183–202 (2009)

[31]
R. I. Boţ.
Conjugate Duality in Convex Optimization.
Lecture Notes in Economics and Mathematical Systems, Vol. 637, Springer, Berlin Heidelberg (2010)

[32]
R. I. Boţ, E. R. Csetnek, D.-K. Nguyen.
Fast OGDA in continuous and discrete time.

[33]
R. I. Boţ, E. R. Csetnek, D.-K. Nguyen.
Fast Augmented Lagrangian Method in the convex regime with convergence guarantees for the iterates.

[34]
R. I. Boţ, D.-K. Nguyen.
Improved convergence rates and trajectory convergence for primal-dual dynamical systems with vanishing damping.
Journal of Differential Equations 303, 369–406 (2021)

[40]
R. I. Boţ, E. R. Csetnek, S.C. László.
A primal-dual dynamical approach to structured convex minimization problems.
Journal of Differential Equations 269(12), 10717–10757 (2020)

[41]
S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein.
Distributed optimization and statistical learning via the alternating direction method of multipliers.
Foundations and Trends in
Machine Learning  3(1), 1–122 (2010)

[43]
A. Cabot, H. Engler, S.Gadat.
On the long time behavior of second order differential equations with asymptotically small dissipation.
Transaction of the American Mathematical Society 361, 5983–-6017 (2009)

[44]
A. Cabot, H. Engler, S.Gadat.
Second order differential equations with asymptotically small dissipation and piecewise flat potentials.
Electronic Journal of Differential Equations 17, 33–38 (2009)

[45]
A Chambolle, C Dossal.
On the convergence of the iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”.
Journal of Optimization theory and Applications 166(3), 968–982 (2016)

[49]
D. Gabay, B. Mercier.
A dual algorithm for the solution of nonlinear variational problems via finite element approximation.
Computers and Mathematics with Applications  2(1),  17–40 (1976)

[50]
T. Goldstein, B. O'Donoghue, S. Setzer, R. Baraniuk.
Fast alternating direction optimization methods.
SIAM Journal on Imaging Sciences 7(3), 1588–1623 (2014)

[51]
O. Güler.
New proximal point algorithms for convex minimization.
SIAM Journal on Optimization 2, 649–664 (1992)

[52]
X. He, R. Hu, Y. Fang.
Convergence rates of inertial primal-dual dynamical methods for separable convex optimization problems.
SIAM Journal on Control and Optimization 59(5), 3278–3301 (2021)

[53]
X. He, R. Hu, Y. Fang.
Inertial accelerated primal-dual methods for linear equality constrained convex optimization problems.
Numerical Algorithms 90, 1669–1690 (2022)

[54]
X. He, R. Hu, Y. Fang.
Fast primal-dual algorithm via dynamical system for a linearly constrained convex optimization problem.

[55]
X. He, R. Hu, Y. Fang.
Perturbed inertial primal-dual dynamics with damping and scaling terms for linearly constrained convex optimization problems.

[57]
Z. Lin, H. Li, C. Fang.
Accelerated Optimization for Machine Learning.
Springer, Singapore (2020)

[58]
R. Madan, S. Lall.
Distributed algorithms for maximum lifetime routing in wireless sensor networks.
IEEE Transactions on Wireless Communications 5, 2185–2193 (2006)

[59]
R. May.
Asymptotic for a second-order evolution equation with convex potential and vanishing damping term.
Turkish Journal of Mathematics 41, 681–785 (2017)

[61]
Y. Nesterov.
A method of solving a convex programming problem with convergence rate $\bO \left( 1 / k^{2} \right)$.
Soviet Mathematics Doklady 27, 372–376 (1983)

[63]
Y. Nesterov.
Introductory Lectures on Convex Optimization. Springer, New York (2004)

[64]
Z. Opial.
Weak convergence of the sequence of successive approximations for nonexpansive mappings.
Bulletin of the American Mathematical Society
73 (1967), 591–597

[65]
B. T. Polyak.
Introduction to Optimization.
Translations Series in Mathematics and Engineering, Optimization Software Inc., New York (1987)

[66]
B. T. Polyak.
Some methods of speeding up the convergence of iteration methods
USSR Computational Mathematics and Mathematical Physics
4(5), 1–17 (1964)

[68]
R. T. Rockafellar.
Augmented Lagrangians and applications of the proximal point algorithm in convex programming.
Mathematics of Operations Research 1(2), 97–116 (1976)

[70]
G. Shi, K. H. Johansson.
Randomized optimal consensus of multi-agent systems.
Automatica 48(12), 3018–3030 (2012)

[72]
W. Su, S. Boyd, E. Candès
A differential equation for modeling Nesterov's accelerated gradient method: theory and insights.
Journal of Machine Learning Research 17(153), 1–43 (2016)

[73]
P. Yi, Y. Hong, F. Liu.
Distributed gradient algorithm for constrained optimization with application to load sharing in power systems.
Systems $\&$ Control Letters 83, 45–52 (2015)

[74]
P. Yi, Y. Hong, F. Liu.
Initialization-free distributed algorithms for optimal resource allocation with feasibility constraints and application to economic dispatch of power systems.
Automatica 74, 259–269 (2016)

[75]
X. Zeng, J. Lei, J. Chen.
Dynamical primal-dual accelerated method with applications to network optimization.
IEEE Transactions on Automatic Control,

[76]
X. Zeng, P. Yi, Y. Hong, and L. Xie.
Distributed continuous-time algorithms for nonsmooth extended monotropic optimization problems
SIAM Journal on Control and Optimization 56(6), 3973–3993 (2018)