# Generalized conformal maps as classical symmetries of Yang-Mills fields

Edward B. Baker III<EMAIL_ADDRESS>
> We show that a class of previously defined maps, called causal and self-dual
> morphisms, form classical symmetries of Yang-Mills fields in four complex
> dimensions. These maps generalize conformal transformations, and admit a
> nonlocal pullback connection that preserves the equations of the theory.
> First it is shown that self-dual morphisms form symmetries of the anti-self-
> dual Yang-Mills equations under this pullback. Then a supersymmetric
> generalization of causal morphisms is defined which preserves solutions of
> the field equations for N=3 supersymmetric Yang-Mills theory. As a special
> case, this implies that a modified definition of causal morphisms form
> symmetries for the ordinary Yang-Mills field equations.

##  1 Introduction

Hidden symmetries have played an important role in the study of Yang-Mills
(YM) theory. As an example, the anti-self dual Yang-Mills (ASDYM) equations
have an infinite class of hidden symmetries which bear some resemblance to the
infinite-dimensional conformal group in two dimensions [1][2][3]. In addition,
an extended conformal symmetry called dual superconformal invariance has been
uncovered in the study of N=4 supersymmetric Yang-Mills (SYM) theory [4][5],
leading to an infinite dimensional Yangian symmetry [6]. This and other
advances have led to powerful tools for the study of N=4 SYM.

Many of these results can be understood best with the use of twistor and
ambitwistor methods, which have been used extensively in the study of Yang-
Mills fields. For example, the Penrose-Ward correspondence reformulates the
ASDYM equations in Twistor space, which leads to the ADHM construction of
instantons [7][8][9]. This construction was generalized to a geometric
formulation of the Yang-Mills field equations in ambitwistor space, with a
natural interpretation in superspace [10][11][12][13]. More recently, twistor
and ambitwistor methods have been used in string theory to understand Yang-
Mills scattering amplitudes and their properties [14][15].

In this paper we investigate a previously defined class of generalized maps
[16] in the context of Yang-Mills theory. These maps are motivated by twistor
and ambitwistor theory, and are called self-dual and causal morphisms,
respectively. Under certain assumptions, one can define a non-local pullback
connection under these transformations that preserves integrability on certain
subspaces. In the case of self-dual morphisms, the maps preserve integrability
on self-dual planes which imply that they are symmetries of the ASDYM
equations. A supersymmetric generalization of causal morphisms is then
developed which preserves integrability on super null lines, implying that
these maps are symmetries of the N=3 SYM field equations. This fact is used to
show that a modified version of causal morphisms are symmetries of the YM
field equations as a special case. It is likely that some of these symmetries
are related to known hidden symmetries for the different cases, but
characterizing these relationships will be left as a topic of future
investigation.

##  2 Self-dual morphisms as symmetries of ASDYM

Self-dual morphisms were introduced in a previous paper, where they were
defined using maps on null surfaces [16]. Here we provide a self contained
summary of these results, using different but equivalent definitions. To
begin, define the twistor correspondence space
$\mathcal{F}=\mathbb{C}^{4}\times\mathbb{C}\mathbb{P}^{1}$ with the usual
double fibration [17][18]

$\mathbb{C}^{4}\xleftarrow{\pi_{1}}\mathcal{F}\xrightarrow{\pi_{2}}\mathcal{P}\mathcal{T},$
(1)

where $\mathcal{P}\mathcal{T}=\mathbb{C}\mathbb{P}^{3}$ is the projective
twistor space of $\mathbb{C}^{4}$. Now define a self-dual embedding as a
totally null holomorphic embedding
$\chi:\mathbb{C}^{2}\rightarrow\mathbb{C}^{4}$ such that vectors at a point
$t\in\mathbb{C}^{2}$ are mapped under $\chi_{\star}$ to vectors of the form
$v^{\alpha\dot{\alpha}}=\lambda^{\alpha}\tilde{\lambda}^{\dot{\alpha}}$ for
$\tilde{\lambda}^{\dot{\alpha}}$ fixed. We will call the image of such a map a
self-dual surface. If $\tilde{\lambda}$ is independent of $t$ then this
surface maps to a self-dual plane (or $\alpha$-plane) $Z$. Furthermore, for
any point on a self-dual embedding there is a tangent $\alpha$-plane passing
through $\chi(t)$ that is characterized by $\tilde{\lambda}(t)$. For brevity,
we say that $\chi$ is tangent to $\tilde{\lambda}$ at $t$. Now define the
self-dual prolongation $j_{s}\chi:\mathbb{C}^{2}\rightarrow\mathcal{F}$ by
$j_{s}\chi=(\chi,\tilde{\lambda})$, where dependence on $t$ is suppressed. The
prolongation satisfies a contact condition, that $\chi$ is tangent to
$\tilde{\lambda}$ for all $t\in\mathbb{C}^{2}$. Conversely, given a surface
$\psi:\mathbb{C}^{2}\rightarrow\mathcal{F}$, we say that it satisfies the
contact condition if $\psi=j_{s}\chi$ for some self-dual embedding $\chi$. A
map $f:\mathcal{F}\rightarrow\mathcal{F}$ is said to preserve the contact
condition if $f\circ j_{s}\chi$ satisfies the contact condition for any
$\chi$.111These constructions are all assumed to be local and defined in some
neighborhood, but are written globally for ease of notation. We then define

######  Definition 1.

A self-dual morphism is a holomorphic map
$f:\mathcal{F}\rightarrow\mathcal{F}$ which preserves the contact condition.

Defined in this way, a self-dual morphism naturally induces maps on self-dual
embeddings and self-dual planes

######  Definition 2.

Given a self-dual morphism $f$ and a self-dual embedding $\chi$, define the
contraction map $f\lrcorner\chi:=\pi_{1}\circ f\circ j_{s}\chi$. Furthermore,
for a self-dual plane $Z$ tangent to $\tilde{\lambda}$, define $f\lrcorner
Z:Z\rightarrow\mathbb{C}^{4}$ by $f\lrcorner Z(x)=\pi_{1}\circ
f(x,\tilde{\lambda})$ where $x\in Z$.

This map on surfaces was the starting point for the definitions in the
previous paper, and the two definitions are equivalent.

Now consider a $GL(n,\mathbb{C})$ connection with vector potential $A$
satisfying the ASDYM equations on $M_{\mathbb{C}}=\mathbb{C}^{4}$. Given a
self-dual morphism $f$, there is a natural definition for a pullback
connection $f^{*}A$. To see this, first restrict to a self-dual plane $Z$,
which can be parameterized linearly by coordinates on $\mathbb{C}^{2}$. The
contraction map $f\lrcorner Z$ then gives a self-dual embedding, and the
pullback connection $(f\lrcorner Z)^{*}A$ is integrable on $Z$ because the
curvature of $A$ vanishes on the self-dual planes tangent to $f\lrcorner Z$ as
a consequence of ASDYM. By varying $Z$ this allows us to define the bundle of
parallel sections on the twistor space, and to use the Penrose-Ward procedure
to define a connection on the pullback bundle, which defines the pullback
connection $f^{*}A$ and gives a solution of ASDYM. This requires that the
bundle of parallel sections is trivial for points $x\in\mathbb{C}^{4}$, which
will be shown with an explicit construction of $f^{*}A$.

For the construction, consider two points $x_{1},x_{2}\in Z$ and their images
$y_{i}=f\lrcorner Z(x_{i})$. Define a Wilson line for the pullback connection
by

$W^{*}_{Z}(x_{1},x_{2})=W_{f\lrcorner
Z}(y_{1},y_{2})=\mathcal{P}\exp\left(\int_{\gamma}A_{\mu}dx^{\mu}\right).$ (2)

Here the path of integration is any path $\gamma$ confined to the image of
$f\lrcorner Z$ starting at $y_{1}$ and ending at $y_{2}$. The Wilson line is
independent of path due to the integrability of the connection on self-dual
surfaces. We can then define the patching matrix used in the Penrose-Ward
correspondence by

$G=W^{*}_{Z}(q,p)=\tilde{H}H^{-1}$ (3)

where

$H=W^{*}_{Z}(p,x),\ \ \tilde{H}=W^{*}_{Z}(q,x).$ (4)

Here $p$ and $q$ are the points of intersection between $Z$ and self-dual
planes $P$ and $Q$ with twistor coordinates $\hat{P}=(0,0,1,0)$ and
$\hat{Q}=(0,0,0,1)$, as defined in the usual patching construction. The
patching matrix descends to the twistor space, and the splitting formula
guarantees that the bundle is trivial, so the bundle satisfies the conditions
of the Penrose-Ward correspondence. We therefore can recover a self-dual
pullback connection $f^{*}A$. This all assumes that the integrals are non-
singular, which depends on the details of the maps, connections and domains
under consideration.

To gain intuition for this construction, it is useful to derive an explicit
formula for $f^{*}A$. To this end, restrict to a self-dual plane $Z$ and use
the formula
$\tilde{\lambda}^{\dot{\alpha}}f^{*}A_{\alpha\dot{\alpha}}=H^{-1}\tilde{\lambda}^{\dot{\alpha}}\partial_{\alpha\dot{\alpha}}H$
from the Penrose-Ward correspondence, in addition to equation (4), to find

$v\cdot f^{*}A\lvert_{x}=(f\lrcorner Z)_{*}v\cdot A\lvert_{f\lrcorner Z(x)},\
\ \ x\in Z,\ v\in T_{x}Z.$ (5)

By varying the self-dual plane through a fixed point $x$ this gives the value
for null vectors $v\in T_{x}\mathbb{C}^{4}$, and the value on other vectors
can be recovered from linearity of the connection. Linearity and self-duality
follow from the Penrose-Ward construction, but it is instructive to derive
these results directly from equation (5), which can be expanded as

$(f\lrcorner Z)_{*}v\cdot A\lvert_{f\lrcorner
Z(x)}=v^{\alpha\dot{\alpha}}\Big{(}\frac{\partial
y^{\beta\dot{\beta}}}{\partial
x^{\alpha\dot{\alpha}}}A_{\beta\dot{\beta}}\Big{)}\Big{\lvert}_{f\lrcorner
Z(x)},\ x\in Z,\ v\in T_{x}Z,$ (6)

where $y^{\beta\dot{\beta}}=f\lrcorner Z(x)^{\beta\dot{\beta}}$. This equation
is degree one in both $\lambda$ and $\tilde{\lambda}$ and globally holomorphic
on $CP^{1}\times CP^{1}$, and so is linear by Liouville’s theorem. Self-
duality is equivalent to the connection being integrable on self-dual planes,
which follows from using equation (5) to show that $[v_{1}\cdot
D^{*},v_{2}\cdot D^{\star}]=0$, where $v_{1,2}\in T_{x}Z$ are linearly
independent vectors tangent to a self-dual plane $Z$. These arguments
generalize well to the $N=3$ supersymmetric case.

##  3 Super causal morphisms and N=3 SYM

In this section we will define a super causal morphism, which is an extension
of the previously defined causal morphisms to superspace. The supersymmetric
generalization is useful because of an interpretation of the N=3 SYM field
equations as an integrability condition on supersymmetric null lines [10][12].
This interpretation allows for a generalization of the arguments of the
previous section to the $N=3$ SYM field equations. Furthermore, solutions of
the usual YM field equations are special cases of the supersymmetric
solutions, and this will be used to show that a modified version of causal
morphisms are also symmetries of the ordinary YM field equations.

The definition of super causal morphisms follows closely to the previous
definitions. To begin, consider the superspace $\mathbb{C}^{4|4N}$ with
coordinates
$z^{A}=(x^{\alpha\dot{\alpha}},\theta^{i\alpha},\tilde{\theta}_{j}^{\dot{\alpha}})$
and supersymmetry generators
$q_{i\alpha}=\frac{\partial}{\partial\theta^{i\alpha}}+i\tilde{\theta}^{\dot{\alpha}}_{i}\frac{\partial}{\partial
x^{\alpha\dot{\alpha}}}$ and
$\tilde{q}_{\dot{\alpha}}^{i}=\frac{\partial}{\partial\tilde{\theta}^{\dot{\alpha}}_{i}}+i\theta^{i\alpha}\frac{\partial}{\partial
x^{\alpha\dot{\alpha}}}$. The lightlike lines through $z\in\mathbb{C}^{4|4N}$
and tangent to $(\lambda,\tilde{\lambda})$ are generated by fermionic
translation operators $T_{i}=\lambda^{\alpha}q_{\alpha i}$ and
$\tilde{T}^{i}=\tilde{\lambda}^{\dot{\alpha}}\tilde{q}_{\dot{\alpha}}^{i}$
which satisfy the algebra

$\\{T_{i},T_{j}\\}=\\{\tilde{T}^{i},\tilde{T}^{j}\\}=0,\ \
\\{T_{i},\tilde{T}^{j}\\}=2i\delta_{i}^{j}D,$ (7)

where
$D=\lambda^{\alpha}\tilde{\lambda}^{\dot{\alpha}}\partial_{\alpha\dot{\alpha}}$.
Define the correspondence space
$(\mathcal{F}^{6|4N},\pi_{1},\mathbb{C}^{4|4N})$ as the bundle over superspace
whose fiber at a point $z$ is the set of super null lines that intersect $z$.
These fibers are isomorphic to $CP^{1}\times CP^{1}$, corresponding to the
projective spinors $\lambda$ and $\tilde{\lambda}$ that generate bosonic
translations along a given null line.

The super null lines described above have one complex dimension and $2N$
fermionic dimensions, which can be parameterized by coordinates
$\sigma=(s,\xi^{i},\tilde{\xi}_{j})\in\mathbb{C}^{1|2N}$. The supersymmetry
generators on this line are $\partial_{s}$,
$q_{i}=\partial_{\xi^{i}}+i\tilde{\xi}_{i}\partial_{s}$ and
$\tilde{q}^{i}=\partial_{\tilde{\xi}_{i}}+i\xi^{i}\partial_{s}$. More
generally, we can also consider a super null curve, which is a morphism
$\chi:\mathbb{C}^{1|2N}\rightarrow\mathbb{C}^{4|4N}$ whose pushforward
$\chi_{*}$ takes the form

$(\partial_{s},q_{i},\tilde{q}^{k})\rightarrow(D,M_{i}^{j}T_{j},\tilde{M}^{k}_{l}\tilde{T}^{l}),$
(8)

at every point $\sigma\in\mathbb{C}^{1|2N}$, where the operators
$(D,T_{j},\tilde{T}^{l})$ are defined for a super null line tangent to $\chi$
at $\chi(\sigma)$. As with the self-dual case, we can then define the
prolongation $j_{c}\chi:\mathbb{C}^{1|2N}\rightarrow\mathcal{F}^{6|4N}$ by
$j_{c}\chi=(\chi,\lambda,\tilde{\lambda})$, and given a curve
$\psi:\mathbb{C}^{1|2N}\rightarrow\mathcal{F}^{6|4N}$ we say that it satisfies
the contact condition if $\psi=j_{c}\chi$ for some nonsingular null curve
$\chi$. As before, a map $f:\mathcal{F}^{6|4N}\rightarrow\mathcal{F}^{6|4N}$
preserves the contact condition if $f\circ j_{c}\chi$ satisfies the contact
condition for any $\chi$. Furthermore, for a super null line $L$ tangent to
$(\lambda,\tilde{\lambda})$, we also define $f\lrcorner L(z)=\pi_{1}\circ
f(z,\lambda,\tilde{\lambda})\ \forall z\in L$ as before.

In the supersymmetric case there is an extra consideration necessary to ensure
integrability of the pullback connection. To see this, consider a morphism
$f:\mathcal{F}^{6|4N}\rightarrow\mathcal{F}^{6|4N}$ which preserves the
contact condition. Given a super null line $L$, we want to demand that the
supersymmetry relations (7) are preserved under $(f\lrcorner L)_{*}$. By
construction, this pushforward takes the form of equation (8), where the
coordinates on $L$ are also chosen to satisfy the supersymmetry relations. To
preserve these relations, we must demand

$M^{i}_{j}\tilde{M}_{k}^{j}=\delta^{i}_{k}.$ (9)

This condition must be satisfied for every super null line $L$, which is the
condition that must be satisfied for integrability.

We can now define

######  Definition 3.

A super causal morphism is a holomorphic map
$f:\mathcal{F}^{6|4N}\rightarrow\mathcal{F}^{6|4N}$ which preserves the
contact condition, and preserves the supersymmetry relations (7) under
$(f\lrcorner L)_{\star}$ for tangent vectors to any super null line $L$.

Now consider an N=3 supersymmetric Yang-Mills field satisfying the field
equations on $\mathbb{C}^{4|12}$. This field can be defined by a
superconnection characterized by a one form $\Phi$ with components
$\Phi_{A}=(\omega_{i\alpha},\tilde{\omega}^{i}_{\dot{\alpha}},A_{\alpha\dot{\alpha}})$,
which defines covariant derivative operators

$\displaystyle Q_{i\alpha}=q_{i\alpha}+\omega_{i\alpha},\
\tilde{Q}^{i}_{\dot{\alpha}}=q^{i}_{\dot{\alpha}}+\tilde{\omega}^{i}_{\dot{\alpha}},\
D_{\alpha\dot{\alpha}}=\partial_{\alpha\dot{\alpha}}+A_{\alpha\dot{\alpha}}.$
(10)

The field equations are equivalent to integrability on super null lines [12],
which for a given line $L$ tangent to
$v^{\alpha\dot{\alpha}}=\lambda^{\alpha}\tilde{\lambda}^{\dot{\alpha}}$ are
given by equations (7) for the translation operators
$T_{i}=\lambda^{\alpha}Q_{i\alpha}$,
$\tilde{T}^{i}=\tilde{\lambda}^{\dot{\alpha}}\tilde{Q}_{\dot{\alpha}}^{i}$ and
$D=\lambda^{\alpha}\tilde{\lambda}^{\dot{\alpha}}D_{\alpha\dot{\alpha}}$.

Now, given a super causal morphism $f$, we can define the pullback connection
$f^{*}\Phi$ similarly to the self-dual case. To do so, consider a super null
line $L$, and the super null curve $f\lrcorner L$ it generates. The bundle of
parallel sections on these super null lines then generates a pullback
connection through the generalized Penrose-Ward construction. To calculate
this pullback connection, we can directly generalize equation (5) to

$v\cdot f^{*}\Phi\lvert_{z}=(f\lrcorner L)_{*}v\cdot\Phi\lvert_{f\lrcorner
L(z)},\ \ \ z\in L,\ v\in T_{x}L.$ (11)

As in the previous section, linearity of the pullback connection follows from
a variant of Liouville’s theorem. Integrability on lines follows from writing
(7) for the pullback translation operators defined on $L$, and then using (11)
and the assumption that $\Phi$ is integrable on lines. This implies that super
causal morphisms are symmetries of the $N=3$ SYM field equations.

##  4 Reduction to Yang-Mills field equations

The geometric interpretation of the YM field equations using field extensions
can be naturally understood by viewing these equations as a special case of
the N=3 SYM field equations for a Yang-Mills super multiplet with the scalar
and spinor fields set to zero [10][12][13][19]. In a similar spirit, it is
possible to use a modified definition of causal morphisms to generate an N=3
super causal morphism which preserves the property that the scalar and spinor
fields equal zero, thus forming a symmetry of the YM field equations.

A causal morphism can be defined as an $N=0$ super causal morphism, or a map
$f:\mathcal{G}\rightarrow\mathcal{G}$ that preserves the contact condition,
where $\mathcal{G}=\mathcal{F}^{6|0}$ is the usual ambitwistor correspondence
space. Given such a function, we can construct an extended morphism
$\hat{f}:\mathcal{F}^{6|4N}\rightarrow\mathcal{F}^{6|4N}$ given by

$\hat{f}(g,\theta^{i\alpha},\tilde{\theta}_{j}^{\dot{\alpha}})=(f(g),[V^{-1}]^{\alpha}_{\beta}\theta^{i\beta},[\tilde{V}^{-1}]^{\dot{\alpha}}_{\dot{\beta}}\tilde{\theta}_{j}^{\dot{\beta}}),\
g\in\mathcal{G},$ (12)

where $V$, $\tilde{V}$ are invertible matrix functions of $g$. Now restrict to
a super null line $L$ tangent to a null vector
$v^{\alpha\dot{\alpha}}=\lambda^{\alpha}\tilde{\lambda}^{\dot{\alpha}}$. Along
$L$, the supersymmetry relations (7) are preserved if and only if

$v^{\beta\dot{\beta}}V^{\alpha}_{\beta}\tilde{V}^{\dot{\alpha}}_{\dot{\beta}}=((f\lrcorner
L_{0})_{*}v)^{\alpha\dot{\alpha}},$ (13)

where $L_{0}$ is the bosonic projection of $L$. The existence of
holomorphically varying matrix functions $V$ and $\tilde{V}$ satisfying this
condition is the extra modification necessary to extend a causal morphism $f$
to $\hat{f}$, and will be assumed.

The above construction yields a symmetry of the $N=3$ SYM field equations, but
we must also show that solutions of the YM field equations, with scalar and
spinor fields set to zero, are preserved by these extended causal morphisms.
To do so, we will use two results proved by Harnad et. al. [13]. In that
paper, theorem 3.3 characterizes the form of the superconnection induced from
a solution of the YM field equations when embedded as a gauge fixed N=3 SYM
connection, which is

$\displaystyle\omega_{i\alpha}=\tilde{\theta}_{i}^{\dot{\alpha}}h_{\alpha\dot{\alpha}}(x^{\beta\dot{\beta}},\tau^{\beta\dot{\beta}}),\
\
\tilde{\omega}^{i}_{\alpha}=\theta^{i\alpha}\tilde{h}_{\alpha\dot{\alpha}}(x^{\beta\dot{\beta}},\tau^{\beta\dot{\beta}}),$
(14)

where
$\tau^{\beta\dot{\beta}}=\sum_{i}\theta^{i\beta}\tilde{\theta}^{\dot{\beta}}_{i}$,
$A_{\alpha\dot{\alpha}}=A_{\alpha\dot{\alpha}}(x^{\beta\dot{\beta}},\tau^{\beta\dot{\beta}})$,
and the gauge condition is
$\theta^{i\alpha}\omega_{i\alpha}+\tilde{\theta}^{i}_{\dot{\alpha}}\tilde{\omega}_{i}^{\dot{\alpha}}=0$.
Conversely, corollary 4.3 shows that a gauge-fixed connection that is
integrable on super null lines and takes the above form corresponds to an
$N=3$ extended solution of the YM field equations.

Based on these considerations, showing that the extended causal morphisms
preserve the form of equation (14) and the gauge condition implies that they
preserve solutions of the YM field equations. To show this, restrict to a
super null line $L$ and use (11) to compute the pullback connection of (14),
which gives

$\displaystyle\begin{split}&\lambda^{\alpha}\hat{f}^{*}\omega_{i\alpha}(z)=\tilde{\theta}_{i}^{\dot{\beta}}\lambda^{\alpha}\left(V^{\beta}_{\alpha}h_{\beta\dot{\beta}}(x^{\prime},\tau^{\prime})\right),\\\
&\tilde{\lambda}^{\dot{\alpha}}\hat{f}^{*}\tilde{\omega}^{i}_{\dot{\alpha}}(z)=\theta^{i\beta}\tilde{\lambda}^{\dot{\alpha}}\left(\tilde{V}^{\dot{\beta}}_{\dot{\alpha}}\tilde{h}_{\beta\dot{\beta}}(x^{\prime},\tau^{\prime})\right),\end{split}$
(15)

where $x^{\prime}$, $\tau^{\prime}$ are evaluated at $\hat{f}\lrcorner L(z)$
and $z\in\mathbb{C}^{4|4N}$. Now consider two points
$z_{1},z_{2}\in\mathbb{C}^{4|4N}$ with $x_{1}=x_{2}$ and $\tau_{1}=\tau_{2}$,
lying on two parallel lines. Under $\hat{f}\lrcorner L$, $\tau$ transforms as
$\tau^{\alpha\dot{\alpha}}\rightarrow[V^{-1}]^{\alpha}_{\beta}[\tilde{V}^{-1}]^{\dot{\alpha}}_{\dot{\beta}}\tau^{\beta\dot{\beta}}$,
so the quantities in parentheses are the same for these two points and
parallel lines, but the line can be varied, so this is true for any
$(\lambda,\tilde{\lambda})$. By linearity, this implies that the connection
has the form of equation (14), as desired. Furthermore, the gauge condition
can be written
$\tau^{\alpha\dot{\alpha}}(h_{\alpha\dot{\alpha}}+\tilde{h}_{\alpha\dot{\alpha}})=0$.
For $\tau^{\alpha\dot{\alpha}}$ proportional to a null vector this condition
is preserved, but this must be true for any null line, so again by linearity
the gauge condition is preserved. This implies that the pullback connection
corresponds to an $N=3$ extended solution of the YM field equations.

##  5 Discussion

We have shown that self-dual, $N=3$ super causal and causal morphisms yield
symmetries of the ASDYM, $N=3$ SYM and YM field equations, respectively. To
further understand these symmetries, it will be necessary to classify their
solutions and to investigate their action on concrete examples of YM fields.
Some partial results were found in the previous paper [16], where examples of
self-dual morphisms were constructed from holomorphic endomorphisms of twistor
space. A method was also developed to construct causal morphisms from these
self-dual morphisms. Although these constructions provide examples of
solutions, it will be important to find a more complete classification. In
particular, it is likely that there are more general examples than those
constructed from endomorphisms of the twistor space, or super ambitwistor
space, which could be analogous to holomorphic functions that preserve the
real line in two dimensions. This preliminary interpretation is based on the
CR ambitwistor space used in [20], but will require further investigation to
make precise. It will also be important to understand how these maps are
related to other well known hidden symmetries for these equations.

There are many additional avenues of further research. Here the action of
these maps was only considered for classical Yang-Mills fields, but the
ultimate goal is to further understand the quantum theory. Furthermore, it
will be interesting to consider how gravitational fields transform under these
maps. In this vein, one could define a causal manifold with coordinate
transformations that are morphisms of these types, in analogy to the
definition of Riemann surfaces for holomorphic functions. Due to the nonlocal
nature of these maps, the theory could lead to interesting new mathematics.

## References

  * [1] L. Dolan, _A new symmetry group of real self-dual yang-mills theory_ , _Physics Letters B_ 113 (1982) 387.
  * [2] L.-L. Chau, G. Mo-Lin, A. Sinha and W. Yong-Shi, _Hidden-symmetry algebra for the self-dual yang-mills equation_ , _Physics Letters B_ 121 (1983) 391.
  * [3] A.D. Popov, _Self-dual yang-mills: Symmetries and moduli space_ , _Reviews in Mathematical Physics_ 11 (1999) 1091.
  * [4] L. Mason and D. Skinner, _Dual superconformal invariance, momentum twistors and grassmannians_ , _Journal of High Energy Physics_ 2009 (2009) 045.
  * [5] N. Arkani-Hamed, F. Cachazo and C. Cheung, _The grassmannian origin of dual superconformal invariance_ , _Journal of High Energy Physics_ 2010 (2010) .
  * [6] J. Drummond, J. Henn and J. Plefka, _Yangian symmetry of scattering amplitudes in n = 4 super yang-mills theory_ , _Journal of High Energy Physics_ 2009 (2009) 046.
  * [7] R.S. Ward, _On self-dual gauge fields_ , _Physics Letters A_ 61 (1977) 81.
  * [8] M. Atiyah, N. Hitchin, V. Drinfeld and Y. Manin, _Construction of instantons_ , _Physics Letters A_ 65 (1978) 185 .
  * [9] M. Atiyah, A. nazionale dei Lincei and S. normale superiore (Italy), _Geometry of Yang-Mills fields_ , Lezioni fermiane, Scuola normale superiore (1979).
  * [10] E. Witten, _An interpretation of classical yang-mills theory_ , _Physics Letters B_ 77 (1978) 394 .
  * [11] J. Isenberg, P.B. Yasskin and P.S. Green, _Non-self-dual gauge fields_ , _Physics Letters B_ 78 (1978) 462 .
  * [12] J. Harnad, J. Hurtubise, M. Legare and S. Shnider, _Constraint equations and field equations in supersymmetric n = 3 yang-mills theory_ , _Nuclear Physics B_ 256 (1985) 609.
  * [13] J. Harnad, J. Hurtubise and S. Shnider, _Supersymmetric yang-mills equations and supertwistors_ , _Annals of Physics_ 193 (1989) 40.
  * [14] E. Witten, _Perturbative gauge theory as a string theory in twistor space_ , _Communications in Mathematical Physics_ 252 (2004) 189.
  * [15] M. Atiyah, M. Dunajski and L.J. Mason, _Twistor theory at fifty: from contour integrals to twistor strings_ , _Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 473 (2017) 20170530.
  * [16] E.B. Baker, _Causal and self-dual morphisms in four complex dimensions_ , 2022\.  10.48550/ARXIV.2203.07952.
  * [17] M. Dunajski, _Solitons, Instantons, and Twistors_ , Oxford Graduate Texts in Mathematics, OUP Oxford (2010).
  * [18] R. Ward and R. Wells, _Twistor Geometry and Field Theory_ , Cambridge Monographs on Mathematical Physics, Cambridge University Press (1991).
  * [19] M. Eastwood, _Supersymmetry, twistors, and the yang-mills equations_ , _Transactions of the American Mathematical Society_ 301 (1987) 615.
  * [20] L. Mason and D. Skinner, _An ambitwistor yang–mills lagrangian_ , _Physics Letters B_ 636 (2006) 60.


# Learning to Attack Powergrids with DERs

Eric MSP Veith  Nils Wenninghoff  Stephan Balduin  Thomas Wolgast  Sebastian
Lehnhoff  OFFIS – Institute for Information Technology, R&D Division Energy,
Escherweg 2, 26121 Oldenburg (Old.), Germany  Carl von Ossietzky University
Oldenburg, Ammerländer Heerstr. 114–118, 26126 Oldenburg (Old.),

###### Abstract

In the past years, power grids have become a valuable target for cyber-
attacks. Especially the attacks on the Ukrainian power grid has sparked
numerous research into possible attack vectors, their extent, and possible
mitigations. However, many fail to consider realistic scenarios in which time
series are incorporated into simulations to reflect the transient behaviour of
independent generators and consumers. Moreover, very few consider the limited
sensory input of a potential attacker. In this paper, we describe a reactive
power attack based on a well-understood scenario. We show that independent
agents can learn to use the dynamics of the power grid against it and that the
attack works even in the face of other generator and consumer nodes acting
independently.

Voltage Control,

Deep Reinforcement Learning,

Attack Vectors,

Distributed Energy Resources,

Vulnerability Analysis,

Cyber-Physical System,

###### keywords:

Research

## Introduction

The attack on the Ukrainian power grid has shown that power grids have become
valuable targets [1, 2]. Cyber-attacks against this critical infrastructure
had previously been discussed in theory, but were not considered an imminent
threat yet. Nowadays, it is considered entirely possible that an attacker
gains acccess to the Supervisory Control and Data Acquisition (SCADA)
system—either physically or by virtually exploiting a bug that leads to a
priviledge escalation—and is subsequently able to control the physical asset
[3]. This is especially important considering that power grids world-wide
transition from well-controlled, central power generation and simple consumers
to a distributed architecture, in which nodes become prosumers and Distributed
Energy Resources (DERs) are the norm. Not only has the number of nodes that
influence the power grid increased, but also the parties involved.

A well-understood analysis of how an inverter-based generator can attack the
power grid has been published by Ju and Lin [4]. Their analysis considers DERs
as potential attack vectors, assuming that an attacker has taken over control
of one or many inverters (e. g., wind turbines or Photovoltaic (PV) systems).
Without knowledge of the grid’s architecture, the authors show analytically
that the attacker is able to use the reactive power control scheme of other
nodes against the grid by introducing a osciallating maximum reactive power
feed-in/consumption behavior. This way, an attacker can do damage of nearly
twice its own reactive power feed-in/consumption capability by ‘leveraging’
other generators.

Ju and Lin’s paper is a prime example for what we call excellent first-
generation analyses: They focus on an isolated issue and treat the power grid
as a still object. I. e., while stringent in their analysis, Ju and Lin do not
consider normal power grid operation: While the attacker does its work, other
generators react according to their voltage control scheme, but no other nodes
are active. During a normal power grid operation, other consumers and
generators would still follow their schedule. In general, one cannot easily
deduce from the “petrified” grid how the influence of time series for weather
or consumption influence the effectiveness of the attack. Two scenarios are
equally possible: That the attack is mitigated by other nodes as their feed-in
or consumption provides enough background noise that renders the attack
ineffective, or that an attack might eventually synchronize with the behavior
of other nodes as to leverage at least momentarily their feed-in or
consumption for the attack, too.

In this paper, we argue that a learning agent based on Deep Reinforcment
Learning (DRL) can learn to discover this attack. We base our research on the
same premisses as Ju and Lin, namely no knowledge of the power grid structure,
sensory data limited to the local attacker node only, and the same reactive
power control scheme as Ju and Lin used and [4, 5] introduced. However, we
extend the scenario by providing a full simulation of a realistic power grid
with acting independent nodes. Thus, our hypotheis is that a “live” power grid
will foil the attack in its original, obvious fashion, but a variant of the
original attack can still be discovered by a learning agent without knowledge
of the rest of the grid. Hence, this paper can serve as a blue-print for a
second generation of more realistic weakness analysis examples.

The remainder of this paper is structured as follows: We review the relevant
literature in Related Work. The section on Experiment Design discusses the
environment, sensor-actuator assignments of the attacker agent, as well as the
time series and learning algorithms employed. Simulation Results shows results
of our simulation and the validation of our theory. Our Conclusion also offers
an outlook to future work.

## Related Work

The literature that covers attacks on power systems can roughly be divided
into two groups: cyber attacks and physical manipulation of the power system
[6].

Especially cyber attacks are investigated extensively. The most important
class of cyber attacks are false data injection attacks to manipulate state
estimation [7]. However, also false data injection attacks to manipulate load
forcasting [8] or automatic voltage control [9] are possible. Further, time
delay attacks, replay attacks, and theft attacks in the power system are
investigated in literature [10]. For a comprehensive review of cyber attacks
refer to Mehrdad et al. [10].

However, physical manipulation of the power system attracts more and more
research interest in recent years. One or multiple compromised generators can
manipulate the decentralized voltage control by leveraging the voltage control
behavior of other non-attacking generators [4]. Such publications present an
analytical approach that focuses on the feasibility or impact of a specific
attack. However, in a previous literature survey, we found that analyzing
Cyber-Physical Systems (CPSs) with learning agents is a neccessary research
topic that allows to uncover new weaknesses despite the complexity of modern
power grids [11]. This is backed up by other scientist’s reseach. E. g., Ni
and Paul use Q-learning to identify the minimum number of line switchings to
achieve a blackout scenario, under the assumption of knowledge about branch
status [12]. Later, they have expanded their approach to sequential attacks
[13]. Wolgast et. al demonstrate how DRL can be used to identify unknown
attack strategies that maximize profit on ancillary service markets [6].

While most previous publications focus on specific attack strategies or
vulnerabilites—e. g., cyber attacks, manipulation of the powerflows, energy
markets—we have previously presented an approach called Adversarial Resilience
Learning (ARL) [14] that allows the agents to explore a system while
maintaining a pressure to learn through an adversarial agent. A complex co-
simulation set-up including a power grid, an energy market, and a
corresponding Information and Communication Technology (ICT) network is based
on this methodology [15].

DRL lends itself very well for these analysis approaches, inspired by the
“superhuman” performance in the game of Go. From the resurrection of (model-
free) reinforcement learning with the 2013 hallmark paper by Mnih et al. [16],
to the publicly-noted achievements of AlphaGo, AlphaGo Zero, AlphaZero, and
(model-based) MuZero [17, 18, 19, 20, 21, 22], DRL has attracted much
attention outside of the Artificial Intelligence (AI) domain. Much of the
attention it has gained comes from the fact that, especially for Go, DRL has
found strategies better than what any human player had been able to, and this
without human teaching or domain knowledge. Baker et al. underpin the
usefulness of DRL, specifically with competing agents, as an analysis tool:
They describe how agents are able to uncover loopholes in the underlying
simulation software, thus discovering a new class of strategies to achieve
their goals [23]. Such modern DRL-based experimentation approaches seldomly
use classic Q-learning [18] or A2C [24] algorithms, but use modern variants
such as Soft Actor Critic (SAC) [25] instead.

## Experiment Design

### Environment

For our experiment, we employ the MIDAS111https://gitlab.com/midas-
mosaik/midas software suite. It incorporates the
PySimMods222https://gitlab.com/midas-mosaik/pysimmods software package that
contains numerous models for power grid units, such as batteries, PV or
Combined Heat and Power (CHP) power plants. The grid model is a CIGRÉ Medium
Voltage (MV) grid model [26] as show in fig. 1.

Each node has a constant load of $560+j320$ kVA attached to it in order to
account for this real power feed-in that occurs naturally because of the
inverter model. The goal is to maintain a voltage magnitude close to
$1.0\text{\,}\mathrm{p}\,\mathrm{u}\,$ on every bus if no action is taken. The
grid has a number of PV plants as inverter-based generators attached, as well
as a super market and a small hotel that serve as consumers. These consumers
follow standard load profiles according to their roles. The PV plants’ output
is dependent on the solar irridation, which is governed by a multi-year
weather dataset from Bremen, Germany.333Publicly available from
https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate_urban/hourly/.
Each PV has apparent power output of
$50\text{\,}\mathrm{M}\mathrm{V}\mathrm{A}$. We deliberately chose this high
value in order to demonstrate the attack within a short simulation time frame.
For the sake of realism, one could readily assume the inverter nodes to
represent wind farms. To ease modelling and without weakening the argument, we
have resorted to PV plants instead.

The attacker controls the buses 3, 4, and 8. Buses 5 to 7, 9, 11, and 13 are
voltage controlled. Each benign voltage controller applies the distributed
control law [27]:

$\bm{q}(t+1)=\left[\bm{q}(t)-\bm{D}(\bm{V}(t)-1)\right]^{+}~{},$ (1)

where the notation $[\cdot]^{+}$ denotes a projection of invalid values to the
range $[\underline{\bm{q}^{g}},\overline{\bm{q}^{g}}]$, i. e., to the feasible
range of setpoints for $q(t+1)$ of each inverter. $\bm{D}$ is a diagonal
matrix of step sizes. We have chosen a step size of 15 for each Q controller.

Figure 1: CIGRÉ benchmark grid used for simulation

### Scenarios

The experimental validation of our hypothesis rests on the following
consecutive scenarios. We define these scenarios to work from the known,
working attack scenario [4] towards a scenario that implements our hypothesis.
For each of the three consecutive scenarios, we change only one
characteristic.

Scenarios are defined through three properties: Its _goal_ describes the
purpose of the scenario; _setup_ offers a concise description of the
experiment setup and simulation environment; _expected result_ allows to build
a chain of arguments that interconnects the three scenarios.

This section contains only the description; evaluation of the actual results
is offered in the Simulation Results section.

#### Scenario 1: Reproduction of an Original Paper

Goal

Reproduce the original paper by Ju and Lin to note the characteristics of the
simulation environment and to verify the effectiveness of the originally
described attack [4].

Setup

Consumers are not connected to the power grid. The weather provider offers
weather data for a sunny summer day at noon, for each simulation step. I. e.,
the PV plants can be operated at peak capacity.

Expected Result

The osciallating behavior of the attacking PV plants forces the Q controllers
into a similar pattern, leading to a voltage band violation.

#### Scenario 2: Original Attack, with Time Series

Goal

Evaluate the influence of time series data on the original attack

Setup

Consumers and PV plants are connected to the grid. All nodes are influenced by
the available time series data, i. e., the PV plants by the weather data, the
consumers by their load profiles.

Expected Result

The effectiveness of the original attack will depend on the current weather;
the resulting data will show a clear yearly pattern.

#### Scenarion 3: Learn to Attack, with Time Series

Goal

Employ a learning agent to re-discover the attack described for Scenario 2.

Setup

As in Scenario 2. However, the attacker is now a learning agent that employs a
DRL algorithm to learn to attack the power grid.

Expected Result

The agent learns to create voltage band violations. We expect the learning
attacker to cope better with the influence of the weather data than the
original attacker.

### Reward Design

In DRL, the design of the reward function is one of the most important steps.
Our experimenting framework, palaestrAI, separates _reward_ from the
_objective_ of an agent. This is motivated by a shortcoming of simpler DRL
experiments such as cartpole: The reward of an agent is given by the
environment because of a state transition. E. g., for Q-learning, we commonly
define the best action in a given state $s$ as its Q value:

$Q(s,a)=r_{s,a}+\gamma\max_{a^{\prime}\in\mathbf{A}}Q(s^{\prime},a^{\prime})~{}.$
(2)

Simple, single-agent DRL setups assume that the reward is given by their
environment and relates to the agent’s goal. In multi-agent environments, this
is not the case. Here, we have to distinguish between the _performance of the
environment_ and the _performance of the agent_ [14]. I. e., if $r_{s,a}$ is
equivalent to the environment’s performance when transitioning from state $s$
given action $a$, then the evaluation of a particular agent that considers
this state transition should be denoted as $p(r_{s,a})$.

For the simulation, we define the reward as a vectorized function of all bus
voltage magnitudes after all agent actions are applied:

$\bm{r}_{s,a}=\left[V_{1},V_{2},\dotsc,V_{n}\right]^{\top}~{}.$ (3)

Let $\bm{i}_{A}$ denote the vector of sensors (inputs) the attacker has access
to. We can then assume a function
$\mathit{filter}:(\bm{r}_{s,a},\bm{i}_{A})\mapsto\bm{r}^{\prime}_{s,a}$ that
reduces the vector $\bm{r}_{s,a}$ such that it includes only those bus voltage
magnitudes the attacker has sensor access to. Then, we express the objective
function of the attacker as:

$p_{A}(\bm{r^{\prime}}_{s,a})=2\left[A\cdot\exp\left(-\frac{\left(\frac{1}{n}\sum_{r\in\bm{r^{\prime}}_{s,a}}(r)-\mu\right)^{2}}{2\sigma^{2}}-C\right)\right]~{}.$
(4)

We set $\mu=1$, $\sigma=-0.05$, $C=-1.2$, and $A=-2.5$. The objective function
is inspired by a Gaussian PDF. Figure 2 shows that, essentially, the attacker
is rewarded for a voltage band violation, while a mean voltage close to
$1.0\text{\,}\mathrm{p}\mathrm{u}$ yields a negative value. This essentially
encourages the attacker to force an extreme state with regards to the voltage
magnitude; moreover, the influce of the weather data will drive the attacking
agent towards the voltage magnitude boundares it can easily enforce.

Figure 2: Objective function of the attacking agent

## Simulation Results

### Scenario 1

Scenario 1 serves as base line in order to reproduce and assertain the attack
documented by Ju and Lin [4]. Figure 3(a) shows how the regular Volt/VAr
controller within an acceptable boundary close to
$1.0\text{\,}\mathrm{p}\mathrm{u}$. Figure 3(b) shows the same buses after the
attacker has begun the osciallating VAr behavior. Clearly, the osciallation of
the $V(t)$ curve is visible. In addition, the second peak is higher as the
first one. The attacker’s behavior is always the same: It alternates between
$100\text{\,}\%$ and $-100\text{\,}\%$ of its possible reactive power
injection/consumption. It confirms the general effectiveness of the attack,
sans realistic time series.

(a) Voltage magnitudes on targeted buses before attack

(b) Voltage magnitude at benign buses during attack

Figure 3: System behavior for the simple alternating attacker

### Scenario 2

Scenario 2 enables time series for weather and (regular) consumers, but does
not change the configuration of the Volt/VAr controller or the attacker.

Figure 4 shows data from the month of July. The voltage magnitude has much
higher values, even exceeding $1.25\text{\,}\mathrm{p}\mathrm{u}$. The
oscillating behavior can still be observered, but the attacker can force only
2 to 3 oscillation periods on the Volt/VAr controllers before the load flow
calculation does not converge anymore.

The simulated month of July is exemplatory in showing that weather or consumer
effects can lead to an increased impact of the attack. Comparing the voltage
magnitude in fig. 4(a) with the solar irridiation plot in fig. 4(b), we
correlate the increased voltage magnitude with the increase solar irridiation
of the course of the day. Considering the distributed VAr control scheme eq.
1, we see that the controller considers the current voltage magnitude as well
as the previous VAr setpoint. Since the VAr setpoints are given as values
relative to the inverter’s current maximum, the controller overshoots the
target, thereby amplifying the attack.

(a) Voltage magnitudes

(b) Solar irridiation

Figure 4: System behavior for the simple alternating attacker with time series

### Scenario 3

Scenario 3 introduces a learning agent as attacker in addition to scenario 3.
To summarize, the third scenario then employs the CIGRÉ network with PV feed-
in, time series both for weather and for consumers, the Volt/VAr controller
logic eq. 1 defines, as well as a learning attacker. The attacker uses SAC as
learning algorithm.

Figure 5(a) confirms that the agent has learned to reproduce the attack. Note,
that the agent was given no indication that the alternating setpoints behavior
would lead to this result.

In fact, the denser nature of the plot in fig. 5(a) shows that the attacker
learns to alternate the setpoints in a much quicker succession. Had the
deterministic attacker in secarios 1 and 2 a holdoff time of 25 steps, none
such was implemented for the SAC-based attacker. This allowed the attacker
agent to learn the value of $\bm{D}$, i. e., the step size of the defender.

That the osciallating Volt/VAr control leads to the desired attack becomes
evident from fig. 5(b). It shows the attacker’s objective (cf. fig. 2). The
objective incentivizes a voltage band violation and punishes values close to
$1.0\text{\,}\mathrm{p}\mathrm{u}$. Equation 4 therefore does not provide an
immediate incentive for the attacker to adapt the osciallating behavior; in
rather disencourages it. However, in face of the Volt/VAr control regime eq.
1, the attacker learns that the osciallating behavior is the only sensible
course of action.

(a) Voltage magnitudes

(b) Solar irridiation

Figure 5: System behavior for the simple alternating attacker with time series

## Discussion and Limitations

Volt/VAr controllers do not provide protection against malicious network
actors. Deterministic Volt/VAr control schemes are vulnerable: Once known, it
can be exploited in theory with any deterministic attacker. However, we show
that even in absence of any knowledge about the grid topology or the control
scheme, attacks are possible.

In practice, the impact of these attacks is mitigated by many other
influencing factors. Usually, a power grid’s design provides enough robustness
against these kind of attacks. However, the vulnerability remains and can be
exploited. With the general tendency to eshew physical grid extension in favor
of a more intelligent grid control, the grid’s inherent “buffer” against these
kind of attacks grows ever smaller. While efficient grid control instead of
physically deploying more cabling and assets is generally favorable since it
is more sustainable, it also works in favor of these kinds of attacks.

The threat to the power grid depends on the degree of influence of the
attacker. Our experiment design (cf. Experiment Design) forms a limitation in
this regard: In order to pose a consistent threat against the CIGRÉ benchmark
grid, the inverter-based generators have been scaled to
$45\text{\,}\mathrm{M}\mathrm{V}\mathrm{A}$ peak output. Without loss of
generability, we argue that our PV plant model can be replaced by wind farms,
where $45\text{\,}\mathrm{M}\mathrm{W}$ peak output constitute a reasonable
value.

Loads can both strengthen and weaken the deterministic attack. If loads and
attackers act in the same direction at the same time, the effect is
additionally amplified; if the attacker acts against the load, it weakens the
attack as the attacker effectively acts like a benign Q controller. Both
effects have an impact on the reaction of the voltage regulator, because it
does not react to the action of the attacker, but to the bus voltage.

Due to the design of the objective and the configuration of the power grid,
the voltage only oscillates in the range above overvoltage. The objective of
the agent evaluates voltages close to $1.0\text{\,}\mathrm{p}\mathrm{u}$
poorly. The grid tends to have an overvoltage due to the configuration.
Undervoltage would only be possible by an action of the agent. During
exploration, a sufficiently large undervoltage does not occur, so the agent
only learns to generate an overvoltage. In a different network configuration,
oscillation between overvoltage and undervoltage would also be possible.
Adjusting the objective could also bring this about. However, this could
minimize the damage, since the oscillation would have to be larger to get into
critical areas. The attack could also be implemented in a network
configuration with a tendency to undervoltage.

DRL can reproduce the attack and exploit the Q controller’s vulnerabilities.
Since the Volt/VAr control scheme employed has the vulnerabilities
demonstrated in scenario 1 and 2, a learning agent can find and exploit them.
In addition, by exploring the solution space and adaptive properties, the
agent is able to optimize the attack.

The adaptive capability gives the attacker an unfair advantage over the
defender. If the defender has a vulnerability, the attacker can search for it,
find it, and exploit it, given enough time. The defender lacks an adaptive and
adequate way to react. Past publications in DRL research have shown that
especially multi-agent autocurricula—our setup also being one—can find any
possible solution vector and even explore and exploit weaknesses in the
underlying simulation environment to this end [23].

This attack shows that adaptive controllers are needed that can react
depending on the situation and adapt to new attack patterns. The task of
defending the power grid against attacks is more complex, since all possible
attack vectors must be covered. However, in the case of learning attackers,
the defender also represents a possible attack vector, as these experiments
show. Complete coverage is therefore difficult to implement, due to the
possible further exploration of the attacker. Instead, robustness should be
pursued through continual adaptation. The continual adaptation and self-
optimization of the defender reduces the risk of becoming a target of the
attack itself.

Like the work of Ju and Lin [4], simplifying assumptions were made in this
scenario. There is no intertia in the power grid; grid codes concerning, e.
g., voltage gradients, are also absent. A Distribution System Operator (DSO)
would have the attacker disconnected after the first oscillation due to
violation of these codes. As future work, we will extend our scenarios with
exactly these grid codes, posing the attacker to find a new attack vector.

This work shows that theoretical attack scenarios can have an impact even as
the degree of realism increases. The identified vulnerabilities should be
addressed in future work. However, due to the abstraction required and the
still very sterile simulation environment, the threat of the identified attack
vector to real power systems is low. However, it demonstrates very clearly how
more realistic, complex simulations and scenarios are urgently needed to
identify potential real-world vulnerabilities and develop countermeasures at
an early stage.

## Conclusion

This paper has considered voltage attacks with distributed DERs and
investigated a learning agent’s ability to discover and exploit a Volt/VAr
control scheme. Through scenarios that build on each other, we experimentally
verified that simple oscillating behavior of a malicious agent can cause a
voltage disruption by forcing other benign agents into the same oscillating
behavior, thus amplifying the attack; moreover, we verified that this behavior
is still relevant even with time series for consumption and weather applied to
the simulation; and finally, we showed that DRL can be used to uncover such a
vulnerability.

We have seen that the DRL agent is able to drive a far more effective attack
by uncovering the step size of the Volt/VAr control scheme. We assume that an
adaptive Volt/VAr controle scheme, possibly also backed by a learning agent,
could mitigate the attack.

In the future, we will investigate this defender. We also plan to employ a
more realistic grid by implementing grid codes and asset constraints.

## Glossary

AI

    Artificial Intelligence
ARL

    Adversarial Resilience Learning
CHP

    Combined Heat and Power
CPS

    Cyber-Physical System
DER

    Distributed Energy Resource
DRL

    Deep Reinforcment Learning
DSO

    Distribution System Operator
ICT

    Information and Communication Technology
MV

    Medium Voltage
PV

    Photovoltaic
SAC

    Soft Actor Critic
SCADA

    Supervisory Control and Data Acquisition

## Funding

This work was funded by the German Federal Ministry of Education and Research
through the project PYRATE (01IS19021A).

## Availability of data and materials

Relevant code and experiment definitions to reproduce all experiment runs for
this paper are available at the following Gitlab repository:
https://gitlab.com/Niwen/attack-paper

## Author’s contributions

Eric MSP Veith conducted experiment runs, analysis, and wrote the
corresponding sections. He is also one of the main contributors to the
framework palaestrAI that was used to carry out the analyses.

Nils Wenninghoff implemented the SAC learning agent. Together with Eric MSP
Veith, he created the initial reward design and was responsible for the Q
control scheme. He is also one of the main contributors to the framework
palaestrAI that was used to carry out the analyses.

Stephan Balduin contributed the power grid model, models for PV, and all time
series. He is the main author of the simulation modelling framework MIDAS.

Thomas Wolgast sanitized the attack design and fine-tuned the scenario design.

Sebastian Lehnhoff was responsbile for the overall plausibility and
verification of the simulation models and testbed.

## Competing interests

The authors declare that they have no competing interests.

## References

  * [1] Styczynski, J., Beach-Westmoreland, N.: When the lights went out: Ukraine cybersecurity threat briefing. Booz Allen Hamilton 12, 20 (2016)
  * [2] Reuters: Ukrainian banks, electricity firm hit by fresh cyber attack. Reuters (2017)
  * [3] E-ISAC: Analysis of the Cyber Attack on the Ukrainian Power Grid: Defense Use Case
  * [4] Ju, P., Lin, X.: Adversarial attacks to distributed voltage control in power distribution networks with DERs, 291–302. doi:10.1145/3208903.3208912. ISBN: 9781450357678
  * [5] Zhu, H., Liu, H.J.: Fast local voltage control under limited reactive power: Optimality and stability analysis 31(5), 3794–3803. doi:10.1109/TPWRS.2015.2504419. Accessed 2021-12-23
  * [6] Wolgast, T., Veith, E.M., Nieße, A.: Towards reinforcement learning for vulnerability analysis in power-economic systems. Energy Informatics 4(S3) (2021)
  * [7] Liang, G., Zhao, J., Luo, F., Weller, S.R., Dong, Z.Y.: A Review of False Data Injection Attacks Against Modern Power Systems. IEEE Transactions on Smart Grid 8(4), 1630–1638 (2017). doi:10.1109/TSG.2015.2495133
  * [8] Chen, Y., Tan, Y., Zhang, B.: Exploiting vulnerabilities of load forecasting through adversarial attacks. In: Proceedings of the Tenth ACM International Conference on Future Energy Systems - e-Energy ’19, pp. 1–11. ACM Press, New York, New York, USA (2019)
  * [9] Chen, Y., Huang, S., Liu, F., Wang, Z., Sun, X.: Evaluation of Reinforcement Learning-Based False Data Injection Attack to Automatic Voltage Control. IEEE Transactions on Smart Grid 10(2), 2158–2169 (2019). doi:10.1109/TSG.2018.2790704
  * [10] Mehrdad, S., Mousavian, S., Madraki, G., Dvorkin, Y.: Cyber-Physical Resilience of Electrical Power Systems Against Malicious Attacks: a Review. Current Sustainable/Renewable Energy Reports 5(1), 14–22 (2018). doi:10.1007/s40518-018-0094-8
  * [11] Veith, E.M., Fischer, L., Tröschel, M., Nieße, A.: Analyzing cyber-physical systems from the perspective of artificial intelligence. In: Proceedings of the 2019 International Conference on Artificial Intelligence, Robotics and Control. AIRC ’19, pp. 85–95. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3388218.3388222
  * [12] Ni, Z., Paul, S., Zhong, X., Wei, Q.: A reinforcement learning approach for sequential decision-making process of attacks in smart grid. In: 2017 SSCI Proceedings, pp. 1–8. IEEE, Piscataway, NJ (2018). doi:10.1109/SSCI.2017.8285291
  * [13] Ni, Z., Paul, S.: A Multistage Game in Smart Grid Security: A Reinforcement Learning Solution. IEEE transactions on neural networks and learning systems 30(9), 2684–2695 (2019). doi:10.1109/TNNLS.2018.2885530
  * [14] Fischer, L., Memmen, J.M., Veith, E.M., Tröschel, M.: Adversarial resilience learning—towards systemic vulnerability analysis for large and complex systems. In: ENERGY 2019, The Ninth International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, pp. 24–32. IARIA XPS Press, Athens, Greece (2019). 1811.06447
  * [15] Veith, E.M., Balduin, S., Wenninghoff, N., Tröschel, M., Fischer, L., Nieße, A., Wolgast, T., Sethmann, R., Fraune, B., Woltjen, T.: Analyzing power grid, ICT, and market without domain knowledge using distributed artificial intelligence. In: ENERGY 2020, The Tenth International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, pp. 1–12. IARIA XPS Press, Lisboa, Portugal (2020). 2006.06074. http://arxiv.org/abs/2006.06074
  * [16] Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
  * [17] Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings (2016). 1509.02971
  * [18] Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., Deepmind, D.S.: Rainbow: Combining improvements in DQN. The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), 3215–3222 (2018)
  * [19] Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016). doi:10.1038/nature16961
  * [20] Silver, D., Schrittwieser, J., Simonyan, K., Nature, I.A.-, 2017, U.: Mastering the game of Go without human knowledge. Nature 550(7676), 354 (2016)
  * [21] Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., Hassabis, D.: Mastering chess and shogi by self-play with a general reinforcement learning algorithm (2017). 1712.01815
  * [22] Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., Lillicrap, T., Silver, D.: Mastering Atari, Go, Chess and Shogi by planning with a learned model, 1–21 (2019). 1911.08265
  * [23] Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., Mordatch, I.: Emergent Tool Use From Multi-Agent Autocurricula. arXiv (2019). https://arxiv.org/abs/1909.07528
  * [24] Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T.P., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous Methods for Deep Reinforcement Learning (2016). 1602.01783
  * [25] Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. 1801.01290. Accessed 2021-12-22
  * [26] Rudion, K., Orths, A., Styczynski, Z.A., Strunz, K.: Design of benchmark of medium voltage distribution network for investigation of dg integration. In: 2006 IEEE Power Engineering Society General Meeting, p. 6 (2006). doi:10.1109/PES.2006.1709447
  * [27] Zhu, H., Liu, H.J.: Fast local voltage control under limited reactive power: Optimality and stability analysis. IEEE Transactions on Power Systems 31(5), 3794–3803 (2016). doi:10.1109/TPWRS.2015.2504419


nestedcomment

# Comments on “A Linear Time Algorithm for the Optimal Discrete IRS
Beamforming” ††thanks: The authors are with the Center for Pervasive
Communications and Computing (CPCC), Department of Electrical Engineering and
Computer Science, University of California, Irvine. ††thanks: This work is
partially supported by NSF grant 2030029. ††thanks: This work has been
submitted to the IEEE for possible publication. Copyright may be transferred
without notice, after which this version may no longer be accessible.

Dogan Kutay Pekcan Ender Ayanoglu, Fellow, IEEE

###### Abstract

Comments on [1] are provided. Updated necessary and sufficient conditions for
its Lemma 1 are given. Consequently, an updated Algorithm 1 is provided with
full specification. Simulation results with improved performance over the
implementation of Algorithm 1 are provided.

###### Index Terms:

Intelligent reflective surface (IRS), reconfigurable intelligent surface
(RIS), discrete beamforming for IRS/RIS.

##  I Introduction

Reference [1] presented an algorithm to solve the problem of finding the
values $\theta_{1},\theta_{2},\ldots,\theta_{N}$ to maximize
$|h_{0}+\sum_{n=1}^{N}h_{n}e^{j\theta_{n}}|$ where $\theta_{n}\in\Phi_{K}$ and
$\Phi_{K}=\\{\omega,2\omega,\ldots,K\omega\\}$ with $\omega=\frac{2\pi}{K}$
and $j=\sqrt{-1}$. The set $\Phi_{K}$ can equivalently be described as
$\\{0,\omega,2\omega,\ldots,(K-1)\omega\\}$. In [1], the values
$h_{n}\in\mathbb{C}$, $n=1,2,\ldots,N$ are the channel coefficients and
$\theta_{n}$ are the phase values added to the corresponding $h_{n}$ by an
intelligent reflective surface (IRS), also known as reconfigurable intelligent
surface (RIS).

##  II Two Statements from [1]

Towards achieving its goal, [1] introduced the following lemma.

Lemma 1: For an optimal solution $(\theta_{1}^{*},\ldots,\theta_{n}^{*})$ to
problem (8), each $\theta_{n}^{*}$ must satisfy

$\theta_{n}^{*}=\arg\min_{\theta_{n}\in\Phi_{K}}|(\theta_{n}+\alpha_{n}-\phase{\mu})\mod
2\pi|$ (11)

where $\phase{\mu}$ stands for the phase of $\mu$ in (10)111To prevent
confusion, we will use the same equation numbers (7)–(13) in [1]. Our own
equation numbers, not available in [1], will begin at (19) and will be
incremented from that number on. Similarly, we will introduce Lemma 2 and
Algorithm 2 in lieu of Lemma 1 and Algorithm 1 in [1]. Note that a lemma or an
algorithm with number 2 does not exist in [1]..

In [1], problem (8) is defined as

$\displaystyle\underset{\mbox{\boldmath$\theta$}}{\rm maximize\ }$
$\displaystyle f({\mbox{\boldmath$\theta$}})$ (8a) $\displaystyle{\rm subject\
to\ }$ $\displaystyle\theta_{n}\in\Phi_{K}\quad{\rm for}\quad n=1,2,\ldots,N$
(8b)

where

$f({\mbox{\boldmath$\theta$}})=\frac{1}{\beta_{0}^{2}}\bigg{|}\beta_{0}e^{j\alpha_{0}}+\sum_{n=1}^{N}\beta_{n}e^{j(\alpha_{n}+\theta_{n})}\bigg{|}^{2},$
(7b)

$h_{n}=\beta_{n}e^{j\alpha_{n}}$ for $n=0,1,\ldots,N$, and
${\mbox{\boldmath$\theta$}}=(\theta_{1},\theta_{2},\ldots,\theta_{N})$. Also,
$g$ is defined as

$g=h_{0}+\sum_{n=1}^{N}h_{n}e^{j\theta_{n}^{*}}$ (9)

and $\mu$ as

$\mu=\frac{g}{|g|}.$ (10)

Lemma 1 does not hold. This can be seen by numerical examples. We give one
such example in Table 1. In this table, we look at the simple case of $K=2$,
$N=2$. According to Lemma 1 in [1], the condition in (11) should satisfy (8)
for this simple case. We draw values of $h_{n}$ according to the first
paragraph of Sec. IV in [1]. We list these values in rows 2–4 of Table 1. We
define

$g_{0}(\theta_{1},\theta_{2})=h_{0}+\sum_{n=1}^{2}h_{n}e^{j\theta_{n}}$ (19)

and list the values of $g_{0}(\theta_{1},\theta_{2})$ for all possible
$\theta_{1},\theta_{2}\in\\{0,\pi\\}$. There are four such values and they are
listed in rows 5–8 of Table 1. The set of values for $\theta_{1}$ and
$\theta_{2}$ that maximize $|g_{0}|$, or equivalently, that achieve $g$ in
(9), are $\theta_{1}=\theta_{2}=\pi$ as in row 8 of Table 1. Note that this
operation results in $\phase{\mu}=2.3719$ radians as shown in column 5 of row
8 of Table 1.

| ${\rm Re}[\cdot]$ | ${\rm Im}[\cdot]$ | $|\cdot|$ |  $\phase{\cdot}\in[0,2\pi)$ (rad.)
---|---|---|---|---
$h_{0}$ | $-2.8267\times 10^{-7}$ | $2.7376\times 10^{-7}$ | $3.9350\times 10^{-7}$ | $2.3722$
$h_{1}$ | $1.0958\times 10^{-10}$ | $-1.0501\times 10^{-11}$ | $1.1008\times 10^{-10}$ | $6.1876$
$h_{2}$ | $-1.2238\times 10^{-11}$ | $-2.6605\times 10^{-11}$ | $2.6634\times 10^{-10}$ | $4.6664$
$g_{0}(\theta_{1}=0,\theta_{2}=0)$ | $-2.8257\times 10^{-7}$ | $2.7348\times 10^{-7}$ | $3.9324\times 10^{-7}$ | $2.3725$
$g_{0}(\theta_{1}=0,\theta_{2}=\pi)$ | $-2.8255\times 10^{-7}$ | $2.7401\times 10^{-7}$ | $3.9359\times 10^{-7}$ | $2.3715$
$g_{0}(\theta_{1}=\pi,\theta_{2}=0)$ | $-2.8279\times 10^{-7}$ | $2.7350\times 10^{-7}$ | $3.9341\times 10^{-7}$ | $2.3729$
$g_{0}(\theta_{1}=\pi,\theta_{2}=\pi)$ | $-2.8277\times 10^{-7}$ | $2.7403\times 10^{-7}$ | ${\bf 3.9377}\times{\bf 10^{-7}}$ | $\bf 2.3719$
Table 1: Sample calculation for attempting to find optimum $\theta_{1}^{*},\theta_{2}^{*},\ldots,\theta_{N}^{*}$ to maximize $|g_{0}|$ where $g_{0}(\theta_{1},\theta_{2},\ldots,\theta_{N})=h_{0}+\sum_{n=1}^{N}h_{n}e^{j\theta_{n}}$ with $\theta_{n}\in\Phi_{K}=\\{0,\frac{2\pi}{K},\ldots,(K-1)\frac{2\pi}{K}\\}$, $n=1,2,\ldots,N$, for $K=2$ and $N=2$. Channel coefficients $h_{n}$, $n=0,1,2$ are calculated using the technique described in [1]. Rows 5–8 present all values of $g_{0}$ with all combinations of $\theta_{1},\theta_{2}\in\Phi_{2}$, showing that $|g|=\max|g_{0}(\theta_{1},\theta_{2})|$ is achieved with $\theta_{1}^{*}=\theta_{2}^{*}=\pi$.  $(\theta_{1}=0)+\alpha_{1}-\phase{\mu}$ | 3.8158
---|---
$\mod((\theta_{1}=0)+\alpha_{1}-\phase{\mu},2\pi)$ | 3.8158
$(\theta_{1}=\pi)+\alpha_{1}-\phase{\mu}$ | 6.9574
$\mod((\theta_{1}=\pi)+\alpha_{1}-\phase{\mu},2\pi)$ | 0.67417
$(\theta_{2}=0)+\alpha_{2}-\phase{\mu}$ | 2.2945
$\mod((\theta_{2}=0)+\alpha_{2}-\phase{\mu},2\pi)$ | 2.2945
$(\theta_{2}=\pi)+\alpha_{2}-\phase{\mu}$ | 5.4361
$\mod((\theta_{2}=\pi)+\alpha_{2}-\phase{\mu},2\pi)$ | 5.4361
$\cos((\theta_{1}=0)+\alpha_{1}-\phase{\mu})$ | -0.7812
$\cos((\theta_{1}=\pi)+\alpha_{1}-\phase{\mu})$ | 0.7812
$\cos((\theta_{2}=0)+\alpha_{2}-\phase{\mu})$ | -0.6672
$\cos((\theta_{2}=\pi)+\alpha_{2}-\phase{\mu})$ | 0.6672
Table 2: Continuation of the sample calculation for attempting to find optimum
$\theta_{1}^{*},\theta_{2}^{*},\ldots,\theta_{N}^{*}$ to maximize $|g_{0}|$.
Rows 1–8 present the calculation of
$\min_{\theta_{n}\in\Phi_{K}}\mod(\theta_{n}+\alpha_{n}-\phase{\mu},2\pi)$ for
$n=1,2,\ldots,N$, as specified in [1] to attempt to find the optimum values of
$\theta_{n}$. This calculation results in values $\theta_{1}=0$ and
$\theta_{2}=\pi$, which are not $\theta_{1}^{*},\theta_{2}^{*}$. Rows 9-12
present the calculation of
$\max_{\theta_{n}\in\Phi_{K}}\cos(\theta_{n}+\alpha_{n}-\phase{\mu})$ to find
$\theta_{1}^{*},\theta_{2}^{*},\ldots,\theta_{N}^{*}$ as discussed in this
comment. This technique finds the optimum values of $\theta_{n},$
$n=1,2,\ldots,N$.

At this point, we would like to emphasize that [1] uses a particular
convention for the phases of complex numbers. They are defined to be in
$[0,2\pi)$, see the text that follows (2) in [1]. We use the same convention
in generating Table 1, see its column 5, as well as in generating Table 2.
With this convention, we list $\theta_{n}+\alpha_{n}-\phase{\mu}$ and
$(\theta_{n}+\alpha_{n}-\phase{\mu})\mod 2\pi$ for possibilities of
$\theta_{n}=0$ and $\theta_{n}=\pi$ and $n=1,2$ in rows 1–8 of Table 2.222Note
that absolute value signs in (11) are not needed since the argument of the
minimum operation in (11) is in $[0,2\pi)$. It can be seen from rows 1–4 of
Table 2 that the method results in $\theta_{1}=\pi$ as the potential
$\theta_{1}^{*}$, which we know from the discussion in the previous paragraph
to be correct. When we carry out the calculation
$(\theta_{2}+\alpha_{2}-\phase{\mu})\mod 2\pi$ in rows 5–8 of Table 2, we find
that the method suggests $\theta_{2}=0$ should be $\theta_{2}^{*}$. However,
we know from the exhaustive search in rows 5–8 of Table 1 that
$\theta_{2}^{*}=\pi$. Thus, Lemma 1 is not correct.

It is possible to come up with a correct lemma similar to Lemma 1. We specify
this lemma below.

Lemma 2: For an optimal solution
$(\theta_{1}^{*},\theta_{2}^{*},\ldots,\theta_{n}^{*})$, it is necessary and
sufficient that each $\theta_{n}^{*}$ satisfy

$\theta_{n}^{*}=\arg\max_{\theta_{n}\in\Phi_{K}}\cos(\theta_{n}+\alpha_{n}-\phase{\mu})$
(20)

where $\phase{\mu}$ stands for the phase of $\mu$ in (10).

Proof: We can rewrite (9) as

$\displaystyle|g|=$ $\displaystyle\
\beta_{0}e^{j(\alpha_{0}-\phase{\mu})}+\sum_{n=1}^{N}\beta_{n}e^{j(\alpha_{n}+\theta_{n}-\phase{\mu})}$
(21) $\displaystyle=$ $\displaystyle\
\beta_{0}\cos(\alpha_{0}-\phase{\mu})+j\beta_{0}\sin(\alpha_{0}-\phase{\mu})$
$\displaystyle+\sum_{n=1}^{N}\beta_{n}\cos(\theta_{n}+\alpha_{n}-\phase{\mu})$
$\displaystyle+j\sum_{n=1}^{N}\beta_{n}\sin(\theta_{n}+\alpha_{n}-\phase{\mu}).$
(22)

Because $|g|$ is real-valued, the second and fourth terms in (22) sum to zero,
and

$|g|=\beta_{0}\cos(\alpha_{0}-\phase{\mu})+\sum_{n=1}^{N}\beta_{n}\cos(\theta_{n}+\alpha_{n}-\phase{\mu})$
(23)

from which (20) follows as a necessary and sufficient condition for Lemma 2 to
hold. $\blacksquare$

Rows 9–12 of Table 2 illustrate that this method finds $\theta_{1}^{*}$ and
$\theta_{2}^{*}$. More extensive calculations can be carried out to show that
an exhaustive search as in rows 5–8 of Table 1 confirms that Lemma 2 holds for
a wide set of $K$ and $N$ values as well as a wide set of channel coefficients
$h_{0},h_{1},\ldots,h_{N}$.

Reference [1] attempts to decide a range of $\mu$ for which
$\theta_{n}^{*}=k\omega$ must hold, making use of Lemma 1. Towards that end,
it first defines a sequence of complex numbers with respect to each
$n=1,2,\ldots,N$ as

$s_{nk}=e^{j(\alpha_{n}+(k-0.5)\omega)},\ {\rm for}\ k=1,2,\ldots,K.$ (12)

Then, [1] defines, for any two points $a$ and $b$ on the unit circle $C$,
${\rm arc}(a:b)$ to be the unit circular arc with $a$ as the initial end and
$b$ as the terminal end in the counterclockwise direction; in particular, it
defines ${\rm arc}(a:b)$ as an open arc with the two endpoints $a$ and $b$
excluded. With this definition, [1] states the following proposition follows
from Lemma 1.

Proposition 1: A sufficient condition for $\theta_{n}^{*}=k\omega$ is

$\mu\in{\rm arc}(s_{nk}:s_{n,k+1}).$ (13)

Reference [1] states that “letting $\theta_{n}=k\omega$ is guaranteed to
minimize the gap $|(\theta_{n}+\alpha_{n}-\phase{\mu})\mod 2\pi|$ whenever
$\mu$ lies in its associated arc, and thus $k\omega$ must be optimal according
to Lemma 1.”

Now, let $K=2$ and thus $\omega=\frac{2\pi}{K}=\pi$, and the two possibilities
for $\theta$ are $\theta^{1}=\pi$ and $\theta^{2}=2\pi$, or equivalently
$\theta^{2}=0$. According to (12), we have

$s_{n1}=e^{j(\alpha_{n}+\frac{\pi}{2})},\quad
s_{n2}=e^{j(\alpha_{n}+\frac{3\pi}{2})}.$ (24)

According to Proposition 1, if $\mu\in{\rm arc}(s_{n1}:s_{n2})$ then
$\theta_{n}^{*}=\omega=\pi$ should hold. Assume $\mu$ is in ${\rm
arc}(s_{n1},s_{n2})$. Then, it can be observed that
$\alpha_{n}-\phase{\mu}\in(\frac{\pi}{2},\frac{3\pi}{2})$, paying attention to
the change of order due to the subtraction of $\phase{\mu}$. In particular,
let $\mu$ be such that $\alpha_{n}-\phase{\mu}\in(\frac{\pi}{2},\pi)$. When
this is the case, note that
$(\theta^{1}+\alpha_{n}-\phase{\mu})\in(\frac{3\pi}{2},2\pi)$ while
$(\theta^{2}+\alpha_{n}-\phase{\mu})\in(\frac{\pi}{2},\pi)$. Thus,
$|(\theta^{2}+\alpha_{n}-\phase{\mu})\mod
2\pi|<|(\theta^{1}+\alpha_{n}-\phase{\mu})\mod 2\pi|$, and according to Lemma
1, $\theta_{n}^{*}=\theta^{2}=0$, in contradiction with Proposition 1. On the
other hand, Proposition 1 is compatible with Lemma 2. To see this, assume
$\mu$ satisfies (12). Then,

$\phase{\mu}\in\Big{(}\alpha_{n}+\Big{(}k-\frac{1}{2}\Big{)}\omega,\alpha_{n}+\Big{(}k+\frac{1}{2}\Big{)}\omega\Big{)}.$
(25)

Since $\omega=\frac{2\pi}{K}$,

$\alpha_{n}-\phase{\mu}\in\Big{(}(-2k-1)\frac{\pi}{K},(-2k+1)\frac{\pi}{K}\Big{)}$
(26)

considering the reversal of order due to the substraction of $\phase{\mu}$.
Now, let $\theta_{n}=k\omega=2k\frac{\pi}{K}$. Then

$\theta_{n}+\alpha_{n}-\phase{\mu}\in\Big{(}-\frac{\pi}{K},\frac{\pi}{K}\Big{)}$
(27)

and thus $\cos(\theta_{n}+\alpha_{n}-\phase{\mu})$ is the largest among all
other possibilities for $\theta_{n}$ because the slice
$(-\frac{\pi}{K},\frac{\pi}{K})$ corresponds to the largest values of the
cosine function among all slices corresponding to different values of
$\theta_{k}\in\Phi_{K}$ for $k=1,2,\ldots,K$.

##  III New Algorithm

We now specify Algorithm 2 to replace Algorithm 1 in [1]. In doing so, not
only do we incorporate Lemma 2 instead of Lemma 1 but also we eliminate the
many uncertainties present in Algorithm 1 of [1].

Algorithm 2 Update for Algorithm 1

1:Initialization: Compute $s_{nk}=e^{j(\alpha_{n}+(k-0.5)\omega)}$ for
$n=1,2,\ldots,N$ and $k=1,2,\ldots,K$.

2:Eliminate duplicates among $s_{nk}$ and sort to get
$0\leq\lambda_{1}<\lambda_{2}<\cdots<\lambda_{L}<2\pi.$

3:Let, for $l=1,2,\ldots,L,$ ${\cal
N}(\lambda_{l})=\\{n|s_{nk}=\lambda_{l}\\}.$

4:Set $\phase{\mu}=0$. For $n=1,2,\ldots,N$, calculate
$\theta_{n}=\arg\max_{\theta_{n}\in\Phi_{K}}\cos(\theta_{n}+\alpha_{n}-\phase{\mu})$.

5:Set $g_{1}=h_{0}+\sum_{n=1}^{N}h_{n}e^{j\theta_{n}}$, ${\tt absgmax}=0$.

6:for $l=2,3,\ldots,L$ do

7: For each $n\in{\cal N}(\lambda_{l})$, let
$(\theta_{n}+\omega\leftarrow\theta_{n})\\!\\!\mod\Phi_{K}$.

8: Let  $g_{l}=g_{l-1}+\sum_{n\in{\cal
N}(\lambda_{l})}h_{n}\big{(}e^{j\theta_{n}}-e^{j(\theta_{n}-\omega)\\!\\!\\!\\!\mod\Phi_{K}}\big{)}$

9: if $|g_{l}|>{\tt absgmax}$ then

10: Let ${\tt absgmax}=|g_{l}|$

11: Store $\theta_{n}$ for $n=1,2,\ldots,N$

12: end if

13:end for

14:Read out $\theta_{n}^{*}$ as the stored $\theta_{n}$, $n=1,2,\ldots,N$.

##  IV Results and Remarks

Because its description is based on Lemma 1, which does not provide an
equivalency condition for finding
$\theta_{1}^{*},\theta_{2}^{*},\ldots,\theta_{N}^{*}$, the performance of
Algorithm 1 will in general not achieve the optimum result for SNR Boost [1].

Figure 1: CDF plots for SNR Boost with Algorithm 1 and Algorithm 2, $K=2$.

We have implemented Algorithm 1 to the best of our interpretation. We have
also implemented Algorithm 2. We present the CDF results for SNR Boost [1] in
Fig. 1 for $K=2$ and $N=16$, $64$, and $256$, using the average of 1,000
realizations of the channel. Clearly, Algorithm 1 is not optimal. Algorithm 2
performs better than Algorithm 2 although the gains decrease with $N$. Plots
for $K=4$ show smaller gains as compared to $K=2$, but still, Algorithm 2
always performs better than Algorithm 1 for the same $K$ and $N$.

We note that it is possible to convert the maximization of
$\cos(\theta_{n}+\alpha_{n}-\phase{\mu})$ to the minimization of a simple
expression. For example, minimization of $f_{1}(x)=\pi-|(x\\!\\!\mod
2\pi)-\pi|$ is the same as maximization of $\cos(x)$ within the context of
Lemma 2. However, this is different than minimization of $|x\\!\\!\mod 2\pi|$
proposed in Lemma 1 of [1]. The reason can be seen by plotting these functions
against $x$. While $f_{1}(x)$ and $\cos(x)$, in addition to being periodic
with period $2\pi$, have even symmetry around odd multiples of $\pi$,
$|x\\!\\!\mod 2\pi|$ (or equivalently, $(x\\!\\!\mod 2\pi)$) does not have
this symmetry.

## References

  * [1] S. Ren, K. Shen, X. Lin, X. Chen, and Z.-Q. Luo, “A linear time algorithm for the optimal discrete IRS beamforming,” _IEEE Wireless Communications Letters_ , vol. 12, no. 3, pp. 496–500, Mar. 2023.