# Non-metacyclic groups acting with nilpotent centralizers

Jhone Caldeira  Instituto de Matemática e Estatística, Universidade Federal de
Goiás, Goiânia-GO, 74690-900, Brazil<EMAIL_ADDRESS>and  Emerson de Melo
Department of Mathematics, University of Brasília, Brasília-DF 70910-900,
Brazil<EMAIL_ADDRESS>
(Date: 2021)

###### Abstract.

Let $A$ be a non-metacyclic finite group. Suppose that $A$ acts coprimely on a
finite group $G$ in such a manner that $C_{G}(a)$ is nilpotent for any $a\in
A^{\\#}$. In the present paper we investigate some conditions on $A$ which
imply that $G$ is nilpotent with “bounded” nilpotency class. More precisely,
we generalize known results on action of $q$-groups and Frobenius groups,
where $q$ is a prime.

###### Key words and phrases:

Automorphisms, centralizers, $p$-groups, Frobenius groups

###### 2010 Mathematics Subject Classification:

Primary 20D45; secondary 20D15, 20F40.

This paper is dedicated to the 60th birthday of Professor Pavel Shumyatsky. We
are grateful for his inspiring ideas on this research topic.

##  1\. Introduction

Suppose that a finite group $A$ acts by automorphisms on a finite group $G$.
The action is coprime if the groups $A$ and $G$ have coprime orders. We denote
by $C_{G}(a)$ the set $\\{g\in G\ |\ g^{a}=g\\},$ the centralizer of $a$ in
$G$ (the fixed-point subgroup). Similarly, we denote by $C_{G}(A)$ the set
$\\{g\in G\ |\ g^{a}=g\ \textrm{for all}\ a\in A\\}.$ In what follows we
denote by $A^{\\#}$ the set of nontrivial elements of $A$. If $A$ is
metacyclic and $C_{G}(a)$ is nilpotent for any $a\in A^{\\#}$, it follows from
the classification of finite simple groups that the group $G$ is soluble (see
[16]). Furthermore, we can easily construct examples showing that $G$ need not
be nilpotent even if the action is coprime. On the other hand, when $A$ is not
metacyclic and acts coprimely on a finite group $G$, recent results have shown
that the properties of $G$ are in a certain sense close to the corresponding
properties of the centralizers of non trivial elements.

Recently, prompted by Mazurov’s problem 17.72 in the Kourovka Notebook [10],
some attention was given to the situation where a Frobenius group $FH$ acts by
automorphisms on a finite group $G$. Recall that a Frobenius group $FH$ with
kernel $F$ and complement $H$ can be characterized as a finite group that is a
semidirect product of a normal subgroup $F$ by $H$ such that $C_{F}(h)=1$ for
every $h\in H\setminus\\{1\\}$. By Thompson’s theorem [15] the kernel $F$ is
nilpotent, and by Higman’s theorem [6] the nilpotency class of $F$ is bounded
in terms of the least prime divisor of $|H|$.

In the following we present some results about coprime action of non-
metacyclic groups with nilpotent centralizers.

1\. Let $q$ be a prime. If $A$ is an elementary abelian $q$-group of rank 3
and $C_{G}(a)$ is nilpotent of class $c$ for any $a\in A^{\\#}$, then the
group $G$ is nilpotent with class bounded solely in terms of $c$ and $q$ [17,
14].

2\. In the recent article [13] the above result was extended to the case where
$A$ is not necessarily abelian. Namely, it was shown that if $A$ is a finite
group of prime exponent $q$ and order at least $q^{3}$ acting on a finite
$q^{\prime}$-group $G$ in such a manner that $C_{G}(a)$ is nilpotent of class
at most $c$ for any $a\in A^{\\#}$, then $G$ is nilpotent with class bounded
solely in terms of $c$ and $q$. In [12], this result has been extended in the
following way: if $\gamma_{\infty}(C_{G}(a))$ has order at most $m$ for any
$a\in A^{\\#}$, then the order of $\gamma_{\infty}(G)$ is bounded solely in
terms of $m$. Furthermore, it was shown that if the Fitting subgroup of
$C_{G}(a)$ has index at most $m$ for any $a\in A^{\\#}$, then the second
Fitting subgroup of $G$ has index bounded solely in terms of $m$.

3\. Let $p$ be a prime and $A=FH$ a Frobenius group with complement $H$ and
whose kernel $F$ is an elementary abelian group of order $p^{2}$. If $FH$ acts
coprimely on a finite group $G$ in such a manner that $C_{G}(H)$ and
$C_{G}(x)$ are nilpotent for any $x\in F^{\\#}$, then $G$ is nilpotent. In
addition, there exists a constant $P=P(c,|H|)$ depending only on $c$ and $|H|$
with the following property: if $p\geq P$, $C_{G}(H)$ and $C_{G}(x)$ are
nilpotent of class at most $c$ for any $x\in F^{\\#}$, then $G$ is nilpotent
with class bounded solely in terms of $c$ and $|H|$ [1]. Examples in [9] show
that the result on the nilpotency class of $G$ is no longer true if $p=2$.

4\. Let $p$ be a prime and $A=FH$ a Frobenius group with complement $H$ and
whose kernel $F$ is an elementary abelian group of order $p^{3}$. If $FH$ acts
coprimely on a finite group $G$ in such a manner that $C_{G}(H)$ and
$C_{G}(x)$ are nilpotent of class at most $c$ for any $x\in F^{\\#}$, then $G$
is nilpotent with class bounded solely in terms of $c$ and $|H|$ [3].

The next result generalizes the results mentioned in Item (2). It is an
immediate consequence of the theory of regular $q$-groups and well-known
results on metacyclic $q$-groups. In fact, in Lemma 2.3 we prove that if $q>3$
is a prime and $A$ is a non-metacyclic $q$-group, then $A$ contains a subgroup
of exponent $q$ and order $q^{3}$ which is sufficient to obtain the next
theorem since the proofs in [13] and [12] are reduced to the case where $A$ is
a group of exponent $q$ and order $q^{3}$. Throughout the paper we use the
expression “$(a,b,\dots)$-bounded” to abbreviate “bounded from above in terms
of $a,b,\dots$ only”.

######  Theorem 1.1.

Let $q>3$ be a prime and $A$ a non-metacyclic finite $q$-group acting by
automorphisms on a finite $q^{\prime}$-group $G$.

  1. i)

If $C_{G}(a)$ is nilpotent of class at most $c$ for any $a\in A^{\\#}$, then
$G$ is nilpotent with $(c,q)$-bounded class.

  2. ii)

If $\gamma_{\infty}(C_{G}(a))$ has order at most $m$ for any $a\in A^{\\#}$,
then the order of $\gamma_{\infty}(G)$ is bounded solely in terms of $m$.

  3. iii)

If the Fitting subgroup of $C_{G}(a)$ has index at most $m$ for any $a\in
A^{\\#}$, then the second Fitting subgroup of $G$ has index bounded solely in
terms of $m$.

As a consequence of the classification of groups with no noncyclic
characteristic abelian subgroups we prove in Lemma 2.5 that if $A=FH$ is a
non-metacyclic Frobenius group, then the kernel $F$ contains a normal
elementary abelian subgroup of rank 2. We use Theorem 1.1 and this fact to
extend the result mentioned in Item (4) to the case where $F$ is not
necessarily abelian.

######  Theorem 1.2.

Let $FH$ be a Frobenius group with a non-metacyclic kernel $F$ of odd order
and complement $H$. Suppose that $FH$ acts coprimely on a finite group $G$ in
such a manner that $C_{G}(H)$ and $C_{G}(x)$ are nilpotent of class at most
$c$ for any $x\in F^{\\#}$. Then $G$ is nilpotent of $(c,|H|)$-bounded class.

Although we do not know if the nilpotency class would not be bounded when the
kernel $F$ is a non-metacyclic group of even order, it is possible to
construct a Frobenius group with a non-metacyclic kernel of order 16 which
does not contain a subgroup of exponent 2 and order 8. The next result
generalizes Item (3) when $FH$ is a supersolvable group. Recall that a group
$N$ is supersolvable if $N$ possesses a normal series with cyclic factors such
that each term is normal in $N$. It is easy to see that a supersolvable group
possesses a chief series whose factors have prime order.

######  Theorem 1.3.

Let $FH$ be a supersolvable non-metacyclic Frobenius group with kernel $F$ and
complement $H$. Suppose that $FH$ acts coprimely on a finite group $G$ in such
a manner that $C_{G}(H)$ and $C_{G}(x)$ are nilpotent of class at most $c$ for
any $x\in F^{\\#}$. Then $G$ is nilpotent of $(c,|H|)$-bounded class.

The paper is organized as follows. In Section 2, we summarize without proofs
some basic results on coprime actions and we prove Theorem 1.1 and Theorem
1.2. In the third section, we prove some results on Lie rings admitting a
Frobenius group of automorphisms. The proof of Theorem 1.3 is given in Section
4.

##  2\. Preliminaries

If $A$ is a group of automorphisms of a group $G$, the subgroup generated by
elements of the form $g^{-1}g^{\alpha}$ with $g\in G$ and $\alpha\in A$ is
denoted by $[G,A]$. It is well-known that the subgroup $[G,A]$ is an
$A$-invariant normal subgroup of $G$. Our first lemma is a collection of well-
known facts on coprime actions (see for example [5]). Throughout the paper we
will use it without explicit references.

######  Lemma 2.1.

Let $A$ be a group of automorphisms of a finite group $G$ such that
$(|G|,|A|)=1$. Then:

  1. i)

$G=C_{G}(A)[G,A]$.

  2. ii)

$[G,A,A]=[G,A]$.

  3. iii)

$A$ leaves invariant some Sylow $p$-subgroup of $G$ for each prime
$p\in\pi(G)$.

  4. iv)

$C_{G/N}(A)=C_{G}(A)N/N$ for any $A$-invariant normal subgroup $N$ of $G$.

  5. v)

If $G$ is nilpotent and $A$ is a noncyclic abelian group, then $G=\prod_{a\in
A^{\\#}}C_{G}(a)$.

We will require the following well-known characterization of metacyclic groups
(see for example [2, Theorem 2.6]).

######  Lemma 2.2.

For $q$ an odd prime, a $q$-group $A$ is metacyclic if and only if
$[A:A^{q}]\leq q^{2}$.

We will also use the following fact about regular $q$-groups: if $G$ is a
$q$-group for some prime $q$ and $|G|\leq q^{q}$, then $G$ is a regular
$q$-group. In particular, $|G/G^{q}|=|\Omega_{1}(G)|$.

######  Lemma 2.3.

Let $q>3$ be a prime and $A$ a non-metacyclic group. Then $A$ contains a
subgroup of exponent $q$ and order $q^{3}$.

###### Proof.

We use induction on the order of $A^{q}$. If $A^{q}=1$ there is nothing to
prove since $A$ is non-metacyclic. Otherwise we can find a normal subgroup $N$
of $A$ of order $q$, which is contained in $A^{q}$. It follows from the
inductive hypothesis that $A/N$ has a subgroup $K$ such that $N\leq K$ and
$K/N$ has exponent $q$ and order $q^{3}$. Since $N$ has order $q$ we obtain
that $K$ has order $q^{4}$. Thus $K$ is a regular group and $\Omega_{1}(K)\geq
q^{3}$ since $K/K^{q}$ has order at least $q^{3}$. ∎

The next theorem is proved in [5, Theorem 5.5.3].

######  Theorem 2.4.

Let $q$ be an odd prime. Let $Q$ be a $q$-group which contains no noncyclic
characteristic abelian subgroups. Then $Q$ is isomorphic to the central
product of a cyclic group and an extra-special group of exponent $q$.

Now we apply Theorem 2.4 to non-metacyclic Frobenius groups.

######  Lemma 2.5.

Let $FH$ be a non-metacyclic Frobenius group with complement $H$ and kernel
$F$. Then $F$ contains a normal elementary abelian $p$-subgroup of order
$p^{2}$ for some prime $p$.

###### Proof.

Since $F$ is nilpotent we can assume that $F$ is a $p$-group for some prime
$p$. If $p=2$, then $Z(F)$ must not be cyclic and the result follows. If $p$
is odd and contains a noncyclic characteristic abelian subgroup the result
also follows. Thus, it remains to prove the case when $F$ contains no
noncyclic characteristic abelian subgroup. In this case by Theorem 2.4, $F$ is
a central product of a cyclic group and an extra-special group of exponent
$p$. Therefore, $\Omega_{1}(F)$ has exponent $p$ and is not cyclic. The proof
is complete. ∎

We are now in a position to show Theorem 1.1 and Theorem 1.2. As mentioned in
the introduction, Theorem 1.1 is an immediate consequence of Lemma 2.3 and the
main results of [13, 12].

Now we prove Theorem 1.2. Let us assume the hypothesis of the theorem. Thus,
$FH$ is a Frobenius group with a non-metacyclic kernel $F$ of odd order and
complement $H$ acting coprimely on a finite group $G$ in such a manner that
$C_{G}(H)$ and $C_{G}(x)$ are nilpotent of class at most $c$ for any $x\in
F^{\\#}$. We wish to show that $G$ is nilpotent of $(c,|H|)$-bounded class.
Since $F$ is nilpotent, it is sufficient to consider the case where $F$ is a
$p$-group for some prime $p$. By Lemma 2.4 $F$ contains a normal elementary
abelian $p$-subgroup $N$ of order $p^{2}$. Thus, considering the Frobenius
group $NH$ and applying [1, Theorem 1.5] we obtain that there exists a
constant $P=P(c,|H|)$ depending only on $c$ and $|H|$ such that if $p\geq P$,
then $G$ is nilpotent with class bounded solely in terms of $c$ and $|H|$. On
the other hand, if $3<p<P$, then we can apply Theorem 1.1 on $F$ to obtain
that $G$ is nilpotent with class bounded solely in terms of $c$ and $|H|$.
Now, assume that $p=3$. We will prove that the rank of $Z(F)$ is at least 3.
If the rank of $Z(F)$ is less than 3, we consider a subgroup of $Z(F)$
isomorphic to $C_{3}$ or $C_{3}\times C_{3}$ and then since $H$ acts on such
subgroup we must have that $H$ contains an element of order 2. Thus, in this
case $F$ is an abelian group of rank at most 2 which means that $F$ is
metacyclic that is a contradiction. Therefore, $Z(F)$ contains an elementary
abelian group of rank at most 3 and the result follows by [13] or [17, 14].

##  3\. Bounding nilpotency class of Lie algebras

We start this section with a criteria for nilpotency of graded Lie algebras.
Let $p$ be a prime number and $L$ be a $\mathbb{Z}/p\mathbb{Z}$-graded Lie
algebra. If $J,Y,J_{1},\ldots,J_{s}$ are subsets of $L$ we use $[J,Y]$ to
denote the subspace of $L$ spanned by the set $\\{[j,y]\ ;\ j\in J,y\in Y\\}$
and for $i\geq 2$ we write $[J_{1},\ldots,J_{i}]$ for
$[[J_{1},\ldots,J_{i-1}],J_{i}]$.

Assume that there exist non-negative integers $u$ and $v$ such that

(1) $[L,\underbrace{L_{0},\ldots,L_{0}}_{u}]=0$

and

(2) $[[L,L]\cap L_{0},\underbrace{L_{a},\ldots,L_{a}}_{v}]=0,\ \mbox{for all}\
a\in\mathbb{Z}/p\mathbb{Z}.$

In [4] we showed that conditions (1) and (2) together are sufficient to
conclude that $L$ is nilpotent of $(p,u,v)$-bounded class. Here we repeat some
arguments for the reader’s convenience.

By [8, Theorem 1], condition (1) implies that $L$ is solvable with
$(p,u)$-bounded derived length. Thus, we can use induction on the derived
length of $L$. If $L$ is abelian, there is nothing to prove. Assume that $L$
is metabelian. In this case, $[x,y,z]=[x,z,y]$, for every $x\in[L,L]$ and
$y,z\in L$.

For each $b\in\mathbb{Z}/p\mathbb{Z}$ we denote $[L,L]\cap L_{b}$ by
$L_{b}^{\prime}$. By [11, Proposition 2] it is sufficient to prove that there
exists a $(p,u,v)$-bounded number $t$ such that

$[L_{b}^{\prime},\underbrace{L_{a},\ldots,L_{a}}_{t}]=0,\ \mbox{for all}\
a,b\in\mathbb{Z}/p\mathbb{Z}.$

If $b=0$, this follows from (2) with $t=v$. Suppose that $b\neq 0$. If $a=0$,
the commutator is zero from (1) with $t=u$. In the case where $a\neq 0$ we can
find a positive integer $s<p$ such that $b+sa=0\ (\textrm{mod }p)$. Therefore,
we have
$[L_{b}^{\prime},\underbrace{L_{a},\ldots,L_{a}}_{v+s}]\subseteq[L_{0}^{\prime},\underbrace{L_{a},\ldots,L_{a}}_{v}]$.
We know that the latter commutator is 0 by (2).

In what follows $FH$ denotes a finite supersolvable Frobenius group with
kernel $F$ and complement $H$. Moreover, we assume that $F$ is an elementary
abelian $p$-group of rank 2 for some prime $p$.

Let $R$ be an associative ring with unity. Assume that the characteristic of
$R$ is coprime with $p$ and the additive group of $R$ is finite (or locally
finite). Let $L$ be a Lie algebra over $R$. The main goal of this section is
to prove the following theorem.

######  Theorem 3.1.

Suppose that $FH$ acts by automorphisms on $L$ in such a way that $C_{L}(H)$
is nilpotent of class $c$ and $C_{L}(x)$ is nilpotent of class $d$ for any
$x\in F^{\\#}$. Then $L$ is nilpotent of $(c,d,|FH|)$-bounded class.

Let $Z$ be a subgroup of prime order $p$ of $F$ such that $Z\triangleleft FH$
and let $\varphi$ be a generator of $Z$.

Now, let $\omega$ be a primitive $p$th root of unity. We extend the ground
ring of $L$ by $\omega$ and denote by $\tilde{L}$ the algebra
$L\otimes_{\mathbb{Z}}\mathbb{Z}[\omega]$. The action of $FH$ on $L$ extends
naturally to $\tilde{L}$. Note that $C_{L}(x)$ is nilpotent of class $d$ for
any $x\in F^{\\#}$ and $C_{\tilde{L}}(H)$ is nilpotent of class $c$. Since $L$
and $\tilde{L}$ have the same nilpotency class, it is sufficient to bound the
class of $\tilde{L}$. Hence, without loss of generality it will be assumed
that the ground ring contains $\omega$ so that we will work with $L$ rather
than $\tilde{L}$.

For each $i=0,\ldots,p-1$ we denote by $L_{i}$ the $\varphi$-eigenspace
corresponding to eigenvalue $\omega^{i}$, that is, $L_{i}=\\{x\in L\ ;\
x^{\varphi}=\omega^{i}x\\}$. We have

$L=\bigoplus_{i=0}^{p-1}L_{i}\ \textrm{ and }\ [L_{i},L_{j}]\subseteq
L_{i+j(\textrm{mod}\ p)}.$

Note that $L_{0}=C_{L}(Z)$. It is easy to see that $FH/Z$ acts on $L_{0}$ and
$C_{L_{0}}(F/Z)=C_{L}(F)$. Also, note that $L_{0}$ is nilpotent of class $d$
by hypothesis.

A proof of the next lemma can be found in [9, Lemma 2.4]. It will be useful to
decompose the subalgebra $L_{0}$ into $C_{L}(F)$ and $C_{L}(ZH^{f})$, where
$f\in F$.

######  Lemma 3.2.

Suppose that a finite group $N$ admits a Frobenius group of automorphisms $KB$
with kernel $K$ and complement $B$ such that $C_{N}(K)=1$. Then $N=\langle
C_{N}(B)^{y};\ y\in K\rangle$.

In the above lemma we can write $N=\langle C_{N}(B^{y});\ y\in K\rangle$,
since $C_{N}(B)^{y}=C_{N}(B^{y})$.

For any $f\in F$ we denote by $V_{f}$ the subalgebra $C_{L}(ZH^{f})\leq
L_{0}$. Note that $ZH^{f}$ is also a Frobenius group and that $ZH^{f}=ZH$
whenever $f\in Z$.

######  Lemma 3.3.

We have $L_{0}=C_{L}(F)+\sum_{f\in F}V_{f}$.

###### Proof.

The subalgebra $L_{0}$ is $FH$-invariant and $L_{0}=C_{L_{0}}(F)+[L_{0},F]$.
So it admits the natural action by the group $FH/Z$. Moreover, $F/Z$ acts
fixed-point-freely on $[L_{0},F]$. Since the additive group of $L$ is finite,
it is immediate from Lemma 3.2 that $[L_{0},F]=\langle
C_{L_{0}}(H^{\tilde{f}});\ \tilde{f}\in F/Z\rangle$. As a consequence we have
that $[L_{0},F]=\langle C_{L}(ZH^{f});\ f\in F\rangle$ and then
$L_{0}=C_{L}(F)+\sum_{f\in F}V_{f}$. ∎

######  Lemma 3.4.

We have $[L,\underbrace{C_{L}(F),\ldots,C_{L}(F)}_{d}]=0$.

###### Proof.

Since $F$ is an elementary abelian $p$-group, it is immediate that
$L=\sum_{x\in F^{\\#}}C_{L}(x)$. Thus, taking into account that all
centralizers $C_{L}(x)$ is nilpotent of class $d$ for any $x\in F^{\\#}$ and
$C_{L}(F)\leq C_{L}(x)$ whenever $x\in F^{\\#}$, we conclude

$[L,\underbrace{C_{L}(F),\ldots,C_{L}(F)}_{d}]=\sum_{x\in
F^{\\#}}[C_{L}(x),\underbrace{C_{L}(F),\ldots,C_{L}(F)}_{d}]=0.$

∎

It is clear that any conjugated $H^{f}$ of $H$ can be considered as a
Frobenius complement of $FH$. Now we describe the action $H^{f}$ on the
homogeneous components $L_{i}$. Since $H$ is cyclic, we can choose a generator
$h$ of $H$ and $r\in\\{1,2,\ldots,p-1\\}$ such that
$\varphi^{h^{-1}}=\varphi^{r}$. Then $r$ is a primitive $q$th root of 1 in
$\mathbb{Z}/p\mathbb{Z}$ (in this section $q$ denotes the order of $H$ and it
is not necessarily a prime number). The group $H$ permutes the homogeneous
components $L_{i}$ as follows: $L_{i}^{h}=L_{ri}$ for all
$i\in\mathbb{Z}/p\mathbb{Z}$. Indeed, if $x_{i}\in L_{i}$, then
$(x_{i}^{h})^{\varphi}=x_{i}^{h\varphi
h^{-1}h}=(x_{i}^{\varphi^{r}})^{h}=\omega^{ri}x_{i}^{h}$. On the other hand,
since $F$ commutes with $\varphi$, we also have $L_{i}^{h^{f}}=L_{ri}$ for all
$i\in\mathbb{Z}/p\mathbb{Z}$ (because if $x_{i}\in L_{i}$, then
$(x_{i}^{h^{f}})^{\varphi}=\omega^{ri}x_{i}^{h^{f}}$). Thus, the action of
$H^{f}$ on the components $L_{i}$ coincides with the action of $H$.

To lighten the notation we establish the following convention.

######  Remark 3.5 (Index Convention).

In what follows, for a given $u_{s}\in L_{s}$ we denote both $u_{s}^{h^{i}}$
and $u_{s}^{(h^{f})^{i}}$ by $u_{r^{i}s}$, since
$L_{s}^{h^{i}}=L_{s}^{(h^{f})^{i}}=L_{r^{i}s}$. Therefore, we can write
$u_{s}+u_{rs}+\cdots+u_{r^{q-1}s}$ to mean a fixed-point of $H^{f}$ for any
$f\in F$.

Proof of Theorem 3.1

We use the arguments presented at the beginning of this section.

######  Lemma 3.6.

There exists a $(c,d,|FH|)$-bounded number $u$ such that
$[L,\underbrace{L_{0},\ldots,L_{0}}_{u}]=0$.

###### Proof.

It suffices to prove that $[L_{b},\underbrace{L_{0},\ldots,L_{0}}_{u}]=0$ for
any $b\in\\{0,1,\ldots,p-1\\}$.

Taking into account that $L_{0}$ is nilpotent of class $d$, we may assume
$b\neq 0$.

By Lemma 3.3, $L_{0}=C_{L}(F)+\sum_{f}V_{f}$. We now by Lemma 3.4, that
$[L,\underbrace{C_{L}(F),\ldots,C_{L}(F)}_{d}]=0$. Now let $u_{b}\in L_{b}$.
Since $u_{b}+u_{rb}+\cdots+u_{r^{q-1}b}\in C_{L}(H^{f})$ and $V_{f}\subseteq
C_{L}(H^{f})$, we have

$[u_{b}+u_{rb}+\cdots+u_{r^{q-1}b},\underbrace{V_{f},\ldots,V_{f}}_{c}]=0.$

Thus, $\sum_{i=0}^{q-1}[u_{r^{i}b},\underbrace{V_{f},\ldots,V_{f}}_{c}]=0$. On
the other hand, $[u_{r^{i}b},\underbrace{V_{f},\ldots,V_{f}}_{c}]\in
L_{r^{i}b}$ and $L_{r^{i}b}\neq L_{r^{j}b}$ whenever $i\neq j$. Therefore, we
obtain $[L_{b},\underbrace{V_{f},\ldots,V_{f}}_{c}]=0$.

Let $S$ be an $FH$-invariant subalgebra of $L_{0}$. Now let $S_{F}=S\cap
C_{L}(F)$ and let $S_{f}=S\cap V_{f}$ for $f\in F$. It follows that
$S=S_{F}+\sum_{f}S_{f}$. We will show that there exists a $(c,d,|FH|)$-bounded
number $u$ such that $[L_{b},\underbrace{S,\ldots,S}_{u}]=0$ using induction
on the nilpotency class of $S$. Since $[S,S]$ is nilpotent of smaller class,
there exists a $(c,d,|FH|)$-bounded number $u_{1}$ such that
$[L_{b},\underbrace{[S,S],\ldots,[S,S]}_{u_{1}}]=0$.

Set $l=d+(c-1)\left|F\right|$ and $W=[L_{b},S_{i_{1}},\ldots,S_{i_{l}}]$ for
some choice of $S_{i_{j}}$ in $S_{F}\cup\\{S_{f};\ f\in F\\}$. It is clear
that for any permutation $\pi$ of the symbols $i_{1},\ldots,i_{l}$ we have
$W\leq[L_{b},S_{\pi(i_{1})},\ldots,S_{\pi(i_{l})}]+[L_{b},[S,S]]$. Note that
the number $l$ is big enough to ensure that $S_{F}$ occurs at least $d$ times
or some $S_{j}\in\\{S_{f};\ f\in F\\}$ occurs at least $c$ times in the list
$S_{i_{1}},\ldots,S_{i_{l}}$. Thus, if $m=\max\\{c,d\\}$ we deduce that there
exists $i$ such that
$W\leq[[L_{b},\underbrace{S_{i},\ldots,S_{i}}_{m}],*,\ldots,*]+[L_{b},[S,S]]$,
where the asterisks denote some of the subalgebras $S_{i_{j}}\in
S_{F}\cup\\{S_{f};\ f\in F\\}$ which, in view of the fact that
$[L_{b},\underbrace{S_{i},\ldots,S_{i}}_{m}]=0$, are of no consequence. Hence,
$W\leq[L_{b},[S,S]]$.

Further, for any choice of $S_{i_{1}},\ldots,S_{i_{l}}\in S_{F}\cup\\{S_{f};\
f\in F\\}$ the same argument shows that

$[W,S_{i_{1}},\ldots,S_{i_{l}}]\leq[W,[S,S]]\leq[L_{b},[S,S],[S,S]].$

More generally, for any $n$ and any $S_{i_{1}},\ldots,S_{i_{nl}}\in
S_{F}\cup\\{S_{f};\ f\in F\\}$ we have

$[L_{b},S_{i_{1}},\ldots,S_{i_{nl}}]\leq[L_{b},\underbrace{[S,S],\dots,[S,S]}_{n}].$

Put $u=u_{1}l$. The above shows that

$[L_{b},S_{i_{1}},\ldots,S_{i_{u}}]\leq[L_{b},\underbrace{[S,S],\dots,[S,S]}_{u_{1}}]=0.$

Of course, this implies that $[L_{b},\underbrace{S,\ldots,S}_{u}]=0$. The
lemma is now straightforward from the case where $S=L_{0}$. ∎

In view of Lemma 3.6, Theorem 3.1 holds if we show the following:

######  Lemma 3.7.

Let $L$ be metabelian. There exists a $(c,d,\left|FH\right|)$-bounded number
$v$ such that $[[L,L]\cap L_{0},\underbrace{L_{a},\ldots,L_{a}}_{v}]=0,\
\mbox{for all}\ a\in\mathbb{Z}/p\mathbb{Z}$.

###### Proof.

Recall that $L_{0}=C_{L}(F)+\sum_{f}V_{f}$ and set $L^{\prime}_{F}=[L,L]\cap
C_{L}(F)$. We have $[L,L]\cap L_{0}=L^{\prime}_{F}+\sum_{f}[L,L]\cap V_{f}$
where $V_{f}=C_{L}(ZH^{f})$. Set $V=C_{L}(ZH)$ and $V^{\prime}=[L,L]\cap V$.

First we prove that $[L^{\prime}_{F},\underbrace{L_{a},\ldots,L_{a}}_{v}]=0$
for any $a\in\mathbb{Z}/p\mathbb{Z}$.

Note that we can write $L_{a}=\sum_{y\in F\setminus Z}C_{L_{a}}(y)$ and let
$k$ be the number of elements of $F\setminus Z$. Therefore,

$[L^{\prime}_{F},\underbrace{L_{a},\ldots,L_{a}}_{k(d-1)+1}]=\sum_{y\in
F\setminus
Z}[L^{\prime}_{F},\underbrace{C_{L_{a}}(y),\ldots,C_{L_{a}}(y)}_{d},*,\ldots,*]=0.$

Here the asterisks denote some subspaces of $L_{a}$ which are not important
since the commutator is 0 anyway.

Second we prove that $[V^{\prime},\underbrace{L_{a},\ldots,L_{a}}_{v}]=0$ for
any $a\in\mathbb{Z}/p\mathbb{Z}$.

For any $u_{a}\in L_{a}$ we have $u_{a}+u_{ra}+\cdots+u_{r^{q-1}a}\in
C_{L}(H)$ (under Index Convention). Thus,

$[V^{\prime},\underbrace{u_{a}+u_{ra}+\cdots+u_{r^{q-1}a},\ldots,v_{a}+v_{ra}+\cdots+v_{r^{q-1}a}}_{c}]=0$

for any $c$ elements $u_{a},\ldots,v_{a}\in L_{a}$.

Let $T$ denote the span of all the sums $x_{a}+x_{ra}+\cdots+x_{r^{q-1}a}$
over $x_{a}\in L_{a}$ (in fact, $T$ is the fixed-point subspace of $H$ on
$\bigoplus_{i=0}^{q-1}L_{r^{i}a}$). Then the latter equality means that

$[V^{\prime},\underbrace{T,\ldots,T}_{c}]=0.$

Applying $\varphi^{j}$ we also obtain

$[V^{\prime},\underbrace{T^{\varphi^{j}},\ldots,T^{\varphi^{j}}}_{c}]=[(V^{\prime})^{\varphi^{j}},\underbrace{T^{\varphi^{j}},\ldots,T^{\varphi^{j}}}_{c}]=0.$

A Vandermonde-type linear algebra argument shows that
$L_{a}\subseteq\sum_{j=0}^{q-1}T^{\varphi^{j}}$. Actually this fact is a
consequence of the following result:

######  Lemma 3.8.

[9, Lemma 5.3] Let $\left\langle\alpha\right\rangle$ be a cyclic group of
order $n$, and $\omega$ a primitive nth root of unity. Suppose that $M$ is a
$\mathbb{Z}[\omega]\left\langle\alpha\right\rangle$-module such that
$M=\sum_{i=1}^{m}M_{t_{i}}$, where $x\alpha=\omega^{t_{i}}x$ for $x\in
M_{t_{i}}$ and $0\leq t_{1}<t_{2}<\cdots<t_{m}<n$. If
$z=y_{t_{1}}+y_{t_{2}}+\cdots+y_{t_{m}}$ for $y_{t_{i}}\in M_{t_{i}}$, then
for some $m$-bounded number $d_{0}$ every element $n^{d_{0}}y_{t_{s}}$ is a
$\mathbb{Z}[\omega]$-linear combination of the elements
$z,z\alpha,\ldots,z\alpha^{m-1}$.

Now we can apply Lemma 3.8 with $\alpha=\varphi$,
$M=L_{a}+L_{ra}+\cdots+L_{r^{q-1}a}$ and $m=q$ to
$w=u_{a}+u_{ra}+\cdots+u_{r^{q-1}a}\in T$ for any $u_{a}\in L_{a}$, because
here the indices $r^{i}a$ can be regarded as pairwise distinct residues modulo
$p$ ($r$ is a primitive $q$th root of 1 modulo $p$). Since $p$ is invertible
in our ground ring, follows that
$L_{a}\subseteq\sum_{j=0}^{q-1}T^{\varphi^{j}}$.

Set $v=(c-1)q+1$. We now claim that
$[V^{\prime},\underbrace{L_{a},\ldots,L_{a}}_{v}]=0$. Indeed, after replacing
$L_{a}$ with $\sum_{j=0}^{q-1}T^{\varphi^{j}}$ and expanding the sums, in each
commutator of the resulting linear combination we can freely permute the
entries $T^{\varphi^{j}}$, since $L$ is metabelian. Since there are
sufficiently many of them, we can place at least $c$ of the same
$T^{\varphi^{j_{0}}}$ for some $j_{0}$ right after $V^{\prime}$ at the
beginning, which gives 0.

Note that in the case where $f\in F\setminus Z$ we can consider the Frobenius
group $ZH^{f}$ and conclude in a similar way that

$[[L,L]\cap V_{f},\underbrace{u_{a},\ldots,v_{a}}_{v}]=0$

for any $v$ elements $u_{a},\ldots,v_{a}\in L_{a}$.

∎

##  4\. Proof of Theorem 1.3

By Lemma 2.5 we can assume that $F$ is an elementary abelian group of order
$p^{2}$ for some prime $p$. We know that $G$ is nilpotent and there exists a
constant $P=P(c,|H|)$ such that if $p>P$, then the nilpotency class of $G$ is
$(c,|H|)$-bounded (see [1, Theorem 1.5]). We wish to show that the nilpotency
class of $G$ is $(c,|H|)$-bounded when $p\leq P$. In particular, we can assume
that $|F|$ is $(c,|H|)$-bounded.

It is easy to see that without loss of generality we may assume that $G$ is a
$p$-group. Consider the associated Lie ring of the group $G$

$L(G)=\bigoplus_{i=1}^{n}\gamma_{i}/\gamma_{i+1},$

where $n$ is the nilpotency class of $G$ and $\gamma_{i}$ are the terms of the
lower central series of $G$. The nilpotency class of $G$ coincides with the
nilpotency class of $L(G)$. The action of the group $FH$ on $G$ induces
naturally an action of $FH$ on $L(G)$ and the subrings $C_{L(G)}(H)$ and
$C_{L(G)}(x)$ are nilpotent of class at most $c$ for any $x\in F^{\\#}$ since
the action is coprime. Theorem 3.1 now tells us that $L(G)$ is nilpotent of
$(c,|H|)$-bounded class. The proof is complete.

## References

  * [1] C. Acciarri, P. Shumyatsky, Double automorphisms of graded Lie algebras, Journal of Algebra, 387 (2013), 1-10.
  * [2] N. Blackburn, On prime-power groups with two generators, Proc. Cambridge Philos. Soc., 54 (1958), 327-337.
  * [3] J. Caldeira, E. de Melo, P. Shumyatsky, On groups and Lie algebras admitting a Frobenius group of automorphisms, J. Pure Appl. Algebra, 216 (2012) 2730-2736.
  * [4] J. Caldeira, E. de Melo, Supersolvable Frobenius groups with nilpotent centralizers, J. Pure Appl. Algebra, 223 (2019), 1210-1216.
  * [5] D. Gorenstein, Finite Groups, New York, Evanston, London: Harper and Row, (1968).
  * [6] G. Higman, Groups and rings which have automorphisms without non-trivial fixed elements, J. London Math. Soc., 32 (1957), 321-334.
  * [7] E. I. Khukhro, Nilpotent Groups and their Automorphisms, Berlin; New York: de Gruyter, (1993).
  * [8] E. I. Khukhro, On solvability of Lie rings with an automorphism of finite order, Siberian. Math. J., 42 (2001), 996-1000.
  * [9] E. I. Khukhro, N. Y. Makarenko, P. Shumyatsky, Frobenius groups of automorphisms and their fixed points, Forum Math., 26 (2014), 73-112.
  * [10] E. I. Khukhro, V. D. Mazurov (Eds), Unsolved Problems in Group Theory, The Kourovka Notebook, 18th edition, Institute of Mathematics, Novosibirsk, (2014).
  * [11] E. I. Khukhro, P. Shumyatsky, Nilpotency of finite groups with Frobenius groups of automorphisms, Monatsh. Math., 163 (2011), 461-470.
  * [12] E. de Melo, Nilpotent residual and Fitting subgroup of fixed points in finite groups, J. Group Theory, 22 (2019), 1059-1068.
  * [13] E. de Melo, P. Shumyatsky, Finite groups and their coprime automorphisms, Proc. Amer. Math. Soc., 145 (2017), 3755-3760.
  * [14] P. Shumyatsky, Finite groups and the fixed points of coprime automorphisms, Proc. Amer. Math. Soc., 129 (2001), 3479-3484.
  * [15] J. G. Thompson, Finite groups with fixed-point-free automorphisms of prime order, Proc. Nat. Acad. Sci. U.S.A., 45 (1959), 578-581.
  * [16] Y. M. Wang, Z. M. Chen, Solubility of finite groups admitting a coprime order operator group (English, with Italian summary), Boll. Un. Mat. Ital. A, 7 (1993), no. 3, 325-331.
  * [17] J. N. Ward, On finite groups admitting automorphisms with nilpotent fixed-point group, Bull. Aust. Math., 5 (1971), 281-282.