linecolor=black,linewidth=2pt

# Towards Autonomous Crop-Agnostic Visual Navigation in Arable Fields

Alireza Ahmadi, Michael Halstead, and Chris McCool All authors are with the
University of Bonn, Bonn 53115 Germany. {alireza.ahmadi, michael.halstead,
<EMAIL_ADDRESS>
###### Abstract

Autonomous navigation of a robot in agricultural fields is essential for every
task from crop monitoring through to weed management and fertilizer
application. Many current approaches rely on accurate GPS, however, such
technology is expensive and also prone to failure (e.g. through lack of
coverage). As such, navigation through sensors that can interpret their
environment (such as cameras) is important to achieve the goal of autonomy in
agriculture.

In this paper we introduce a purely vision based navigation scheme which is
able to reliably guide the robot through row-crop fields. Independent of any
global localization or mapping, this approach is able to accurately follow the
crop-rows and switch between the rows, only using on-board cameras. With the
help of a novel crop-row detection and a novel crop-row switching technique,
our navigation scheme can be deployed in a wide range of fields with different
canopy types in various growth stages. We have extensively tested our approach
in five different fields under various illumination conditions using our
agricultural robotic platform (BonnBot-I). And our evaluations show that we
have achieved a navigation accuracy of $3.82cm$ over five different crop
fields.

Keywords — Robotics and Automation in Agriculture and Forestry; Agricultural
Automation; Vision-Based Navigation.

##  I Introduction

Novel agricultural robotic technologies need to ensure they meet the needs of
the key stakeholder, farmers. Usually, this means being cost effective both as
a platform and as a labor (or mechanical) replacement technology. However,
another critical element is that the technology can be deployed to a variety
of fields and environmental conditions with minimal intervention (supervision)
from the farmer. In particular, navigating through the field is central to the
real-world deployment of agricultural robotics.

Hence, automated field navigation for a robot is essential for every task. For
autonomous navigation it is common for platforms to use precise real-time
kinematic (RTK) GNSS as it is being used in fully controlled and engineered
agricultural sites where they heavily rely on structural information [1, 2].
But, robotic technologies will never be guaranteed accurate GPS in every
field, first due to its expense and second because of the coverage. Since,
currently most of fields seeding is completed using traditional approaches
without the use of auto-seeding geo-referenced systems, creating a gap between
GPS capabilities and farming requirements [1]. As such, utilizing GPS
technology in unregistered fields increases the risk of damaging crops [3].
Therefore, generally applicable methods to navigate the field, such as local
crop-row following, are required.

In our prior work [4] we introduced a GPS independent technique that was able
to traverse a single crop-row and switch between adjacent rows. However, the
prior work was only applicable to a single crop-row system which is
inefficient and does not take advantage of the overall planting structure in
the field. For instance, normally multiple crop-rows are planted in a lane for
the platform (robot or tractor) to traverse; a lane consists of multiple crop-
rows.

Figure 1: An overview of BonnBot-I autonomously following lanes of crops using
two symmetrically mounted monocular camera in front and back.

In this paper we introduce a novel crop-agnostic vision based navigation
scheme as illustrated in Fig. 1. Our method is capable of extracting
repetitive patterns in row-crop fields and navigates along multiple crop-rows.
It utilizes a novel feature based crop-row switching mechanism that enables
the robot to autonomously and reliably change between the lanes. The system
presented in this paper is deployed on BonnBot-I for a variety of row-crops
and provides the following contributions:

  * •

a robust multi-crop-row detection strategy, capable of dealing with cluttered
and weedy scenes for a variety of crop types;

  * •

a crop-agnostic crop-row following approach which enables autonomous guidance
evaluated on a robot (BonnBot-I) in the field; and

  * •

a multi-crop-row lane switching strategy which enables BonnBot-I to switch to
a new lane independent of any global positioning system.

These approaches are evaluated in real-world fields and we plan to open-source
our implementation for usage by the community.

##  II Related Work

Autonomous agricultural robots could improve productivity [5], enable targeted
field interventions [6] and facilitate crop monitoring [7]. For uptake of
these platforms they should be deploy-able in a variety of scenarios including
different cultivars, crop-row structures, and seeding patterns [8]. A key
enabling technology is reliable navigation through the whole field [9, 10].

One potential solution to the navigation problem is to make use of the Global
Navigation Satellite System (GNSS). Such an approach has been used for both
agricultural machinery [11] and robotic platforms [2]. The downside of this
approach is that it relies on an expensive sensor and suffers from limitations
such as possible GNSS outages and reliance on geo-referenced auto-seeding.
Thus, crop-based navigation techniques leveraging the field structure were
investigated for autonomous guidance [12, 4] and in-field interventions [13].

In an attempt to use the structure of a field, Barawid  _et al._ [14]
investigated 2D LiDAR based navigation systems for orchards and a similar
strategy was used in a simulated environment by [15] for traversing row-crop
fields. While these approaches enable side-applications such as obstacle
avoidance, frame drift in self-similar environments can cause issues [16],
including crop damage.

To avoid crop damage through GNSS or LiDAR failures, RGB based navigation
approaches directly exploit the available visual information. These techniques
can vary significantly in terms of cost, algorithm simplicity, and
availability. Classical machine vision approaches detect crop-rows with
juvenile crops [13] or detect crop-rows under challenging illumination
conditions [17]. Other classical approaches include crop stem locations from
Multi-spectral images [18], and Plant Stem Emerging Point (PSEP) using hand-
crafted features [19]. While these approaches are generally real-time and can,
to varying degrees, navigate a lane they do require hand selected features
which reduces the learning abilities of the techniques. In recent research
deep learning approaches have superseded their traditional counterparts as
shown by [20].

Kraemer  _et al._ [20] used a deep learning approach to reconcile the PSEP
features by exploiting the likelihood maps of a deep neural networks (DNN).
Also utilising DNNs, [21] proposed a convolutional neural network (CNN) for
strawberry crop-row detection with accurate navigation. Lin  _et al._ [22]
also showcase the potential of CNNs to reliably navigate a tea field by
classifying tea rows. These approaches are often more accurate than their
traditional counterparts for detecting or segmenting specific plants. However,
in contrast to traditional approaches, CNNs require a significant amount of
labelled data and more computational resources both for training and
inference, while being less dynamic in nature.

To perform vision based navigation two common approaches exist: proportional-
integral-derivative (PID) or visual servoing. Billingsley  _et al._ [12]
extracted the row of plants using a Hough transform, this information was then
fed to a PID controller for navigation through a sugar-beet field. The visual-
servoing [23] technique was also exploited for autonomous car guidance in
urban areas [24] by road lane following with obstacle avoidance using
monocular camera. These methods regularize the controlled agents motion within
a loop directly based on the current visual features.

The technique proposed in this paper draws inspiration from our previous work
[4] where we are able to both navigate a single crop-row and switch lanes at
the end. We extend on this work to including being crop agnostic and able to
follow multiple crop-rows in a single lane. Also, as real-time performance is
important, we rely on traditional machine vision techniques.

##  III Autonomous Navigation in Row-Crop Fields

Our navigation strategy is influenced by the robot that we use. In this work
we use a retrofitted Thorvald platform [25] which we refer to as BonnBot-I.
Below we provide, in brief, the specifications of the BonnBot-I platform and
then present the high level guidance strategy of our proposed navigation
scheme.

###  III-A BonnBot-I Platform

BonnBot-I is a prototype crop monitoring and weeding platform. The platform is
a lightweight Thorvald system which has been adapted for arable farming and
phenotyping fields in Europe. For this, the width of the platform was set to
$1.4m$ wheel-centre-to-wheel-centre with a vertical clearance of $0.86m$ to
ensure the platform could operate in the field during the early growth cycle
of most plants. The length of the robot was set to $1.4m$ ensuring there is
room for all the weeding tools, and has a maximum speed of $1.5\,m/s$.
BonnBot-I has multiple sensors, of interest for navigation is a GNSS (with an
IMU) and two RGB-D cameras which are mounted equally distant from the center
of rotation of the robot symmetrically (in front and back) illustrated in Fig.
1.

Figure 2: In-field navigation strategy, (1) following crop rows with front
camera, (2) exiting crop rows using back camera, (3) switching to the next
crop rows, and (4) following new crop rows with back camera.

###  III-B In Field Guidance Strategy

A benefit of crop-rows is that they are generally planted in consistent long
parallel structures. A downside to this parallel structure is that there is no
connection between them. Therefore, the platform needs to not only follow the
crop-row without damaging the crop but also autonomously switch between them.
To achieve multi-crop-row following we employ a similar algorithm to our
previous work in [4] for a single crop-row.

Fig. 2 outlines our multi-crop-row following strategy. Starting in a corner of
the field, the platform autonomously follows the current set of crop-rows (a
lane) using vision based navigation techniques 1 until the front facing camera
detects the end of the current lane. The rear camera then guides the robot to
the exit point, end of the lane, actively guiding the robot at all times 2.
Using the omni-directional capability, the robot then switches to the next set
of crop-rows 3 to be traversed. The benefits of the omni-directional platform
again prevails here as we can directly navigate to the new lane without
turning around 4, this also outlines the benefit of symmetrically mounted
sensors at the front and rear. In the next section we describe the vision-
based crop-row following and crop row switching algorithms.

##  IV Vision-Based Guidance in Farming Fields

We propose a multi-crop-row following approach that can successfully traverse
a row-crop field. Inspired by our prior work on single crop-row following [4],
we greatly extend this to achieve multi-crop-row following. In achieving this
we have developed a multi-crop-row detection approach described in Sec. IV-A,
integrate information from multiple detected crop-rows to perform visual-
servoing based crop-row following in Sec. IV-B, and present a multi-crop-row
switching approach in Sec. IV-C.

###  IV-A Multi-Crop-Row Detection

The first step in a successful vision system capable of traversing a field is
the assumption that crop-rows are planted in a parallel fashion. To have a
completely crop agnostic navigation system the varying distances between the
rows for the different crops is an important element. Therefore, it is
imperative to have a system that can detect the number of crop rows before
instantiating the navigation algorithm.

We perform crop-row detection by employing a sliding window based approach.
This extracts the location of the prominent crop-rows while being robust to
the appearance of weeds between them. Our detection approach consists of three
steps. First, we perform vegetation segmentation followed by connected
components operations to find individual regions (plants) and their center
points. This allows us to remain agnostic to the crop that has been planted.
Second, we automatically detect the number of crop-rows by employing an
estimate of the moving-variance which we use to describe the field structure.
Finally, we track the detected crop-rows by centering a parallelogram on each
row while the robot is traversing the lane. We detail each of these steps
below.

(a) Lemon-balm

(b) Coriander

Figure 3:  The result of vegetation segmentation (in green) with plant
boundaries (in bold green) and the resultant plant centers (magenta dots). In
(a) individual plants are easy to see and (b) is a case where crop boundaries
have to be estimated.

####  IV-A1 Vegetation Mask and Crop Center Extraction

In the first stage, we summarize a row by the position of the individual
plants along it. Each plant is represented by its center point. We obtain this
by first computing the vegetation mask of the input RGB image using the Excess
Green Index (ExG) [26]. To separate foreground and background pixels in the
image based on ExG we employ Otsu’s method [27]. Then, each connected
component in the vegetation mask is converted to an object of interest (plant)
with the unique center point obtained from the center of mass.

One issue associated with this technique occurs when multiple “plants” are
absorbed into a single large region. A single region representing an entire
crop-row negatively impacts later stages such as line fitting (described in
the next Section). To reconcile this we divide contours into smaller
subsections horizontally if they exceed a predefined maximum height, as shown
in Fig. 3. Ultimately, this step allows us to cater for a larger variety of
canopy types.

####  IV-A2 Detecting Individual Crop Rows

Fig. 4 illustrates the crop-row detection algorithm, First, a sliding window
$\Psi$ scans the image from right to left in steps of $\mathbf{S}$. The size
of the sliding window $w,h$ and the scanning steps $\mathbf{S}$ are set to
ensure a large overlap between adjacent steps in the scans. At each step, we
compute a line based on crop centers inside the sliding window using least-
squares method and find the intersection point $\textit{I}_{n}$ of the line
$L_{n}$ with the bottom axis of the image. All lines that do not intersect
with the bottom of the image are discarded, ensuring that only crop-rows that
can impact navigation are included. The estimated intersection $I_{n}=[x,y]$
is associated with the angle $\phi_{n}$ of corresponding line
$L_{n}=(I_{n},\phi_{n})$.

We use the hypothesized crop lines to represent the local structure of the
field and then employ a Moving Variance [28] operator, such that,

$\sigma^{2}(\phi_{n})=\dfrac{\sum_{i=n-k/2}^{n+k/2}(\phi_{n}-\bar{\phi}_{n})^{2}}{k};\;\bar{\phi}_{n}=\frac{\sum_{i=n-k/2}^{n+k/2}\phi_{i}}{k}$
(1)

where each variance $\sigma^{2}(\phi_{n})$ is calculated over a sliding window
of length $k$ across neighboring elements of $\phi_{n}$. The moving variance
operator yields peaks when there is discord between the local hypothesised
crop lines, this occurs between the crop-row lines. Troughs occur when there
is consistent agreement regarding the hypothesised crop lines, this occurs in
the presence of crop-rows. We refer to this as the field structure signal and
is depicted in Fig. 4.

To detect the peaks ($\blacktriangle$) and troughs ($\triangledown$) of the
field structure signal we use peak prominence with a constant threshold. To
detect the troughs, the signal is flipped (via negation) and peak prominence
is applied with the same threshold. The detection of troughs is more complex
as crop-rows can yield multiple peaks. We resolve this by computing the
weighted average of the possible solutions in the local neighbourhood, where
the local neighbourhood is defined to be adjacent sampling positions with
similar standard deviation values. We define $\mathcal{L}$ to be the set of
detected crop-row lines. An example of this is given in Fig. 4 where the final
trough is denoted by $\blacksquare$.

####  IV-A3 Crop-Row Tracking

For each detected crop-row line $L_{n}$ we form a parallelogram
$\mathcal{P}_{n}$ with a width relative to the canopy width of the crops
forming line $L_{n}$. And for every new camera image, we update the position
of line $L_{n}$ based on the crop centers within the parallelogram
$\mathcal{P}_{n}$ at its latest position. The $\mathcal{L}$ is then tracked
and used to update the robot’s position to ensure that the crop is always
beneath the robot, using a visual-servoing controller. Finally, when all of
the crop centers are in the top half of the image we consider that we have
reached the end of the crop row and employ the multi-crop-row switching
algorithm (Sec. IV-C).

Figure 4: The sliding window $\Psi$ scans input image every $\mathbf{S}$ step.
The moving-variance of hypothesis lines intersecting the bottom of $\Psi$ are
calculated and using prominence peaks ($\blacktriangle$) and troughs
($\triangledown$) are found in the region of interest (blue) using prominence
constraint. The weighted average of multiple troughs leads to the final trough
final trough $\blacksquare$.

###  IV-B Visual-Servoing Based Crop-Row Following

To guide the robot within the crop-rows, we utilize our previously developed
in-row guidance approach [4] which relies on the Image-Based Visual Servoing
(IBVS) controller [23]. The guiding feature $\mathcal{\mathbf{L}}$ is the
average line of the set of detected crop-row lines $\mathcal{L}$ illustrated
in Sec. IV-A computed from the current camera image. The desired feature
$\mathcal{\mathbf{L}}^{*}=[0,\,\frac{H}{2},\,0]$ is located at the bottom
center of the image $\mathcal{I}$, illustrated in Fig. 5. And, the IBVS
controls law leads the current feature
$\mathcal{\mathbf{L}}=[a,\,\dfrac{H}{2},\,\Theta]$ to towards
$\mathcal{\mathbf{L}}^{*}$, where $H$ denotes the height of the image and $a$
is the deviation from the image center of the intersection point $\mathbf{I}$.
By continuously regulating the robot’s motion based on this controller and
observation features explained in Sec. IV-A3, we ensure the robot stays in the
middle of the path and follows the desired crop rows to their end. At the end
of crop-rows, we switch to the next lane by using the switching technique
described in the next section.

###  IV-C Multi-Crop-Row Switching

To autonomously navigate over an entire row-crop field a robot must be able to
both navigate down lanes and shift between them. The process of changing lanes
needs to be robust to different numbers of crop-rows contained in a lane and
ensure complete coverage is obtained without missing or double counting crop-
rows. In our previous work [4] the task of changing lanes was managed
reliably, however, it was only designed to handle a single crop-row in a lane.
Furthermore, this task was hard coded and required significant space to
maneuver. The approach described here takes advantage of the side-ways
movement of BonnBot-I allowing easier transitions while requiring less space.
We use a feature-based crop-row recognition technique based on SIFT features
described below.

We start by considering the robot to have found the end of the crop-row; stage
3 of the navigation scheme Sec. III-B. The multi-crop-row detection algorithm
Sec. IV-A provides us with the number of crops rows $C$ that have been
traversed. To find new crop rows, lane switching, we need to move across $C$
rows and then restart the crop row following algorithm. To do this we describe
each crop row by a set of SIFT features and follow the algorithm described
below.

Assuming we are moving left-to-right, we store the features of the right most
parallelogram in the image forming a feature set $\mathcal{G}$. The robot then
starts moving to the right-side with a constant $u_{y}=0.15\,m/s$ velocity.
Upon receiving a new image we detect the crop-rows in a similar manner
outlined in Sec. IV-A2 and then only consider the right most side of the
image. We extract a new set of SIFT features from the right most parallelogram
in the image forming a feature set $\mathcal{G}^{*}$. The new feature set,
$\mathcal{G}^{*}$, is potentially a new crop-row. To determine if
$\mathcal{G}^{*}$ is a new crop-row we compare it to the stored SIFT features
$\mathcal{G}$. If a new crop-row has been detected, we update the stored
features ($\mathcal{G}$=$\mathcal{G}^{*}$) and continue this process until we
have moved across $C$ new crop-rows.

Figure 5: The image frame $\mathcal{I}$ and $\mathbf{L}=[\mathbf{I},\,\Theta]$
denotes the dominant line estimated from the visible crop-rows.

(a) Sugar-beet

(b) Coriander

(c) Potato

(d) Beans

Figure 6: Illustrations of four crops (top row) with their corresponding field
structure signal (bottom row). The detected peaks (blue triangle) and troughs
(orange square) obtained via their prominence in the signal are also provided.
Denote that, the field structure signal only include the values of lines
intersecting with the bottom axes of the image.

To compare the SIFT features, we use a FLANN-matcher to obtain the best
matches $\mathbf{\Omega}$ between the two sets $\mathcal{G}$ and
$\mathcal{G}^{*}$. This results in the Euclidean distances between
$\mathcal{G}$ and $\mathcal{G}^{*}$ being stored in $\mathbf{\Omega}$. We then
take the average of the $m$ matches in $\mathbf{\Omega}$ which are above a
threshold $\lambda$. This is used to provide a distance measure between the
two sets of features:

$D(\mathcal{G},\mathcal{G}^{*})=\dfrac{1}{m}\sum_{i=1}^{m}\mathbf{\Omega}_{m}$
(2)

When $D(\mathcal{G},\mathcal{G}^{*})$ exceeds a threshold $\tau_{c}$ we assume
a new crop-row has been found, $\tau_{c}$ is a crop type specific constant.

##  V Experimental Evaluations

We performed three experiments to show the capability and robustness of our
proposed approaches. First, we evaluate the reliably and consistency of our
proposed multi-crop-row detection approach by applying it to different fields
with varying numbers of crop-rows (per lane) and canopy shapes. Second, we
evaluate the navigation accuracy of the robot along several multi-crop-row
paths, relying solely on RGB cameras. Finally, the validity of the lane
switching technique is analysed, including the ability to detect and switch
between different crops using the same approach.

###  V-A Experimental Setup

All experiments in this section were completed on BonnBot-I at Campus Klein-
Altendorf of University of Bonn. To fast track the field experiments we used a
1:1 scale Gazebo simulation model with a realistic field. From this
simulation, along with our previous work in [4] for the camera setup, we were
able to derive our algorithmic hyper-parameters. On BonnBot-I, the front and
back navigation cameras are fixed at a height of $1.0\,m$ and tilt angle
$\rho=65^{\circ}$. Both camera resolutions are $1280\times 720$ with a capture
rate of $15\,fps$. For all experiments, the width $w$ of sliding window $\Psi$
was kept constant $w=W/10=128$ with a height of $h=720$ pixels. This window
size and $\mathbf{S}=100$ ensures $\simeq 92\%$ overlap between consecutive
sliding windows. Also, we empirically set $k=10$ in Eq. (1) which in
simulation provided the best trade-off between sample consistency and
neighbourhood relationship. As the primary goal of this platform is to perform
weeding we set its velocity to be a constant $v_{x}^{*}=0.3\,m/s$. We use
differential velocity control within the crop-rows and omni-directional
control for switching between the lanes. Our approach is implemented using
Python and PyCuda ensuring real-time operation and runs on a user grade
computer (Cincose DS-1202).

###  V-B Multi-Crop-Row Detection

The first experiment is a qualitative analysis on the ability to detect crop-
rows in the field using the technique described in Sec. IV-A. The goal of this
technique is to exploit the dominant crop locations and accurately detect the
best location for traversing a lane (i.e. keeping the crop-rows under the
platform). Due to weeds growing between the crop-rows this can be a
challenging proposition in real fields.

Fig. 6 depicts four crop types: sugar-beet, coriander, potato, and beans. Due
to space limitations we refrained from adding Lemon-balm results in Fig. 6.
The illustrated crops have diverse canopy types (see Fig. 6) and are arranged
in two different patterns ($35\,cm$ or $55\,cm$ between crop rows) leading to
either two or three crop-rows to navigate.

In Fig. 6 it can be seen that the crop-rows of beans were the most difficult
to detect. This is due to the presence of weeds in the crop-rows and the
height of the crop. While this was a difficult crop to detect our detection
technique was still able exploit the crop information to find the crop-rows,
denoted by the orange squares in Fig. 6.

For all crops we were able to consistently detect both the peak (no crop-row)
and trough (crop-row) locations regardless of the presence of weeds. This is
especially evident in coriander where even with the small distance within the
crop-row (between the coriander plants) we are still able to detect the crop-
rows. This is an example where dividing single large regions into sub-regions
is essential.

Sugar-beet, Fig. 6-a, is another interesting use case. Visually it is
considerably more difficult to discern the crop locations, however, this
technique was still able to extract the required locations (crop-rows).
Overall, this technique for crop-row detection successfully located the
required troughs in order to navigate a lane, providing accurate information
required for the other stages of our system.

###  V-C Navigating Along The Crop-Rows

To analyse the performance of our crop-row navigation technique we require
accurate ground truth information. To collect the ground truth information the
robot was manually driven down each of the row-crop fields for all crop types
and stored for later evaluation. The associated GPS measurements are then used
as the “correct” position (accurate to $1cm$). Even though manual operation
can cause some errors we consider this to be an appropriate ground truth to
compare to as the crop-rows are not guaranteed to be planted in a straight
line.

We investigate the navigation performance on five crops: sugar-beet,
coriander, potato, beans, and lemon-balm. This introduces a range of
challenges such as being captured under different weather conditions,
different weed density, and varying growth stages. Tab. I outlines the
performance on these crops. In general these approaches are consistently able
to replicate the ground truth (manual) navigation via our novel crop-row
navigation technique.

The sugar-beet navigation was the most complicated crop to follow due to a
number of challenging factors. First, the early growth stage, as seen in Fig.
6, made it more complicated to detect the crop lines. Secondly, not all of the
sugar-beet had germinated and this led to gaps or “dead space” along the rows.

TABLE I: Lane following performance of BonnBot-I using proposed method in five different crop rows in Campus Klein-Altendorf of University of Bonn. Crop | Length |  | Avg. and std. dev.
---
distance to crop rows
| Avg. and std. dev.
---
angular error
Beans | 52 m | 3.49 $\pm$ 2.89 cm | 3.73 $\pm$ 3.21 deg
Potato | 37 m | 2.18 $\pm$ 3.01 cm | 4.91 $\pm$ 1.63 deg
Coriander | 54 m | 2.91 $\pm$ 2.38 cm | 2.57 $\pm$ 1.05 deg
Sugar-beet | 69 m | 8.41 $\pm$ 3.79 cm | 3.25 $\pm$ 1.27 deg
Lemon-balm | 40 m | 2.12 $\pm$ 1.58 cm | 3.21 $\pm$ 2.83 deg

From a navigational perspective the bean crop had a large standard deviation
when considering angular error. The weather conditions played a crucial part
in this as heavy winds consistently changed the location of the leaves of the
crops. This limitation in the navigation technique leads to large angular
variations while traversing the lane.

Overall, the average deviation from the ground truth was $3.82cm$ or
approximately $10\%$ of the crop-row distance. This minor fluctuation is
sufficient to ensure safe navigation without damaging crops. Finally, due to
this navigational accuracy this technique was consistently able to traverse
all the crops in the field and not miss any plant within the rows.

###  V-D Multi-Crop-Row Switching

The third experiment is designed to show the capability of the multi-crop-row
switching method introduced in Sec. IV-C. The differentiation between
different crop rows in an outdoor environment is the most crucial step in this
process. Hence, we evaluated the performance of crop recognition using the
similarity distance metric Eq. (2). We do this by evaluating the similarity
between different instances of the same crop row ($+ve$ samples) instances of
different crop rows ($-ve$ samples) for a range of fields and crop types.

Our results, in Tab. II, highlights the validity of our approach. It can be
seen that in all cases the distribution of the $+ve$ and $-ve$ samples is
clearly separable. The most challenging crop was beans, where the means were
closest and the $-ve$ samples had the largest standard deviation, we attribute
this to its cluttered and branchy shape. However, even in this case there is
still a clear separation between the $+ve$ and $-ve$ samples.

TABLE II: Table of the mean ($\mu$) and std. dev ($\sigma$) for the same ($+ve$) and different ($-ve$) rows.  Crop |  $\mu\pm\sigma$ of $+ve$ cases |  $\mu\pm\sigma$ of $-ve$ cases
---|---|---
Beans | 0.26 $\pm$ 0.004 | 0.33 $\pm$ 0.037
Potato | 0.15 $\pm$ 0.018 | 0.27 $\pm$ 0.040
Coriander | 0.19 $\pm$ 0.008 | 0.30 $\pm$ 0.016
Sugar-beet | 0.14 $\pm$ 0.014 | 0.29 $\pm$ 0.015
Lemon-balm | 0.20 $\pm$ 0.025 | 0.29 $\pm$ 0.012

We have also manually measured the distance which the robot needed to perform
a transition maneuver both in simulation and in the real fields for more than
10 different performances, which outlined an average of $2.1$m to $2.6$m. By
considering the robot’s length being $1.4$m, supports our claim that the new
lane switching method is requiring less space to be executed.

##  VI Conclusion

In this paper, we presented a novel approach to enable autonomous navigation
in row-crop fields empowering precision farming and crop monitoring tasks.
This approach exploits the crop-row structure using only the local observation
from the on-board cameras. Furthermore, it does not require any global or
local position awareness for guiding the robot. To achieve this, we have
proposed a novel multi-crop-row detection strategy that can deal with
cluttered and weedy scenes. We also proposed a novel lane switching strategy
which enables BonnBot-I to switch to a new lane independent of any global
positioning system or human intervention. This enabled us to perform
autonomous field navigation which we evaluated on a robot (BonnBot-I) in the
field. Evaluations were performed on BonnBot-I in five different real fields
with various crop and canopy types with an average navigation accuracy of
$3.82cm$.

## Acknowledgements

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation) under Germany’s Excellence Strategy - EXC 2070 –
390732324. This work was also possible with the help of Julius Knechtel and
Gereon Tombrink.

## References

  * [1] “Garford(2014).robocrop inrow weeder,” https://garford.com/, accessed: 2020-10-22.
  * [2] O. Bawden, J. Kulk, R. Russell, C. McCool, A. English, F. Dayoub, C. Lehnert, and T. Perez, “Robot for weed species plant‐specific management,” _Journal of Field Robotics_ , vol. 34, pp. 1179–1199, 2017.
  * [3] M. Bakken, V. R. Ponnambalam, R. J. Moore, J. G. O. Gjevestad, and P. J. From, “Robot-supervised learning of crop row segmentation.”
  * [4] A. Ahmadi, L. Nardi, N. Chebrolu, and C. Stachniss, “Visual servoing-based navigation for monitoring row-crop fields,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 4920–4926.
  * [5] E. J. Van Henten, J. Hemming, B. Van Tuijl, J. Kornet, J. Meuleman, J. Bontsema, and E. Van Os, “An autonomous robot for harvesting cucumbers in greenhouses,” _Autonomous robots_ , vol. 13, no. 3, pp. 241–258, 2002.
  * [6] M. Pérez-Ruíz, D. C. Slaughter, F. A. Fathallah, C. J. Gliever, and B. J. Miller, “Co-robotic intra-row weed control system,” _Biosystems engineering_ , vol. 126, pp. 45–55, 2014.
  * [7] M. Bayati and R. Fotouhi, “A mobile robotic platform for crop monitoring,” _Adv. Robot. Autom_ , vol. 7, no. 2, 2018.
  * [8] T. Utstumo, F. Urdal, A. Brevik, J. Dørum, J. Netland, Ø. Overskeid, T. W. Berge, and J. T. Gravdahl, “Robotic in-row weed control in vegetables,” _Computers and electronics in agriculture_ , vol. 154, pp. 36–45, 2018.
  * [9] B. Åstrand and A. J. Baerveldt, “A vision based row-following system for agricultural field machinery,” _Mechatronics_ , vol. 15, no. 2, pp. 251–269, 2005.
  * [10] J. Billingsley and M. Schoenfisch, “The successful development of a vision guidance system for agriculture,” _cea_ , vol. 16, no. 2, pp. 147–163, 1997\.
  * [11] B. Thuilot, C. Cariou, P. Martinet, and M. Berducat, “Automatic guidance of a farm tractor relying on a single cp-dgps,” _Autonomous robots_ , vol. 13, no. 1, pp. 53–71, 2002.
  * [12] J. Billingsley and M. Schoenfisch, “The successful development of a vision guidance system for agriculture,” _Computers and electronics in agriculture_ , vol. 16, no. 2, pp. 147–163, 1997.
  * [13] B. Åstrand and A.-J. Baerveldt, “A vision based row-following system for agricultural field machinery,” _Mechatronics_ , vol. 15, no. 2, pp. 251–269, 2005.
  * [14] O. C. Barawid Jr, A. Mizushima, K. Ishii, and N. Noguchi, “Development of an autonomous navigation system using a two-dimensional laser scanner in an orchard application,” _Biosystems Engineering_ , vol. 96, no. 2, pp. 139–149, 2007.
  * [15] F. B. Malavazi, R. Guyonneau, J.-B. Fasquel, S. Lagrange, and F. Mercier, “Lidar-only based navigation algorithm for an autonomous agricultural robot,” _Computers and electronics in agriculture_ , vol. 154, pp. 71–79, 2018.
  * [16] M. Bakken, R. J. Moore, and P. From, “End-to-end learning for autonomous crop row-following,” _IFAC-PapersOnLine_ , vol. 52, no. 30, pp. 102–107, 2019\.
  * [17] H. T. Søgaard and H. J. Olsen, “Determination of crop rows by image analysis without segmentation,” _Computers and electronics in agriculture_ , vol. 38, no. 2, pp. 141–158, 2003.
  * [18] S. Haug, P. Biber, A. Michaels, and J. Ostermann, “Plant stem detection and position estimation using machine vision,” in _Workshop Proc. of Conf. on Intelligent Autonomous Systems (IAS)_ , 2014, pp. 483–490.
  * [19] H. S. Midtiby, T. M. Giselsson, and R. N. Jørgensen, “Estimating the plant stem emerging points (pseps) of sugar beets at early growth stages,” _Biosystems engineering_ , vol. 111, no. 1, pp. 83–90, 2012.
  * [20] F. Kraemer, A. Schaefer, A. Eitel, J. Vertens, and W. Burgard, “From plants to landmarks: Time-invariant plant localization that uses deep pose regression in agricultural fields,” _arXiv preprint arXiv:1709.04751_ , 2017.
  * [21] V. R. Ponnambalam, M. Bakken, R. J. Moore, J. Glenn Omholt Gjevestad, and P. Johan From, “Autonomous crop row guidance using adaptive multi-roi in strawberry fields,” _Sensors_ , vol. 20, no. 18, p. 5249, 2020.
  * [22] Y.-K. Lin and S.-F. Chen, “Development of navigation system for tea field machine using semantic segmentation,” _IFAC-PapersOnLine_ , vol. 52, no. 30, pp. 108–113, 2019.
  * [23] B. Espiau, F. Chaumette, and P. Rives, “A new approach to visual servoing in robotics,” _ieee Transactions on Robotics and Automation_ , vol. 8, no. 3, pp. 313–326, 1992.
  * [24] D. A. de Lima and A. C. Victorino, “A visual servoing approach for road lane following with obstacle avoidance,” in _17th International IEEE Conference on Intelligent Transportation Systems (ITSC)_. IEEE, 2014, pp. 412–417.
  * [25] L. Grimstad and P. J. From, “Thorvald ii-a modular and re-configurable agricultural robot,” _IFAC-PapersOnLine_ , vol. 50, no. 1, pp. 4588–4593, 2017.
  * [26] D. Woebbecke, G. Meyer, K. V. Bargen, and D. A. Mortensen, “Color indices for weed identification under various soil, residue, and lighting conditions,” _Trans. of the American Society of Agricultural Engineers (ASAE)_ , vol. 38, no. 1, pp. 259–269, 1995.
  * [27] C. Reimann, P. Filzmoser, and R. G. Garrett, “Background and threshold: critical comparison of methods of determination,” _Science of the total environment_ , vol. 346, no. 1-3, pp. 1–16, 2005.
  * [28] J. MacGregor and T. Harris, “The exponentially weighted moving variance,” _Journal of Quality Technology_ , vol. 25, no. 2, pp. 106–118, 1993.


Gravitational and dark wave emission at binary merger

Kunio<EMAIL_ADDRESS>Kin-ya<EMAIL_ADDRESS>and
Motohiko<EMAIL_ADDRESS>
Department of Mathematics, Tokyo Woman’s Christian University,
Tokyo 167-8585, Japan

† Research Institute for Interdisciplinary Science, Okayama University,
Tsushima-naka 3-1-1 Kita-ku Okayama 700-8530, Japan

ABSTRACT

The recently proposed formalism of extended Jordan-Brans-Dicke gravity makes
it possible to calculate energy loss rate due to both gravitational wave and
scalar field (giving the origin of dark energy) wave emission at merger of a
black hole and a neutron star; a binary system of no scalar hair and a star
with the scalar charge. The scalar field emission changes orbit parameters of
the binary system, thereby changes detectable gravitational wave emission.
When neutron stars carry significantly large scalar charge, significant dark
wave (namely, scalar field wave) emission occurs at the same time of
gravitational wave emission. It is found that solutions of coupled
differential equations predict non-vanishing remnant dark charge after the
gravitational collapse. This gives two interesting possibilities: (1) the no-
hair conjecture of black hole is violated, or (2) a bosonic cloud is formed
outside the event horizon of black hole. The bosonic cloud proposed in the
literature is a gigantic atom made of gravitationally bound dark energy quanta
surrounding a black hole. One can either constrain, or even determine,
parameters of extended Jordan-Brans-Dicke gravity from accumulated
gravitational wave observations of merging black hole and neutron star. 444
Key words Gravitational wave emission, Dark energy, No hair theorem, Jordan-
Brans-Dicke gravity, Black hole merger with neutron star, bosonic cloud

Introduction

General relativity (GR) is the leading theory of gravity, passing all tests in
the weak gravity regime [1], and test at the initial stage of gravitational
wave (GW) emission in the strong gravity regime [2, 3]. There is however a
good reason to consider scalar-tensor gravity as a powerful alternative, due
to recent developments in cosmology; the dark energy problem and inflation.

A class of scalar-tensor gravity, an extended Jordan-Brans-Dicke (eJBD)
theory, has recently been identified [5], which pass stringent observational
constraints on time variation of fundamental parameters. These parameters are
time-invariants in GR. So far no deviation from GR has been found, giving
constraints on eJBD theory. Concordance of calculated energy budget evolution
with cosmological observations found in [5] suggests that the time varying
mass of eJBD field is small, of order the Hubble rate $\sim 10^{-33}$ eV at
the present epoch, thus making the original massless JBD theory [6], (however
modified by its coupling and finite potential), a relevant theory for analysis
at recent epochs.

In the present work we discuss GW emission at binary merger in JBD gravity.
The energy loss due to the scalar field emission accompanying GW emission may
become comparable to, or even dominate over, GW; it may thus drastically
change result of pure GW amplitude signals in GR. BH-BH merger proceeds via
pure GW emission adopting the no-hair conjecture. It is thus promising to
examine GW emission at BH- neutron star (NS) merger, since NS is allowed to
have the scalar charge called the dark charge, too. Discussion of GW emission
rate at NS-NS merger is difficult due to increased calculational uncertainty.
Our simple analysis of BH-NS merger presented in this work may enable one to
place useful constraint on model parameters in eJBD gravity if GW detection
data from BH-NS merger are accumulated. One needs to introduce two-component
O(2) symmetric eJBD fields in order to define a conserved dark charge
unambiguously.

We use the natural unit of $c=1=\hbar$ unit unless otherwise stated. A typical
energy loss rate of gravitational wave emission is 1 solar mass/ msec
$=1.8\times 10^{57}$ erg/sec.

Static solution with scalar hair

We start our discussion from the no-hair conjecture of isolated compact
objects. Spherically symmetric vacuum solutions of JBD gravity are well known;
for example [7]. Their metric and JBD field are given by

$\displaystyle
ds^{2}=f(r)dt^{2}-\frac{dr^{2}}{f(r)}-g(r)(d\theta^{2}+\sin^{2}\theta
d\phi^{2})\,,$ (1) $\displaystyle f(r)=(1-\frac{l}{r})^{\alpha}\,,\hskip
14.22636ptg(r)=r^{2}(1-\frac{l}{r})^{1-\alpha}\,,$ (2)
$\displaystyle\chi(r)=\chi(\infty)+\frac{qM}{2}\ln(1-\frac{l}{r})\,,\hskip
14.22636pt\alpha^{2}+q^{2}=1\,.$ (3)

$G$ is the gravitational constant. The mass $M$ and the JBD charge $D$, called
the dark charge hereafter, at the center of massive body are defined from
asymptotic behaviors at $r\rightarrow\infty$:

$\displaystyle g_{tt}=1-\frac{2GM}{r}+\cdots\,,\hskip
14.22636pt\chi=\chi(\infty)-\frac{GDM}{r}+\cdots\,.$ (4)

Parameters of the solution in (1) $\sim$ (3) are given in terms of $M$ and
$D$;

$\displaystyle l=2G\sqrt{M^{2}+D^{2}}\,,\hskip
8.5359ptq=\frac{D}{\sqrt{M^{2}+D^{2}}}\,,\hskip
8.5359pt\alpha=\frac{M}{\sqrt{M^{2}+D^{2}}}\,.$ (5)

Divergence in the metric (1) at $r=l$ gives a curvature singularity with the
scalar quantity $R_{\mu\nu\rho\sigma}R^{\mu\nu\rho\sigma}=\infty$: it is a
naked singularity not covered by event horizon. The no-hair conjecture, often
called no-hair theorem in the literature, states that physically allowed
solution at gravitational collapse is black hole without the curvature
singularity. 111 Unstable linearized perturbation modes around general static
solution having the curvature singularity were found [8]. Since dynamical
gravitational collapse proceeds without developing unstable modes, this
suggests that formation of black hole ends without the curvature singularity;
namely, the Schwarzschild metric is formed, as discussed in [8]. This argument
may be taken as a support for no-hair conjecture.  The Schwarzshild solution
defined by the condition $\alpha=1\,,q=0$ (hence $D=0$), is thus the unique
static solution giving a candidate black hole. In an isolated NS or a white
dwarf a finite dark charge may exist with $D\neq 0$. In this case neither the
event horizon nor the curvature singularity is present, since the
Schwarzschild radius is within the neutron star surface where the above vacuum
solution becomes irrelevant, the presence of matter precluding the event
horizon and the curvature singularity.

Scalar hair in eJBD gravity

We introduce two-component real eJBD fields:
$\chi=\chi_{1}+i\chi_{2}=\chi_{r}e^{i\theta}$ with real $\chi_{i}(i=1,2,r)$
and $\theta$ fields. In models of eJBD gravity that we consider in the present
work the spatially homogeneous background of $\chi_{r}$ evolves with
cosmological evolution, and near the present cosmic epoch it is given by
$\chi_{r}=\chi_{0}(t)\sim At^{1/\delta}\,,\delta>1$, with $A\,,\delta$ two
constants. Time variation arises from a non-trivial potential in the
reformulated eJBD theory [5]. The dark eJBD field $\chi$ couples to matter
fields of particle physics (fermions, gauge bosons, and Higgs boson) via
interaction lagrangian ${\cal L}^{\rm E}_{st}$ in the Einstein metric frame,

$\displaystyle{\cal L}^{\rm E}_{st}=F^{p_{g}}{\cal L}_{g}+F^{p_{\rm f}}{\cal
L}_{f}-F^{p_{H}}V_{\rm J}(H)+F^{p_{\rm
dH}}(D_{\mu}H)^{\dagger}(D^{\mu}H)+F^{p_{Y}}{\cal L}_{y}$ $\displaystyle\hskip
28.45274pt+i\frac{1}{2}p_{f}\,\sum_{i=1,2}\partial_{\chi_{i}}\ln
F\,\partial_{\mu}\chi_{i}\,(3j_{B}^{\mu}+j_{L}^{\mu})\,,$ (6)

where ${\cal L}_{g}$ is SU(3)$\times$ SU(2)$\times$ U(1) gauge field
contribution, and ${\cal L}_{f}$ refers to kinetic lagrangian of fermions,
while ${\cal L}_{y}$ is Yukawa coupling of fermion to Higgs boson, other
${\cal L}_{i}$’s being Higgs potential and Higgs kinetic lagrangian.
$3j_{B}^{\mu}+j_{L}^{\mu}=\sum_{f}\bar{f}\gamma^{\mu}f$ is the universal
fermion current summed over all fundamental fermions $f$. The conformal
function is taken as $F(\chi)=1+c(\chi_{r}/M_{\rm P})^{2}$ with $c>0$ a
constant,222 Most generally, there can be five independent conformal functions
and five independent powers [5] consistent with low energy limit of super-
string theory [10]. We simplified this general case by introducing a common
conformal function $F$.  and $M_{\rm P}=1/\sqrt{8\pi G}\sim 2.4\times
10^{18}\,{\rm GeV}$ the reduced Planck mass. The lagrangian density (6) is a
function of eJBD fields, $\chi_{r}^{2}$ and $\partial\chi_{r}^{2}$, while its
kinetic terms contain $(\partial\chi_{r})^{2}$ and
$(\chi_{r}\partial\theta)^{2}$.

Spatially inhomogeneous eJBD field develops inside compact objects such as
neutron star and white dwarf where many nucleons and electrons act as the
source term of $\chi$ field. Relevant lagrantian parts in (6) for the present
work are nucleon and electron mass terms and coupling term in the second line
of (6), since these make up ordinary matter of a massive astrophysical body.
The main nucleon mass originates from quantum chromo-dynamic interaction given
in ${\cal L}_{g}$, with a small additional contribution of u- and d-quark
masses arising from ${\cal L}_{y}$ in the spontaneously broken phase.

The power factors $p_{i}$ and the conformal function $F(\chi)$ are expected to
be derivable from super-string theory at a far more advanced stage of
theoretical developments, but at present it is completely unknown except at
the string tree level [10]. Null-observation on time variation of standard
particle physics parameters places bounds on the powers of conformal
functions, of order
$|p_{g}|/\delta<10^{-7}\,,|p_{Y}|/\delta<10^{-2}\,,\delta>1\,,|p_{Y}|<1/2$,
from independent observations and laboratory tests [5].

The dark charge is unambiguously defined in terms of non-trivial Nambu-
Goldsone mode [9]. O(2) symmetry of eJBD $\chi$ field is spontaneously broken
inside compact objects by chameleon effect [11], and the dark charge
$Q_{i}^{\chi}$ of compact object $i$ is given by the space-integrated time
component of the conserved current $J_{\mu}$,

$\displaystyle
J_{\mu}=-i\frac{1}{2}(\chi^{\dagger}\partial_{\mu}\chi-\partial_{\mu}\chi^{\dagger}\chi)=\chi_{1}\partial_{\mu}\chi_{2}-\chi_{2}\partial_{\mu}\chi_{1}=\chi_{r}^{2}\partial_{\mu}\theta\,,\hskip
14.22636ptQ_{i}^{\chi}=\int_{i}d^{3}x\,J_{0}\,,$ (7)

inside a compact object $i$. The field equation for the angular $\theta-$part,
or equivalently the conservation law, $\partial^{\mu}J_{\mu}=0$, makes it
possible to have non-vanishing, time dependent $Q_{i}^{\chi}$ along with non-
vanishing $\vec{\nabla}\cdot\vec{J}$ inside compact objects such that the
Gauss-law holds.

On the other hand, $D$ that appears in (4) is derived by solving $\chi$ field
equation,

$\displaystyle\frac{1}{\sqrt{-g}}\partial_{\mu}\partial_{\nu}(\sqrt{-g}g^{\mu\nu}\delta\chi)=\langle\left(\frac{\delta^{2}{\cal
L}^{\rm
E}_{st}}{\delta\chi^{2}}\right)_{\chi=\chi_{0}}\rangle_{i}\delta\chi\,,$ (8)
$\displaystyle\left(\frac{\delta^{2}{\cal L}^{\rm
E}_{st}}{\delta\chi^{2}}\right)_{\chi=\chi_{0}}\simeq\frac{c}{M_{\rm
P}^{2}}\left(2p_{g}{\cal L}_{\rm
gluon}-(2p_{Y}+p_{f})\sum_{f=u,d,e}m_{f}\bar{f}f+2p_{f}\sum_{f=u,d,e}\partial\cdot\bar{f}i\gamma
f\right)\,,$ (9)

with $\chi=\chi_{0}+\delta\chi$. The quantity $\langle\cdots\rangle_{i}$ is
the expectation value integrated over a compact object $i$. The last term in
(9) is derived by a partial integration, and vanishes if one uses the Dirac
equation $i\partial\cdot\gamma f=m_{f}f$, namely on-the-mass-shell condition.
The metric here is taken the same as in (1). The right hand side quantity of
(8) is non-vanishing only inside compact objects. Spatial integration of this
quantity gives a quantity of order, $2c\bar{p}M_{i}Q^{\chi}_{i}$ with
$\bar{p}$ an average of $p_{i}$ factors. We define a quantity $\eta_{0}$ by
$\eta_{0}=2c\bar{p}$. If one chooses $p_{g}=p_{Y}=0$ in order to maintain the
simplest consistency with the null-observation of time variation, the right
hand side of (9) becomes $-cp_{f}/M_{\rm P}^{2}$ times
$\sum_{f=u,d}m_{f}\bar{f}f$, which gives a fraction of quark mass contribution
$\sim 0.01$ of neutron star. The important parameter $Q^{\chi}_{i}$ in the
case of $p_{g}=p_{Y}=0$ is of order $0.01/2$ and $\eta_{0}=-cp_{f}$ for the
neutron star.

As in the literature [12], we assume that black holes have no hair, while
neutron stars, or any other astrophysical bodies without event horizon, can
have dark charges $D$. The purpose of the present work is how to derive
constraint on $D/M_{i}=O(2c\bar{p}Q_{i}^{\chi})$ from gravitational emission
from BH-NS merger, assuming that the scalar charge emission is impossible to
observe by gravitational wave detectors presently available. If one can
constraint these power factors from GW observations, one would have hints on
low energy limit of string theories.

Gravitational and dark wave emission from binary system

Formulas of quadrupole gravitational wave (denoted by $GW$) emission and
dipole dark wave (denoted by $DW$) emission rates are given in [12].333 The
formula of dark wave emission given in this paper is based on the work [14]:

$\displaystyle
L_{GW}=\frac{32}{5}\frac{G^{4}\mu^{2}M^{3}}{R^{5}}\,f_{g}(e)\,,\hskip
8.5359ptL_{DW}=\frac{G^{3}\mu^{2}D^{2}}{R^{4}}\,f_{d}(e)\eta_{0}^{2}\,,$ (10)
$\displaystyle
f_{g}(e)=\frac{1+73e^{2}/24+37e^{4}/96}{(1-e^{2})^{7/2}}\,,\hskip
8.5359ptf_{d}(e)=\frac{1+e^{2}/2}{(1-e^{2})^{5/2}}\,,$ (11)

where $e$ is eccentricity, $\Omega=2\pi/P$ with $P$ the orbital period, 444
According to [16] the orbital angular momentum $J$ changes following a
relation $J^{2}=GM\mu^{2}(1-e^{2})\,R\propto R$ for a constant eccentricity.
Remnant compact object can be rotating BH described by a Kerr solution.  and
$M=M_{1}+M_{2}\,,\mu=M_{1}M_{2}/M$ (total and reduced masses). $\eta_{0}$ is
the quantity of order $2c\bar{p}$. We assume that the eccentricity is
unchanged during the entire emission. In the case of NS-NS merger the energy
loss rate is proportional to $\eta_{0}^{4}(D_{1}-D_{2})^{2}/M_{NS}^{2}$.

We consider BH-NS merger as the binary system. The effective gravitational
mass $M_{g}$ is given by $M_{g}=M_{1}+M_{2}-GM_{1}M_{2}/(2R)$ in the binary
system: this formula is valid in non-relativistic Keplerian motions. When the
radius $R$ is much larger than the Schwarzshild radius $2GM_{g}$, variation of
gravitational mass is given by variation of $R$. Hence one has a differential
equation for the radius $R$ change:

$\displaystyle\frac{dR}{dt}=-\frac{64}{5}\frac{G^{3}\mu
M^{2}}{R^{3}}f_{g}(e)-\frac{2G^{2}M_{BH}D^{2}}{M_{NS}R^{2}}f_{d}(e)\eta_{0}^{2}\,.$
(12)

Solution without the dark wave emission (the case $f_{d}(e)=0$) is given by

$\displaystyle R(t)=\left(4K_{GR}(t_{e}-t)\right)^{1/4}\,,\hskip
14.22636ptK_{GR}=\frac{64}{5}G^{3}\mu
M^{2}\,f_{g}(e)=\frac{8}{5}r_{S}^{3}\frac{\mu}{M}\,f_{g}(e)\,.$ (13)

Time scale in the last stage of gravitational collapse is $r_{S}\sim
10^{-5}\mu{\rm sec}(2GM/3{\rm km})$.

To solve the problem of gravitational collapse at BH-NS merger, we need to
derive the differential equation for dark scalar wave emission.

Dark wave emission in isolated neutron star or white dwarf

Consider an isolated neutron star or a white dwarf. Neutron star is supported
by the degeneracy pressure of neutrons against gravitational collapse, while
white dwarf by the electron degeneracy pressure. This picture has been
confirmed by many observations when GR holds without the presence of eJBD
field: both objects are stable in isolation. Stability of compact objects in
isolation are guaranteed due to very small $\chi$ coupling of gravitational
strength order to matter [13]. Nevertheless, they may emit the dark wave in
dynamical process of the gravitational collapse at merger with black hole.

The fundamental calculation of dark wave emission rate by [14] gives the
gravitational mass change, while we also need dark charge emission rate. These
two rates are related. An intuitive way to convert from the energy to the
charge is to multiply the factor $D_{NS}/m_{NS}$ for this conversion. 555 The
paper [14] adopts for the matter system N-body geodesic action in which
particle masses may depend on JBD field, hence $m_{i}(\chi)$. Derivative
$dm_{i}/d\chi$ plays the fundamental role in evaluating dipole dark wave
emission rate, consistent with our conversion factor $D_{NS}/m_{NS}$.

It is possible to justify this conversion factor another way, by adopting an
adiabatic change of static configuration in eJBD gravity. In this picture one
takes parameters of static solutions as slowly changing functions of time. In
particular, the parameter $l$ in the static solution above, hence its function
$M^{2}+D^{2}=l^{2}/(2G)^{2}$ slowly decreases in time. Since
$\Delta(M^{2}+D^{2})\sim M\Delta M+D\Delta D$, we take this to imply that the
gravitational mass variation with weight $M$ is accompanied by $D$ charge
variation with weight $D$, hence

$\displaystyle\frac{dD}{dt}=\frac{D}{M_{NS}}(\frac{dM_{g}}{dt})_{DW}=-\frac{D}{M_{NS}}L_{DW}=-G^{3}\frac{M_{BH}^{2}}{M_{NS}}f_{d}(e)\eta_{0}^{2}\frac{D^{3}}{R^{4}}\,,$
(14)

for BH-NS binary.

Coupled differential equations and asymptotic behavior towards gravitational
collapse

Using dimensionless variables,

$\displaystyle r=\frac{R}{2GM}=\frac{R}{r_{S}}\,,\hskip
8.5359ptx=\frac{D}{M}\,,\hskip 8.5359pt\tau=\frac{t}{2G\mu}\,,$ (15)

the set of differential equations to be solved is

$\displaystyle\frac{dr}{d\tau}=-\left(\frac{8}{5}f_{g}(e)\frac{1}{r^{3}}+\frac{1}{2}f_{d}(e)\eta_{0}^{2}\frac{x^{2}}{r^{2}}\right)(\frac{\mu}{M})^{2}\,,$
(16)
$\displaystyle\frac{dx}{d\tau}=-\frac{1}{8}(\frac{M_{BH}}{M})^{3}f_{d}(e)\eta_{0}^{2}\,\frac{x^{3}}{r^{4}}\,.$
(17)

This mathematical system is not analytically tractable. But one can attempt to
derive the end point behavior and confirm the result by numerical
integrations. We adopt the following ansatz for solution towards the end time
$\tau_{*}$ of the gravitational collapse:

$\displaystyle x\sim x_{*}+a\ln(\tau_{*}-\tau)\approx x_{*}\,,\hskip
14.22636ptr\sim b(\tau_{*}-\tau)^{\beta}\,.$ (18)

Here it is assumed that time span $\tau_{*}-\tau$ is limited by an inequality,
$|a\ln(\tau_{*}-\tau)|\leq x_{*}$. Inserting these into the coupled equations,
we impose consistency of time powers and their coefficients, to derive

$\displaystyle\beta=\frac{1}{4}\,,\hskip
14.22636ptb=(\frac{32}{5}f_{g})^{1/4}\sqrt{\frac{\mu}{M}}\,,\hskip
14.22636pta=\frac{5}{4}\frac{f_{d}}{f_{g}}\eta_{0}^{2}\,\frac{M_{BH}^{3}}{M\mu^{2}}\,x_{*}^{3}\,,$
(19) $\displaystyle
r(\tau)\rightarrow(\frac{32}{5}f_{g}\frac{\mu^{2}}{M^{2}})^{1/4}\,(\tau_{*}-\tau)^{1/4}\,.$
(20)

The minimum size of remnant compact object is derived from that observers can
watch the region outside the event horizon, $R\geq 2GM_{g}$ (the Schwarzschild
radius taking a mass $M_{g}$), from which one derives an equation for the
minimum $M_{g}$: $M_{g}=M-GM\mu/(4GM_{g})$. This gives the minimum
$M_{g}=M(1+\sqrt{1-\mu/M})/2\equiv M_{\rm min}$.

In Fig(1) we illustrate a result of numerical integration, and give a
comparison with the behavior in GR. The mass parameters are taken from event
GW200115 [4], interpreted to be BH-NS merger event. Computation is stopped at
the time when the orbit radius reaches the minimum value $R=2GM_{\rm min}$,
where $x(\tau)$ is found to be positive and finite. This indicates that haired
compact object remains after gravitational collapse.

Figure 1:  Dark charge $x$ in solid black and binary orbit radius $r$ in
dashed orange against a rescaled time $\tau$ (to transform away
$8f_{g}\mu^{2}/5M^{2}\rightarrow 1$ in $r-$equation) for parameters,
$\eta_{0}=1\,,e=0.3\,,m_{NS}/m_{BH}=1.4/6$. Assumed initial conditions are
$x=5\,,r=50$. For comparison the GR formula of radius
$\propto(\tau_{*}-\tau)^{1/4}$ is shown in dash-dotted blue, and the end point
value $x$ and $r_{*}=(1+\sqrt{1-\mu/M})/4$ are in dotted black.

The orbit change of binary systems generates the gravitational wave amplitude
$h(\tau)$ ( metric component usually denoted by $h^{TT}$, the transverse
traceless component of metric fluctuation from the flat Minkowski spacetime,
$g_{\mu\nu}-\eta_{\mu\nu}$) according to [15, 16]

$\displaystyle\frac{\dot{P}}{P}=-\frac{3}{5}\frac{f_{g}(e)}{r^{4}}\,,\hskip
8.5359pth\sim\frac{R(t_{0})}{R(\tau)}\,\cos 2\pi\frac{\tau}{P}\,,$ (21)

using $\tau$ variable for time. This $h$ is taken at the emission site. This
does not include the ringdown phase caused by quasi-normal modes. These
formulas help to understand template behaviors of gravitational wave signals
in observed events [2, 3].

Gravitational $h-$wave amplitude given by (21) is illustrated in Fig(2) using
the same set of parameters as in Fig(1). This example is not meant to give a
definite conclusion on constraint of eJBD gravity, since a more elaborate,
fully relativistic calculation of orbit motion is required. But it is
suggestive that the parameter choice taken here seems testable from
accumulated GW observations at the BH-NS merger. It is further found that the
product of GW envelope amplitude and period $\propto P(\tau)/R(\tau)$ is a
good measure to distinguish eJBD gravity from GR, as seen from Fig(2),
although how to match the ringdown phase is highly non-trivial to us.

Although we present, in our three figures, an example among many numerical
computations, the existence of remnant scalar-hair and a sharper rise of the
amplitude-period product in GR, as observed in Fig(2), have been verified in
all of many other initial combinations of the dark charge and the radius.
These two features of eJBD gravity are generic independent of initial
conditions.

Figure 2:  Gravitational $h-$amplitude in our unit of time $\tau$ in
$\cos(2\pi\tau/P(\tau 0)/r(\tau)$ for JBD gravity in solid black and the
product of period $\times$ $h-$envelope amplitude (scaled down by a factor
$1\times 10^{-4}/2\pi$): the product result with dark wave emission in dotted
black and result of GR in dash-dotted blue. The same set of parameters as in
Fig(1) are used. Computation is stopped at a time $\tau_{f}$ which satisfies
the orbit radius $r(\tau_{f})$ reaches $5\times 2GM_{g}(\tau_{f})$ (5 times
the realistic Schwarzschild radius with calculated $M_{g}(\tau_{f})\sim
0.92\times M$). The ratio of GR to eJBD gravity for product values changes by
$\sim 290$ at the stopped time from the initial value.

A remnant candidate: dark bosonic cloud

There is no sign of naked curvature singularity at $r\geq r_{*}$, outside the
Shwarzschild radius, during the gravitational collapse. In reality, the
solution must be, for small radius $R$, matched to solution inside a massive
body of finite size. Inside massive bodies gravity is weakened, and the body
would be shrunk until it is supported by pressure term.

There are two possibilities for remnant compact objects after the
gravitational collapse at the BH-NS merger. The one is formation of black hole
having a dark scalar hair, violating the no-hair conjecture. The other
interesting possibility for the merged remnant is the bosonic cloud made of
black hole and eJBD scalar bosons [17]. 666 The idea was originally proposed
in [18]. This a microscopic Penrose process [19] that extracts both energy and
angular momentum from rotating black holes.  This is a gigantic atom quantum
mechanically bound by gravitational interaction. Unlike electrons around a
nucleus any number of scalar particles can occupy in a single energy level.
Hence coherent atomic transitions, superradiant graviton emission, may be very
much enhanced giving rise to characteristic GW emission rate [17]. If eJBD
quanta of a mass range, $10^{-20}\sim 10^{-10}$eV, are gravitationally trapped
outside the Schwarzshild radius of final BH, its presence may become
detectable. Since eJBD field acquires a density dependent mass of order
$\sqrt{G\rho_{m}}\sim 1\times 10^{-19}{\rm eV}\sqrt{\rho_{m}/{\rm
gr\,cm}^{-3}}$ (as in [11]), the quoted mass range may be realized in
accretion medium surrounding a black hole. Thus, if the remaining dark charge
$D$ is all outside the radius $2GM_{g}$ of gravitational collapse, bound eJBD
quanta may all be ionized to escape from BH, to finally dispel dark charge
consistent with the no-hair conjecture.

If experimenters find symptoms of dark wave emission from GW detection, it may
become important to find out a direct detection method of dark wave arrival
simultaneously with GW arrival on earth. It would be further exciting if the
super-radiant GW signal is detected in the ringdown phase after the collapse.

## Acknowledgement

The work is in part supported by JSPS KAKENHI Grant Nos. 19H01899 (KK and KO),
21H01107 (KO), and 21K03575 (MY).

## References

  * [1] C.M. Will, Pramana 63, 731 (2004); The Confrontation between General Relativity and Experiment, arXiv:gr-qc/0103036v1 (2002).
  * [2] The LIGO Scientific Collaboration and the Virgo Collaboration, Phys.Rev. D100 10, 104036 (2019). Tests of General Relativity with the Binary Black Hole Signals from the LIGO-Virgo Catalog GWTC-1, arXiv: 1903.04467 [gr-qc] (2019). B.P. Abbott et al, Phys.Rev.Lett. 116 22, 221101 (2016), Phys.Rev.Lett. 121 12, 129902 (2018) (erratum). Tests of general relativity with GW 150914, arXiv: 1602.03841 [gr-qc] (2018).
  * [3] For GR test of NS-NS merger, LIGO Scientific and Virgo Collaboration, Phys.Rev.Lett. 123 1, 011102 (2019). Tests of General Relativity with GW170817, arXiv: 1811.00364 [gr-qc] (2019).
  * [4] R. Abbott et al., Astrophysical Journal Letters, 915:L5 (24pp), (2021) https://doi.org/10.3847/2041-8213/ac082e © 2021.
  * [5] K. Kaneta, K. Oda, and M. Yoshimura, arXiv: 2304.08656 (2023). Constraints on extended Jordan-Brans-Dicke gravity.
  * [6] P. Jordan, Z. Phys. 157, 112 (1959). C. Brans and R. Dicke, Phys. Rev. 124, 925(1961).
  * [7] O. Heckmann, P. Jordan, and R. Fricke, Astroph. Z. Naturforsch. 28, 113 (1951). H. Buchdahl, Phys.Rev. 115, 1325 (1959). M. Yoshimura, Phys. Rev. D34, 1021 (1986). Classification of Static Vacuum Metric With Ricci Flat Compactification.
  * [8] M. Yoshimura Phys.Scripta T36, 276 (1991). Unstable space-time in the Jordan-Brans-Dicke gravity.
  * [9] M. Yoshimura, Phys.Rev. D105, 083522 (2022).
  * [10] T. Damour and A.M. Polyakov, Nucl. Phys. B423 ,532 (1994).
  * [11] J. Khoury and A. Weltman, Phys.Rev. D69, 044026 (2004). Phys.Rev.Lett. 93, 171104(2004); Chameleon fields: Awaiting surprises for tests of gravity in space.
  * [12] M.W. Horbatsch and C.P. Burgess, JCAP 05, 010 (2012); arXiv: 1111.4009[gr-qc] (2011).
  * [13] N. Ishizuka and M. Yoshimura, Prog.Theor.Phys. 84, 233 (1990). Axion and dilaton emissivity from nascent neutron stars.
  * [14] T. Damour and G. Esposito-Farese, Class. and Quant. Grav. 9, 2093 (1992).
  * [15] L.D. Landau and E.M. Lifshitz, The classical theory of fields, 4th revised english edition, Elsevier (1975).
  * [16] S.L. Shapiro and S.A. Teukolsky, Black holes, white dwarfs, and neutron stars, John Wiley and Sons, New York (1983).
  * [17] D. Baumann, G. Bertone, J. Stout, and G.M. Tomaselli, Sharp Signals of Boson Clouds in Black Binary Spirals, arXiv: 2206.01212 [gr-qc], (2022). M. Isi, L. Sun, R. Brito, and A. Melatos, Phys.Rev. D 99, 084042 (2019); Phys.Rev. D 102 , 049901 (erratum) (2020). Directed searches for gravitational waves from ultralight bosons, arXiv: 1810.03812 [gr-qc], (2019). R. Brito, S. Ghosh, E. Barausse, E. Berti, V. Cardoso, I. Dvorkin, A. Klein, and P. Pani, Phys. Rev. D 96, 064050 (2017). Gravitational wave searches for ultralight bosons with LIGO and LISA, A. Arvanitaki, M. Baryakhtar, and X. Huang, Phys. Rev. D 91, 084011 (2015). Discovering the QCD axion with black holes and gravitational waves.
  * [18] A. Arvanitaki, S. Dimopoulos, S. Dobovsky, N. Kloper, and J. March-Russell, Phys.Rev. D81, 123530(2010); arXiv: 0905.4720[hep-th].
  * [19] R. Penrose, Riv. Nuovo Cim. 1, 252 (1969). [Gen. Rel. Grav. 34, 1141 (2002)].