# Understanding the Impact of UGC Specificities on Translation Quality

José Carlos Rosales Núñez1,2,3 Djamé Seddah3 Guillaume Wisniewski4
1 CNRS, LISN 2 Université Paris-Saclay
3 Inria Paris 4 Université de Paris, LLF, CNRS
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

This work takes a critical look at the evaluation of user-generated content
automatic translation, the well-known specificities of which raise many
challenges for MT. Our analyses show that measuring the average-case
performance using a standard metric on a UGC test set falls far short of
giving a reliable image of the UGC translation quality. That is why we
introduce a new data set for the evaluation of UGC translation in which UGC
specificities have been manually annotated using a fine-grained typology.
Using this data set, we conduct several experiments to measure the impact of
different kinds of UGC specificities on translation quality, more precisely
than previously possible.

##  1 Introduction

This work takes a critical look at the evaluation of user-generated content
(UGC) automatic translation. The well-known specificities of UGC (high rate of
OOVs, rare, grammatical constructs, …) raise many challenges for Machine
Translation and has been the topic of many recent works Rosales Núñez et al.
(2019); Specia et al. (2020).

Several UGC parallel corpora Michel and Neubig (2018a); Rosales Núñez et al.
(2019) have been introduced to evaluate the robustness of MT, some of which,
such as Fujii et al. (2020), are specially annotated to identify UGC
idiosyncrasies allowing to measure the impact of a given specificity. Our
analyses (§2), indeed, show that measuring the average-case performance using
a standard metric on a UGC test set falls far short of giving a reliable image
of the UGC translation quality: explaining the observed performance gap
requires a particular evaluation framework made of tailored metrics and
specific test sets in which UGC idiosyncrasies have been precisely annotated.

That is why, following this line of works, we introduce PMUMT, a new parallel
data set for the evaluation of UGC translation between French and English in
which UGC specificities have been manually annotated using a fine-grained
typology. PMUMT is larger, relies on a more refined error typology and, more
importantly, its annotations are more detailed than existing noisy parallel
corpora. Its annotation scheme enables us to generate automatically parallel
corpora in which the kind and number of UGC specificities are precisely
controlled. Contrary to many works studying the robustness of NMT systems by
adding artificial noise to canonical corpora, PMUMT is made of attested UGC
examples.

Using this framework, we conduct several experiments on three out-of-the-box
NMT architectures in a zero-shot scenario, to measure more precisely than what
was possible before the impact of the different kinds of UGC specificities on
translation quality. Surprisingly enough, our experiments (§3) on natural data
show that out-of-the-box models exhibit unexpected strong robustness against
several kinds of noise, questioning several results reported in the literature
Michel and Neubig (2018a); Belinkov and Bisk (2018). We believe that this data
set and its associated evaluation framework will pave the way for a better
understanding of the interactions at play in neural machine translation of
noisy user-generated content contexts.

##  2 Testing Out-of-the-Box NMT models on UGC

$\downarrow$ Metric / Test set $\rightarrow$ | PFSMB† | PMUMT† | MTNT† | 4SQ† | NewsTest | OpenSubsTest
---|---|---|---|---|---|---
3-gram KL-Div | 1.563 | 1.442 | 0.471 | 0.500 | 0.406 | 0.006
%OOV | 12.63 | 11.47 | 6.78 | 3,46 | 3.81 | 0.76
PPL | 599.48 | 596.12 | 318.24 | 293.67 | 288.83 | 62.06
Table 1: Domain-related measure on the source side (FR), between used Test
sets and other noisy UGC corpora using OpenSubtitles as training set. Dags
indicate UGC corpora. 4SQ is the 4Square UGC data set introduced in Berard et
al. (2019). PPL: perplexity, KL-Div: Kullback-Leibler divergence.

|  | WMT |  | OpenSubtitles
---|---|---|---|---
|  |  PFSMB † | MTNT† | News⋄ | OpenTest |  |  PFSMB † | MTNT† | News | OpenTest⋄
BPE-based models
abSeq2seq |  |  09.9 | 21.8 | 27.5 | 14.7 |  | 17.1 | 27.2 | 19.6 | 28.2
abcd \+ <UNK> rep. |  | 17.1 | 24.0 | 29.1 | 16.4 |  | 26.1 | 28.5 | 24.5 | 28.2
abTransformer |  | 15.4 | 21.2 | 27.4 | 16.4 |  | 27.5 | 28.3 | 26.7 | 31.4
Character-based models
abchar2char |  |  07.1 | 13.9 | 18.1 |  08.8 |  | 23.8 | 25.7 | 17.8 | 26.3

Table 2: Bleu scores for our models. The ${\dagger}$ symbol indicates the UGC
test sets, and $\diamond$ in-domain test sets.

###  2.1 Experimental Setting

#### Training Data

Because of the lack of a large parallel data set of noisy sentences, we train
our systems on ‘standard’ parallel data sets: WMT Bojar et al. (2016) and
OpenSubtitles Lison et al. (2018). The former contains canonical texts (2.2M
sent.) and the latter (9.2M sent.) is made of informal dialogues found in
popular sitcoms.

#### UGC Test Sets

To evaluate the different NMT models, we consider two data sets of manually
translated UGC: MTNT Michel and Neubig (2018a) and the Parallel French Social
Media Bank corpus (PFSMB) Rosales Núñez et al.
(2019)111https://gitlab.inria.fr/seddah/parallel-french-social-mediabank which
extends the French Social Media Bank Seddah et al. (2012) with English
translations. These two data sets raise many challenges for MT systems: they
notably contain characters that have not been seen in the training data (e.g.
emojis), rare character sequences (e.g. inconsistent casing or usernames) as
well as many OOVs denoting URL, mentions, hashtags or more generally named
entities (NE). Most of the time, OOVs are exactly the same in the source and
target sentences.

#### NMT Models 222Models parameters are detailed in the appendix.

In our experiments, we use three translation models. The first two models are
standard NMT models that take as input BPE tokenized sentences: the model used
in (Michel and Neubig, 2018a), a Seq2seq bi-LSTM architecture with global
attention decoding as implemented in XNMT Neubig et al. (2018) as well as a
vanilla Transformer model as implemented in the OpenNMT toolkit Klein et al.
(2018).

We also consider a char-based model, namely the char2char of Lee et al.
(2017). Using char-based models which are, by nature, open-vocabulary to
translate UGC is intuitively appealing as these models are designed
specifically to address the problem of translating OOVs and to deal with noisy
input Belinkov and Bisk (2018).

As the Seq2seq model we consider in our experiments is not able to translate
OOVs, we introduce, as part of our translation pipeline, a post-processing
step in which the translation hypothesis is aligned with the source and <UNK>
tokens are replaced by their aligned source token. In the case of our
Transformer model, OpenNMT performs this automatically.

###  2.2 Results

Table 2 reports the Bleu scores Papineni et al. (2002) 333All Bleu scores are
calculated by Post (2018)’s SacreBleu using the intl tokenization of the
different models we consider both on canonical and non-canonical test sets.
Contrary to the first results of Michel and Neubig (2018a), the quality of UGC
translation does not appear to be so bad: the drop in performance observed on
non-canonical corpora is of the same order of magnitude as the drop observed
when translating out-of-domain data.

These results seem to indicate that, counter-intuitively, translating UGC does
not raise any specific challenges. We however believe that they are biased by
the evaluation metric used: as UGC contains many mentions, URLs emoticons, or
named entities that are the same in the source and in the target sentence,
Bleu scores estimated on a canonical and on a non-canonical can not be
directly compared: Bleu scores on non-canonical data are artificially high as
systems are rewarded for simply coping source tokens, which is the most
natural solution to translate OOVs. For instance, the Bleu score between the
_sources_ and references of the PFSMB is 15.1 while it is only 2.7 on the WMT
test set. That is why, we believe that the usual MT metrics overestimate the
translation quality on UGC and we introduce, in the next section, a new corpus
and a new way to measure the real impact of UGC specificities on translation
quality.

##  3 Analyzing the Impact of UGC on Translation Quality

|  char del/add |  missing diacritics |  phonetic writing |  tokenization |  wrong tense |  special char |  agreement |  casing |  emoji |  named entity |  contraction |  repetition |  interjections
---|---|---|---|---|---|---|---|---|---|---|---|---|---
s2s |  0.80 (28.7) |  0.95 (33.9) |  0.93 (27.3) |  0.96 (30.8) |  0.94 (30.7) |  0.88 (26.1) |  0.95 (27.1) |  0.75 (27.7) |  0.91 (31.0) |  0.86 (31.7) |  0.95 (30.8) |  0.90 (30.2) |  0.93 (29.2)
c2c |  0.99 (32.5) |  0.99 (29.6) |  0.86 (25.2) |  1.00 (31.9) |  0.97 (28.8) |  0.81 (24.6) |  0.96 (28.9) |  0.86 (28.0) |  0.83 (26.2) |  0.94 (32.7) |  0.91 (30.4) |  0.95 (26.2) |  0.91 (28.7)
TX |  0.98 (35.3) |  1.02 (34.0) |  1.03 (33.2) |  0.98 (32.9) |  1.02 (33.7) |  0.92 (29.2) |  0.97 (33.8) |  0.90 (26.9) |  0.75 (28.3) |  0.99 (35.4) |  0.93 (31.1) |  0.89 (36.8) |  0.86 (30.2)
Table 3: Bleu score ratios between pairs of noisy and normalized sets of
sentences, containing only one UGC specificity. Bleu scores on noisy sets are
shown in parenthesis.

In order to understand the impact of UGC specificities on translation quality,
we have annotated a new corpus in which UGC peculiarities are identified in
each source sentence and ‘normalized’ to a canonical form.

###  3.1 A Corpus Annotated with UGC Specificities

#### The PMUMT corpus

To understand the impact of UGC peculiarities, we manually annotated 400
source sentences sampled from the PFSMB: one of the authors, fluent in French
and with good knowledge of UGC, has identified spans in the sentence that
differ from canonical French and characterized these specificities using the
fine-grained typology of Sanguinetti et al. (2020) (see Table 4). Since the
whole annotation process was done by a single person, no inter-annotator
agreement can be calculated. Nevertheless, results of our pilot study for each
individual UGC peculiarity (cf. Table 3 and Table A.3 in the Appendix, for a
cross-metrics analysis), show that MT performance consistently performs better
on our normalized corpus than on the original noisy set.

code | kind of specificities
---|---
1 | Letter deletion/addition
2 | Missing diacritics
3 | Phonetic writing
4 | Tokenisation error
5 | Wrong verb tense
6 | #; @, URL
7 | Wrong gender/grammatical number
8 | Inconsistent casing
9 | Emoji
10 | Named Entity
11 | Contraction
12 | Graphemic/punctuation stretching
13 | Interjections
Table 4: Typology of UGC specificities used in our manual annotation.

Each span containing an UGC specificity has been ‘normalized’ to a form closer
to canonical French.444To ensure that this normalization has actually made our
corpus closer to a canonical corpus, we have computed the perplexity of the
original sentences and of the normalized sentences estimated by a 5-gram
Kneser-Ney language model trained on the OpenSubtitles corpus: the normalized
version has a perplexity of 2,214 (and 11.60% of its token are OOVs) far lower
that the original version (with a perplexity of 8,546 and an OOV ratio of
19.60%). Table A.1 shows some examples of annotated (source) sentence. A
normalized form of each target (i.e. English) sentence has also been produced
to ensures that the target can be generated from the ‘normalized’ source.

In the end, the annotation of this corpus represents 200h of work, comprising
an iterative improvement and debugging of the annotations to achieve the
corpus’ current version. 555The annotated corpus and code collection can be
found in
https://github.com/josecar25/PMUMT_annotated_UGC_corpus/

The resulting corpus contains more than $1,310$ annotations. On average, each
sentence contains $2.8$ UGC peculiarities. Figure 1 describes the distribution
of UGC peculiarities in the corpus.

char del/addmissing diacriticsphonetic writingtokenizationwrong tensespecial
charagreementcasingemojinamed
entitycontractionrepetitioninterjections$100$$200$$300$$113$$111$$120$$68$$76$$174$$61$$40$$55$$292$$81$$62$$77$Counts
Figure 1:  Distribution of UGC specificites of the FR UGC sources in PMUMT.

#### Controlling the Number of Specificities per Sentence

Comparing the predictions of an NLP system taking either the normalized
sentences or the original non-canonical sentences as input allows us to
measure the impact of UGC on this system. However, it is impossible to perform
a fine-grained analysis in which, for example, the impact of different types
of specificities are compared, since UGC sentences generally contain several
specificities of different types and the interactions between them cannot be
easily neutralized.

That is why we have also constructed automatically a second version of our
corpus to help us analyze the interactions between the UGC specificities in a
sentence: by substituting only some of the span we have annotated, we can
create corpora in which the number and the kind of specificities present in
each sentence is tightly controlled. In this framework, each original sentence
can be (partially) rewritten into as many sentences as there are UGC
specificities in it.

This possibility of partial substitution greatly reduces the amount of data to
be annotated for our analyses: instead of having to annotate a large amount of
data to find enough sentences fulfilling the requested criteria, we are able
to generate these sentences from our original annotation of 400 sentences. We
believe that this approach could be of great interest to perform fine-grained
error analysis for NLP systems dealing with UGC.

###  3.2 Impact of UGC Peculiarities on Translation Quality

We used the PMUMT corpus to evaluate the impact of UGC peculiarities on
translation quality: we have reported in Table 5 the Bleu scores achieved by
the considered systems on both the 400 original sentences and the 400
normalized sentences. As expected, translations of normalized sentences, that
are more similar to the training data, are of better quality than translations
of original (noisy) sentences: the Bleu scores achieved when translating
normalized UGC content are close to those obtained on the in-domain test-set.

For all systems, considering the non-canonical original sentences results in a
drop in translation quality of the same order of magnitude, which shows that,
even if these models build sentence representations from completely different
information, the presence of UGC peculiarities has a similar impact on all of
them.

| original | normalized
---|---|---
Seq2seq | 25.8 | 32.4
char2char | 24.1 | 30.5
Transformer | 28.6 | 33.6
Table 5: Bleu scores on the original and normalized source sentences of the
PMUMT corpus.

#### Individual UGC Errors

To get a more precise picture of the impact of UGC on translation quality, we
have computed, for each kind of peculiarities, the Bleu scores achieved on the
corpus built to contain only this peculiarity and the Bleu score computed on
the ‘normalized’ version of the same sentences. Table 3 reports the ratio
between these two scores (detailed results are reported in Table A.3 in the
supplementary material).

The impact of a given kind of UGC specificity on translation quality is very
different from one system to another: it appears that the source sentences
representation that MT systems learn to construct are not sensitive to the
same kind of noise or errors in the source sentence and even seem to be
complementary. For instance, inconsistent casing strongly penalizes the
Seq2seq model but has only a limited impact on the char2char model. On the
contrary, the presence of characters specific to online conversation such as @
or # results in a substantial decrease of translation quality for char2char,
but has less impact for Seq2seq or Transformer, suggesting that char-based
models are not able to properly modeled characters that hardly appear in the
training set.

Interestingly, the Transformer model appears to be very robust to a wide array
of UGC peculiarities, even if it was not designed specifically to handle noisy
inputs: in particular, the presence of named entities, spelling errors (i.e.
substitution, deletion or insertion of letters), agreement error (of verb
tense or in gender and number) as well as tokenization errors hardly hurt
translation quality. Similarly, the char2char model succeed in translating
correctly sentences with letter addition or suppression, showing that the
model actually manage to learn sentence representations that are robust to
spelling errors even if such errors are not present at training time. This
results is at odd with the conclusion drawn by Belinkov and Bisk (2018) on
artificial data.

#### Combination of Peculiarities

To better understand the impact of combinations of UGC peculiarities on
translation quality, Table 6 reports the ratio between the Bleu scores
computed on the translation of a corpus in which there are exactly $N$
different UGC peculiarities in a sentence and on the translation of the
normalized version of these sentences. It appears that for all our systems
translation quality decreases linearly with the number of specificities,
suggesting that the impacts of the different specificities are independent of
each other. Surprisingly enough, the gap between the char2char and Transformer
is getting smaller with the number of specificities in each sentence.

| 1 | 2 | 3 | 4+
---|---|---|---|---
# sents. | 1,306 | 1,776 | 1,439 | 1,089
s2s |  0.90 (30.1) |  0.83 (27.0) |  0.77 (24.2) |  0.75 (23.2)
c2c |  0.92 (29.5) |  0.87 (26.6) |  0.83 (24.3) |  0.83 (23.2)
TX |  0.96 (32.8) |  0.89 (30.0) |  0.86 (28.3) |  0.84 (26.5)
Table 6: Bleu score ratio between pairs of normalized and noisy sentences
containing $N$ specificities. Bleu scores on noisy sentences are shown in
parenthesis.

##  4 Conclusions

This work introduces PMUMT a new corpus of UGC translation in which UGC
specificities are manually annotated using a fine-grained typology. Thanks to
our detailed annotation process, we were able to build a new framework that
allows us to automatically generate several parallel corpora in which the
number and kind of UGC specificities is precisely controlled.

Our experiments show that, contrary to what was previously believed, out-of-
the-box NMT models are robust to many different kind of UGC specificities and
that the different architectures we tested are complementary, in the sense
that they are not sensitive to the same specificities. In our future work, we
plan to explore the intricacies of the robustness that seem to be linked to
specific UGC idiosyncrasies. We make this data set and its associated
evaluation framework
public666https://github.com/josecar25/PMUMT_annotated_UGC_corpus/ as we
believe it can pave the way for a better understanding of the interactions at
play in neural machine translation of noisy user-generated content contexts.

## Acknowledgments

We thank the reviewers for their valuable feedback. This work was funded by
the French Research Agency via the ANR ParSiTi project (ANR-16-CE33-0021).

## References

  * Belinkov and Bisk (2018) Yonatan Belinkov and Yonatan Bisk. 2018.  Synthetic and natural noise both break neural machine translation.  In _Proceedings of the 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_.
  * Berard et al. (2019) Alexandre Berard, Ioan Calapodescu, Marc Dymetman, Claude Roux, Jean-Luc Meunier, and Vassilina Nikoulina. 2019.  Machine translation of restaurant reviews: New corpus for domain adaptation and robustness.  In _Proceedings of the 3rd Workshop on Neural Generation and Translation@EMNLP-IJCNLP 2019, Hong Kong, November 4, 2019_ , pages 168–176. Association for Computational Linguistics.
  * Bojar et al. (2016) Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno-Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana L. Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin M. Verspoor, and Marcos Zampieri. 2016.  Findings of the 2016 conference on machine translation.  In _Proceedings of the First Conference on Machine Translation, WMT 2016, colocated with ACL 2016, August 11-12, Berlin, Germany_ , pages 131–198.
  * Fujii et al. (2020) Ryo Fujii, Masato Mita, Kaori Abe, Kazuaki Hanawa, Makoto Morishita, Jun Suzuki, and Kentaro Inui. 2020.  PheMT: A phenomenon-wise dataset for machine translation robustness on user-generated contents.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 5929–5943, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Klein et al. (2018) Guillaume Klein, Yoon Kim, Yuntian Deng, Vincent Nguyen, Jean Senellart, and Alexander M. Rush. 2018.  Opennmt: Neural machine translation toolkit.  In _Proceedings of the 13th Conference of the Association for Machine Translation in the Americas, AMTA 2018, Boston, MA, USA, March 17-21, 2018 - Volume 1: Research Papers_ , pages 177–184.
  * Koehn (2004) Philipp Koehn. 2004.  Statistical significance tests for machine translation evaluation.  In _Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing_ , pages 388–395, Barcelona, Spain. Association for Computational Linguistics.
  * Lee et al. (2017) Jason Lee, Kyunghyun Cho, and Thomas Hofmann. 2017.  Fully character-level neural machine translation without explicit segmentation.  _TACL_ , 5:365–378.
  * Lison et al. (2018) Pierre Lison, Jörg Tiedemann, and Milen Kouylekov. 2018.  Opensubtitles2018: Statistical rescoring of sentence alignments in large, noisy parallel corpora.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018._
  * Michel and Neubig (2018a) Paul Michel and Graham Neubig. 2018a.  MTNT: A testbed for machine translation of noisy text.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 543–553, Brussels, Belgium. Association for Computational Linguistics.
  * Michel and Neubig (2018b) Paul Michel and Graham Neubig. 2018b.  MTNT: A testbed for machine translation of noisy text.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018_ , pages 543–553.
  * Neubig et al. (2018) Graham Neubig, Matthias Sperber, Xinyi Wang, Matthieu Felix, Austin Matthews, Sarguna Padmanabhan, Ye Qi, Devendra Singh Sachan, Philip Arthur, Pierre Godard, John Hewitt, Rachid Riad, and Liming Wang. 2018.  XNMT: the extensible neural machine translation toolkit.  In _Proceedings of the 13th Conference of the Association for Machine Translation in the Americas, AMTA 2018, Boston, MA, USA, March 17-21, 2018 - Volume 1: Research Papers_ , pages 185–192.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA_ , pages 311–318. ACL.
  * Popović (2015) Maja Popović. 2015.  chrF: character n-gram F-score for automatic MT evaluation.  In _Proceedings of the Tenth Workshop on Statistical Machine Translation_ , pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.
  * Post (2018) Matt Post. 2018.  A call for clarity in reporting BLEU scores.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 186–191, Belgium, Brussels. Association for Computational Linguistics.
  * Rosales Núñez et al. (2019) José Carlos Rosales Núñez, Djamé Seddah, and Guillaume Wisniewski. 2019.  Comparison between NMT and PBSMT performance for translating noisy user-generated content.  In _Proceedings of the 22nd Nordic Conference on Computational Linguistics_ , pages 2–14, Turku, Finland. Linköping University Electronic Press.
  * Sanguinetti et al. (2020) Manuela Sanguinetti, Lauren Cassidy, Cristina Bosco, Özlem Çetinoglu, Alessandra Teresa Cignarella, Teresa Lynn, Ines Rehbein, Josef Ruppenhofer, Djamé Seddah, and Amir Zeldes. 2020.  Treebanking user-generated content: a UD based overview of guidelines, corpora and unified recommendations.  _CoRR_ , abs/2011.02063.
  * Seddah et al. (2012) Djamé Seddah, Benoît Sagot, Marie Candito, Virginie Mouilleron, and Vanessa Combet. 2012.  The french social media bank: a treebank of noisy user generated content.  In _COLING 2012, 24th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, 8-15 December 2012, Mumbai, India_ , pages 2441–2458.
  * Specia et al. (2020) Lucia Specia, Zhenhao Li, Juan Pino, Vishrav Chaudhary, Francisco Guzmán, Graham Neubig, Nadir Durrani, Yonatan Belinkov, Philipp Koehn, Hassan Sajjad, Paul Michel, and Xian Li. 2020.  Findings of the WMT 2020 shared task on machine translation robustness.  In _Proceedings of the Fifth Conference on Machine Translation_ , pages 76–91, Online. Association for Computational Linguistics.

## Supplementary Materials

➀ | src |  JohnDoe389 (10) qui n’arrive pas a (2) depasser (2) 1 a (2) FlappyBird (10) … ptddddr (12,13)
---|---|---
| ref |  JohnDoe389 who can’t score more than 1 at FlappyBird … lmaooooo
| N. src |  Jean qui n’arrive pas à dépasser 1 à Jean …
| N. ref |  Jean who can’t score more than 1 at Jean…
➁ | src |  #CaMeVénèreQuand (6) le matin a (2) 7h on me parle alors que je suis pas encore réveiller. (5)
| ref |  #ItAnnoysMeWhen in the moring at 7 am someone talks to me although I didn’t wake up yet.
| N. src |  le matin à 7h on me parle alors que je suis pas encore réveillé.
| N. ref |  in the moring at 7 am someone talks to me although I didn’t wake up yet.
➂ | src |  vu sa tete (2) c (3) normal kon (3) est (3) jms (11) parler (5) d’elle !
| ref |  in light of her face it’s normal no one ever spoke about her!
| N. src |  vu sa tête c’est normal qu’on a jamais parlé d’elle !
| N. ref |  in light of her face it’s normal no one ever spoke about her!
➃ | src |  y a ma cousine qui joue a (2) flappy bird (10) mdrrrrrrrrrrr (12, 13) elle et plus nuuul (12,7) que moi
| ref |  my cousin plays flappy bird loooooooooool she’s more hopeless than me
| N. src |  y a ma cousine qui joue à Jean Jean elle et plus nulle que moi
| N. ref |  my cousin plays Jean Jean she’s more hopeless than me
Table A.1: Examples from our annotated noisy UGC corpus. Source sentences have
been annotated with UGC specificities of Table 4 (in blue) according to their
numerical code. For each example, the original source and reference (src and
ref) and their corresponding normalized version (N. src and N. ref) are shown.
Letter deletion/addition/change
---
➀ | src | j’arrive pas à boir normalemen
| norm | j’arrive pas à boire normalement
| ref | I can’t drink normally
| s2s | I can’t drink normal.
| c2c | I can’t drink normally.
| Tx | I can’t drink normal men.
➁ | src | Je conseille à toux ceux qui ont l’esprit disons, un peu fermé de regarder sur les "Français d’origine contrôlée"
| norm | Je conseille à tous ceux qui ont l’esprit disons, un peu fermé de regarder sur les "Français d’origine contrôlée"
| ref | I advise everyone with a, let’s say a little narrow mind to watch about the "Français d’origine contrôlée"
| s2s | I suggest cough those who have minds say, a little closed to look at the "frances of controlled origins"
| c2c | I counsel those who have the mind, a little close to looking at the French original controlled original controlled.
| Tx | I advise anyone with a mind, say, a little closed to look at the controlled French.
➄ | src | le côté suis tro cool au quotidien et je relach tout quan j’ai bu
| norm | les gens qui m’aiment me détestent quand j’ai bu
| ref | my side very cool in everyday life and loosen everything when I’ve been drinking
| s2s | I’ve been drinking all the time and I’ve been drinking everything quan I’ve been drinking.
| c2c | the side of the daily cool side and relacing everything when I’ve been drinking
| Tx | I’m the cool side. I’m the cool one.
Tokenization
➅ | src |  J’sais pas vous, mais de voir la joie des grands joueurs comme Zlatan, Motta, Verratti je trouve ça magnifique
| norm | Je sais pas vous, mais de voir la joie des grands joueurs comme Jean, Jean, Jean je trouve ça magnifique
| ref |  I don’t know about you, but seeing the joy of great players like Zlatan, Motta, Verratti I think it’s wonderful
| s2s |  I don’t know you, but seeing the joy of the great players like Zlatan, Motta, Verratti, I think it’s beautiful.
| c2c |  I don’t know about you, but to see the joy of great players like Zlatan, Motta, Varratt, I think it’s beautiful.
| Tx |  I don’t know about you, but seeing the joy of big players like Zlatan, Motta, Verratti, I think it’s beautiful.
➆ | src | pendant que vous me laissez en chien à l’atelier mon score de flappy bird fait que d augmenter
| norm | pendant que vous me laissez en chien à l’atelier mon score de Jean fait que d’augmenter
| ref | while you’re bailing out on me at the workshop my flappy bird score is just increasing
| s2s | when you leave me as a dog when you leave me as a dog at the workshop.
| c2c | while you leave me dog at the workshop my flappy bird score is that increasing
| Tx | while you leave me as a dog at the workshop my flappy bird score is just up.
➇ | src | il ma dit que c’était normal aussi et que ça allait redescendre,
| norm | il m’a dit que c’était normal aussi et que ça allait redescendre,
| ref |  he told me it was normal too and that it would come down,
| s2s |  He said it was normal, too, and it was going to go down,
| c2c |  He told me it was normal, too, and it was going back,
| Tx |  He told me it was normal, too, and it was gonna come down,
Inconsistent casing
➈ | src |  Jean DANS VOS YEUX | ➉ | src |  JE VIENS DE VOIR Jean ET Jean JE PEUX PLUS
| norm |  Jean dans vos yeux |  | norm | Je viens de voir Jean je peux plus
| ref |  Jean IN YOUR EYES |  | ref | I JUST WATCHED Jean AND Jean CAN’T TAKE IT
| s2s |  Jean D in VOSY |  | s2s | I’m going to kill Jean and Jean I can’t believe it.
| c2c |  Jean in your eyes |  | c2c | I’m here to see Jean And Jean I can no longer.
| Tx |  Jean in your eyes |  | Tx | I just saw Jean and Jean again.
Domain-specific characters and emojis
➊ | src | Avec mes magnifiques jumeaux Jean et Jean @maxcarver @Charlie_Carver $\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\varheartsuit$
| norm | Avec mes magnifiques jumeaux Jean et Jean
| ref | With my wonderful twins Jean and Jean @maxcarver @Charlie_Carver $\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\varheartsuit$
| s2s | with my gorgeous Jean and Jean @maxarver @Carlie_Carver @Carlie_Carver #
| c2c | with my beautiful Jean twins, Jean Jean and Jean Charlier Charlier Carver.
| Tx | with my beautiful twins Jean and Jean imexcarver Charlie_Charver @Charver
Table A.2: Examples from our noisy UGC corpus showing the Transformer,
char2char and Seq2seq predictions. Present UGC specificities of Table 4 (in
blue) are marked in bold.

As some of the data sets contain as few as 40 sentences, we have also computed
the 95% confidence interval for all Bleu scores in Table A.3 using the
bootstrapping method described in Koehn (2004). The width of all intervals is
smaller than 0.30 for the Bleu scores (roughly 1% of the corresponding score)
and than 0.006 for the ratios, which shows that we can trustfully compare
their values.

Similarly, we have included results for the chrF Popović (2015) and Multi-
bleu-detok.perl evaluation metrics since SacreBLEU showed a ratio between
performances on noisy and clean text versions, indicating that the noisy
version could be easier to translate than its normalization. In this regard,
we can see in Table A.3 that at least one of the three reported metrics gives
a ratio value equal or smaller than $1.0$ within the 95% confidence interval
error (CI Err.), suggesting that our normalization introduce limited
artificial noise, comparable to the difference between correlated evaluation
metrics.

| Metric |  char del/add |  missing diacritics |  phonetic writing |  tokenization |  wrong tense |  special char |  agreement |  casing |  emoji |  named entity |  contraction |  repetition |  interjections
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
s2s | MB |  0.78 (25.3) |  0.94 (31.1) |  0.92 (23.2) |  0.95 (30.6) |  0.98 (28.6) |  0.83 (24.8) |  0.95 (25.1) |  0.77 (26.7) |  0.87 (29.6) |  0.87 (30.9) |  0.91 (28.4) |  0.83 (29.9) |  0.90 (26.5)
chrF |  0.93 (46.7) |  0.97 (53.1) |  0.89 (43.1) |  0.95 (50.9) |  0.99 (50.6) |  0.91 (44.3) |  0.98 (49.3) |  0.76 (40.2) |  0.94 (51.1) |  0.94 (51.7) |  0.93 (47.3) |  0.93 (47.3) |  0.96 (48.0)
SB |  0.80 (28.7) |  0.95 (33.9) |  0.93 (27.3) |  0.96 (30.8)) |  0.94 (30.7) |  0.88 (26.1) |  0.95 (27.1) |  0.75 (27.7) |  0.91 (31.0) |  0.86 (31.7) |  0.95 (30.8) |  0.90 (30.2) |  0.93 (29.2)
c2c | MB |  1.00 (29.5) |  1.00 (27.4) |  0.85 (22.5) |  0.99 (29.7) |  0.97 (26.9) |  0.80 (23.5) |  0.97 (25.5) |  0.91 (27.7) |  0.83 (25.1) |  0.95 (31.7) |  0.88 (26.6) |  0.93 (28.0) |  0.91 (25.7)
chrF |  0.99 (48.5) |  1.00 (50.6) |  0.92 (44.8) |  0.95 (50.1) |  0.99 (49.1) |  0.84 (44.0) |  0.98 (49.9) |  0.78 (40.6) |  0.93 (49.5) |  0.95 (51.6) |  0.92 (48.8) |  0.90 (47.8) |  0.95 (49.7)
SB |  0.99 (32.5) |  0.99 (29.6) |  0.86 (25.2) |  1.00 (31.9) |  0.97 (28.8) |  0.81 (24.6) |  0.96 (28.9) |  0.86 (28.0) |  0.83 (26.2) |  0.94 (32.7) |  0.91 (30.4) |  0.95 (26.2) |  0.91 (28.7)
TX | MB |  0.96 (30.3) |  1.01 (33.0) |  0.98 (33.2) |  0.98 (31.5) |  1.01 (31.6) |  0.90 (28.4) |  0.97 (31.4) |  0.98 (25.8) |  0.72 (26.7) |  1.06 (35.7) |  0.90 (28.4) |  0.81 (25.9) |  0.83 (27.0)
chrF |  0.95 (48.2) |  1.00 (52.3) |  0.98 (46.6) |  0.99 (51.0) |  1.01 (52.4) |  0.93 (46.5) |  0.97 (50.9) |  0.80 (30.7) |  0.88 (49.1) |  1.00 (52.6) |  0.93 (48.9) |  0.87 (46.2) |  0.92 (46.2)
SB |  0.98 (35.3) |  1.02 (34.0) |  1.03 (33.2) |  0.98 (32.9) |  1.02 (33.7) |  0.92 (29.2) |  0.97 (33.8) |  0.90 (26.9) |  0.75 (28.3) |  0.99 (35.4) |  0.93 (31.1) |  0.89 (36.8) |  0.86 (30.2)
CI Err. | (E-3) |  4.5 (0.17) |  1.5 (0.13) |  2.7 (0.11) |  2.6 (0.17) |  2.4 (0.15) |  1.8 (0.12) |  1.7 (0.23) |  5.7 (0.30) |  3.0 (0.23) |  2.2 (0.11) |  2.2 (0.16) |  2.5 (0.24) |  3.1 (0.22)
Table A.3: Bleu score ratios between pairs of noisy and normalized sets of
sentences, containing only one UGC specificity. Bleu scores on noisy sets are
shown in parenthesis. Three different metrics are shown for comparison:
MultiBleu-detok.perl (MB) , chrF and SacreBleu (SB). Error for 95% confidence
intervals (CI Err).

$1$$2$$3$$4to7$$0.8$$0.9$Number of present UGC specificities per
sent.Ratios2sc2ctx

Figure A.1: Noisy/Clean Bleu scores’ ratios for accumulated number of UGC
specificities present per sentence for each model, corresponding to the
results in Table 6. The 4to7 bin groups more than 4 types to provide a larger
subcorpus, which weighted average is 4.34 UGC specificities per sentence.

##  Appendix A Reproducibility

#### Data

All the UGC test sets and source code for our experiments are provided in the
supplementary materials. For training data, we let the reader refer to each
project’s website for WMT777https://www.statmt.org/wmt15/translation-task.html
(consisting of Europarlv7, Newcommentariesv10 and Open
Subtitles888http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/moses/en-
fr.txt.zip, both accessed on November, 2019. Regarding clean test sets, we
used newstest15 from WMT and a subset of 11,000 unique phrases from Open
Subtitles. We make the former test available in the link provided above for
exact performance comparison.

#### Computation

The NMT systems were trained using 1 Tesla V100, during an average of 72 hours
to converge to the final solution for the char2char model and 56 hours for the
BPE-based baselines.

###  A.1 NMT Models

#### Character-based models

char2char models were trained as out-of-the box systems using the
implementations provided by Lee et al. (2017) .999https://github.com/nyu-
dl/dl4mt-c2c

#### BPE-based models

We consider, as baseline, two standard NMT models that take, as input,
tokenized sentences. The first one is a seq2seq bi-LSTM architecture with
global attention decoding. The seq2seq model was trained using the XNMT
toolkit Neubig et al. (2018).101010We decided to use XNMT, instead of OpenNMT
in our experiments in order to compare our results to the ones of Michel and
Neubig (2018b). It consists in a 2-layered Bi-LSTM layers encoder and
2-layered Bi-LSTM decoder. It considers, as input, word embeddings of 512
components and each LSTM units has 1,024 components.

We also study a vanilla Transformer model using the implementation proposed in
the OpenNMT framework Klein et al. (2018). It consists of 6 layers with word
embeddings of 512 components, a feed-forward layers made of 2,048 units and 8
self-attention heads.

#### Hyper-parameters

In Table A.4, we list the training variables set for our experiments. They
match their corresponding default hyper-parameters.

Batch size | 64
---|---
Optimizer | Adam
Learning rate | 1e-4
Epochs | 10 (best of)
Patience | 2 epochs
Gradient clip | [-1.0, 1.0]
Table A.4: Hyper-parameters used for training the NMT systems.

#### Pre-processing

For the BPE models, we used a 16K merging operations tokenization employing
sentencepiece111111https://github.com/google/sentencepiece. For word-level
statistics we segmented the corpora using the Moses tokenizer
121212https://github.com/moses-smt/mosesdecoder.


# Advanced Analysis of Radar Cross-Section Measurements in Reverberation
Environment

C. Charlo, S. Méric, F. Sarrazin, E. Richalot, J. Sol and P. Besnier This work
is supported by AID/DGA, France. It is also supported in part by the European
Union through the European Regional Development Fund, in part by the Ministry
of Higher Education and Research, in part by the Région Bretagne, and in part
by the Département d’Ille et Vilaine through the CPER Project SOPHIE/STIC &
Ondes.Corentin Charlo, Stéphane Méric, Jérôme Sol and Philippe Besnier are
with Univ Rennes, INSA Rennes, CNRS, IETR-UMR 6164, F-35000 Rennes, France
(e-mail: {ccharlo, smeric, jsol<EMAIL_ADDRESS>Sarrazin is
with Univ Rennes, CNRS, IETR-UMR 6164, F-35000 Rennes, France
(e-mail:francois.sarrazin@univ-rennes.fr).Elodie Richalot is with Univ Gustave
Eiffel, CNRS, ESYCOM, F-77454 Marne-la-Vallée, France (e-mail:
elodie.richalot-taisne@univ-eiffel.fr).

###### Abstract

Reverberation chambers (RCs) were recently reported as a low-cost alternative
to anechoic chambers (ACs) to perform radar cross-section (RCS) pattern
measurements. The method consists of using transmitting and receiving antennas
pointing towards a target under test placed on a rotating mast. As a classical
RCS characterization, the echo signal is analysed based on two measurements
with and without the target in the RC. In the hypothesis of an ideal diffuse
field generated in the RC, this signal difference appears as the echo signal
hidden in a Gaussian noise. In case of a point-like backscattering target,
observing this signal over a given frequency bandwidth allows the
identification of the target response as a sinusoidal signal over this
bandwidth whose period is related to the antenna-target distance measured from
the measurement calibration plane positions. Therefore, the extraction of the
magnitude of this sinusoidal signal requires a proper estimation of this
distance. Furthermore, a sinusoidal regression processing relies on the
approximation of a constant envelope over the selected frequency bandwidth,
imposing some restrictions. In this paper, we introduce a two-step method that
consists in identifying the most appropriate distance according to the
target’s orientation before estimating the magnitude of the sinusoidal signal.
We highlight improvement of RCS estimation on a point-like back-scattering
target compared to the one-step procedure applied so far. In addition, it is
shown that the analysis performed regarding the estimated distance provides a
physical insight into the position of the equivalent backscattering point.

###### Index Terms:

Radar Measurements, Quasi-monostatic Radar, Radar Cross-Section, Reverberation
Chamber

##  I Introduction

Radar cross-section (RCS) is a well-known physical parameter that quantifies
the scattering behavior of an object when illuminated by an electromagnetic
wave. Its estimation is of high interest for military applications (e.g.
stealth detection) and civil applications (e.g. automotive or air traffic
control radars). RCS measurement requires the use of anechoic chambers (ACs)
in order to retrieve the line-of-sight contribution between the measurement
antenna and the target while minimizing the contributions of all the other
propagation paths. However, such measurement environments are costly and not
widely available outside some academic or military research laboratories.
Therefore, alternative approaches are of interest in order to facilitate RCS
measurement.

Reverberation chambers (RCs) have been successfully used to achieve various
types of electromagnetic characterization as detailed abundantly in the
literature [1, 2, 3, 4]. An RC can be seen as an oversized electromagnetic
cavity allowing the superposition of numerous modes with random complex
weights according to the variation of boundary conditions. Mechanical,
frequency or source stirrings (or a combination of them) yield to a
statistically uniform and isotropic field. This test environment is of high
interest for electromagnetic immunity tests since it provides high-field
strength and does not require any movement of the equipment under test. Beyond
electromagnetic compatibility applications, RCs are now used in many other
domains as antenna characterization [5, 6, 7, 8, 9], over-the-air testing of
wireless devices [10, 11, 12], or exposure system for bio-electromagnetic
studies [13, 14, 15].

Although well suited to estimate average absorption quantities, some papers
recently introduced novel approaches to estimate RCS patterns within RCs. To
that purpose, different methods have been developed in order to extract the
line-of-sight backscattered contribution of the target from the diffuse field
provided by the RC. In [16], the $K$-factor has been used in order to extract
the line-of-sight signal between the transmitting and the receiving antennas
from the scattered component associated to reflections and diffractions on RC
walls. This direct path component has then been used to extract the RCS of a
target [17]. This method is relevant to estimate high K-factor and therefore
high RCS magnitude but the RCS dynamic range is limited [18]. Another approach
consists in moving the target inside the RC in the direction of the measuring
antenna, so that the backscatterred signal from the target can be associated
to a virtually recreated Doppler effect [19]. It partially solves the problem
of the $K$-factor method but requires a linear rail to alternately move the
target closer and further away in the direction of the measuring antenna and
for different viewing angles. A time-gating method has been proposed in [20]
which is similar to the current method used in an anechoic chamber and,
therefore, not specific to RCs. In [21], some of the current co-authors have
introduced a monostatic measurement principle based on a sinusoidal regression
applied on the difference of the complex-valued frequency signals recorded in
the empty chamber (calibration step) and the target-loaded chamber
(measurement step). The theory has been further improved accounting for the
mechanical stirrer effect and contribution in [22] and extended to quasi-
monostatic configurations in [23].

The current method is based on a sinusoidal regression process whose maximum
amplitude versus the considered signal period leads to the target RCS. The
regression is performed on a selected frequency bandwidth around the central
frequency, and it has been demonstrated in [22] that the larger the bandwidth,
the higher the accuracy of the regression process itself. However, it is shown
in this paper that a large frequency bandwidth degrades the accuracy of the
retrieved RCS pattern by smoothing out the side lobes. Therefore, a two-step
signal processing is introduced to identify the appropriate antenna-target
distance on the one hand, and to improve the RCS amplitude estimation over all
azimuth angles of the target, on the other hand.

This manuscript is organised as follows. Section II briefly recalls the main
hypotheses and the principle of RCS measurement in RCs as detailed in [22].
Section III describes the measurement environment, the target under study and
also the data acquisition process. These measurement data are recorded in a
quasi-monostatic configuration, at once with both antennas in horizontal (HH)
and vertical (VV) polarizations. Then, section IV is dedicated to the impact
of the frequency bandwidth selection on the estimation of the distance between
the target and the antennas, whereas section V studies its impact on the RCS
estimation itself. The two-step signal processing proposed in section VI is a
direct consequence of the analysis in the two previous sections. This advanced
method is applied to the measured data in section VII highligthing a major
improvement of RCS results if appropriate bandwidths are selected,
specifically for local minima estimation of the RCS pattern. Conclusions are
drawn in Section VIII.

##  II Theory of RCS measurement in RCs

In the following theoretical developments, the RC is supposed to be well
oversized (with respect to the wavelength), so that Hill’s hypothesis of an
ideal random (diffuse) field can be assumed [24]. In practice, goodness-of-fit
test provides a weak rejection of the null-hypothesis for a Gaussian field
distribution. Furthermore, the target is assumed to behave as an equivalent
scattering point-source. The RCS pattern of such a target is evaluated from
the difference of the scattering parameters measured by two fixed antennas in
the RC without the target (thereafter called ”empty room”) and the RC loaded
by the target for each azimuth position of the latter. The theory of RCS
measurement within an RC is detailed in monostatic configuration in [21, 22]
and has been extended to the quasi-monostatic configuration in [23]. The
difference $\Delta S(f_{0})$ of the transmission parameters measured within
the charged room $S_{xy}^{\mathrm{T}}(f_{0})$ and the empty room
$S_{xy}(f_{0})$ writes for a given azimuth position of the target:

$\Delta S(f_{0})=S_{xy}^{\mathrm{T}}(f_{0})-S_{xy}(f_{0})=\\\
B(f_{0})+A(f_{0})\times\exp\left(-\mathrm{j}\frac{2\pi f_{0}}{\delta
f}\right)\times\exp(\mathrm{j}\phi_{0})$ (1)

with

$B(f_{0})=\sqrt{m_{x}(f_{0})m_{y}(f_{0})}\times\sqrt{\eta_{x}(f_{0})\eta_{y}(f_{0})}\\\
\times\left(H^{\mathrm{T}}(f_{0})-H(f_{0})\right)$ (2)
$A(f_{0})=\sqrt{\sigma^{\mathrm{T}}(f_{0})}\frac{c\sqrt{m_{x}(f_{0})m_{y}(f_{0})G_{x}(f_{0})G_{y}(f_{0})}}{(4\pi)^{3/2}R^{\prime
2}f_{0}}$ (3)

The VNA ports and related antennas are indicated by the $x$ and $y$ indexes,
and the presence of the target by the superscript $\mathrm{T}$. The difference
$\Delta S(f_{0})$ consists of the addition of two distinct terms. The first
term $B(f_{0})$ on the right-hand side of (1) is related to the RC diffuse
field. It is itself composed of three factors as shown in (2). The first two
ones involve the free-space (FS) antenna mismatching coefficients
$m_{i}(f_{0})=1-\lvert S_{ii\mathrm{FS}}(f_{0})\rvert^{2}$, and the antenna
radiation efficiencies $\eta_{i}(f_{0})$ with $i=\\{x,y\\}$, respectively. The
last factor is proportional to the difference of the diffuse RC transfer
functions with the target $(H^{\mathrm{T}}(f_{0}))$ and without the target
$(H(f_{0}))$. The second term $A(f_{0})\times\exp\left(-\mathrm{j}\frac{2\pi
f_{0}}{\delta f}\right)\times\exp(\mathrm{j}\phi_{0})$ corresponds to the
radar echo which is proportional to the square root of the RCS of the target
$\sigma^{\mathrm{T}}(f_{0})$, $A(f_{0})$ being a real-valued parameter. The
terms $G_{x}(f_{0})$ and $G_{y}(f_{0})$ in (3) are the gains of the $x$ and
$y$ antennas, respectively. The term $R^{\prime}$ is the physical distance
between the target and the antennas. In the frequency domain, this delayed
echo can be approximated as a sinusoidal function around the central frequency
$f_{0}$, whose periodicity $\delta f$ depends on the electrical distance $R$
with respect to the relation $\delta f=c/(2R)$. Finally, $\phi_{0}$ is
considered as a constant phase.

In our case, we further assume that the RC electromagnetic field statistical
properties are not modified by the target. As a result, both variables
$H^{\mathrm{T}}(f_{0})$ and $H(f_{0})$ are distributed according to a centered
Gaussian probability density function with the same variance. Therefore, the
sinusoidal waveform is embedded in a Gaussian noise once the frequency
variations of the antenna gains and mismatch factors are compensated for as
described in (4).

$\Delta S^{\mathrm{N}}(f_{0})=\Delta S(f_{0})\frac{(4\pi)^{3/2}R^{\prime
2}f_{0}}{c\sqrt{m_{x}(f_{0})m_{y}(f_{0})G_{x}(f_{0})G_{y}(f_{0})}}$ (4)

that can be decomposed as:

$\Delta
S^{\mathrm{N}}(f_{0})=D(f_{0})+\sqrt{\sigma^{\mathrm{T}}(f_{0})}\times\exp\left(-\mathrm{j}\frac{2\pi
f_{0}}{\delta f}\right)\times\exp(\mathrm{j}\phi_{0})$ (5)

Through scattering parameters measurements over a certain bandwidth centered
at $f_{0}$, applying a sinusoidal regression procedure on the difference
$\Delta S^{\mathrm{N}}(f_{0})$ permits to extract the magnitude of the target
RCS $\sigma^{\mathrm{T}}(f_{0})$. The magnitude of the sinusoidal wave is
valued from the average of both amplitudes extracted from the real and the
imaginary parts of $\Delta S^{\mathrm{N}}(f_{0})$. However, before estimating
the sine component amplitude, we need to determine the distance $R$ that
imposes its frequency period. As illustrated in Fig. 1b, this distance $R$ can
be seen as the distance $R^{\prime}$ between the target and the antenna
apertures plus the electrical distance $\delta R$ between the antenna
apertures and the setup calibration planes. In the earlier publications [21,
22], the amplitude $A(f_{0})$ was estimated by fixing a distance $R$ whatever
the target orientation. Nevertheless, in [23], we highlighted that this
distance varies according to the target orientation and we also introduced a
method to estimate the distance between the target and the antennas by
measuring the target response on a very large bandwidth. Nevertheless, using a
very large frequency bandwidth is not suitable for the RCS estimation as
further highlighted in the following section.

##  III Measurement Setup and Data Acquisition

###  III-A Measurement Setup

Measurements are made in the largest RC of the IETR. The dimensions of the
chamber are $2.9$ m $\times$ $3.7$ m $\times$ $8.7$ m. The RC has a lowest
usable frequency of $0.25$ GHz and behaves as an ideal RC above $0.8$ GHz if
empty. It means that a goodness-of-fit test provides a weak rejection of the
null-hypothesis for a Gaussian field above $0.8$ GHz. The RCS measurement is
performed using two identical horn antennas (ETS-Lindgren model 3115) oriented
towards the target and at a close distance from each other regarding the
target point-of-view. Antennas are used in two different polarizations, the
horizontal polarization (HH) and the vertical polarization (VV). It has been
verified that the coupling between the two measurement antennas due to the
close distance between them does not affect the RCS measurement result [23]. A
rotating mast supporting the target is placed at the distance $R^{\prime}=3$ m
from both antennas (Fig. 1a).

(a) Picture of the measurement set up in HH polarization in the RC

(b) Schematic description with distance definitions

Figure 1: Picture in HH polarization (a) and schematic description with
distance definitions (b) of the test setup with a metallic plate of
$0.152\times 0.150$ m2 in the RC.

.

The orientation of the antenna-to-mast axis is chosen arbitrarily but not
perpendicular to any cavity wall to avoid any spurious echo from the back wall
behind the target.

###  III-B Target under study and theoretical response

The selected target is a rectangular metallic plate of length $l=0.152$ m and
height $h=0.150$ m. Its theoretical RCS behavior is well-known (e.g. using
physicalcal optic approximation [25, 26] or geometrical diffraction theory
[26]). Thus, discarding the Gaussian noise of the RC, the theoretical response
exhibits a wide RCS dynamic range over the azimuth plane. According to the
physical optics approximation [25], the monostatic target RCS
$\sigma^{\mathrm{T,Th}}(f_{0},\theta)$ in HH configuration as a function of
the plate surface $lh$, the frequency $f_{0}$ and the azimuth position
$\theta$, is given by:

$\sigma^{\mathrm{T,Th}}(f_{0},\theta)=\frac{4\pi(lh)^{2}f_{0}^{2}}{c^{2}}\cos^{2}(\theta)\operatorname{sinc}^{2}\left(\frac{2\pi
l\sin(\theta)f_{0}}{c}\right)$ (6)

As a consequence, the theoretical normalized $\Delta S^{\mathrm{NTh}}(f_{0})$
expression for this plate discarding the Gaussian field in the RC
($B(f_{0})=0$) writes:

$\Delta
S^{\mathrm{NTh}}(f_{0})\approx\sqrt{\sigma^{\mathrm{T,Th}}(f_{0},\theta)}\times\exp{\left(\frac{-\mathrm{j}2\pi\times
f_{0}}{\delta f}\right)}$ (7)

The term $\Delta S^{\mathrm{NTh}}(f_{0})$ phase shift variation versus
frequency is written as a function of $\delta f=c/2R$ as described in (1).
Fig. 2 shows the real part of (7) over a bandwidth of $4$ GHz centered around
$f_{0}=10$ GHz for $R=3.3$ m and for two angular positions of
$\theta=0^{\circ}$ and $\theta=35^{\circ}$.

(a) $\theta=0^{\circ}$

(b) $\theta=+35^{\circ}$

Figure 2: Real part of the theoretical noiseless difference of S-parameters
($\Delta S^{\mathrm{NTh}}(f_{0})$) in absence and presence of a metallic
rectangular plate at (a) $\theta=0^{\circ}$ and (b) $\theta=35^{\circ}$.

For $\theta=0^{\circ}$ (Fig. 2a), the amplitude of the sinusoidal wave raises
up linearly with frequency. Due to this smooth variation, the sinusoidal
fitting in a least square sense would lead to similar amplitudes whatever the
sub-bandwidth selected for this fitting. On the contrary, in Fig. 2b
($\theta=35^{\circ}$), the amplitude of the noiseless sinusoidal oscillations
strongly varies over the frequency band and even vanishes at specific
frequencies. Therefore, the amplitude extracted from the sinusoidal fit
strongly depends on this frequency bandwidth. The latter must be restricted so
that $\Delta S^{\mathrm{NTh}}(f_{0})$ amplitude remains approximately constant
to ensure a correct RCS estimation at angles for which the RCS variation is
important versus frequency. This paper aims at solving this issue of the
suited frequency band selection over a large angle range as described in the
next sections.

###  III-C Data acquisition

Both antennas are connected to a vector network analyzer (VNA). To respect the
far field Fraunhofer distance, the distance $R^{\prime}$ between the target
under test and the antennas must be greater than $2D^{2}/\lambda$ where $D$ is
the largest dimension of the target and $\lambda$ is the minimum wavelength in
the measurement bandwidth. As measurements are carried out from $8$ GHz to
$12$ GHz, $R^{\prime}$ is fixed at $3$ m and respects this limit. This
condition is supposed to be sufficient to assume the ballistic wave being a
locally-plane wave at the target position. A $1$ kHz IF filter and a frequency
step of $100$ kHz are used ($40001$ frequency points in total). Indeed, the RC
correlation bandwidth has been experimentally estimated to roughly $100$ kHz
at $10$ GHz, hence the chosen frequency step. The RC mechanical stirrer is
maintained in the same position throughout all experiments (i.e. no mechanical
stirring is used throughout this paper). Firstly, we carry out S-parameter
measurement of the empty RC $(S_{xy}(f_{0}))$. Secondly, we add the target on
the mast in the RC and we perform the measurement
$(S^{\mathrm{T}}_{xy}(f_{0}))$ for azimuth angles between
$\theta=-100^{\circ}$ and $\theta=+100^{\circ}$ with a $1^{\circ}$ rotation
step.

##  IV Distance estimation

This section is dedicated to the accurate estimation of the distance between
the target and the antennas. Using the algorithm described in [23], the
estimated distance is taken as the one leading to the maximum extracted sine
wave amplitude over a distance range encompassing the total volume of the
target. In our study, the distance (or the corresponding period of the
sinusoidal fit) is varied between $3.25$ m and $3.50$ m by a step of $0.001$
m. As the distance between both antennas is much smaller, the coupling between
the two antennas has no effect on the distance estimation of the target. Thus,
at each azimuth angle, the distance of the maximal amplitude is recorded as
the optimal one. In the following, the effect of the frequency bandwidth used
for the amplitude extraction is studied, comparing the optimal distance
estimation for frequency windows of width varying from $\Delta f=0.5$ GHz to
$\Delta f=4$ GHz, all centered at $f_{0}=10$ GHz. This study is performed for
both HH and VV polarizations.

###  IV-A HH polarization

Fig. 3 exhibits the estimated distance $R$ over the azimuth angles $\theta$
for different frequency bandwidths ($\Delta f=\\{0.5,1,2,4\\}$ GHz) in the HH
configuration along with the theoretical positions of the plate leading and
trailing edges.

Figure 3: Distance evolution corresponding to the maximal amplitude of the
sinusoidal regression (performed in the $3.25$ m to $3.50$ m range) as a
function of the azimuth position $\theta$ and for different considered
frequency windows in the HH configuration.

Clear conclusions can be drawn. The distance estimated using the largest
bandwidth ($4$ GHz) leads to the smoothest variation and the most physically
coherent result. Indeed, the extracted distance is seen to coincide with
either the trailing edge or, at some particular angles, the leading edge of
the rotating metallic plate, that is consistent with the possible contribution
of the varying current distribution on the plate. On the contrary, the
distance is poorly estimated for the frequency window of $0.5$ GHz, exceeding
the limits prescribed by the positions of the target leading and trailing
edges. The two other frequency bandwidths ($1$ GHz and $2$ GHz) lead to
intermediate estimations with less non-physical results than for $0.5$ GHz but
not as accurate as for $4$ GHz. This is explained by the fact that the
distance resolution $\delta d$ depends on the bandwidth $BW$ such as $\delta
d=c/(2\times BW)$. Therefore, the greater the bandwidth, the finer the
resolution. Moreover, we can notice that the trailing edge is the main
scattering part of the plate for $\lvert\theta\rvert\geq 10^{\circ}$, whereas
the leading edge is dominant for $\lvert\theta\rvert<10^{\circ}$. The main
scattering point follows the theoretical position of the leading edge or the
trailing edge. The abrupt change of distance between
$\theta=-90^{\mathrm{\circ}}$ and $\theta=-80^{\circ}$ is about $0.146$ m. As
expected, it is very close to the length $l$ of the plate ($0.152$ m).

These observations are further illustrated focusing on specific angles
$\theta$ that correspond to local RCS minimum and maximum according to (7).
The amplitude of the sinusoidal regression fitting over the $3.25$ m to $3.50$
m distance range of HH polarization measurement results is plotted in Fig. 4
for a local RCS maximum ($\theta=+8^{\circ}$) and for a local RCS minimum
($\theta=+12^{\circ}$).

Figure 4: Amplitude estimated by the sinusoidal regression fitting in the
$3.25$ to $3.5$ m distance range for $0.5$ GHz and $4$ GHz frequency windows
and two different azimuth angles of the metallic plate in the HH measurement
configuration. At $\theta=+12^{\mathrm{\circ}}$, the position of the two main
scattering contributors, i.e. the leading edge and the trailing edge, are
apparent.

Considering the local maximum RCS located at $\theta=+8^{\circ}$, Fig. 4 shows
that the distance is much more precisely assessed since the frequency
bandwidth is large. Then we are able to precisely localize the target area
which mainly contributes to the backscattered echo. Around this local RCS
maximum contribution, the distance is correctly identified with the largest
bandwidth. On the contrary, the conclusion is different for the local minimum
RCS at $\theta=+12^{\circ}$. As shown in Fig. 4, two main scattering
contributions of the plate are obviously visible located at
$\theta=+12^{\circ}$ for the frequency bandwidth of $4$ GHz and that is not
the case for the frequency bandwidth of $0.5$ GHz. For this latter, the
estimated distance is the mean distance between the two edges and the
antennas. For $\theta=+12^{\circ}$, the target may be represented by the
contribution of two scattering zones. The two maxima obtained with a bandwidth
of $4$ GHz are located at $3.345$ m and $3.391$ m and are therefore $0.046$ m
apart. The theoretical distance of the leading edge at the same angle is
$3.352$ m [23]. This is close to the estimated distance. As far as the
trailing edge is concerned, its theoretical distance is $3.383$ m. Once again,
it corresponds approximately to the second maximum of the curve. If we now
consider paths associated to both estimated distances (corresponding to the
path from the antenna to one target edge then back to the receiving antenna),
the difference between the phase shifts of the traveling waves is of
$\Delta\phi=\frac{2\pi\times 0.046}{\lambda}\approx 3.1\pi$. As a consequence,
both contributions are approximately out of phase, as expected for a RCS
minimum.

Fig. 5 presents the fluctuating amplitude provided by the sinusoidal
regression fit at each distance and each azimuth position $\theta$ with the
largest frequency bandwidth ($4$ GHz).

Figure 5: Amplitude of the sinusoidal regression fit in the $3.25$ to $3.50$ m
range as a function of the azimuth angle
($-90^{\circ}\leq\theta\leq+90^{\circ}$) in the HH polarization considering
the largest frequency bandwidth ($4$ GHz). The amplitude is normalized to its
maximal value at each angle.

For each azimuth position $\theta$, that corresponds to the variation of the
amplitude estimated by the sinusoidal regression fit at a fixed azimuth
position for a distance varying from $3.25$ to $3.50$ m as in Fig. 4. The
amplitude is normalized by its maximum value at each angle enabling a clear
visualization of the distance evolution over all azimuth positions of the
target. The cartography exhibited in Fig. 5 highlights the location of the
highest amplitude (equal to 1) of the sinusoidal regression for each value of
$\theta$. The trailing edge position has a significant contribution for all
angles whereas the leading edge contributes only at specific angles. Regarding
the position of the regression maximal amplitude, a similar pattern to that of
Fig. 3 is obtained.

###  IV-B VV polarization

The same analysis is carried out in the VV configuration. The distance
evolution of the maximal amplitude issued from the sinusoidal fit is shown for
different frequency bandwidths ($\Delta f=\\{0.5,1,2,4\\}$ GHz) in Fig. 6.

Figure 6: Distance evolution corresponding to the maximal amplitude of the
sinusoidal regression (performed in the $3.25$ m to $3.50$ m range) as a
function of the azimuth position $\theta$ and for different considered
frequency windows in the VV configuration.

On the one hand, the VV configuration differs from the HH one since the
leading edge constitutes the major contributor to the backscattering wave. On
the other hand, like in the HH measurement configuration, only the large
frequency bandwidths make possible to retrieve the theoretical distance of the
leading edge. As in the HH polarization, the difference between the distances
estimated at $\theta=0^{\circ}$ and $\theta=+90^{\circ}$ is about $7.5$ cm,
that corresponds to the half-plate length. Besides, it remains possible to
identify the position of the two edges from the amplitude evolution of the
sinusoidal fit as a function of distance. This is clearly visible in Fig. 7
for the azimuth position of $\theta=+12^{\circ}$.

Figure 7: Amplitude estimated by the sinusoidal regression fit in the $3.25$ m
to $3.5$ m distance range for $0.5$ GHz and $4$ GHz frequency windows in the
VV configuration when the plate azimuth angle is of $12^{\circ}$. At that
position, we can see the position of the main scattering contributions i.e.
the predominant leading edge on the left and the less contributing trailing
edge on the right.

The two edges are indeed identified like in the HH configuration by the two
amplitude maxima. However, the two edges are not associated to the same
amplitude in the VV polarization, the leading edge is predominant. Fig. 8 is
equivalent to Fig. 5 in the VV configuration and provides a normalized
cartography of the fluctuating amplitude of the sinusoidal regression fit in
the same conditions. The leading edge is predominant and the trailing one is
barely visible.

Figure 8: Amplitude of the sinusoidal regression fit in the $3.25-3.50$ m
range as a function of the azimuth angle
($-90^{\circ}\leq\theta\leq+90^{\circ}$) in the VV polarization considering
the largest frequency bandwidth ($4$ GHz). The amplitude is normalized to its
maximal value at each angle.

###  IV-C Summary and milestone conclusion

The target RCS amplitude is provided through a sinusoidal regression fitting
applied on (5). As the regression fitting applies for a sine wave whose period
depends on the distance between the antennas and the major backscattering
contributor, the accurate estimation of this distance $R$ is a prerequisite
for the signal amplitude extraction. We have shown here that it is not
possible to estimate correctly the distance with a small frequency bandwidth
and that the best distance estimation is obtained with a frequency bandwidth
of $4$ GHz whatever the polarization configuration. Therefore, from now on,
the largest frequency bandwidth of $4$ GHz is systematically used to estimate
the distance between the target and the antennas.

Figure 9: Comparison of the estimated distances corresponding to the maximal
amplitude of the sinusoidal regression fit on a bandwidth of $4$ GHz as a
function of the target azimuth angle $\theta$ varying from $-90^{\circ}$ to
$+90^{\circ}$ for the two different measurement configurations.

In Fig. 9, we exhibit the distance estimated in HH and VV configurations using
the $4$ GHz bandwidth and illustrate the position of the main scattering point
in each polarization for different target azimuth angles. The distances
retrieved in both configurations are similar for $\theta$ around $0^{\circ}$
and $\pm 90^{\circ}$. Apart from these angular ranges, the major contributor
is the leading edge for the HH polarisation and is the trailing edge for the
VV polarisation.

##  V Impact of the frequency bandwidth on RCS estimation

In the previous section, the selection of the largest frequency bandwidth has
been shown to be the better choice to accurately estimate the antenna-target
distance, i.e. the periodicity of the sinusoidal waveform to be fitted. This
section is dedicated to the frequency bandwidth analysis for estimating the
RCS amplitude itself, once the scattering point distance is known. First, we
extract the RCS by using the same frequency bandwidth of $4$ GHz. In Fig. 10,
we exhibit a comparison between the RCS extracted from measurements in RC
using a $4$ GHz bandwidth for distance and amplitude extractions and a
measurement of the RCS in an AC.

Figure 10: RCS evolution as a function of $\theta$ in the range from
$-90^{\circ}$ to $+90^{\circ}$: comparison between the RCS extracted from RC
measurement using a $4$ GHz window for the distance estimation as well as the
amplitude fitting and the RCS measured in an AC.

Although there is a good agreement between both estimations around the RCS
main lobe, the RCS extracted from RC measurements is smoothed and fails to
discriminate the local minima and maxima. Consequently, the use of a large
bandwidth is not efficient to estimate the RCS pattern. Based on Fig. 2, this
result could be anticipated as the RCS amplitude $\sigma(f_{0},\theta)$
fluctuates versus frequency in a non-monotonic way. Given $\theta$ and
$f_{0}$, the lower the frequency bandwidth, the smaller the fluctuations of
the sine wave amplitude. Consequently, a better accuracy of the RCS amplitude
is expected. To quantify the frequency variation of the target RCS according
to the chosen frequency bandwidth ($BW$), we use the analytical physical
optics (PO) expression of the plate RCS and compute the RCS excursion over
$BW=\\{0.1,0.5,4\\}$ GHz at $f_{0}=10$ GHz as follows:

$\centering\Delta\sigma(f_{0},\theta)=\frac{\underset{df}{\text{max}}(\sigma(f,\theta))-\underset{df}{\text{min}}(\sigma(f,\theta))}{\underset{df}{\text{max}}(\sigma(f,\theta))+\underset{df}{\text{min}}(\sigma(f,\theta))}\@add@centering$
(8)

with $df=[(f_{0}-\frac{BW}{2});(f_{0}+\frac{BW}{2})]$.

In Fig. 11, the plate RCS is plotted in blue at the central frequency
$f_{0}=10$ GHz and for $0^{\circ}\leq\theta\leq 25^{\circ}$.

Figure 11: Theoretical RCS (PO formulation) and the relative error evolution
w.r.t. the azimuth orientation $\theta$ in the range from $0^{\circ}$ to
$25^{\circ}$ and according to different frequency bandwidths.

This figure highlights the increase of RCS fluctuations according to (8) for
increasing bandwidths. This RCS excursion is bounded between $0$ and $1$. The
relative min/max difference is especially important at angles of RCS local
minima where RCS frequency derivative is high. Thus, the RCS is better
estimated for a small frequency bandwidth. However, as described in [22], the
number of uncorrelated realizations must be high enough for a correct
estimation of the RCS amplitude. A trade-off must be found to ensure both low
RCS fluctuation and low amplitude estimation uncertainty. Therefore, the two-
step method presented in the next section is introduced to solve this issue.

##  VI The two-step method

In the previous sections, we have shown that the most suited frequency
bandwidth choice is different for the estimation of the distance between the
target and the antennas and the extraction of the backscattering signal
amplitude leading to the RCS value. A two-step approach is thus proposed to
account for these different requirements. It consists of:

  1. 1.

Estimating the distance (i.e., the periodicity of the sinusoidal waveform to
be fitted). This step is performed using data over the whole measurement
bandwidth, i.e. $4$ GHz in our case as determined in section IV. It is called
the distance bandwidth in the rest of this paper.

  2. 2.

Extracting the sine-wave amplitude through a regression fit, while restricting
the processed data to a smaller bandwidth, called the amplitude bandwidth in
the rest of this paper.

The two different post-processing methods used in RCs are summarized in the
Fig 12.

Figure 12: Flow-chart of the one-step method (left) and of the two-step method
(right) in RC.

The reduction of the amplitude bandwidth aims at limiting the variation of the
target RCS highlighted in Section V. Moreover, a sufficient number of data is
necessary in the fitting step for an accurate amplitude extraction. To
determine the amplitude bandwidth leading to the most accurate RCS extraction,
the antenna-target distance is firstly estimated using a $4$ GHz bandwidth;
then, the RCS of the target is extracted for various frequency bandwidths
ranging from $0.05$ GHz to $4$ GHz in order to find the best trade-off. Its
impact on the amplitude estimation is observed through the relative difference
between the RCS values obtained from RC and AC measurements.

###  VI-A HH Polarization

First RC measurements are performed with both horn antennas in HH polarization
and the extracted RCS values are studied for various amplitude bandwidths.
Fig. 13 shows the evolution of the difference between RCS values obtained from
RC and AC measurements according to the amplitude bandwidth chosen to extract
the sine wave amplitude. This average relative difference $\Delta$ is computed
as follows:

$\centering\Delta=\left<\frac{\left|\sigma_{\mathrm{AC}}-\sigma_{\mathrm{RC}}\right|}{\sigma_{\mathrm{AC}}}\right>\@add@centering$
(9)

Figure 13: Relative difference between extracted RCS values from RC and AC
measurements in the HH and VV polarizations versus the frequency bandwidth
used for the sine-wave amplitude extraction. Amplitude bandwidths ranging from
0.05 to 4 GHz are considered.

Fig. 13 highlights that the relative difference evolution is not monotonic as
it tends to decrease for amplitude bandwidths from 0.05 GHz to about 0.9 GHz
and then to slightly increase for amplitude bandwidths from 0.9 GHz up to 4
GHz. The decrease at the lowest bandwidth range is explained by a too low
number of considered noisy data during the regression process. As already
shown in [22], a sufficient number of uncorrelated realizations is necessary
to extract accurately the amplitude, leading to a large enough frequency
bandwidth. After this decrease, the slope of the curve changes and the
relative difference increases with the frequency bandwidth. This behaviour is
linked to the frequency variation of the RCS and thus of the sine wave
amplitude. From Fig. 13, the best choice for the amplitude bandwidth is
situated in the interval from $0.5$ GHz to $1.5$ GHz. It has to be underlined
that this indicated interval is specific to the RC in which the measurements
are carried out and to the RCS of the target used in this measurement.
However, a rectangular metallic plate exhibits large frequency fluctuations
and can be considered as a limit case usable for the definition of the two-
step method and particularly of the upper frequency value of this interval.
The lowest frequency value strongly depends on the number of independent
realizations of that RC or, in other words, on the RC correlation bandwidth.
Here the $0.5$ GHz bandwidth corresponds approximately to a set of $5000$
independent realizations.

###  VI-B VV Polarization

In this section, the same study is performed with both horn antennas in the VV
polarization. The evolution of the relative error according to the amplitude
bandwidth is presented in Fig. 13 with the blue dotted curve. As in the HH
polarization case, the fitting process requires a sufficient number of
uncorrelated realizations to extract accurately the RCS; therefore the
relative difference between AC and RC results is the largest with the smallest
frequency bandwidth. Then, the relative error decreases as the frequency
bandwidth increases reaching a minimum around $0.9$ GHz. Beyond, the relative
error goes up. The result is less impressive but consistent with that obtained
in the HH polarization. This difference is only due to the smallest
fluctuation of the RCS over the azimuth angle in the VV polarization. As a
consequence, the smoothing effect on the extracted amplitude of a large
frequency bandwidth is less harmful to the RCS accuracy. The conclusion drawn
for HH configuration still holds: the amplitude bandwidth should be located in
the range from $0.5$ GHz to $1.5$ GHz.

##  VII Final RCS Results

In this section, the two-step method is applied using a $4$ GHz distance
bandwidth and $0.9$ GHz amplitude bandwidth. This smaller bandwidth is an
appropriate choice to minimize the relative difference between AC and RC
results with our RC and the considered rectangular metallic plate target.

###  VII-A HH Polarization

The two-step method is compared to the single-step one, i.e. choosing the same
bandwidth of $4$ GHz for both distance and amplitude estimations. Fig. 14
presents the RCS in the HH polarization obtained with two different bandwidths
along with the one measured in the AC (considered here as a reference).

Figure 14: RCS extracted using a distance computed on a $4$ GHz window and a
sine-wave amplitude estimated using a $0.9$ GHz bandwidth (two-step method) or
a $4$ GHz bandwidth (one-step method) versus the plate position in the
$-90^{\circ}$ to $+90^{\circ}$ azimuth range and in the HH polarization.

As expected, the result obtained with the one-step method does not replicate
the fluctuations of the reference RCS (AC measurement), i.e., the RCS side
lobes are smoothed out. On the contrary, the RCS obtained with the two-step
method better agrees with the reference RCS, enabling to retrieve local maxima
and minima. Therefore, it is demonstrated here that the two-step method
increases the accuracy of the RCS estimation within RCs. To quantitatively
compare the two RCS estimations within the RC, we compute the average relative
difference $\Delta$ according to (9). An average relative difference of
$234\%$ is reached between AC and RC measurements with a frequency bandwidth
of $4$ GHz (one-step method). This average relative difference is strongly
reduced down to $50\%$ with the smaller frequency bandwidth (two-step method).
This remaining error can be mainly explained by larger relative errors at low
RCS values and around local minima.

###  VII-B VV Polarization

The two-step method is now applied to the VV configuration in the same way.
Once again, Fig. 15 highlights the improvement of the extraction accuracy with
the two-step method. Whereas the relative difference is of $32\%$ with the
one-step method, it is decreased to $20\%$ with the two-step method. As
already observed, the differences are smaller in this polarization as the
minimal RCS values are larger than in the HH polarization. Thus, Fig. 15 shows
a satisfying agreement with the RCS measured in AC if the RCS amplitude is
computed on a small frequency bandwidth of $0.9$ GHz.

Figure 15: RCS extracted using a distance computed on a $4$ GHz window and a
sine-wave amplitude estimated using a $0.9$ GHz bandwidth (two-step method) or
a $4$ GHz bandwidth (one-step method) versus the plate position in the
$-90^{\circ}$ to $+90^{\circ}$ azimuth range and in the VV polarization.

Besides, it has to be noticed that the relative difference is increased by the
small angular offset observed on the measurements in regard to RCS maximum
location.

###  VII-C RCS extraction at different frequencies $f_{0}$

The developed method is now used to extract from the same measurement data the
RCS at different frequencies, that are $9$ GHz and $11$ GHz. First, we extract
the distance (or period) using the large bandwidth of $4$ GHz between $8$ GHz
and $12$ GHz. Then, the amplitude of the sinusoidal regression fit is
estimated with a frequency bandwidth of $0.9$ GHz centered around $9$ GHz on
the one hand or around $11$ GHz on the other hand. The HH polarization results
are shown in Fig. 16 and are also compared with the AC measurement.

(a) RCS at $9$ GHz in HH polarization

(b) RCS at $11$ GHz in HH polarization

Figure 16: Comparison of RCS extracted from RC measurements using a distance
bandwidth of $4$ GHz between $8$ GHz and $12$ GHz and an amplitude bandwidth
of $0.9$ GHz for different central frequencies ($9$ GHz and $11$ GHz) and RCS
obtained from AC measurements according to the plate position in the
$-90^{\circ}$ to $+90^{\circ}$ azimuth range and in the HH polarization.

These results indicate that the RCS estimation can be performed in RC for
arbitrary values of the central frequency, using a limited frequency windows
for amplitude estimation. The obtained results show that it is technically
feasible to estimate the RCS pattern over the $9$ GHz and $11$ GHz frequency
ranges.

The measurement data in the VV configuration are then processed to extract the
RCS pattern at $9$ GHz and $11$ GHz. The obtained results are shown in the
Fig. 17. Again, RCS local minima and maxima are well retrieved and a
satisfying agreement with the AC measurement is observed.

(a) RCS at $9$ GHz in VV polarization

(b) RCS at $11$ GHz in VV polarization

Figure 17: Comparison of RCS extracted from RC measurements using a distance
bandwidth of $4$ GHz between $8$ GHz and $12$ GHz and an amplitude bandwidth
of $0.9$ GHz for different central frequencies ($9$ GHz and $11$ GHz) and RCS
obtained from AC measurements according to the plate position in the
$-90^{\circ}$ to $+90^{\circ}$ azimuth range and in the VV polarization.

##  VIII Conclusion

This paper deals with the post-processing enhancement of an RCS measurement
method performed within RCs. The method is based on a differential measurement
(with and without the target) in a given frequency band, on which a sinusoidal
regression is applied in order to extract the RCS amplitude. A first step
consists in evaluating the antenna-target distance which sets the sine period,
whereas a second step is dedicated to the RCS amplitude estimation. While the
frequency bandwidth has been arbitrarily chosen in previous works, this
article investigates the proper selection of the bandwidth used for distance
and RCS amplitude estimations.

It is shown that a large frequency bandwidth allows a better estimation of the
antenna-target distance, by providing a finer distance resolution. In
particular, this allows discriminating multiple distinct scattering zones, as
for example the two edges of a plate. However, it is also shown that enlarging
the frequency bandwidth leads to a frequency-averaged RCS estimation. Indeed,
although it does not affect much the main lobe retrieval, it strongly degrades
the rest of the RCS pattern by smoothing out the secondary lobes. Therefore, a
two-step method is introduced in order to take into account these two
contradictory constraints.

The two-step method consists in defining two distinct frequency bandwidths:
one for the distance estimation (the distance bandwidth) and one for the
amplitude estimation (the amplitude bandwidth). The distance estimation is
firstly performed using an as-large-as-possible distance bandwidth in order to
better estimate the position of the main scattering contribution. Once the
distance has been precisely evaluated, the RCS amplitude estimation is
secondly performed using a reduced amplitude bandwidth in order to avoid the
frequency-averaging effect. However, although an as-narrow-as-possible
amplitude bandwidth is theoretically targeted, a minimum bandwidth is
mandatory in order to ensure a sufficient number of RC uncorrelated states (on
which the regression accuracy relies). It has been shown that in the case of
our specific setup, a $0.9$ GHz amplitude bandwidth exhibits the best trade-
off, while the distance bandwidth is fixed to 4 GHz.

RCS patterns of a rectangular metallic plate (152 mm $\times$ 150 mm) at $10$
GHz measured in an RC for $-90^{\circ}\leq\theta\leq 90^{\circ}$ (VV and HH
polarizations) have been compared to the ones measured in an AC. Compared to
the one-step method, the newly introduced two-step method enables reducing the
average relative difference from $234\%$ to $50\%$ in the HH polarization and
from $32\%$ to $20\%$ in the VV polarization. These results allow considering
RCs as a reliable alternative to anechoic chambers for RCS measurements. The
reliability and accuracy of the proposed method is confirmed for targets
assimilated as a point-like scatterer. Future works will be dedicated to the
study of resonating or multiple point-like scatterers.

## Acknowledgment

This work is supported by AID/DGA, France. It is also supported in part by the
European Union through the European Regional Development Fund, in part by the
Ministry of Higher Education and Research, in part by the Région Bretagne, and
in part by the Département d’Ille et Vilaine through the CPER Project
SOPHIE/STIC & Ondes.

## References

  * [1] P. Corona, G. Latmiral, E. Paolini and L. Piccioli, ”Use of a Reverberating Enclosure for Measurements of Radiated Power in the Microwave Range,” in IEEE Transactions on Electromagnetic Compatibility, vol. EMC-18, no. 2, pp. 54-59, May 1976.
  * [2] Hill, David A. ”Electromagnetic fields in cavities: deterministic and statistical theories”, John Wiley & Sons, 2009.
  * [3] Besnier, Philippe and Démoulin, Bernard, ”Electromagnetic Reverberation Chambers”, ISTE Wiley & Sons, 08 2011.
  * [4] Andrieu, Guillaume ”Electromagnetic Reverberation Chambers: Recent advances and innovative applications”, The Institution of Engineering and Technology, 02 2021.
  * [5] A. K. Puls, J. M. Ladbury and W. F. Young, ”Antenna Radiation Pattern Measurements Using a Reverberation Chamber,” in 2018 AMTA Proceedings, 2018, pp. 1-6.
  * [6] M. García-Fernández, D. Carsenat, and C. Decroze, ”Antenna radiation pattern measurements in reverberation chamber using plane wave decomposition,” in IEEE Transactions on Antennas and Propagation, vol. 61, no. 10, pp. 5000?5007.
  * [7] Q. Xu et al., ”3-D Antenna Radiation Pattern Reconstruction in a Reverberation Chamber Using Spherical Wave Decomposition,” in IEEE Transactions on Antennas and Propagation, vol. 65, no. 4, pp. 1728-1739, April 2017.
  * [8] W. Krouka, F. Sarrazin, J. Sol, P. Besnier and E. Richalot, ”Comparison of Antenna Radiation Efficiency Measurement Techniques in Reverberation Chamber Using or Not a Reference Antenna,” in 2020 14th European Conference on Antennas and Propagation (EuCAP), 2020, pp. 1-4.
  * [9] A. Reis, F. Sarrazin, P. Besnier, P. Pouliguen and E. Richalot, ”Contactless Antenna Gain Pattern Estimation From Backscattering Coefficient Measurement Performed Within a Reverberation Chamber,” IEEE Trans. Antennas Propag., vol. 70, no. 3, pp. 2318-2321, March 2022.
  * [10] K. A. Remley et al., ”Configuring and Verifying Reverberation Chambers for Testing Cellular Wireless Devices,” in IEEE Transactions on Electromagnetic Compatibility, vol. 58, no. 3, pp. 661-672, June 2016.
  * [11] A. Skårbratt, J. Åsberg and C. Orlenius, ”Over-the-air performance testing of wireless terminals by data throughput measurements in reverberation chamber,” in Proceedings of the 5th European Conference on Antennas and Propagation (EUCAP), 2011, pp. 615-619.
  * [12] X. Chen, P. -S. Kildal, C. Orlenius and J. Carlsson, ”Channel Sounding of Loaded Reverberation Chamber for Over-the-Air Testing of Wireless Devices: Coherence Bandwidth Versus Average Mode Bandwidth and Delay Spread,” in IEEE Antennas and Wireless Propagation Letters, vol. 8, pp. 678-681, 2009.
  * [13] R. Aminzadeh, J. Sol, P. Besnier, M. Zhadobov, L. Martens and W. Joseph, ”Estimation of Average Absorption Cross Section of a Skin Phantom in a mm-Wave Reverberation Chamber,” in 2019 13th European Conference on Antennas and Propagation (EuCAP), 2019, pp. 1-3.
  * [14] K. Jung, T. Kim, J. Kim, H. Doh, Y. Chung, J. Choi, and J. Pack, ”Development and validation of reverberation chamber type whole-body exposure system for mobile-phone frequency” in Electromagnetic biology and medicine, vol. 27, no. 1, pp. 73?82, 2007.
  * [15] P. F. Biagi, L. Castellana, T. Maggipinto, G. Maggipinto, T. Ligonzo, L. Schiavulli, D. Loiacono, A. Ermini, M. Lasalvia, G. Perna, and V. Capozzi, ”A reverberation chamber to investigate the possible effects of ”in vivo” exposure of rats to 1.8 GHz electromagnetic fields: a preliminary study,” in Progress In Electromagnetics Research, vol. 94, pp. 133?152, 2009.
  * [16] C. Lemoine, E. Amador, and P. Besnier, ?On the K-factor estimation for Rician channel simulated in reverberation chamber,? in IEEE Trans. on Antennas and Propagation, vol. 59, no. 3, pp. 1003-1012, 2011.
  * [17] A. Sorrentino, G. Ferrara, M. Migliaccio, and S. Cappa. ”Measurements of backscattering from a dihedral corner in a reverberating chamber” . in Applied Computational Electromagnetics Society Newsletter, 33:91?94, 01 2018.
  * [18] C. Lemoine, E. Amador, P. Besnier, J. -M. Floc’h and A. Laisné, ”Antenna Directivity Measurement in Reverberation Chamber From Rician K-Factor Estimation,” in IEEE Transactions on Antennas and Propagation, vol. 61, no. 10, pp. 5307-5310, Oct. 2013,
  * [19] M. Á. García-Fernández, D. Carsenat and C. Decroze, ”Antenna Gain and Radiation Pattern Measurements in Reverberation Chamber Using Doppler Effect,” in IEEE Transactions on Antennas and Propagation, vol. 62, no. 10, pp. 5389-5394, Oct. 2014.
  * [20] Soltane A, Andrieu G, and Reineix A., ”Monostatic radar cross-section estimation of canonical targets in reverberating room using time-gating technique ”. in International Symposium on Electromagnetic Compatibility, EMC EUROPE; 2018 Sep; Amsterdam, The Netherlands. IEEE; 2018. p. 355?359.
  * [21] P. Besnier, J. Sol, and S. Méric, “Estimating radar cross-section of canonical targets in reverberation chamber”, in 2017 International Symposium on Electromagnetic Compatibility - EMC EUROPE, 2017, pp. 1-5.
  * [22] A. Reis, F. Sarrazin, E. Richalot, S. Méric, J. Sol, P. Pouliguen, and P. Besnier,”Radar Cross Section Pattern Measurements in a Mode-Stirred Reverberation Chamber: Theory and Experiments”, in IEEE Transactions on Antennas and Propagation, vol. 69, no. 9, pp. 5942?5952.
  * [23] C. Charlo, P. Besnier and S. Méric, ”Quasi-monostatic Radar Cross-Section Measurement in Reverberation Chamber,” in 2021 18th European Radar Conference (EuRAD), 2022, pp. 94-97.
  * [24] D. A. Hill, ”Plane wave integral representation for fields in reverberation chambers,” in IEEE Transactions on Electromagnetic Compatibility, vol. 40, no. 3, pp. 209-217, Aug. 1998.
  * [25] J. W. Crispin and A. L. Maffett, ”Radar cross-section estimation for simple shapes,” in Proceedings of the IEEE, vol. 53, no. 8, pp. 833-848, Aug. 1965.
  * [26] R. Ross, ”Radar cross section of rectangular flat plates as a function of aspect angle,” in IEEE Transactions on Antennas and Propagation, vol. 14, no. 3, pp. 329-335, May 1966.